#  EffOCR: An Extensible, Open-Source Package for Efficiently Digitizing World
Knowledge

Tom Bryan1, Jacob Carlson1, Abhishek Arora1, Melissa Dell1,2∗
1 Harvard University; Cambridge, MA, USA.
2 National Bureau of Economic Research; Cambridge, MA, USA.
All authors contributed equally. ∗Corresponding author:
<EMAIL_ADDRESS>
###### Abstract

Billions of public domain documents remain trapped in hard copy or lack an
accurate digitization. Modern natural language processing methods cannot be
used to index, retrieve, and summarize their texts; conduct computational
textual analyses; or extract information for statistical analyses, and these
texts cannot be incorporated into language model training. Given the diversity
and sheer quantity of public domain texts, liberating them at scale requires
optical character recognition (OCR) that is accurate, extremely cheap to
deploy, and sample-efficient to customize to novel collections, languages, and
character sets. Existing OCR engines, largely designed for small-scale
commercial applications in high resource languages, often fall short of these
requirements. EffOCR (EfficientOCR), a novel open-source OCR package, meets
both the computational and sample efficiency requirements for liberating texts
at scale by abandoning the sequence-to-sequence architecture typically used
for OCR, which takes representations from a learned vision model as inputs to
a learned language model. Instead, EffOCR models OCR as a character or word-
level image retrieval problem. EffOCR is cheap and sample efficient to train,
as the model only needs to learn characters’ visual appearance and not how
they are used in sequence to form language. Models in the EffOCR model zoo can
be deployed off-the-shelf with only a few lines of code and include
lightweight models designed for mobile phones that are extremely cheap to
deploy. Importantly, EffOCR also allows for easy, sample efficient
customization with a simple model training interface and minimal labeling
requirements due to its sample efficiency. We illustrate the utility of EffOCR
by cheaply and accurately digitizing 20 million historical U.S. newspaper
scans, evaluating zero-shot performance on randomly selected documents from
the U.S. National Archives, and accurately digitizing a Japanese document
collection for which all other OCR solutions failed.

##  1 Introduction

Vast document collections remain trapped in hard copy or lack accurately
digitized texts. For example, the U.S. National Archives holds approximately
13.28 billion pages of textual records, most of which are in the public
domain.111For documents published in the United States, the public domain
includes any content published by a U.S. government officer/employee in the
course of official duties, all content published more than 95 years ago, and
some content published before 1989 that either wasn’t published with a notice
or did not renew copyright. This is common, for instance, in the case of
publications like local newspapers Ockerbloom (2019). See the supplementary
materials for details. These documents are preserved because they are central
to the workings of the U.S. government, have long-term research value, or
provide valuable information for the public, but working with most of them is
costly and time-consuming. The U.S. National Archives are not unique: many
other countries have national archives with public domain collections
numbering in the billions of pages, not to mention state and local archives
and libraries. Without accurate machine-readable data, modern natural language
processing (NLP) tools cannot be used to index, retrieve, and summarize
materials; conduct computational textual analyses; or extract information for
statistical investigations. Public domain texts, if accurately digitized,
could also provide massive scale information for training large language
models, with no risks of copyright infringement.

Using optical character recognition (OCR) to digitize public domain
collections on a large scale entails several challenges.

Cost: First, the OCR solution must be cheap to deploy, given document
collections whose size numbers in the millions or even billions of pages.
Commercial engines - as well as large open-source OCR models - fall well short
of this requirement. Using them to digitize large-scale collections would
require astronomical budgets.

Accuracy: Second, digitized texts need to be sufficiently accurate for end
users’ objectives, which are highly diverse. Accuracy can be particularly
central for quantitative applications, for which small errors can create major
statistical outliers. Models for lower resource languages, if they exist, tend
to perform much worse than models for high resource settings like English.

Sample efficient, easy training: Documents are highly heterogeneous in terms
of their fonts or handwritings, languages, scripts, backgrounds, and artifacts
from scanning and aging. There are a diversity of documents for which no
existing OCR solution works zero-shot, particularly in low resource languages.
Yet stakeholders who would like to digitize these documents rarely have
familiarity with deep learning frameworks. Bringing high quality OCR to low
resource settings requires a simple API for training and a sample efficient
architecture, with an accessible compute and annotation burden.

A diversity of pre-trained and tuneable models: Users have diverse accuracy
needs, scaling requirements, and budgets. A comprehensive OCR solution would
make it easy to compare the accuracy and deployment costs of models of varying
sizes so that users can choose the one that best suits their needs for a
particular application.

To meet these objectives, we developed EffOCR, an open-source OCR package
designed for researchers, libraries, and archives seeking a computationally
and sample efficient OCR solution for digitizing diverse document collections.
EffOCR has two key ingredients: 1) a novel OCR architecture and 2) a carefully
designed interface to facilitate off-the-shelf OCR usage, customization via
model training if necessary, and easy sharing of OCR models.

The novel EffOCR model architecture is treated in detail in Carlson et al.
(2023), where we compare accuracy, sample efficiency, and deployment costs to
a range of popular OCR engines. In short, OCR predominantly models text
recognition as a sequence-to-sequence (seq2seq) problem, in which learned
representations from a vision model are taken as inputs to a learned language
model. Learning how vision embeddings are used in sequence to form language
requires large amounts of data. For example, the predominant transformer
sequence-to-sequence OCR package Li et al. (2021b) was trained on 684 million
text lines using 32 32GB V100 GPU cards. State-of-the-art seq2seq OCR is
sample-inefficient to tune and infeasible for users to extend to low resource
languages, which may not even have a transformer large language model (LLM)
that can be used to initialize the model, as language modeling advances are
concentrated in less than two dozen languages Joshi et al. (2020). The typical
stakeholder working with low resource documents has a minimal budget for
training and limited experience with deep learning frameworks, underscoring
the need for a much more sample efficient framework with an easy-to-use API.

Additionally, seq2seq OCR requires autoregressive decoding, which makes
inference slower than it would be, all else equal, with parallel decoding.

EffOCR abandons the seq2seq OCR model that predominates in the literature,
instead modeling OCR as a word or character level image retrieval problem.
EffOCR first localizes words using highly accurate, scalable object detection
methods Ultalytics (2023); Chen et al. (2019); Wu et al. (2019). Recognition
is then modeled as a contrastively trained image retrieval problem, where
image embeddings of the same character or word have similar representations,
regardless of their style. EffOCR is trained primarily on digital fonts,
combined with a modest number of character and word crops from real-world
documents. At inference time, characters/words are recognized by computing
their nearest neighbor in an offline dictionary of exemplar embeddings created
with a digital font. Carlson et al. (2023) show, using English, Japanese, and
Polytonic Greek benchmarks, that the EffOCR architecture is accurate, highly
sample efficient, cheap to train, and extremely fast to deploy when using
backbones designed for mobile phones.

To meet the challenges of digitizing large-scale and low-resource document
collections, the EffOCR package contains the following components:

  1. 1.

An off-the-shelf toolkit for applying OCR models with just a few lines of code

  2. 2.

A repository of pre-trained OCR models that underlies off-the-shelf usage

  3. 3.

ONNX runtime support for fast deployment

  4. 4.

Comprehensive tools for efficient model tuning

  5. 5.

Supports models from popular backends Chen et al. (2019); Ultalytics (2023)
for initializing localization and any timm-supported model for initializing
recognition

  6. 6.

Easy sharing of models, to promote reusability, reproducibility, and
extensibility

EffOCR has been extensively tested. For example, we have used it to cheaply
digitize 20 million pages of historical public domain U.S. newspaper scans
that are extremely heterogeneous, posting the massive-scale output to Hugging
Face.222https://huggingface.co/datasets/dell-research-harvard/AmericanStories
Creating this dataset within our modest budget while meeting accuracy
requirements would have been impossible without EffOCR. We have also examined
performance in settings where no existing OCR solutions provide usable output,
and tested zero-shot performance on a random selection of U.S. National
Archive documents, with a model that did not see any similar content during
training. Tutorials are available at https://effocr.github.io/.

EffOCR has a GNU General Public License. It is being actively maintained and
crowd-sourcing of annotations to expand the pre-trained model zoo to other
languages and settings, including handwriting, is underway.

The rest of this paper is organized as follows. Section 2 briefly compares
EffOCR to existing, popular OCR solutions. Section 3 describes the key
features of the OCR package, and Section 4 examines several use cases: using
EffOCR to digitize 20 million historical newspaper scans, using EffOCR zero-
shot on randomly selected collections from the U.S. National Archives, and
using EffOCR to digitize a historical Japanese publication for which all
existing OCR solutions fail. Finally, Section 5 discusses the limitations of
the EffOCR package.

##  2 Comparisons to Other OCR Engines

There is a vast literature on OCR. Of primary interest here are widely used
OCR softwares, which are the most plausible alternatives to EffOCR.

EffOCR\- as the name suggests - is tailored towards applications requiring
computational or sample efficiency. Carlson et al. (2023) conduct detailed
experiments comparing the EffOCR architecture to other widely used solutions,
considering accuracy, sample efficiency, and computational efficiency. We
refer the interested reader to that paper for details, summarizing the two key
themes that emerge here.

Customization is highly relevant: As the preponderance of researchers still
using data entry firms suggests, sometimes no existing OCR solution provides
acceptable accuracy. For typewritten Japanese documents from the mid-20th
century, that are of considerable relevance to studying Japan’s remarkable
20th century growth performance, Carlson et al. (2023) show that the best
performing engine (Baidu, the leading commercial OCR for Asian languages) gets
over half of characters wrong. The widespread failure of OCR to provide
acceptable results is also evidenced by a large post-OCR error correction
literature (e.g., Lyu et al. (2021); Nguyen et al. (2021); van Strien. et al.
(2020)).

EffOCR is significantly more sample efficient than leading open-source OCR
engines: EasyOCR JaidedAI (2021), TrOCR Li et al. (2021b), and PaddleOCR Du et
al. (2022), as shown in the supplementary materials.333EasyOCR uses a seq2seq
convolutional recurrent neural network (CRNN) framework Shi et al. (2016),
TrOCR uses a seq2seq encoder-decoder transformer Li et al. (2021b), and
PaddleOCR’s uses Single Vision Text Recognition (SVTR), which like EffOCR
abandons seq2seq, dividing text images into small (non-character) patches,
using mixing blocks to perceive inter- and intra-character patterns, and
recognizing text by linear prediction Du et al. (2022). Learning to recognize
the visual features of individual characters is a highly parsimonious problem,
making EffOCR cheap to tune or train from scratch. Because EffOCR does not
need to understand language, it is straightforward to extend to new languages
and scripts, including those that lack a transformer large language model to
initialize state-of-the-art seq2seq. The convolutional models in the EffOCR
model zoo can be trained on a Google Colab account, whereas training TrOCR on
684 million text lines required 32 32GB V100 cards.

A central aim of EffOCR is to democratize access to OCR to low resource
languages and settings that are difficult to study because existing solutions
are not suitable to these use cases. While we do not have the resources to
train OCR models for all these settings, our simple APIs for training models
and uploading them to the EffOCR model hub can encourage the crowdsourcing of
this effort.

The most accurate OCR engines in high resource settings (e.g., English) are
costly to deploy at scale: TrOCR (Base) is a highly accurate state-of-the-art
English OCR. With 334 million parameters, it is nearly 50 times slower to
deploy than our pre-trained lightweight EffOCR English word recognition model,
while offering only relatively modest gains on the evaluation tasks in Carlson
et al. (2023).444TrOCR has a small model (62M parameters), but Carlson et al.
(2023) find it is outperformed by the 334M parameter base model by a wide
margin on historical documents. For English, Google Cloud Vision (GCV) - a
proprietary commercial product - dominated all open-source solutions
(including EffOCR), but would have been orders of magnitude more costly to
deploy. In our experience, it is frequently outside academic budgets for
larger projects.

Lightweight EffOCR models are also faster than Tesseract and PaddleOCR - with
the comparison to EasyOCR depending on the hardware used for deployment. This
is despite having around 8x more parameters than Tesseract and around 4x more
than EasyOCR (parameter count is similar to PaddleOCR). This is achieved
through parallel rather than sequential decoding and ONNX integration. EffOCR
is also significantly more accurate on tasks like digitizing the 20 million
U.S. historical newspaper scans.

Users for whom neither computational nor sample efficiency is of concern -
because they are working in a well-resourced context and don’t face cost
constraints for the scale of their problem - are not our target audience and
may well find an existing OCR engine like Google Cloud Vision better meets
their needs. In practice, academic or large-scale archival digitization of
document collections often involves low-resource languages or settings, tight
budget constraints, or both.

##  3 The EffOCR Library

###  3.1 Off-the-shelf Usage

At the core of EffOCR is an off-the-shelf toolkit. EffOCR is a modular
framework, that first localizes lines, characters, and (for some models) words
using object detection, and then recognizes characters and words by embedding
their crops and retrieving their nearest neighbor from an offline index of
exemplar embeddings created from a digital font.

Localization: EffOCR supports two widely used backends for localization
inference: MMDetection Chen et al. (2019), which includes state-of-the-art
object detection models, and Yolo Ultalytics (2023), which includes fast,
efficient object detection models. Users can deploy line, word, and character
models from the pre-trained model zoo, that use Yolo v8 Ultalytics (2023)
(optimized for efficiency), Yolo v5 Jocher (2020) (fewer dependencies) or
Cascade R-CNN Cai and Vasconcelos (2018) (optimized for accuracy). Pre-trained
localization models are available for alphabetic English/Latin, Polytonic
Greek, and CJK characters (which vary significantly in their aspect ratios and
groupings).

Recognition: EffOCR recognizes word and character crops using contrastively
trained image retrieval models. The EffOCR model zoo currently contains 30
pre-trained models, covering English, Polytonic Greek, and horizontally and
vertically-written Japanese. We chose these languages to examine the utility
of EffOCR in a high resource setting, in a setting where existing solutions
fail, and in an intermediate case.

The EffOCR pre-trained models use a variety of backbones: two lightweight
convolutional backbones that are very efficient to deploy Howard et al.
(2019); Maaz et al. (2022), a state-of-the-art CNN encoder Liu et al. (2022),
and three vision transformers Ali et al. (2021); Li et al. (2022); Liu et al.
(2021). For English, there is a word level model that defaults to character
recognition when the word is below a default (tuneable) cosine similarity
threshold, as well as a character-only model.

The documentation provides more guidance on model selection. A description of
the training dataset is provided alongside with the trained models such that
users can quickly identify the most suitable models for their tasks.

EffOCR can be used off-the-shelf with just a few lines of code:

⬇

1import effocr

2engine = effocr.EffOCR(

3 line_detector = "./line_model",

4 localizer = "./localizer",

5 char_recognizer = "./char_recognizer",

6 word_recognizer = "./word_recognizer"

7)

8results = engine.infer(’image.jpg’)

ONNX ONNX (2021) integration is an important component, as it allows for
efficient CPU deployment and interoperability between deep learning
frameworks. All EffOCR stages can optionally employ ONNX-format models and
ONNX-runtime inference, and models can be converted to ONNX format within the
package. ONNX-runtime increases CPU throughput by up to four times Jocher
(2020) for YOLO models used in EffOCR, which allows for cost-effective cloud
deployment for processing large document sets. ONNX compatibility allows
additional model speedups through graph optimizations, quantization, and
pruning.

###  3.2 Customized Model Training

Many low-resource settings are poorly served by existing OCR engines, and a
central aim of EffOCR is to democratize OCR for these settings by providing a
simple interface for custom model training that can be used by researchers and
others who have limited experience with deep learning frameworks. Custom
training can be initialized using a Yolo object detection model for
localization and any timm image encoder model for recognition. In the near
future, support for training localization models with MMDetection will be
added. This futureproofs EffOCR, as new models are developed.

EffOCR supports logging of a training run on Weights and Biases Biewald
(2020). It takes industry standard coco json labels as inputs, and hence is
compatible with the outputs of a range of both open-source and proprietary
labeling softwares. It also exports output in the same format, so that users
can easily correct model predictions if desired to speed up labeling.

Model training with EffOCR is highly efficient, e.g., the convolutional
backbones can be trained on Google Colab. We trained all models on either a
single Nvidia RTX 3090 or A6000 card.

###  3.3 Visualization, Storage and Export

EffOCR comes with a tool to visualize the OCR, side-by-side with the original
image, as well as to visualize the line, word, and character predictions.
These greatly facilitate quality checking the output and troubleshooting
potential problems.

EffOCR offers users different options for data export. The default outputs of
EffOCR include line coordinates, word coordinates, character coordinates, and
the text associated with each of these annotations. The text for the full
image is also assembled in the correct order. Users may choose to export only
the assembled text, only text annotations associated with a given level of
bounding box (line, word, or character), or all of the above.

###  3.4 User Contributions

By making OCR sample efficient and easy to train, EffOCR aims to promote the
reusability and reproducibility of OCR pipelines. This is particularly
important for low resource settings and languages, where there is little
commercial incentive for product development and few alternatives to crowd-
sourcing models. EffOCR users can upload their self-trained models to the
EffOCR Hugging Face hub. Whenever a model is saved, a model card is
automatically generated that follows best practices outlined in Hugging Face’s
Model Card Guidebook.555https://huggingface.co/docs/hub/model-card-guidebook
Moreover, the automatically generated card contains instructions on how to use
the model in the context of EffOCR and model-specific architecture and
training details in the interest of reproducibility.

###  3.5 Integration with Layout Parser

OCR engines typically detect lines, versus detecting and classifying different
layout objects in a document. Many documents have complex layouts - e.g.,
newspapers have headlines, articles, captions, ads, and headers arranged in
complex multicolumn layouts, and tables likewise have different types of
information arranged in oftentimes complex layouts. These structures
necessitate applying object detection models for document layout analysis,
which have been trained to detect the coordinates of each layout object and
classify its type (e.g., headline, articles, etc).

To facilitate combining EffOCR with deep learning-based document layout
models, wrappers will be integrated into a popular open-source layout
detection package, Layout Parser Shen et al. (2020), that will allow Layout
Parser users to call any EffOCR model. Layout Parser also has wrappers to call
GCV and Tesseract, which will allow users to easily compare EffOCR output to
these other packages to decide what best meets their accuracy and cost
objectives. Layout Parser and EffOCR were designed by the same lab,
facilitating long-run coordination between the packages.

##  4 Applications

Figure 1: This figure shows a diversity of examples processed with EffOCR,
with predicted transcriptions on the right.

Scalability: We have tested the utility of EffOCR with various real-world
applications. In the first application, we cheaply and accurately digitized 20
million newspaper page scans from Library of Congress’s Chronicling America
collection Library of Congress (2022). The resulting dataset, American
Stories, is available for download on Hugging
Face.666https://huggingface.co/datasets/dell-research-harvard/AmericanStories.
Figure 1 illustrates why this is a challenging task: newspapers are extremely
heterogeneous in their fonts and image quality. Dell et al. (2023) provide a
detailed analysis of the quality of the resulting text dataset.

We first trained character EffOCR using synthetic data plus a labeled set of
291 newspaper lines Carlson et al. (2023), created in a couple of hours. We
then bootsrapped word level annotations by creating them with the character
level EffOCR model, filtering out lines with a high non-word rate.

With EffOCR, combined with layout analysis using Layout Parser, we could
digitize the dataset with a $60K USD cloud compute budget (plus pipeline
development costs). GCV makes significant layout errors when fed full
newspaper scans and achieves best performance when fed individual lines. At
current prices, digitizing the collection at the line level, since GCV charges
per image, would have cost over $23 million USD. TrOCR Base, the most accurate
open-source OCR, would have exceeded our budget by a factor of nearly 50.

Zero-Shot Performance: Second, we show that our English lightweight word-level
model has strong zero-shot performance on randomly selected document
collections from the U.S. National Archives. This model saw only newspapers in
training, to test true zero shot performance. We selected a single textline
from each of 300 random documents from separate National Archive record
groups. EffOCR achieved a 11.2% CER on the diverse collection, compared with a
11.8% CER from Tesseract (Best), a 12.1% CER from EasyOCR, and a 51% CER from
TrOCR (Small), which appeared to struggle with blurry and partially obscured
text. We suspect the results could be significantly improved by including a
random sample of documents from the National Archives in training, to broaden
the set of real world documents that the model is exposed to.

All open-source models performed significantly worse than GCV (1.2% CER), but
as discussed earlier cost concerns presently preclude its use at scale.
Despite being engineered for low-resource, few-shot contexts, EffOCR remains
competitive in high-resource, zero-shot situations.

Low Resource Settings: Finally, we use EffOCR to digitize historical Japanese
firm level records for vertically written Japanese documents Teikoku Koshinjo
(1957), where the best available solution (from Baidu OCR) mispredicts over
half of characters. We use the evaluation set in Carlson et al. (2023), which
consists of randomly selected segments that were double labeled.

Using a training set of 898 labeled table cells, we achieve a CER of 0.7%, 80
times more accurate than the best existing solution. As a result, we are able
to study a variety of questions about Japan’s remarkable growth performance
that would have been impossible to examine without EffOCR.

To further examine the limits of sample efficiency, we calculate the character
classification error when the (character) model only sees one (or up to 5)
labeled character(s) for each of the characters that appear in the training
set, which comprise 77% of the characters in the test set. This results in
character classification errors of 13.4% and 2.0% respectively. While the
model does clearly benefit from seeing multiple crops of characters that
appear frequently, this illustrates viable few shot performance.

##  5 Limitations

If large portions of a document are illegible, vision-only OCR will not be
suitable and language understanding may be helpful for inferring content. For
high resource languages such as English when cost is not a concern, users may
get the best mileage from a leading commercial product such as GCV.

Currently, the EffOCR model zoo has pre-trained models supporting typewritten
English, Japanese, and Polytonic Greek. Over the coming months, we will be
crowd-sourcing annotations (including handwriting) from package users and
colleagues. We will use them, along with digital fonts, to pre-train
additional models. In addition, users are encouraged to contribute their
models.

EffOCR does not currently support handwriting. We started with typewritten
documents because there are billions of public domain typeface documents that
are of considerable interest to researchers and the general public. We are
planning to expand the model zoo to include handwriting and users have already
offered to contribute annotations. Synthetic handwriting generators, e.g.
Bhunia et al. (2021), can provide extensive data for pre-training for scripts
that they support, analogous to the use of digital fonts for typeface
documents. We will make synthetic handwriting datasets available so that
package users can also use them for training their own custom models.

## Supplementary Materials

##  S-1 Model Architecture and Model Zoo

Figure S-1 shows the EffOCR model architecture, and Table S-1 summarizes the
models in the EffOCR model zoo. Readers seeking technical details for the
EffOCR models contained in the pre-trained model zoo are referred to the
detailed supplementary materials in Carlson et al. (2023).

##  S-2 Sample Efficiency

To examine how efficiently EffOCR learns in comparison to leading open source
architectures, we train different OCR models from scratch using varying
amounts of annotated data. EffOCR-C (Base) is compared to SVTR (implemented
via PaddleOCR) Du et al. (2022), CRNN (implemented via EasyOCR) Shi et al.
(2016), and TrOCR Li et al. (2021b). All architectures are pre-trained from
scratch on 8,000 synthetic text lines, starting from pre-trained checkpoints
not customized for OCR when supported by the framework. They are then fine-
tuned on the study’s benchmark datasets, with varying train-test-validation
splits: 70%-15%-15%, 50%-25%-25%, 20%-40%-40%, 5%-47.5%-47.5%, and 0%-50%-50%
(i.e., zero-shot). These exercises are performed for the English newspaper
character level models and horizontal Japanese, as vertical Japanese is not
supported by the comparison architectures.

Figure S-2 plots the percentage of the benchmark dataset used in training on
the x-axis and the CER on the y-axis. On just 99 labeled table cells for
Japanese and 21 labeled rows for LoCCA (the 5% train split), EffOCR’s CER is
only 5% (Japanese) and 7% (English), showing viable few shot performance. The
other architectures remain unusable. EffOCR performs nearly as well using 20%
or training data as using 70%, where it continues to outperform all other
alternatives. This illustrates that its parsimonious architecture learns
efficiently.

##  S-3 Training Config Details

The EffOCR package exposes a wide variety of training options and
hyperparameters to users. A few key elements are described here, readers
looking for more details are directed to the package documentation.

Recognizer Training Options:

  * •

timm_model_name Model name from timm Wightman (2019) package used as a base
encoder for the recognizer.

  * •

render_dict Folder to store crop renders and gold training data locally.

  * •

font_dir_path Local path to draw tff (font) files from, which are used to
create character/word renders.

  * •

hns_txt_path Local file path to draw hard negative samples from. Hard negative
text files are created by default at the end of recognizer training. Most
recognizer training applications use two stages, an initial run and a hard
negative sampling run.

  * •

latin_suggested_args Uses default arguments for alphabetic writing systems
such as Latin, Greek, and Cyrillic.

In addition to these options, a wide variety of standard model training
parameters are exposed, including learning rate, optimizer options, weight
decay, batch size, device selection, and number of training epochs.

Localizer Training Options:

  * •

vertical Whether model should expect characters aligned horizontally (as in
English and many Latin scripts) or vertically (as in many character-based
scripts).

  * •

no_words Detect only characters, not words. Recommended for languages without
word groupings.

  * •

iou_thresh Training and validation IOU threshold for character/word detection.

  * •

conf_thresh Training and validation confidence threshold for character/word
detection.

As with the recognizer, other standard training parameters are exposed. In
particular, adjusting the image input shape may be valuable for particularly
long or short lines.

Hyperparameters and training procedure used to generate models listed in the
Model Zoo (Table S-1) are listed in Carlson et al. (2023).

##  S-4 Visualization

Figure S-3 shows the EffOCR visualization interface.

##  S-5 American Stories

Figure S-4 plots the number of articles in the American Stories dataset,
created with EffOCR, across time.

##  S-6 The Public Domain

Table S-2 provides detailed information about the requirements for information
published in the United States to be in the public domain, in order to give
readers a better sense of these collections.

##  S-7 Inference Speed

EffOCR implements two features designed to increase computational efficiency.
First, both localization and recognition inference is run in a multithreaded
fashion, ensuring that compute resources are fully utilized. Second, EffOCR
provides support for ONNX runtime and ONNX-format models, which provide up to
a 3x speedup on a CPU compared to native PyTorch runtime ONNX (2021). GPUs are
typically cost prohibitive for digitization at scale.

Table S-3 provides a comparison between EffOCR and other commonly used OCR
frameworks’ python implementations. It is important to note that these numbers
- across softwares - can vary significantly dependending on the hardware
resources available. All comparisons are made on four 2200 MHz CPU cores,
selected to represent a plausible and relatively affordable research compute
setup. EffOCR performance is competitive with other widely used frameworks,
with EffOCR (Small) having the fastest performance. Tesseract Ooms (2023)
testing used the pytesseract package with default settings. EasyOCR JaidedAI
(2021) testing used the easyocr package with default English settings.
PaddleOCR PaddlePaddle (2022) testing used the paddleocr package with
use_angle_cls option and default English settings. TrOCR Li et al. (2021a)
testing used the transformers package implementation, with trocr-base-printed
and trocr-small-printed models for Base and Small tests, respectively. EffOCR
testing used default settings with pretrained ONNX English newspaper models
from the model zoo.

Figure S-1: EffOCR and Seq2Seq Model Architectures. This figure represents the
EffOCR architecture, as compared to a typical sequence-to-sequence OCR
architecture.

Figure S-2: Sample Efficiency. This figure plots the percentage of the
benchmark dataset used in training against the character error rate, for
different OCR model architectures: CRNN (EasyOCR), SVTR (PaddleOCR), TrOCR
(Transformer OCR), and EffOCR small and base convolutional models.

Figure S-3: Visualization. This figure shows the EffOCR visualization
interface.

Figure S-4: American Stories. This figure plots the number of articles in the
American Stories dataset, created with EffOCR, across time.

Training Set | Line Detection | Localizer | Word Recognition | Character Recognition
---|---|---|---|---
| YOLO | YOLO | MaskRCNN | MobileNetV3 | EdgeNeXt | MobileNetV3 | EdgeNeXt | ViT | ConvNeXt | XCiT
English Newspapers | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓
English Mixed Archival | ✓ | - | ✓ | ✓ | - | ✓ | - | - | - | -
Japanese Vertical | ✓ | ✓ | ✓ | N/A | N/A | ✓ | ✓ | ✓ | ✓ | ✓
Japanese Horizontal | ✓ | ✓ | ✓ | N/A | N/A | ✓ | ✓ | ✓ | ✓ | ✓
Polytonic Greek | ✓ | ✓ | - | N/A | N/A | ✓ | - | - | ✓ | -

Table S-1:  Models Currently Available in the EffOCR Model Zoo. Note Japanese
models do not use word-level recognition.

Date of Publication | Conditions | Copyright Term
---|---|---
Public Domain |  |
Anytime | Works prepared by an officer/employee of the | None
| U.S. Government as part of their official duties |
Before 1928 | None | None. Copyright expired.
1928 through 1977 | Published without a copyright notice | None. Failure to comply with required formalities
1978 to 1 March 1989 | Published without notice and | None. Failure to comply with required formalities
| without subsequent registration within 5 years |
1928 through 1963 | Published with notice | None. Copyright expired
| but copyright was not renewed |
Copyrighted |  |
1978 to 1 March 1989 | Published without notice, but with | 70 (95) years after the death of author (corporate author)
| subsequent registration within 5 years |
1928 through 1963 | Published with notice | 95 years after publication
| and the copyright was renewed |
1964 through 1977 | Published with notice | 95 years after publication
1978 to 1 March 1989 | Created after 1977 and published with notice | 70 (95) years after the death of author (corporate author)
|  | or 120 years after creation, if earlier
1978 to 1 March 1989 | Created before 1978 and first published | The greater of the term specified in the previous entry
| with notice in the specified period | or 31 December 2047
From 1 March 1989 through 2002 | Created after 1977 | 70 (95) years after the death of author (corporate author)
|  | or 120 years after creation, if earlier
From 1 March 1989 through 2002 | Created before 1978 and | The greater of the term specified in the previous entry
| first published in this period | or 31 December 2047
After 2002 | None | 70 (95) years after the death of author (corporate author)
|  | or 120 years after creation, if earlier

Table S-2: This table summarizes U.S. copyright law, based on a similar table
produced by the Cornell libraries. For concision, we focus on works initially
published in the United States. A variety of other cases are also covered at
https://guides.library.cornell.edu/copyright.

Model | Textline/s | Article/s
---|---|---
EffOCR Base | 0.46 | 0.02
EffOCR Small | 21.07 | 1.08
Tesseract | 4.47 | 0.21
EasyOCR | 19.80 | 1.03
PaddleOCR | 13.56 | 0.61
TrOCR (Base) | 0.43 | 0.02
TrOCR (Small) | 0.97 | 0.05
Table S-3: Comparison of EffOCR speeds with other popular OCR frameworks in
CPU environment. Tests included both Textline (single lines of text) and
Article (5-40 lines of text) examples.

## References

  * Ali et al. (2021) Alaaeldin Ali, Hugo Touvron, Mathilde Caron, Piotr Bojanowski, Matthijs Douze, Armand Joulin, Ivan Laptev, Natalia Neverova, Gabriel Synnaeve, Jakob Verbeek, et al. 2021.  Xcit: Cross-covariance image transformers.  _Advances in neural information processing systems_ , 34.
  * Bhunia et al. (2021) Ankan Kumar Bhunia, Salman Khan, Hisham Cholakkal, Rao Muhammad Anwer, Fahad Shahbaz Khan, and Mubarak Shah. 2021.  Handwriting transformers.  _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 1086–1094.
  * Biewald (2020) Lukas Biewald. 2020.  Experiment tracking with weights and biases.  Software available from wandb.com.
  * Cai and Vasconcelos (2018) Zhaowei Cai and Nuno Vasconcelos. 2018.  Cascade r-cnn: Delving into high quality object detection.  _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 6154–6162.
  * Carlson et al. (2023) Jacob Carlson, Tom Bryan, and Melissa Dell. 2023.  Efficient ocr for building a diverse digital history.  _arXiv preprint arXiv:2304.02737_.
  * Chen et al. (2019) Kai Chen, Jiaqi Wang, Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, et al. 2019.  Mmdetection: Open mmlab detection toolbox and benchmark.  _arXiv preprint arXiv:1906.07155_.
  * Dell et al. (2023) Melissa Dell, Jacob Carlson, Tom Bryan, Emily Silcock, Abhishek Arora, Zejiang Shen, Luca D’Amico-Wong, Quan Le, Pablo Querubin, and Leander Heldring. 2023.  American stories: A large-scale structured text dataset of historical u.s. newspapers.
  * Du et al. (2022) Yongkun Du, Zhineng Chen, Caiyan Jia, Xiaoting Yin, Tianlun Zheng, Chenxia Li, Yuning Du, and Yu-Gang Jiang. 2022.  Svtr: Scene text recognition with a single visual model.  _arXiv preprint arXiv:2205.00159_.
  * Howard et al. (2019) Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, et al. 2019.  Searching for mobilenetv3.  _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 1314–1324.
  * JaidedAI (2021) JaidedAI. 2021.  Easyocr.  https://github.com/JaidedAI/EasyOCR.
  * Jocher (2020) Glenn Jocher. 2020.  YOLOv5 by Ultralytics.
  * Joshi et al. (2020) Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020.  The state and fate of linguistic diversity and inclusion in the nlp world.  _arXiv preprint arXiv:2004.09095_.
  * Li et al. (2021a) Minghao Li, Tengchao Lv, Lei Cui, Yijuan Lu, Dinei Florencio, Cha Zhang, Zhoujun Li, and Furu Wei. 2021a.  Trocr github repository.  https://github.com/microsoft/unilm/tree/master/trocr.
  * Li et al. (2021b) Minghao Li, Tengchao Lv, Lei Cui, Yijuan Lu, Dinei Florencio, Cha Zhang, Zhoujun Li, and Furu Wei. 2021b.  Trocr: Transformer-based optical character recognition with pre-trained models.  _arXiv preprint arXiv:2109.10282_.
  * Li et al. (2022) Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He. 2022.  Exploring plain vision transformer backbones for object detection.  _arXiv preprint arXiv:2203.16527_.
  * Library of Congress (2022) Library of Congress. 2022.  Chronicling America: Historic American Newspapers.
  * Liu et al. (2021) Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021.  Swin transformer: Hierarchical vision transformer using shifted windows.  _arXiv preprint arXiv:2103.14030_.
  * Liu et al. (2022) Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. 2022.  A convnet for the 2020s.  _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 11976–11986.
  * Lyu et al. (2021) Lijun Lyu, Maria Koutraki, Martin Krickl, and Besnik Fetahu. 2021.  Neural ocr post-hoc correction of historical corpora.  _Transactions of the Association for Computational Linguistics_ , 9:479–483.
  * Maaz et al. (2022) Muhammad Maaz, Abdelrahman Shaker, Hisham Cholakkal, Salman Khan, Syed Waqas Zamir, Rao Muhammad Anwer, and Fahad Shahbaz Khan. 2022.  Edgenext: efficiently amalgamated cnn-transformer architecture for mobile vision applications.  In _European Conference on Computer Vision_ , pages 3–20. Springer.
  * Nguyen et al. (2021) Thi Tuyet Hai Nguyen, Adam Jatowt, Mickael Coustaty, and Antoine Doucet. 2021.  Survey of post-ocr processing approaches.  _ACM Comput. Surv._ , 54(6).
  * Ockerbloom (2019) John Mark Ockerbloom. 2019.  Newspaper copyrights, notices, and renewals.
  * ONNX (2021) ONNX. 2021.  Onnx runtime.  https://www.onnxruntime.ai.  Version: x.y.z.
  * Ooms (2023) J Ooms. 2023.  Tesseract: Open source ocr engine.
  * PaddlePaddle (2022) PaddlePaddle. 2022.  PaddleOCR.
  * Shen et al. (2020) Zejiang Shen, Kaixuan Zhang, and Melissa Dell. 2020.  A large dataset of historical japanese documents with complex layouts.  _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , pages 548–549.
  * Shi et al. (2016) Baoguang Shi, Xiang Bai, and Cong Yao. 2016.  An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition.  _IEEE transactions on pattern analysis and machine intelligence_ , 39(11):2298–2304.
  * Teikoku Koshinjo (1957) Teikoku Koshinjo. 1957.  _Teikoku Ginko Kaisha Yoroku_.  Teikoku Koshinjo.
  * Ultalytics (2023) Ultalytics. 2023.  Yolo v8 github repository.  https://github.com/ultralytics/ultralytics.
  * van Strien. et al. (2020) Daniel van Strien., Kaspar Beelen., Mariona Coll Ardanuy., Kasra Hosseini., Barbara McGillivray., and Giovanni Colavizza. 2020.  Assessing the impact of ocr quality on downstream nlp tasks.  In _Proceedings of the 12th International Conference on Agents and Artificial Intelligence - Volume 1: ARTIDIGH,_ , pages 484–496. INSTICC, SciTePress.
  * Wightman (2019) Ross Wightman. 2019.  Pytorch image models.  https://github.com/rwightman/pytorch-image-models.
  * Wu et al. (2019) Yuxin Wu, Alexander Kirillov, Francisco Massa, Wan-Yen Lo, and Ross Girshick. 2019\.  Detectron2.  https://github.com/facebookresearch/detectron2.


for all $A$, $b$, and $\alpha$. We express the number of black holes sampled
as a fraction of the total, removing the dependence on $A$ and $b$:

$\frac{1}{N_{\rm
total}}\frac{d^{2}N}{d\Omega}(\theta)=\frac{(\sin\theta)^{\alpha-1}}{4\pi}.$
(D.5)

It is then straightforward to observe that the fraction of black holes in the
sky instantaneously observed by a single baseline decreases with $\alpha$.
However, it is less straightforward to compute the cumulative number of unique
sources a baseline resolves over the course of some rotation through space,
because the sources sampled at each baseline orientation are partially
redundant with the source sampled at prior baseline orientations. Put another
way, the instantaneous sampling of black holes provided by a single baseline
is given by a sweeping of the projected baseline through $\phi$, whereas the
sampling over a change in orientation is a sweep through a new angle, which we
call $\chi$. The angle $\chi$ is related to $\theta$ by

$\chi=\tan^{-1}\left(\sin\phi\tan\theta\right).$ (D.6)

$\chi$ is defined so that it aligns with $\theta$ when $\phi=\pi/2$. An
illustration of $\chi$ and the redundant sampling are shown (for the
$\phi=\pi/2$ cross-section) in the middle panel of Figure 18.

The rotation of the baseline in space can then be described by a change in the
axial angle $\Delta\chi$. Computing Equation 23 for the total sources sampled
after this rotation (shown by the dashed red surface in the middle panel of
Figure 18) can be simplified by breaking the integral into two regions: first,
the longest projected baseline sweeps out a partial spheroid, while the rest
of the baselines form arcs that intersect at the cusps of the dashed red line
in Figure 18. We refer to the sources sampled by the partial spheroid as
$N_{s}$ and those sampled by the cusped curves as $N_{4}$. As $\Delta\chi$
increases, $N_{s}$ increases and $N_{4}$ decreases. The cusped curves have a
four-fold symmetry, so we integrate over a convenient curve (that between
$\theta=\pi/2$ and $\theta=\pi$), and Equation 23 reduces to

$N(\Delta\chi)=4N_{4}(\Delta\chi)+N_{s}(\Delta\chi),$ (D.7)

where

$N_{4}(\Delta\chi)=\int_{0}^{\pi}\text{d}\phi\int_{\pi/2}^{\theta_{\text{max}}(\Delta\chi)}\sin\theta\frac{d^{2}N}{d\Omega}(\theta)\,\text{d}\theta$
(D.8)

is the contribution from each of the cusped curves,

$N_{s}(\Delta\chi)=\frac{\Delta\chi}{\pi}\times
4\pi\left[\frac{d^{2}N}{d\Omega}\left(\theta=\frac{\pi}{2}\right)\right]$
(D.9)

is the contribution from the partial spheroid, and

$\theta_{\text{max}}(\Delta\chi)=\tan^{-1}\left(\frac{\tan\left(\pi-\frac{\Delta\chi}{2}\right)}{\sin\phi}\right)$
(D.10)

is the value of $\theta$ at the first leading cusp. The geometry of this
computation is shown between the dotted gray lines in the middle panel of
Figure 18.

The right panel of Figure 18 shows the cumulative source sampling for several
values of the power-law index $\alpha$ from Equation D.5. We use $\alpha=2$
for the schematic diagrams in Figure 18 because the sampled surface around an
instantaneous baseline in this case reduces to a torus; $\alpha=4$ corresponds
to a sky density of black holes that scales volumetrically (i.e., proportional
to $\theta_{r}^{-3}$), which is closer to the actual behavior. For plausible
values of $\alpha$, we find that the difference between a fully swept sampling
of the sky and the instantaneous baseline sampling is not more than a factor
of 2.


# Simultaneous scintillation light and charge readout of a pure argon filled
Spherical Proportional Counter

R. Bouet  J. Busto  V. Cecchini  C. Cerna  A. Dastgheibi-Fard  F. Druillole
C. Jollet  P. Hellmuth  I. Katsioulas  P. Knights  I. Giomataris  M. Gros  P.
Lautridou  A. Meregaglia  X. F. Navick  T. Neep  K. Nikolopoulos  F. Perrot
F. Piquemal  M. Roche  B. Thomas  R. Ward  M. Zampaolo  CENBG, Université de
Bordeaux, CNRS/IN2P3, 33175 Gradignan, France  CPPM, Université d’Aix-
Marseille, CNRS/IN2P3, F-13288 Marseille, France  SUBATECH, IMT-Atlantique,
Université de Nantes, CNRS/IN2P3, France  LSM, CNRS/IN2P3, Université
Grenoble-Alpes, Modane, France  School of Physics and Astronomy, University of
Birmingham, B15 2TT, UK  IRFU, CEA, Université Paris-Saclay, F-91191 Gif-sur-
Yvette, France

###### Abstract

The possible use of a Spherical Proportional Counter for the search of
neutrinoless double beta decay is investigated in the R2D2 R&D project. Dual
charge and scintillation light readout may improve the detector performance.
Tests were carried out with pure argon at 1.1 bar using a 6$\times$6 mm2
silicon photomultiplier. Scintillation light was used for the first time to
trigger in a spherical proportional counter. The measured drift time is in
excellent agreement with the expectations from simulations. Furthermore the
light signal emitted during the avalanche development exhibits features that
could be exploited for event characterisation.

###### keywords:

Spherical TPC , neutrino , neutrinoless double beta decay , scintillation

††journal: N.I.M. A

##  1 Introduction

The use of Spherical Proportional Counters (SPC) in direct dark matter
searches has been going on for a decade within the NEWS-G collaboration [1, 2,
3, 4]. More recently, the possibility to use such a technology to search for
$\beta\beta 0\nu$ decay has been investigated [5] and an R&D programme is
ongoing [6]. The aim is to tackle the various technical challenges to be faced
for the construction of a large SPC, up to 1 m radius, filled with 136Xe at 40
bar and simultaneously to demonstrate that energy resolution of 1$\%$ FWHM,
and the recognition and radial localization of the two searched-for beta
tracks are possible with this very simple setup. The proposed R&D programme
addresses the different topics sequentially, before designing the final
apparatus optimized for the targeted physics program.

So far, very encouraging results on the energy resolution (1.1$\%$ FWHM) have
been obtained in Ar at 1 bar, and a coarse localization of the interactions
within the detector seemed possible [6]. In parallel, the NEWS-G collaboration
has demonstrated with SPC low-pressure operations that identification and
analysis of events initiated by 2 distinct electrons was routinely reached [7,
8]. Likewise, the search for KK axions in their expected di-photon decay
channel was carried out [9]. These experimental advances suggests that the
identification of double traces induced by a double beta decay would be
possible.

In this study, our objective is to verify the possibility of using a pure
noble gas and to characterise the detector performance obtained at atmospheric
pressure before turning to higher pressure. The light emission being one of
the distinguishing characteristics of the excitation in a noble gas, we
implemented a minimal system to detect scintillation light, and coincidences
with the SPC detector signal were observed.

In this paper the experimental setup is described with emphasis on the light
readout system. We compare between observed data and simulation and discuss
the light emitted in the avalanche process. We also evaluate the impact of
quencher-free gas on the charge signal shape and resolution.

##  2 Experimental setup

Figure 1: Operating principle of the SPC detector taken from Ref. [10].

The detector operating principle consists of a spherical grounded vessel,
filled with gas and with a central anode at a positive high voltage as shown
in Fig. 1. Particles passing through the gas ionize it and produce electrons
which drift under the influence of the electric field to the central anode.
When the electrons reach a distance of few mm from the central anode,
depending on the high voltage (HV), they create an avalanche leading to the
positive ions signal. More details can be found in Refs. [1, 11].

The prototype built at CENBG, in the framework of the R2D2 R&D, consists of a
spherical stainless steel vessel of 20 cm radius with a central anode of 2 mm
diameter. At the bottom of the detector there is an opening which is used to
insert a 210Po source emitting $\alpha$ particles with an energy of 5.3 MeV.
This prototype was used to assess the energy resolution in Ar at different
pressures and further details on the setup can be found in Ref. [6].

###  2.1 Light readout system

Figure 2: SiPM setup.

(a) (b)

Figure 3: Picture of the open readout box with the MPPC C12332-01 readout
circuit (3(a)) and SiPM readout circuit setup (3(b)).

(a) (b)

Figure 4: Signal of SiPM with a threshold of about 10 photoelectrons with
(4(a)) and without (4(b)) the CLPFL-0010-BNC CRYSTEK low-pass filter of 10
MHz.

The light yield of Ar gas is 14700 photons per MeV, assuming that
electronegative impurities are kept at the ppm level [12]. In our case the
detector was pumped to a vacuum of $5\times 10^{-5}$ mbar, and the use of pure
Ar (i.e. purity at the level of 99.9999% given by supplier) allowed to reach a
purity sufficient for light detection and a homogeneous gain, i.e. independent
of $\alpha$ track radial distance. No further purification of the gas was
attempted. Data taking was started 6.5 h after filling the SPC to ensure
stable operation. The detector stability was monitored to ensure that the gain
response remains constant over time.

To observe argon scintillation light a photodetector sensitive to 128 nm
photons is required. Generally, liquid argon experiments aiming at detecting
scintillation light, exploit Tetraphenyl-Butadiene (TPB) coating of surfaces
to shift the wavelength of the light into a range visible to Photomultiplier
Tubes (PMTs). Such a procedure is however delicate and our chosen solution was
to use commercial VUV silicon photomultipliers (SiPM) from Hamamatsu. Indeed
SiPM S13370, with a size of $6\times 6$ mm2 is a product of the VUV4 family
with a photon detection efficiency (PDE) of 14% at 128 nm.

In the R2D2 setup the SiPM was mounted on the source support as shown in Fig.
2. Such a position has several advantages including the maximisation of the
number of detected photons emitted by the $\alpha$ track (tracks passing in
front of the SiPM), and the ease of cabling power supply and signal readout
through the feedthrough located at the bottom of the detector. There is,
however, a drawback: the SiPM had to be operated at a voltage of 55 V and it
is located in a region where the electric field was weak, at the level of 0.1
V/cm. Since the SiPM surface was at 55 V most of the electrons from the
$\alpha$-particle track would have drifted towards the SiPM instead of
drifting towards the central anode. To overcome this issue a grid with holes
of 1 mm and an optical transparency of 65%, was installed in front of the SiPM
acting as a Faraday cage. Dedicated tests showed that turning the SiPM on and
off had no impact on the signal and on the energy resolution proving the
efficiency of the grid in shielding the SiPM field.

Since the SiPM is operated at room temperature a self-trigger rate at the
level of kHz is expected for thresholds up to 4–5 photoelectrons (p.e.)
depending on the SiPM. This, however, was not an issue for this application
where the threshold could be raised to about 10 photoelectrons. A further
challenge was related to the electronic noise due to the cables connecting the
SiPM to the Hamamatsu readout circuit. Such a circuit (MPPC C12332-01), self-
regulated with respect to temperature variations, was used to read out the
SiPM signal, but it was located outside the detector, as shown in Fig. 3(a).
This choice was imposed by the requirement to minimize the presence of
material inside the detector. However, to reduce the noise, the distance
between the SiPM and the readout circuit was minimized: the circuit was
connected directly at the feedthrough as shown in Fig. 3(b). Inside the
detector, between the feedthroughs and the SiPM there was a distance of about
30 cm which could not be reduced. Several options were explored, but the
solution put forward was to use a single coaxial cable, using the inner part
of the cable for the signal transmission and the outer part for the power
supply of the SiPM.

The SiPM signal is relatively fast and its width is of about 200 ns. A high
frequency noise due to the distance between the SiPM and the readout circuit
was observed and eliminated with a low-pass filter of 10 MHz (CLPFL-0010-BNC)
from CRYSTEK (Fig. 4). Such a configuration allowed reaching the best
experimental condition in order to look at the SiPM signal in coincidence with
the SPC signal.

##  3 Data analysis

Different runs were taken at different pressures and HV values. For pressures
below 500 mbar $\alpha$-particle tracks are too long (i.e. more than 6 cm) to
give enough detectable light on the SiPM, and therefore the runs used in the
presented analysis were all taken at the maximal allowed detector pressure of
1.1 bar. Two HV regimes were investigated: HV at the level of 1800 V where the
avalanche was not large enough to give detectable light on the SiPM, and HV of
about 2200 V where a clear avalanche light signal was observed as explained in
Sec. 3.3. Those two HV regimes correspond to a detector gain of about 15 and
50, respectively.

###  3.1 SiPM calibration

Figure 5: Signal recorded by the same readout electronic chain used for the
SiPM for a 10 mV pulse. Figure 6: Amplitude of SiPM dark noise signals. The 1
p.e. and 2 .p.e. peaks are clearly visible.

The first point to be addressed was the SiPM calibration. Considering that the
SiPM was operated in an unusual condition, at a large distance from the
readout card, the noise prevented having the typical SiPM charge spectrum
where the different number of photoelectrons are clearly seen.
Another reason why the typical SiPM charge spectrum was not observed is that
the detector DAQ uses a CALI card [13] at a sampling rate of 2 MHz (i.e. bins
of 500 ns) which records the SiPM signal of $\sim 200$ ns in a single time
bin. The signal amplitude is therefore smeared by the coarse time sampling
transforming the typical “comb-like” structure of the SiPM readout into a
continuous distribution.

To calibrate the SiPM a two-step procedure was therefore applied. First a
defined pulse of 10 mV, from a KEYSIGHT 33600A waveform generator, was used as
input in the same readout electronics chain used for the SiPM. In order to
account for the slow sampling rate of the DAQ with respect to the signal
width, a signal similar to the real one was used, namely a pulse with a rise-
time of 70 ns and a fall-time of 118.8 ns at a rate of 1 kHz. The obtained
distribution shown in Fig. 5 gives a conversion factor of 1 mV equal to $12.9$
Data Acquisition Units (ADU). Linearity was also checked and demonstrated
within the input uncertainty between 10 and 30 mV.
The second step consisted of removing the SiPM from the detector and measuring
its dark noise signal with the same readout electronics but directly connected
to the readout circuit in order to remove the noise due to the cables. This
was done using a Tektronix oscilloscope in order to retrieve the expected
charge readout pattern. A random trigger was used and the maximum amplitude of
the waveform in a window of about 700 ns before the trigger was recorded. The
amplitude of the signal corresponding to a given number of photoelectrons is
not affected by the window width, however the probability of having a given
number of photoelectrons does depend on it. The larger the window the higher
the probability of having a large signal corresponding to 2 photoelectrons or
more. The results is shown in Fig. 6 where peaks corresponding to 1 and 2
photoelectrons are clearly visible: the amplitude corresponding to 1 p.e.
corresponds to about 1.2 mV. The pedestal is not seen since it is below
threshold. A conversion can therefore be established between and DAQ units,
such that 1 photoelectron corresponds to about 15.5 ADU.

###  3.2 Drift time analysis

Figure 7: Raw waveforms of SiPM (blue) and anodie signal (orange) for one
event taken at 1.1 bar and 2200 V. The $\Delta t$ of about 300 $\mu$s is
shown. For illustration purpose the SiPM signal is multiplied by a factor of
5. Figure 8: Drift time obtained applying different HV on the central anode.
The different histograms are normalized to one for a direct comparison
independently on the number of triggered events of the run. Figure 9: Drift
time from Garfield++ simulations for as a function of the anode HV for
different starting radial distances from 17 to 20 cm (colored solid lines).
The experimental data are also shown and the bar width corresponds to the RMS
of the drift time distributions of Fig. 8. Figure 10: Radial energy
depositions from $\alpha$-particles simulated by Geant4 at 1.1 bar in argon.

The first dataset was taken triggering on the SiPM with a threshold of 200 ADU
which corresponds to about 13 p.e. At the time of measurements, the 210Po
source had an activity of 0.4 Bq but only tracks passing in front of the SiPM
produced detectable light; selecting this subsample of events reduced the
trigger rate by a factor of 10 to 40 mHz.

Triggering on the SiPM selects a subsample of $\alpha$ tracks in a specific
direction, however all these events should result in a signal on the central
anode. This is indeed what we observed and one example of the waveforms for
such events is shown in Fig. 7.

The time difference $\Delta t$ between the SiPM signal and the SPC signal was
computed and is shown in Fig. 8. The width of the $\Delta t$ distribution
depends on two factors: the spread due to the electron diffusion during the
drift, and the $\alpha$ tracks emission angle. Considering that
$\alpha$-particle tracks have a range of about 3 cm at 1.1 bar, the ionization
electron distance from the anode spans between 17 cm and 20 cm.

A Garfield++ [14] simulation was carried out in order to benchmark the drift
velocity of electrons. The drift time for different values of the anode HV was
computed starting from radial positions spanning from 17 cm to 20 cm with
respect to the detector center. The results are shown in Fig. 9 along with the
measured data, indicating a mean distance of the $\alpha$-particle tracks of
about 19 cm from the detector centre. This was confirmed by a Geant4 [15]
simulation of the $\alpha$ tracks: the radial position of the energy
depositions exhibits a peak at 19.76 cm with a mean at 19.15 cm as shown in
Fig. 10.

###  3.3 Avalanche light analysis

Figure 11: Width of the avalanche light signal versus rise-time of the central
anode signal.

When the detector is operated at sufficiently high voltage, 2200 V in our case
considering the argon gas pressure of 1.1 bar, the light signal emitted during
the avalanche is large enough to be detected by the 6$\times$6 mm2 SiPM
located at the cathode surface. Such a light is emitted at the central anode
and is observed also for $\alpha$ tracks which do not pass in front of the
SiPM. It is therefore possible to trigger on the avalanche light to have a
higher rate of events, at the level of 0.2 Hz expected from the source
activity, and the trigger signal is in this case simultaneous with the central
anode signal independently on the drift time.
Triggering on the avalanche light is of course not helpful for the radial
track reconstruction, but such a signal could potentially be exploited to have
an additional handle to reconstruct the topology of the event. The width of
the waveform is indeed proportional to the time between the first and the last
drifted electrons which in turn depends on the radial projection of the track.
To validate such a feature we compared the width of the avalanche light signal
to the rise-time of the central anode signal which is known to provide
information on the radial position of the energy deposits as demonstrated by
the NEWS-G collaboration and previous R2D2 works [4, 6]. The results shown in
Fig. 11 show a clear correlation between the two variables demonstrating that
the avalanche light waveforms contains indeed information on the event
topology. A dedicated analysis to exploit this feature will be the topic of
future work.

###  3.4 Secondary electrons signal

Figure 12: Waveform, after signal processing (deconvolution), showing the bump
due to the secondary electrons. The bump is modelled as a Gaussian function
(red curve) over an exponential function (green curve) corresponding to the
tail of the primary ionization peak.

(a) (b)

Figure 13: 13(a) Time difference between the primary ionization signal and the
bump due to secondary electron signal.The red is a Gaussian fit. 13(b)
Distribution of the width difference ($\sigma_{dif}$) between the peak of
primaries and the induced photon peak, fitted by a Gaussian.

The runs taken at 2200 V on the central anode, with a significant amount of
light emitted in the avalanche process, allowed to study another phenomenon,
namely the secondary electrons signal.
Photons produced during the avalanche reach the sphere surface and could eject
electrons which would be drifted to the central anode producing a second
signal. A digital devonvolution of the charge preamplifier output waveform
allowed to recover the charge time distribution which was masked by the RC
effect of the charge integrator (see [6] for the method). The resulting
deconvolved waveform exhibits a bump on the tail of the primary ionization
signal, as seen in Fig. 12. . For further analysis, this photon-induced
contribution was modeled with a Gaussian distribution superimposed on an
exponential component describing the tail of the primary ionization peak. The
time difference between the secondary electron signal and the primary
ionization one was computed as shown in Fig. 13(a). The mean value at about
316 $\mu$s corresponds indeed to the drift time expected from electrons
starting at the sphere surface (i.e. distance of 20 cm from the anode) for a
central anode at 2200 V as shown in Fig. 9.
It is possible to extract the value of the signal time spread due to diffusion
($\sigma_{dif}$). Indeed if the electrons were all produced at the same time
the Gaussian width would be completely due to the electron diffusion. This is
not the case and the time spread of the production is given at first
approximation by the time spread of the primary ionization signal. The
diffusion can therefore be evaluated as:
$\sigma_{dif}=\sqrt{\sigma_{photon}^{2}-\sigma_{ion}^{2}}$ where
$\sigma_{photon}$ and $\sigma_{ion}$ are the widths of the Gaussian fits of
the secondary electron signal and of the primary ionization signal
respectively. The mean value of the diffusion over the full 20 cm drift is 32
$\mu$s (see Fig. 13(b)).
It is noted that in case of high gain operation, secondary electron emissions
could become challenging since it may bring the detector into a Geiger mode.
The addition of a quencher to the gas allows to mitigate this effect, however,
in a future Xe-filled SPC, the presence of a quencher could result in an
additional degree of complication concerning xenon purity since the use of a
purifier, such as a hot getter, could modify the quencher fraction over time.
For this reason, considering the foreseen low gain operation, the use of pure
xenon is currently preferred. The impact on the energy resolution of the
secondary electron signal was therefore an important point to be addressed.
The waveform integral was studied, for fully contained tracks, and a
resolution of 1.5% FWHM was found at 5.3 MeV as shown in Fig. 14. If the
secondary electron signal is subtracted (modelled with a Gaussian function)
the resolution is degraded to 2.2%.
Such a result is of great interest for the development of the future detector
for $\beta\beta 0\nu$ decay search since it suggests that pure xenon may be
used also in proportional mode and the use of a quencher is not mandatory to
achieve the desired energy resolution (at least within our modest gain
operating regime, and the high energy deposits expected with $\beta\beta 0\nu$
search).
This is true for what concerns the use of quencher in order to avoid secondary
electrons stripped from the sphere by scintillation light. This result
illustrates the possibility to operate the detector without quencher, but the
quencher presence has also an impact on the resolution, affecting the number
of collected electron via Penning effect [16]. Indeed, previous results shown
a better resolution with 98%Ar:2%CH4 [6] with respect to the resolution
presented here, obtained in pure argon.

Figure 14: Resolution obtained integrating the waveform over the full range.

##  4 Conclusions

For the first time a scintillation light signal is used as trigger in a SPC.
The presented results, although still partial, constitute an encouraging first
step towards the use of an SPC filled with pure noble gas. Studies on the
waveform of the light signal have been carried out and the possibility of
crossing information provided by the anode signal was investigated.
Observations showed an excellent agreement between the ionization electron
drift time and the expectation for the GARFIELD++ simulations.

An additional important outcome of this study is that, in a suitable regime of
gain, the use of a quencher to avoid secondary electrons is not mandatory, at
least at the tested pressure. The presence of secondary electrons emitted from
the cathode by photons produced in the avalanche, does not spoil the detector
energy resolution.

Further work is required to validate these findings for a xenon high pressure
detector aiming at the detection of $\beta\beta 0\nu$.

## Acknowledgments

The authors would like to thank the IdEx Bordeaux 2019 Emergence program for
the OWEN grant for the “Development of a custom made electronics for a single
channel time projection chamber detector aiming at the discovery of
neutrinoless double beta decays, and for possible applications in industry”.
In addition we thank the CNRS International Emergency Action (IEA) for the
“E-ACHINOS” grant supporting the collaboration between CENBG and University of
Birmingham. We thank the CENBG technical staff. We thank M. Chapellier for
providing the Po source and for useful discussions. This project has received
funding from the European Union’s Horizon 2020 research and innovation
programme under the Marie Skłodowska-Curie grant agreement no 841261
(DarkSphere). K. Nikolopoulos acknowledges support by the European Research
Council (ERC).

## References

  * [1] I. Giomataris, et al., A Novel large-volume Spherical Detector with Proportional Amplification read-out, JINST 3 (2008) P09007.  arXiv:0807.2802, doi:10.1088/1748-0221/3/09/P09007.
  * [2] G. Gerbier, NEWS: a new spherical gas detector for very light WIMP detection, in: 9th Rencontres du Vietnam: Windows on the Universe, 2013, pp. 353–357.
  * [3] G. Gerbier, et al., NEWS : a new spherical gas detector for very low mass WIMP detection, astro-ph.IM/1401.7902arXiv:1401.7902.
  * [4] Arnaud, et al., Spherical Proportional Counter: A review of recent developments, J. Phys. Conf. Ser. 1029 (1) (2018) 012006.  doi:10.1088/1742-6596/1029/1/012006.
  * [5] A. Meregaglia, et al., Study of a spherical Xenon gas TPC for neutrinoless double beta detection, JINST 13 (01) (2018) P01009.  arXiv:1710.04536, doi:10.1088/1748-0221/13/01/P01009.
  * [6] R. Bouet, et al., R2D2 spherical TPC: first energy resolution results, JINST 16 (03) (2021) P03012.  arXiv:2007.02570, doi:10.1088/1748-0221/16/03/P03012.
  * [7] M.-C. Piro, D. Durnford, The search for Light Dark Matter with NEWS-G, in: TAUP21: XVII International Conference on Topics in Astroparticle and Underground Physics.
  * [8] K. Nikolopoulos, Reaching the neutrino floor for sub-GeV dark matter with spherical proportional counters fully electroformed underground, in: TAUP21: XVII International Conference on Topics in Astroparticle and Underground Physics.
  * [9] F. V. de Sola, Status and Future of NEWS-G, GDR Deep Underground Physics kick-off meeting, in: https://indico.in2p3.fr/event/23971/contributions/95745/.
  * [10] Katsioulas, et al., Development of a simulation framework for spherical proportional counters, JINST 15 (06) (2020) C06013.  arXiv:2002.02718, doi:10.1088/1748-0221/15/06/C06013.
  * [11] Savvidis, et al., Low energy recoil detection with a spherical proportional counter, Nucl. Instrum. Meth. A 877 (2018) 220–226.  arXiv:1606.02146, doi:10.1016/j.nima.2017.09.014.
  * [12] C. Amsler, et al., Luminescence quenching of the triplet excimer state by air traces in gaseous argon, JINST 3 (2008) P02001.  arXiv:0708.2621, doi:10.1088/1748-0221/3/02/P02001.
  * [13] E. Armengaud, et al., Performance of the EDELWEISS-III experiment for direct dark matter searches, JINST 12 (08) (2017) P08010.  arXiv:1706.01070, doi:10.1088/1748-0221/12/08/P08010.
  * [14] R. Veenhof, GARFIELD, recent developments, Nucl. Instrum. Meth. A 419 (1998) 726–730.  doi:10.1016/S0168-9002(98)00851-1.
  * [15] S. Agostinelli, et al., GEANT4–a simulation toolkit, Nucl. Instrum. Meth. A 506 (2003) 250–303.  doi:10.1016/S0168-9002(03)01368-8.
  * [16] O. Şahin, I. Tapan, E. N. Özmutlu, R. Veenhof, Penning transfer in argon-based gas mixtures, JINST 5 (05) (2010) P05002.  doi:10.1088/1748-0221/5/05/P05002.


* Saito et al. (2020) Shunsuke Saito, Tomas Simon, Jason Saragih, and Hanbyul Joo. 2020.  Pifuhd: Multi-level pixel-aligned implicit function for high-resolution 3d human digitization. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 84–93.
  * Salimans et al. (2016) Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. 2016\.  Improved techniques for training gans. In _Advances in neural information processing systems_. 2234–2242.
  * Si et al. (2018) Chenyang Si, Wei Wang, Liang Wang, and Tieniu Tan. 2018\.  Multistage adversarial losses for pose-based human image synthesis. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 118–126.
  * Siarohin et al. (2019a) Aliaksandr Siarohin, Stéphane Lathuilière, Enver Sangineto, and Nicu Sebe. 2019a.  Attention-based Fusion for Multi-source Human Image Generation.  (2019).
  * Siarohin et al. (2019b) Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe. 2019b.  Animating arbitrary objects via deep motion transfer. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 2377–2386.
  * Siarohin et al. (2019c) Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe. 2019c.  First order motion model for image animation.  _Advances in Neural Information Processing Systems_ 32 (2019), 7137–7147.
  * Siarohin et al. (2018) Aliaksandr Siarohin, Enver Sangineto, Stéphane Lathuilière, and Nicu Sebe. 2018.  Deformable gans for pose-based human image generation. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 3408–3416.
  * Son Chung et al. (2018) Joon Son Chung, Arsha Nagrani, and Andrew Zisserman. 2018\.  VoxCeleb2: deep speaker recognition.  _arXiv e-prints_ (2018), arXiv–1806.
  * Song et al. (2019a) Dan Song, Tianbao Li, Zhendong Mao, and An-An Liu. 2019a.  SP-VITON: shape-preserving image-based virtual try-on network.  _Multimedia Tools and Applications_ (2019), 1–13.
  * Song et al. (2021) Linsen Song, Wayne Wu, Chaoyou Fu, Chen Qian, Chen Change Loy, and Ran He. 2021\.  Everything’s Talkin’: Pareidolia Face Reenactment.  _arXiv e-prints_ (2021), arXiv–2104.
  * Song et al. (2022) Linsen Song, Wayne Wu, Chen Qian, Ran He, and Chen Change Loy. 2022.  Everybody’s talkin’: Let me talk as you want.  _IEEE Transactions on Information Forensics and Security_ (2022).
  * Song et al. (2019b) Sijie Song, Wei Zhang, Jiaying Liu, and Tao Mei. 2019b.  Unsupervised person image generation with semantic parsing transformation. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 2357–2366.
  * Song et al. (2019c) Yang Song, Jingwen Zhu, Dawei Li, Andy Wang, and Hairong Qi. 2019c.  Talking face generation by conditional recurrent adversarial network. In _Proceedings of the 28th International Joint Conference on Artificial Intelligence_. 919–925.
  * Sun et al. (2019b) Feng Sun, Jiaming Guo, Zhuo Su, and Chengying Gao. 2019b.  Image-Based Virtual Try-on Network with Structural Coherence. In _2019 IEEE International Conference on Image Processing (ICIP)_. IEEE, 519–523.
  * Sun et al. (2019a) Wei Sun, Jawadul H Bappy, Shanglin Yang, Yi Xu, Tianfu Wu, and Hui Zhou. 2019a.  Pose Guided Fashion Image Synthesis Using Deep Generative Model.  _arXiv e-prints_ (2019), arXiv–1906.
  * Sun et al. (2022) Yang-Tian Sun, Hao-Zhi Huang, Xuan Wang, Yu-Kun Lai, Wei Liu, and Lin Gao. 2022\.  Robust pose transfer with dynamic details using neural video rendering.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ (2022).
  * Suwajanakorn et al. (2017) Supasorn Suwajanakorn, Steven M Seitz, and Ira Kemelmacher-Shlizerman. 2017.  Synthesizing obama: learning lip sync from audio.  _ACM Transactions on Graphics (TOG)_ 36, 4 (2017), 1–13.
  * Szegedy et al. (2016) Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016.  Rethinking the inception architecture for computer vision. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 2818–2826.
  * Tang et al. (2020) Hao Tang, Song Bai, Li Zhang, Philip HS Torr, and Nicu Sebe. 2020.  Xinggan for person image generation. In _European Conference on Computer Vision_. Springer, 717–734.
  * Tang et al. (2019) Hao Tang, Dan Xu, Gaowen Liu, Wei Wang, Nicu Sebe, and Yan Yan. 2019\.  Cycle in cycle generative adversarial networks for keypoint-guided image generation. In _Proceedings of the 27th ACM International Conference on Multimedia_. 2052–2060.
  * Tewari et al. (2020) Ayush Tewari, Ohad Fried, Justus Thies, Vincent Sitzmann, Stephen Lombardi, Kalyan Sunkavalli, Ricardo Martin-Brualla, Tomas Simon, Jason Saragih, Matthias Nießner, et al. 2020\.  State of the art on neural rendering. In _Computer Graphics Forum_ , Vol. 39. Wiley Online Library, 701–727.
  * Thies et al. (2020) Justus Thies, Mohamed Elgharib, Ayush Tewari, Christian Theobalt, and Matthias Nießner. 2020\.  Neural voice puppetry: Audio-driven facial reenactment. In _European Conference on Computer Vision_. Springer, 716–731.
  * Thies et al. (2019) Justus Thies, Michael Zollhöfer, and Matthias Nießner. 2019\.  Deferred neural rendering: Image synthesis using neural textures.  _ACM Transactions on Graphics (TOG)_ 38, 4 (2019), 1–12.
  * Thies et al. (2015) Justus Thies, Michael Zollhöfer, Matthias Nießner, Levi Valgaerts, Marc Stamminger, and Christian Theobalt. 2015.  Real-time expression transfer for facial reenactment.  _ACM Transactions on Graphics (TOG)_ 34, 6 (2015), 1–14.
  * Thies et al. (2016) Justus Thies, Michael Zollhofer, Marc Stamminger, Christian Theobalt, and Matthias Nießner. 2016\.  Face2face: Real-time face capture and reenactment of rgb videos. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 2387–2395.
  * Tolosana et al. (2020) Ruben Tolosana, Ruben Vera-Rodriguez, Julian Fierrez, Aythami Morales, and Javier Ortega-Garcia. 2020\.  Deepfakes and beyond: A survey of face manipulation and fake detection.  _Information Fusion_ 64 (2020), 131–148.
  * Tripathy et al. (2020) Soumya Tripathy, Juho Kannala, and Esa Rahtu. 2020\.  Icface: Interpretable and controllable face reenactment using gans. In _Proceedings of the IEEE/CVF winter conference on applications of computer vision_. 3385–3394.
  * Tripathy et al. (2021) Soumya Tripathy, Juho Kannala, and Esa Rahtu. 2021\.  Facegan: Facial attribute controllable reenactment gan. In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_. 1329–1338.
  * Tulyakov et al. (2018) Sergey Tulyakov, Ming-Yu Liu, Xiaodong Yang, and Jan Kautz. 2018\.  Mocogan: Decomposing motion and content for video generation. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 1526–1535.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need. In _Advances in neural information processing systems_. 5998–6008.
  * Velickovic et al. (2017) Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017.  Graph attention networks.  _stat_ 1050 (2017), 20.
  * Verdoliva (2020) Luisa Verdoliva. 2020\.  Media forensics and deepfakes: an overview.  _IEEE Journal of Selected Topics in Signal Processing_ 14, 5 (2020), 910–932.
  * Villegas et al. (2017) Ruben Villegas, Jimei Yang, Yuliang Zou, Sungryull Sohn, Xunyu Lin, and Honglak Lee. 2017\.  Learning to generate long-term future via hierarchical prediction. In _Proceedings of the 34th International Conference on Machine Learning-Volume 70_. JMLR. org, 3560–3569.
  * Vlasic et al. (2006) Daniel Vlasic, Matthew Brand, Hanspeter Pfister, and Jovan Popovic. 2006.  Face transfer with multilinear models.  In _ACM SIGGRAPH 2006 Courses_. 24–es.
  * Vobeckỳ et al. (2019) Antonín Vobeckỳ, Michal Uřičář, David Hurych, and Radoslav Škoviera. 2019\.  Advanced Pedestrian Dataset Augmentation for Autonomous Driving. In _2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)_. IEEE, 2367–2372.
  * Vougioukas et al. (2019a) Konstantinos Vougioukas, Stavros Petridis, and Maja Pantic. 2019a.  End-to-End Speech-Driven Realistic Facial Animation with Temporal GANs. In _CVPR Workshops_. 37–40.
  * Vougioukas et al. (2019b) Konstantinos Vougioukas, Stavros Petridis, and Maja Pantic. 2019b.  Realistic speech-driven facial animation with gans.  _International Journal of Computer Vision_ (2019), 1–16.
  * Walker et al. (2017) Jacob Walker, Kenneth Marino, Abhinav Gupta, and Martial Hebert. 2017.  The pose knows: Video forecasting by generating pose futures. In _Proceedings of the IEEE international conference on computer vision_. 3332–3341.
  * Wan et al. (2013) Vincent Wan, Robert Anderson, Art Blokland, Norbert Braunschweiler, Langzhou Chen, BalaKrishna Kolluru, Javier Latorre, Ranniery Maia, Björn Stenger, Kayoko Yanagisawa, et al. 2013\.  Photo-realistic expressive text to talking head synthesis.. In _INTERSPEECH_. 2667–2669.
  * Wang et al. (2018b) Bochao Wang, Huabin Zheng, Xiaodan Liang, Yimin Chen, Liang Lin, and Meng Yang. 2018b.  Toward characteristic-preserving image-based virtual try-on network. In _Proceedings of the European Conference on Computer Vision (ECCV)_. 589–604.
  * Wang et al. (2020a) Jiahang Wang, Tong Sha, Wei Zhang, Zhoujun Li, and Tao Mei. 2020a.  Down to the last detail: Virtual try-on with fine-grained details. In _Proceedings of the 28th ACM International Conference on Multimedia_. 466–474.
  * Wang et al. (2020b) Kaisiyuan Wang, Qianyi Wu, Linsen Song, Zhuoqian Yang, Wayne Wu, Chen Qian, Ran He, Yu Qiao, and Chen Change Loy. 2020b.  Mead: A large-scale audio-visual dataset for emotional talking-face generation. In _European Conference on Computer Vision_. Springer, 700–717.
  * Wang et al. (2019) Ting-Chun Wang, Ming-Yu Liu, Andrew Tao, Guilin Liu, Jan Kautz, and Bryan Catanzaro. 2019\.  Few-shot video-to-video synthesis. In _Proceedings of the 33rd International Conference on Neural Information Processing Systems_. 5013–5024.
  * Wang et al. (2018a) Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Guilin Liu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. 2018a.  Video-to-Video Synthesis.  _Advances in Neural Information Processing Systems_ 31 (2018).
  * Wang et al. (2021) Ting-Chun Wang, Arun Mallya, and Ming-Yu Liu. 2021\.  One-shot free-view neural talking-head synthesis for video conferencing. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 10039–10049.
  * Wang et al. (2004) Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004\.  Image quality assessment: from error visibility to structural similarity.  _IEEE transactions on image processing_ 13, 4 (2004), 600–612.
  * Wiles et al. (2018) Olivia Wiles, A Sophia Koepke, and Andrew Zisserman. 2018\.  X2face: A network for controlling face generation using images, audio, and pose codes. In _Proceedings of the European conference on computer vision (ECCV)_. 670–686.
  * Wu et al. (2021) Kun Wu, Chengxiang Yin, Zhengping Che, Bo Jiang, Jian Tang, Zheng Guan, and Gangyi Ding. 2021.  Human Pose Transfer with Disentangled Feature Consistency.  _arXiv e-prints_ (2021), arXiv–2107.
  * Wu et al. (2018) Wayne Wu, Yunxuan Zhang, Cheng Li, Chen Qian, and Chen Change Loy. 2018.  Reenactgan: Learning to reenact faces via boundary transfer. In _Proceedings of the European conference on computer vision (ECCV)_. 603–619.
  * Wu et al. (2019) Zhonghua Wu, Guosheng Lin, Qingyi Tao, and Jianfei Cai. 2019\.  M2e-try on net: Fashion from model to everyone. In _Proceedings of the 27th ACM International Conference on Multimedia_. 293–301.
  * Xie et al. (2020) Zhenyu Xie, Jianhuang Lai, and Xiaohua Xie. 2020.  LG-VTON: Fashion Landmark Meets Image-Based Virtual Try-On. In _Chinese Conference on Pattern Recognition and Computer Vision (PRCV)_. Springer, 286–297.
  * Xu et al. (2020) Chengming Xu, Yanwei Fu, Chao Wen, Ye Pan, Yu-Gang Jiang, and Xiangyang Xue. 2020\.  Pose-Guided Person Image Synthesis in the Non-Iconic Views.  _IEEE Transactions on Image Processing_ 29 (2020), 9060–9072.
  * Yan et al. (2017) Yichao Yan, Jingwei Xu, Bingbing Ni, Wendong Zhang, and Xiaokang Yang. 2017.  Skeleton-aided articulated motion generation. In _Proceedings of the 25th ACM international conference on Multimedia_. 199–207.
  * Yang et al. (2018) Ceyuan Yang, Zhe Wang, Xinge Zhu, Chen Huang, Jianping Shi, and Dahua Lin. 2018\.  Pose guided human video generation. In _Proceedings of the European Conference on Computer Vision (ECCV)_. 201–216.
  * Yang et al. (2020b) Han Yang, Ruimao Zhang, Xiaobao Guo, Wei Liu, Wangmeng Zuo, and Ping Luo. 2020b.  Towards Photo-Realistic Virtual Try-On by Adaptively Generating-Preserving Image Content. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 7850–7859.
  * Yang et al. (2021) Lingbo Yang, Pan Wang, Chang Liu, Zhanning Gao, Peiran Ren, Xinfeng Zhang, Shanshe Wang, Siwei Ma, Xiansheng Hua, and Wen Gao. 2021\.  Towards fine-grained human pose transfer with detail replenishing network.  _IEEE Transactions on Image Processing_ 30 (2021), 2422–2435.
  * Yang et al. (2020a) Lingbo Yang, Pan Wang, Xinfeng Zhang, Shanshe Wang, Zhanning Gao, Peiran Ren, Xuansong Xie, Siwei Ma, and Wen Gao. 2020a.  Region-adaptive texture enhancement for detailed person image synthesis. In _2020 IEEE International Conference on Multimedia and Expo (ICME)_. IEEE, 1–6.
  * Yang et al. (2019) Lingbo Yang, Zhenghui Zhao, Shiqi Wang, Shanshe Wang, Siwei Ma, and Wen Gao. 2019\.  Disentangled Human Action Video Generation via Decoupled Learning. In _2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW)_. IEEE, 495–500.
  * Yi et al. (2020) Ran Yi, Zipeng Ye, Juyong Zhang, Hujun Bao, and Yong-Jin Liu. 2020.  Audio-driven Talking Face Video Generation with Learning-based Personalized Head Pose.  _arXiv e-prints_ (2020), arXiv–2002.
  * Yildirim et al. (2019) Gokhan Yildirim, Nikolay Jetchev, Roland Vollgraf, and Urs Bergmann. 2019.  Generating high-resolution fashion model images wearing custom outfits. In _Proceedings of the IEEE International Conference on Computer Vision Workshops_. 0–0.
  * Yin et al. (2021) Weidong Yin, Ziwei Liu, and Leonid Sigal. 2021.  Person-in-context synthesis with compositional structural space. In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_. 2827–2836.
  * Yoo et al. (2016) Donggeun Yoo, Namil Kim, Sunggyun Park, Anthony S Paek, and In So Kweon. 2016.  Pixel-level domain transfer. In _European Conference on Computer Vision_. Springer, 517–532.
  * Yoon et al. (2021) Jae Shin Yoon, Lingjie Liu, Vladislav Golyanik, Kripasindhu Sarkar, Hyun Soo Park, and Christian Theobalt. 2021.  Pose-Guided Human Animation from a Single Image in the Wild. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 15039–15048.
  * Yu et al. (2019b) Lingyun Yu, Jun Yu, and Qiang Ling. 2019b.  Mining audio, text and visual information for talking face generation. In _2019 IEEE International Conference on Data Mining (ICDM)_. IEEE, 787–795.
  * Yu et al. (2019c) Li Yu, Yueqi Zhong, and Xin Wang. 2019c.  Inpainting-Based Virtual Try-on Network for Selective Garment Transfer.  _IEEE Access_ 7 (2019), 134125–134136.
  * Yu et al. (2019a) Ruiyun Yu, Xiaoqi Wang, and Xiaohui Xie. 2019a.  Vtnfp: An image-based virtual try-on network with body and clothing feature preservation. In _Proceedings of the IEEE International Conference on Computer Vision_. 10511–10520.
  * Zablotskaia et al. (2019) Polina Zablotskaia, Aliaksandr Siarohin, Bo Zhao, and Leonid Sigal. 2019.  DwNet: Dense warp-based network for pose-guided human video generation.  _arXiv e-prints_ (2019), arXiv–1910.
  * Zakharov et al. (2020) Egor Zakharov, Aleksei Ivakhnenko, Aliaksandra Shysheya, and Victor Lempitsky. 2020.  Fast Bi-layer Neural Synthesis of One-Shot Realistic Head Avatars. In _European Conference on Computer Vision_. Springer, 524–540.
  * Zakharov et al. (2019) Egor Zakharov, Aliaksandra Shysheya, Egor Burkov, and Victor Lempitsky. 2019.  Few-shot adversarial learning of realistic neural talking head models. In _Proceedings of the IEEE International Conference on Computer Vision_. 9459–9468.
  * Zanfir et al. (2018) Mihai Zanfir, Alin-Ionut Popa, Andrei Zanfir, and Cristian Sminchisescu. 2018.  Human appearance transfer. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 5391–5399.
  * Zeng et al. (2020) Xianfang Zeng, Yusu Pan, Mengmeng Wang, Jiangning Zhang, and Yong Liu. 2020.  Realistic face reenactment via self-supervised disentangling of identity and pose. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 34. 12757–12764.
  * Zhang et al. (2018) Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. 2018.  The unreasonable effectiveness of deep features as a perceptual metric. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 586–595.
  * Zhang et al. (2013) Weiyu Zhang, Menglong Zhu, and Konstantinos G Derpanis. 2013\.  From actemes to action: A strongly-supervised representation for detailed action understanding. In _Proceedings of the IEEE International Conference on Computer Vision_. 2248–2255.
  * Zhang et al. (2019) Yulei Zhang, Qingjie Zhao, and You Li. 2019.  Multi-view Based Pose Alignment Method for Person Re-identification. In _Chinese Intelligent Automation Conference_. Springer, 439–447.
  * Zhao et al. (2018b) Bo Zhao, Xiao Wu, Zhi-Qi Cheng, Hao Liu, Zequn Jie, and Jiashi Feng. 2018b.  Multi-view image generation from a single-view. In _Proceedings of the 26th ACM international conference on Multimedia_. 383–391.
  * Zhao et al. (2018a) Long Zhao, Xi Peng, Yu Tian, Mubbasir Kapadia, and Dimitris Metaxas. 2018a.  Learning to forecast and refine residual motion for image-to-video generation. In _Proceedings of the European conference on computer vision (ECCV)_. 387–403.
  * Zhao et al. (2020) Wenbin Zhao, Qing Xie, Yanchun Ma, Yongjian Liu, and Shengwu Xiong. 2020.  Pose Guided Person Image Generation Based on Pose Skeleton Sequence and 3D Convolution. In _2020 IEEE International Conference on Image Processing (ICIP)_. IEEE, 1561–1565.
  * Zheng et al. (2019a) Haitian Zheng, Lele Chen, Chenliang Xu, and Jiebo Luo. 2019a.  Unsupervised Pose Flow Learning for Pose Guided Synthesis.  _arXiv e-prints_ (2019), arXiv–1909.
  * Zheng et al. (2015) Liang Zheng, Liyue Shen, Lu Tian, Shengjin Wang, Jingdong Wang, and Qi Tian. 2015\.  Scalable person re-identification: A benchmark. In _Proceedings of the IEEE international conference on computer vision_. 1116–1124.
  * Zheng et al. (2019b) Na Zheng, Xuemeng Song, Zhaozheng Chen, Linmei Hu, Da Cao, and Liqiang Nie. 2019b.  Virtually trying on new clothing with arbitrary poses. In _Proceedings of the 27th ACM International Conference on Multimedia_. 266–274.
  * Zheng et al. (2019c) Zhedong Zheng, Xiaodong Yang, Zhiding Yu, Liang Zheng, Yi Yang, and Jan Kautz. 2019c.  Joint discriminative and generative learning for person re-identification. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 2138–2147.
  * Zheng et al. (2021) Zerong Zheng, Tao Yu, Yebin Liu, and Qionghai Dai. 2021\.  Pamir: Parametric model-conditioned implicit representation for image-based human reconstruction.  _IEEE transactions on pattern analysis and machine intelligence_ 44, 6 (2021), 3170–3184.
  * Zheng et al. (2017) Zhedong Zheng, Liang Zheng, and Yi Yang. 2017.  Unlabeled samples generated by gan improve the person re-identification baseline in vitro. In _Proceedings of the IEEE International Conference on Computer Vision_. 3754–3762.
  * Zhou et al. (2019b) Hang Zhou, Yu Liu, Ziwei Liu, Ping Luo, and Xiaogang Wang. 2019b.  Talking face generation by adversarially disentangled audio-visual representation. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 33. 9299–9306.
  * Zhou et al. (2021b) Hang Zhou, Yasheng Sun, Wayne Wu, Chen Change Loy, Xiaogang Wang, and Ziwei Liu. 2021b.  Pose-controllable talking face generation by implicitly modularized audio-visual representation. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 4176–4186.
  * Zhou et al. (2021a) Mohan Zhou, Yalong Bai, Wei Zhang, Tiejun Zhao, and Tao Mei. 2021a.  Responsive Listening Head Generation: A Benchmark Dataset and Baseline.  _arXiv preprint arXiv:2112.13548_ (2021).
  * Zhou et al. (2019a) Xingran Zhou, Siyu Huang, Bin Li, Yingming Li, Jiachen Li, and Zhongfei Zhang. 2019a.  Text guided person image synthesis. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 3663–3672.
  * Zhou et al. (2020) Yang Zhou, Xintong Han, Eli Shechtman, Jose Echevarria, Evangelos Kalogerakis, and Dingzeyu Li. 2020.  MakeltTalk: speaker-aware talking-head animation.  _ACM Transactions on Graphics (TOG)_ 39, 6 (2020), 1–15.
  * Zhou et al. (2019c) Yipin Zhou, Zhaowen Wang, Chen Fang, Trung Bui, and Tamara Berg. 2019c.  Dance dance generation: Motion transfer for internet videos. In _Proceedings of the IEEE International Conference on Computer Vision Workshops_. 0–0.
  * Zhu et al. (2017a) Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017a.  Unpaired image-to-image translation using cycle-consistent adversarial networks. In _Proceedings of the IEEE international conference on computer vision_. 2223–2232.
  * Zhu et al. (2017b) Shizhan Zhu, Raquel Urtasun, Sanja Fidler, Dahua Lin, and Chen Change Loy. 2017b.  Be your own prada: Fashion synthesis with structural coherence. In _Proceedings of the IEEE international conference on computer vision_. 1680–1688.
  * Zhu et al. (2019) Zhen Zhu, Tengteng Huang, Baoguang Shi, Miao Yu, Bofei Wang, and Xiang Bai. 2019\.  Progressive pose attention transfer for person image generation. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 2347–2356.