# Constraints on the positron emission from pulsar populations with AMS-02
data

Silvia Manconi  Institute for Theoretical Particle Physics and Cosmology, RWTH
Aachen University
Sommerfeldstr. 16, 52056 Aachen, Germany

###### Abstract

Electron and positron fluxes in cosmic rays are currently measured with
unprecedented precision by AMS-02 up to TeV energies, and represent unique
probes for the local properties of our Galaxy. The interpretation of their
spectra is at present still debated, especially for the excess of positrons
above 10 GeV. The hypothesis that pulsars can significantly contribute to this
excess has been consolidated after the observation of gamma-ray halos at TeV
energies of a few degree size around Geminga and Monogem pulsars. However, the
spatial and energetic Galactic distribution of pulsars and the details of the
positron production, acceleration and release from these sources are not yet
fully understood. I will describe how we can use the high-precision AMS-02
positron data to constrain the main properties of the Galactic pulsar
population and of the positron acceleration needed to explain the observed
fluxes. This is achieved by simulating a large number of Galactic pulsar
populations, following the most recent self-consistent modelings for the
pulsar spin-down and evolution properties, calibrated on catalog observations.
By fitting the positron AMS-02 data together with a secondary component due to
collisions of primary cosmic rays with the interstellar medium, we determine
the physical parameters of the pulsars dominating the positron flux, and
assess the impact of different assumptions on radial distributions, spin-down
properties, Galactic propagation scenarios and positron emission time.

##  1 Introduction

This contribution is based on this paper [1] done in collaboration with Luca
Orusa, Mattia Di Mauro, Fiorenza Donato (University of Turin and INFN, Turin),
and on additional reasoning by the speaker, following the material presented
during the talk at the conference. I refer to the paper for an extended
discussion of the model, the results, and for a more exhaustive bibliography,
reduced to minimal here due to space limitations.

###  1.1 The local $e^{+}$ flux

High-energetic cosmic-ray (CR) positrons ($e^{+}$) contribute to the total CR
flux detected at Earth at the percent level. Nevertheless, the unprecedented
precision of the measurement of the local $e^{+}$ flux by PAMELA, Fermi-LAT
and most recently by AMS-02 [2] allowed an in-depth study of this antimatter
component of CRs. This suggests the presence of primary $e^{+}$ sources in our
Galaxy. In fact, the observed flux exceeds the so-called secondary flux (as
computed in the standard CR transport models, see later) produced by inelastic
collisions of CR nuclei in the interstellar medium (ISM) above about 10 GeV.
For an overview on the most recent CR observations and on the prospects for
current and future observatories, we refer to other contributions to this
conference [3].

The quest for the interpretation of the $e^{+}$ flux has been a dynamic part
of the CR community’s debate since more than 10 years. The intense radiative
losses suffered by high energetic $e^{+}$ while propagating in the Galaxy
limit the horizon for GeV-TeV $e^{+}$ to few kiloparsecs (kpc). Thus, it is
generally accepted that AMS-02 $e^{+}$ observations require their primary
source to be local, i.e. located within a few kpc from the Earth. Many
mechanisms have been explored in the literature to account for the $e^{+}$
flux observed by AMS-02. Among them, pulsars and their pulsar wind nebulae
(PWNe) have been consolidating as significant factories of high-energy CR
$e^{\pm}$ in the Galaxy, and thus as main candidates to explain the $e^{+}$
excess. A not-exhaustive list of the main alternative proposals includes dark
matter annihilations in the Galactic halo [4], modifications of the standard
secondary production mechanism [5], as well as production of secondary
antimatter in supernova remnants, also discussed at this conference [6].

###  1.2 Pulsars and their nebulae as $e^{+}$ factories

Many theoretical and observational arguments support the idea that pulsars and
their nebulae are $e^{+}$ (and electrons 111Galactic pulsars are expected to
contribute to the data on the $e^{+}+e^{-}$ and electron ($e^{-}$) spectrum as
well, see e.g.[7, 8]. However, the AMS-02 $e^{+}$ data up to 1 TeV are
currently the most precise observable to constrain the characteristics of
Galactic pulsar populations, being this the main scope of the present work.
The $e^{+}+e^{-}$ data is indeed dominated by the $e^{-}$ produced by
supernova remnants (SNR)[7, 8].) factories.

First, the pulsar spin-down mechanism effectively produces $e^{\pm}$ pairs,
which are possibly accelerated to multi-TeV energies at the termination shock
between the relativistic wind and the surrounding medium (see [9] for recent
reviews on the acceleration mechanism). We note that the details of the
$e^{\pm}$ production, acceleration and release from pulsars and their PWNe are
yet not fully understood [9], as well as the spatial and energetic
distribution of pulsar [10, 11]. CR $e^{+}$ are thus a golden channel to study
primary sources, and in particular PWNe.

The observation of gamma-ray halos at TeV energies of a few degree size around
nearby pulsars [12], such as the ones of Geminga and Monogem, reported by HAWC
[13] and by Fermi-LAT at tens of GeV [14] further corroborates the presence of
$e^{+}$ (and electrons) accelerated, then escaped at few tens of parsec away
from the pulsar location. The observed emission is interpreted as coming from
CR $e^{\pm}$ escaping from the PWN and inverse Compton scattering (ICS) low-
energy photons of the interstellar radiation field (ISRF), see other
contributions to this conference for an extended discussion [12].

Finally, several independent works have demonstrated that pulsar models can
provide a good description of AMS-02 $e^{+}$ and $e^{-}$ data. This conclusion
has been reached both by considering the contribution of few nearby sources,
as well as the cumulative emission from pulsars as observed in existing
catalogs (e.g. [15, 16] and references therein) or in simulations [17, 8], and
including or not possible effects from suppressed diffusion around sources
[18], as suggested by the measured angular profile of gamma-ray pulsar halos
[12]. While energy losses limit the distance traveled by high-energy $e^{\pm}$
to few hundreds of parsecs, where we expect few Galactic sources contributing
significantly, current source catalogs might be not complete. Previous
computations calculating the contribution of $e^{+}$ from the ATNF catalog
sources could therefore suffer for underestimation due to the incompleteness
of the catalog. Simulations of the Galactic source population of pulsars are
thus needed to extensively test the pulsar interpretation of the observed
$e^{+}$ flux in order to overcome the limitations of previous studies, see
also [19]. Following the arguments introduced until now, high-precision
$e^{+}$ data can now be used to constrain the main properties of the Galactic
pulsar population and of the PWN acceleration. In this contribution we build
upon previous works [17, 18, 8] and extend them significantly in various novel
aspects [1].

##  2 Phenomenological models for pulsar populations

We simulate Galactic pulsars following the injection and propagation model
summarized below, and produce mock catalogs for five simulation setups (ModA-
B-C-D-E). For each realization we compute the $e^{+}$ flux from every PWN. A
summary of the simulated quantities is illustrated in Table 1 in [1] and
outlined in what follows.

###  2.1 Pulsar $e^{+}$ injection

Pulsars are rotating neutron stars with a strong surface magnetic field, and
magnetic dipole radiation is believed to provide a good description for its
observed loss of rotational energy [9]. We consider a model in which $e^{\pm}$
are continuously injected at a rate that follows the pulsar spin-down energy.
The injection spectrum $Q(E,t)$ of $e^{\pm}$ at energy $E$ and time $t$ is
described as:

$Q(E,t)=L(t)\left(\frac{E}{E_{0}}\right)^{-\gamma_{e}}\exp\left(-\frac{E}{E_{c}}\right)$
(1)

where the cut-off energy $E_{c}$ is fixed at $10^{5}$ TeV, $E_{0}=1$ GeV and
$\gamma_{e}$ is the $e^{\pm}$ spectral index. The magnetic dipole braking
$L(t)$ is described by the function
$L(t)={L_{0}}/{\left(1+\frac{t}{\tau_{0}}\right)^{\frac{n+1}{n-1}}}$, where
$\tau_{0}$ is the characteristic time scale and $n$ defines the magnetic
braking index. The total energy emitted by the source only into $e^{+}$ is
given by $E_{tot}=\eta W_{0}=\int_{0}^{T}dt\int_{E_{1}}^{\infty}dEEQ(E,t)$,
through which we obtain the value of $L_{0}$, fixing $E_{1}$=0.1 GeV. The
parameter $\eta$ encodes the efficiency of conversion of the spin-down energy
into $e^{+}$(which is half of the efficiency of conversion into $e^{\pm}$).
$W_{0}$ is the initial rotational energy of a pulsar with a moment of inertia
$I$ and rotational frequency $\Omega_{0}=2\pi/P_{0}$, $W_{0}=E_{\rm
rot,0}=\frac{1}{2}I{\Omega_{0}}^{2}\,,$ and it’s determined for each mock
pulsar using the pulsar spin down model below. Since the spectral index
$\gamma_{e}$ of accelerated particles is uncertain, and may vary significantly
for each PWN, it is sampled from uniform distributions within [1.4-2.2]. The
value of $\eta$ for each source is sampled from a uniform distribution in the
range [0.01-0.1].

###  2.2 Pulsar spin down

In each simulation, the total number of sources is fixed at $N_{\rm
PSR}=t_{max}\dot{N}_{PSR}$, where $t_{max}$ is the maximum simulated age and
$\dot{N}_{PSR}$ is the pulsar birth rate. We here assume the maximum age of
the sources to be $t_{max}$ = $10^{8}$ yr, and $\dot{N}_{PSR}$ = 0.01 yr-1.

The spin-down luminosity $\dot{E}=dE_{\rm rot}/dt$ of a pulsar is the rate at
which the rotational kinetic energy is dissipated, and it’s related to the
pulsar period and period derivative as $\dot{E}=\frac{dE_{\rm
rot}}{dt}=I\Omega\dot{\Omega}=-4\pi^{2}I\frac{\dot{P}}{P^{3}}\,.$ Assuming a
small deviation from the dipole nature of the magnetic field $B$ of the
pulsar, the evolution of the star may be parameterized as
$P^{n-2}\dot{P}=ak(B\sin\alpha)^{2}\,.$ [20], where the angle $\alpha>0$
describes the inclination of the magnetic dipole with respect to the rotation
axis, $a$ is a constant of unit sn-3 and $k$ takes the value of $9.76\times
10^{-40}$ s G-2 for canonical characteristics of neutron stars. Within this
model, the spin-down luminosity evolves with time $t$ as
$\dot{E}(t)=\dot{E}_{0}\left(1+\frac{t}{\tau_{0}}\right)^{-\frac{n+1}{n-1}}.$
Finally, the prediction on the typical decay time $\tau_{0}$ is derived to be
$\tau_{0}=\frac{P_{0}}{(n-1)\dot{P}_{0}}.$ By extending the functions
implemented in the Python module gammapy.astro.population [21, 22], we sample
the values of $P_{0},B,n$ and $\alpha$ from the distributions provided in [11]
(CB20), which will be our benchmark model, see Table 1 in [1] for the
parameter intervals.

In order to assess the effect of different distributions for $P_{0},B,n$ and
$\alpha$, we consider the model in [23] (FK06 hereafter). We note that both
CB20 and FK06 models have been calibrated to reproduce the characteristics of
the sources detected in the ATNF catalog [24], like the $P$, $\dot{P}$, $B$,
flux densities at 1.4 GHz, Galactic longitudes and Galactic latitudes
distributions. CB20 is the most updated model and considers the variation of
more parameters with respect to FK06.

###  2.3 Release, and propagation around sources

In our benchmark model we will consider only sources with ages above 20 kyr,
since $e^{\pm}$ accelerated to TeV energies in the termination shock are
believed to be confined in the nebula or in the SNR until the merge of this
system with the ISM, estimated to occur some kyr after the pulsar formation.
We thus leave out sources for which the $e^{\pm}$ pairs might be still
confined in the parent remnant. However, this effective treatment does not
account for possible spectral or time-dependent modifications of the released
particles. To understand the consequences of this assumption on the
interpretation of the AMS-02 $e^{+}$ flux, we also test the hypothesis that
only the $e^{\pm}$ produced after the escaping of the pulsar from the SNR
contribute to the flux at the Earth. Following [8], we define $t_{BS}$ as the
time at which the source leaves the parent SNR due to its proper motion and
eventually forms a bow-shock nebula. The formalism is reported in [25], to
which we refer for further details. To test this scenario we additionally
simulate for each source its birth-kick velocity, adopting its distribution as
reported in [23] (FK06VB) and implemented in gammapy.astro.population [21,
22].

###  2.4 Pulsar spatial distribution

Using gammapy.astro.population [21, 22] we adopt the radial surface density of
pulsars $\rho_{L}(r)$ proposed by [10]. As a comparison, we also consider the
radial surface density $\rho_{F}(r)$ in [23]. We sample the position ${\bf r}$
of each source combining the radial surface density with the spiral arm
structure of the Milky Way of [23] (see their Table 2 for the spiral arm
parameters), as implemented in gammapy.astro.population [21, 22]. We test only
one spiral arm structure, since the most important aspect in the computation
of the $e^{+}$ flux is the source density in the arms nearby the Sun, instead
of the position of the arms themselves. The distance of each source is
$d$=$|{\bf r}-{\bf r}_{\odot}|$, with ${\bf r}_{\odot}=(8.5,0,0){\rm kpc}$.

##  3 Transport in the Galaxy

Once charged particles are injected in the Galaxy, they can propagate and
eventually reach the Earth. The number density per unit time, volume
$N_{e}(E,{\bf r},t)$ of $e^{\pm}$ at an observed energy $E$, a position ${\bf
r}$ in the Galaxy, and time $t$, which is the solution to the propagation
equation considering only diffusion and energy losses, is given by [14]:

$N_{e}(E,{\bf
r},t)=\int_{0}^{t}dt^{\prime}\frac{b(E_{s})}{b(E)}\frac{1}{(\pi\lambda^{2}(t^{\prime},t,E))^{\frac{3}{2}}}\exp\left(-\frac{|{\bf
r}-{\bf r_{s}}|^{2}}{\lambda^{2}(t^{\prime},t,E)}\right)Q(E_{s},t^{\prime})$
(2)

where the integration over $t^{\prime}$ accounts for the PWN releasing
$e^{\pm}$ continuously in time. The energy $E_{s}$ is the initial energy of
$e^{\pm}$ that cool down to $E$ in a loss time
$\Delta\tau\equiv\int_{E}^{E_{s}}\frac{dE^{\prime}}{b(E^{\prime})}=t-t_{obs}.$
The $b(E)$ term is the energy loss rate, ${\bf r_{s}}$ indicates the source
position, and $\lambda$ is the typical propagation length defined as
$\lambda^{2}=\lambda^{2}(E,E_{s})\equiv
4\int_{E}^{E_{s}}dE^{\prime}\frac{D(E^{\prime})}{b(E^{\prime})}$ where
$D(E)=D_{0}E^{\delta}$ is the diffusion coefficient taken as a power-law in
energy. The $e^{\pm}$ energy losses include ICS off the ISRF and the
synchrotron emission on the Galactic magnetic field. The flux of $e^{\pm}$ at
the Earth for a source of age $T$ and distance $d=|{\bf r_{\odot}-r_{s}}|$ is
given by $\Phi_{e^{\pm}}(E)=\frac{c}{4\pi}\mathcal{N}_{e}(E,{\bf
r=r_{\odot}},t=T).$ We consider as benchmark case the propagation parameters
as derived in [26] from a fit to the latest AMS-02 data for the B/C,
antiproton and proton data. Energy losses are computed on the interstellar
photon populations at different wavelengths following [27], by taking into
account the Klein-Nishina formula for ICS, and on the Galactic magnetic field
with intensity $B=3$ $\mu$G. As a comparison, we will also implement the SLIM-
MED model derived in [28], with the ISRFs taken from [29] and $B=1$ $\mu$G.

##  4 Simulation and fit strategies

We produce simulations for the following setups: ModA (benchmark): Spin-down
and pulsar evolution properties are taken from CB20 [11], while the radial
surface density of sources is modelled with [10]. $\eta$ and $\gamma_{e}$ are
extracted from uniform distributions, while the propagation in the Galaxy is
taking into account following [26]. ModB (radial distribution effect): Same as
ModA but with the radial surface density of sources from [23]. ModC (spin-down
properties effect): Same as ModA, but spin-down properties are taken from FK06
[23]. ModD (propagation effect): Same as ModA apart for propagation in the
Galaxy, modelled as in [28] (their model SLIM-MED). ModE (kick velocity
effect): Same as ModA, but considering only the $e^{\pm}$ emitted after the
escaping of pulsars from the SNR. The birth kick velocities are sampled
adopting the distribution FK06VB reported in [23].

For each simulation setup we build and test 1000 simulations. We compute the
$e^{+}$ flux at the Earth as the sum of the primary component due to PWNe
emission, and a secondary component due to the fragmentation of CRs on the
nuclei of the ISM, taken from [26] or [28] consistently with the propagation
model employed. The secondary component enters in our fits with a free
normalization factor $A_{S}$, which we generously let to vary between 0.01 and
3. We also let the total flux generated by the sum of all PWNe to be shifted
by an overall normalization factor $A_{P}$. The values of $A_{P}$ and $A_{S}$
are obtained for each simulation with the fit procedure. We fit AMS-02 data
[2] above 10 GeV, in order to avoid strong influence from solar modulation and
other possible low energy effects. Nevertheless, we correct our predictions
for solar modulation effects following the force field approximation and
leaving the Fisk potential $\phi$ free to vary between 0.4 and 1.2 GV. The
comparison of our predictions with the AMS-02 $e^{+}$ data is performed by a
standard $\chi^{2}$ minimization procedure.

##  5 Results

###  5.1 Fit to AMS-02 data

The fit of the predictions for the total $e^{+}$ flux to the AMS-02 data is
performed for all the 1000 simulations built for each scenario A-B-C-D-E. In
all the tested setups, the number of mock galaxies with a $\chi^{2}_{\rm
red}<1$ (2) does not exceed 1% (4%). In order to inspect the effects of
different simulated Galactic populations, we plot in Figure 1 the total
$e^{+}$ flux for all the pulsar realizations within ModA, and having
$\chi^{2}_{\rm red}<$1.5 on AMS-02 data. For energies lower than 200 GeV,
differences among the realizations are indistinguishable. The data in this
energy range are very constraining. Instead, above around 300 GeV the
peculiarities of each galaxy show up, thanks to the larger relative errors in
the data. Above 1 TeV the predictions are unconstrained by data. Nevertheless,
all the simulations predict globally decreasing fluxes, as expected by energy
losses and continuous $e^{\pm}$ injection. However, at these energies the
total flux gets dominated by the secondary component.

Figure 1: Left: Total (secondary plus PWNe) $e^{+}$ flux obtained from all the
8 simulations within ModA with $\chi^{2}_{\rm red}<1.5$, along with AMS-02
data (black points). Right: Mean number of PWNe that satisfy the AMS-02 errors
criterion in the single energy bin of AMS-02 data. We also show the $68\%$
containment band for simulations with $\chi^{2}_{\rm red}<1.5$.

###  5.2 How many dominant sources?

We inspect the average number of sources which contribute the most to the
$e^{+}$ and thus can shape the AMS-02 flux. We adopt two complementary
criteria to estimate the number of sources that are responsible for the most
significant contribution of the PWNe $e^{+}$ emission: (1)AMS-02 errors: we
count all the sources that produce a flux higher than the experimental flux
error in at least one energy bin above 10 GeV; (2)Total flux 1%: we count the
sources that produce the integral of $\Phi_{e^{\pm}}(E)$ between 10 and 1000
GeV higher than 1% of the total integrated $e^{+}$ flux measured by AMS-02. In
Figure 1 we report the average number of PWNe with the standard deviation (68%
containment band) that contribute in the different energy bins of AMS-02, for
configurations with $\chi^{2}_{\rm red}<1.5$, adopting the AMS-02 errors
criterion. On average, 2-3 sources shine with a flux at least at the level of
AMS-02 $e^{+}$ data errors. We also find a decreasing number of dominant
sources with increasing energy for all the setup reported. This result is
partially induced by the larger experimental errors at high energy, which
raise the threshold for the minimum flux that a PWN has to produce in order to
satisfy the AMS-02 errors criterion. Moreover, being the age simulated in a
uniform interval, the number of young sources responsible for the highest
energy fluxes is smaller than for old pulsars, whose $e^{+}$ have suffered
greater radiative cooling. Overall, it indicates that only a few sources with
a large flux are required in order to produce a good fit to the data.

###  5.3 Characteristics of sources dominating the $e^{+}$ flux

For each Galactic realization of ModA with $\chi^{2}_{\rm red}<1.5$, we report
in Figure 6a[1] the distance, age and maximum $E^{3}\Phi_{e^{+}}(E)$ of the
PWNe satisfying the AMS-02 errors criterion. The data require 1 or 2 sources
with high maximum $E^{3}\Phi_{e^{\pm}}(E)$, with ages between 400 kyr and 2000
kyr and distances to the Earth less than 3 kpc. These sources produce fluxes
peaked between 100 GeV and 500 GeV, allowing good explanation to the data.
Fluxes from farther PWNe contribute less to the data. Sources with small
maximum $E^{3}\Phi_{e^{\pm}}(E)$ and with ages between 2000 kyr and $10^{4}$
kyr also satisfy the criterion, with flux peaks below 100 GeV where the
secondaries are still the dominant component. We do not find any particular
difference between all the simulation setups, except for ModD and ModE. Since
the SLIM-MED propagation implemented in ModD produces smoother fluxes, we find
also some realizations with few more sources contributing with a bright flux
to the $e^{+}$ data. In Figure 6b we report the distances, age and maximum
$E^{3}\Phi_{e^{\pm}}(E)$ values of the dominant PWNe for the mock galaxies
with the worst $\chi^{2}$. These cases give best-fit to the data with the
maximum values allowed by the priors for $A_{S}$ and the lowest values of
$A_{P}$. Moreover, there are not sources which satisfy the AMS-02 errors
criterion with an age between 400 kyr and 2000 kyr. In these galaxies, the
trend of $E^{3}\Phi_{e^{\pm}}(E)$ at high energies remains constant or
decreases, and does not contribute sufficiently to the data above 50 GeV. To
compensate this effect, the fit procedure demands the highest value of
$A_{S}$. We also inspect the distribution of best-fit efficiencies vs initial
spin-down energy of each PWN that satisfies the AMS-02 errors criterion, for
each simulation with $\chi^{2}_{\rm red}<1.5$. The reported efficiencies are
obtained multiplying the simulated $\eta$ values associated to a single source
with the $A_{P}$ obtained from the best fit of the corresponding galaxy. The
efficiencies have a scattered distribution, and in most cases they have a
value between 0.01 and 0.1, confirming the goodness of the $\eta$ interval
initially chosen. Data hint at a slight anti-correlation between $\eta$ and
$W_{0}$. In order to check that the characteristics of these pulsars are
consistent with observations, we compute $\dot{E}$ from $W_{0}$, finding
values quite common in nature. The ATNF catalog [24] lists about 60 sources
with $\dot{E}$ values higher than the maximum values obtained from sources in
our simulations, namely $\dot{E}\sim 10^{36}$ erg s-1. We do not directly
compare the $W_{0}$ values, since for the sources of the ATNF catalog to
compute $W_{0}$ we need to assume arbitrarily the value of $n$ and $\tau_{0}$.
Instead, in our simulations we sample $n$ and we compute $\tau_{0}$ from the
simulated parameters like $P_{0}$, that is also strictly connected to $W_{0}$.

##  6 Summary & Conclusions

Precise AMS-02 $e^{+}$ data give us unprecedented insights on primary CR
antimatter sources, but raise also new questions on the CR production and
transport in our Galaxy. In this contribution, the AMS-02 $e^{+}$ data are
used to constrain the main properties of the Galactic pulsar population and of
the PWN acceleration needed to explain the observed CR flux. To this aim, we
simulate a large number of Galactic pulsar populations, calibrated on ATNF
catalog observations. The result is fitted to AMS-02 $e^{+}$ data. Our
simulations are conducted under different hypothesis about the pulsar spin-
down and evolution properties, their radial distribution, and changing the
propagation models for $e^{\pm}$ propagation in the Galaxy, following the most
recent self-consistent modelings available in the literature.

Independently of the simulation scenario, we find that the vast majority of
the galaxies realizations produce several wiggles in the total contribution
and therefore they do not fit well the data. In all the tested setups, the
number of mock galaxies with a $\chi^{2}_{\rm red}<1$ (2) does not exceed 1%
(4%). The different features of the flux from single PWNs are due to the
peculiar combination of the input parameters. We notice that the secondary
flux, while decreasing with energy, practically forbids the realization of
sharp cut-offs in the $e^{+}$ spectrum above TeV energies. The galaxy
realizations that fit properly the AMS-02 $e^{+}$ data have between 2-3
sources that produce a $e^{+}$ yield at the level of the data errors.
Moreover, these pulsars provide a smooth spectrum that cover a wide energy
range. We find that the dominant contribution comes from sources located
between 1 and 3 kpc from the Earth. Despite the smaller effect from radiative
cooling, the flux from sources within 1 kpc is lower due to the paucity of
sources set along the spiral arms of the Galaxy. Sources dominating the
observed spectrum have ages between 400 kyr and 2000 kyr and distances to the
Earth less than 3 kpc. These sources produce fluxes peaked between 100 GeV and
300 GeV, where AMS–02 data are the most constraining. Finally, we do not find
any particular distribution for the pulsar efficiencies. In most cases they
have a value between 0.01 and 0.1 consistently with what found in our previous
papers [14].

Our results are compared with other recent attempts of simulating pulsar
populations [17, 15, 8] in the last section of [1]. To move forward, we
anticipate refinements on the acceleration, release and propagation models
around pulsars using multiwavelength probes, as well as a detailed analysis of
pulsar in source catalogs possibly matching the characteristics suggested by
the simulations.

## References

## References

  * [1] Luca Orusa, Silvia Manconi, Fiorenza Donato, and Mattia Di Mauro.  Constraining positron emission from pulsar populations with AMS-02 data.  JCAP, 12(12):014, 2021.
  * [2] M. Aguilar, L. Ali Cavasonza, G. Ambrosi, et al.  Towards understanding the origin of cosmic-ray positrons.  Phys. Rev. Lett., 122:041102, Jan 2019.
  * [3] Manuela Vecchi.  Latest results on nuclei mesurements with AMS-02.  VHEPU 2022.
  * [4] M. Di Mauro, F. Donato, N. Fornengo, and A. Vittino.  Dark matter vs. astrophysics in the interpretation of AMS-02 electron and positron data.  JCAP, 5:031, May 2016.
  * [5] Paolo Lipari.  Spectral shapes of the fluxes of electrons and positrons and the average residence time of cosmic rays in the Galaxy.  Phys. Rev., D99(4):043005, 2019.
  * [6] Philipp Mertsch.  Electron+positron spectra with CALET, DAMPE, AMS.  VHEPU 2022.
  * [7] S. Manconi, M. Di Mauro, and F. Donato.  Multi-messenger constraints to the local emission of cosmic-ray electrons.  Journal of Cosmology and Astroparticle Physics, 2019(04):024–024, Apr 2019.
  * [8] Carmelo Evoli, Elena Amato, Pasquale Blasi, and Roberto Aloisio.  Galactic factories of cosmic-ray electrons and positrons.  Physical Review D, 103(8), Apr 2021.
  * [9] Elena Amato.  The theory of Pulsar Wind Nebulae: recent progress.  PoS, HEPROVII:033, 2020.
  * [10] D. R. Lorimer, A. J. Faulkner, A. G. Lyne, R. N. Manchester, M. Kramer, M. A. McLaughlin, G. Hobbs, A. Possenti, I. H. Stairs, F. Camilo, and et al.  The parkes multibeam pulsar survey - vi. discovery and timing of 142 pulsars and a galactic population analysis.  Monthly Notices of the Royal Astronomical Society, 372(2):777–800, Oct 2006.
  * [11] Anirban Chakraborty and Manjari Bagchi.  Understanding the Galactic population of normal pulsars: A leap forward.  arXiv:2012.13243, 12 2020.
  * [12] Mattia Di Mauro.  Inverse Compton halos around pulsars and consequences for the AMS-02 positron excess.  VHEPU 2022.
  * [13] A. U. Abeysekara et al.  Extended gamma-ray sources around pulsars constrain the origin of the positron flux at Earth.  Science, 358(6365):911–914, 2017.
  * [14] Mattia Di Mauro, Silvia Manconi, and Fiorenza Donato.  Detection of a $\gamma$-ray halo around Geminga with the Fermi -LAT data and implications for the positron flux.  Phys. Rev. D, 100(12):123015, 2019.
  * [15] O. Fornieri, D. Gaggero, and D. Grasso.  Features in cosmic-ray lepton data unveil the properties of nearby cosmic accelerators.  Journal of Cosmology and Astroparticle Physics, 2020(02):009–009, feb 2020.
  * [16] Mattia Di Mauro, Fiorenza Donato, and Silvia Manconi.  Novel interpretation of the latest AMS-02 cosmic-ray electron spectrum.  Phys. Rev. D, 104(8):083012, 2021.
  * [17] Ilias Cholis, Tanvi Karwal, and Marc Kamionkowski.  Studying the milky way pulsar population with cosmic-ray leptons.  Physical Review D, 98(6), Sep 2018.
  * [18] Silvia Manconi, Mattia Di Mauro, and Fiorenza Donato.  Contribution of pulsars to cosmic-ray positrons in light of recent observation of inverse-Compton halos.  Phys. Rev. D, 102(2):023015, 2020.
  * [19] Philipp Mertsch.  Stochastic cosmic ray sources and the TeV break in the all-electron spectrum.  JCAP, 11:045, 2018.
  * [20] J. P. Ridley and D. R. Lorimer.  Isolated pulsar spin evolution on the diagram.  Monthly Notices of the Royal Astronomical Society, 404(2):1081–1088, May 2010.
  * [21] C. Nigro, C. Deil, R. Zanin, T. Hassan, J. King, J. E. Ruiz, L. Saha, R. Terrier, K. Brügge, M. Nöthe, and et al.  Towards open and reproducible multi-instrument analysis in gamma-ray astronomy.  Astronomy and Astrophysics, 625:A10, Apr 2019.
  * [22] Christoph Deil et al.  Gammapy - A prototype for the CTA science tools.  PoS, ICRC2017:766, 2018.
  * [23] C.-A. Faucher-Giguere and V. M. Kaspi.  Birth and Evolution of Isolated Radio Pulsars.  ApJ, 643:332–355, May 2006.
  * [24] R. N. Manchester, G. B. Hobbs, A. Teoh, and M. Hobbs.  The Australia Telescope National Facility Pulsar Catalogue.  AJ, 129:1993–2006, April 2005.
  * [25] E. van der Swaluw, A. Achterberg, Y. A. Gallant, T. P. Downes, and R. Keppens.  Interaction of high-velocity pulsars with supernova remnant shells.  A&A, 397(3):913–920, Jan 2003.
  * [26] Mattia Di Mauro and Martin Wolfgang Winkler.  Multimessenger constraints on the dark matter interpretation of the Fermi-LAT Galactic center excess.  Phys. Rev. D, 103(12):123005, 2021.
  * [27] Silvia Vernetto and Paolo Lipari.  Absorption of very high energy gamma rays in the Milky Way.  Phys. Rev., D94(6):063009, 2016.
  * [28] Yoann Génolini, Mathieu Boudaud, Marco Cirelli, Laurent Derome, Julien Lavalle, David Maurin, Pierre Salati, and Nathanael Weinrich.  New minimal, median, and maximal propagation models for dark matter searches with Galactic cosmic rays.  Phys. Rev. D, 104(8):083005, 2021.
  * [29] T. Delahaye, J. Lavalle, R. Lineros, F. Donato, and N. Fornengo.  Galactic electrons and positrons at the Earth: new estimate of the primary and secondary fluxes.  A&A, 524:A51, December 2010.


11institutetext: 1 Turnitin, 2101 Webster St #1800, Oakland, CA 94612, USA
11email<EMAIL_ADDRESS>11email<EMAIL_ADDRESS>
# Full Page Handwriting Recognition via Image to Sequence Extraction

Sumeet S. Singh(✉) 11 0000-0002-5323-9678 Sergey Karayev 11

###### Abstract

We present a Neural Network based Handwritten Text Recognition (HTR) model
architecture that can be trained to recognize full pages of handwritten or
printed text without image segmentation. Being based on Image to Sequence
architecture, it can extract text present in an image and then sequence it
correctly without imposing any constraints regarding orientation, layout and
size of text and non-text. Further, it can also be trained to generate
auxiliary markup related to formatting, layout and content. We use character
level vocabulary, thereby enabling language and terminology of any subject.
The model achieves a new state-of-art in paragraph level recognition on the
IAM dataset. When evaluated on scans of real world handwritten free form test
answers - beset with curved and slanted lines, drawings, tables, math,
chemistry and other symbols - it performs better than all commercially
available HTR cloud APIs. It is deployed in production as part of a commercial
web application.

##  1 Overview

With the advancement of Deep Neural Networks, the field of HTR has progressed
by leaps and bounds. Neural Networks have enabled algorithm developers to
increasingly rely on features and algorithms learned from data rather than
hand crafted ones. This makes it easier than ever before to create HTR models
for new datasets and languages. That said, HTR models still embody domain-
specific inductive biases, assumptions and heuristics in their architectures
regarding the layout, size and orientation of text. We aim to address some of
these limitations in this work.

###  1.1 Challenges of Real World Full Page HTR

(a)  (d)

(b)

(c)

Figure 1: Samples from our Free Form Answers dataset. (a) Full page text with
drawing. (b) Full page computer source code. (c) Diagrams and text with
embedded math. (d) Math and text regions, embedded math and stray artifacts.

|
---|---

(a) Test image

(b) Target label

(c) Model’s prediction

Figure 2: Test image with two text and one math regions. The model
successfully skips the math region and transcribes the two text regions, with
some mistakes.

Our target data - Free Form Answers dataset - was derived from scans of test
paper submissions of STEM subjects, from school level all the way up to post-
graduate courses. It consists of images containing possibly multiple blurbs of
handwritten text, math equations, tables, drawings, diagrams, side-notes,
scratched out text and text inserted using an arrow / circumflex and other
artifacts, all put together with no reliable layout (Figures 1 and 2). Length
of transcription ranges from 0 to 1100 characters averaging around 160.

####  1.1.1 Problem Framework

In order to get a handle on this seemingly random structure, we define each
independent blurb as a region and view each image as being composed of one or
more regions. The regions are further classified as text, math, drawing,
tables and deleted (scratched) text. Non textual regions and deleted text
(hereafter untranscribed regions ) are optionally demarcated with special
auxiliary tags but left untranscribed otherwise. Text regions range from a few
characters to multiple paragraphs and are possibly interspersed or embedded
with untranscribed regions. Text was transcribed in the order it was meant to
be read111This becomes relevant when text is not horizontal or when inserted
using a circumflex or arrow. and line ends, empty lines, spaces and
indentations are also faithfully transcribed. These can be programmatically
removed later if so desired. Since the regions can be randomly situated on the
image, we define a predictable region sequencing order the natural reading
order as the order in which somebody would read the answer aloud. This order
is implicitly captured in the transcription and the model must learn to
reproduce it.

With the above framework in mind, the problem becomes: 1) Implicitly identify
and classify regions on the image 2) Extract text from each text region and
emit it in natural reading order 3) Ignore / skip unrecognized artifacts and
4) Produce auxiliary markup. We call the above formulation _extractive_ ,
since it seeks to extract the identified and desirable content but ignore the
unknown and undesirable. This formulation is generic enough to cover simple to
complex scenarios but also teaches the model to skip over artifacts that were
not encountered in the dataset.

###  1.2 Limitations of Existing Architectures

Most state-of-the-art HTR models rely on a prior image segmentation method to
cleanly isolate pieces of text (words, lines or paragraphs). There are several
problems with this approach.

First, image segmentation in such models is a separate system, usually based
on hand-crafted features and heuristics which do not hold up when the data
changes significantly. Some methods go even further and correct segmented text
for slant, curve, and height, using the same problematic features and
heuristics.

Second, and more importantly, clean segmentation of units of text is not even
possible in many cases of real world handwriting. For instance, observe that
lines are curved or interspersed with non-textual symbols and artifacts in
Figures 1 and 2. Further discussion of such limitations can be found in the
literature [3, 4].

Third, stitching a complete transcription from the individually transcribed
text regions introduces yet another system, with its own potentials for
errors, and brittleness to changing the data distribution (for example, right-
to-left languages would require a different stitching system).

Lastly, formatting and indentation tends to get lost in this three-step
process because text segments are usually concatenated using space or newline
as a separator. This would be unacceptable when transcribing Python language
source code, for instance.

In our end-to-end model all of the above steps are implicit and learned from
data. Adapting the model to a new dataset or adding new capabilities is just a
matter of retraining or fine-tuning with different labels or different data.

###  1.3 Model Overview

(a)

(b)

Figure 3: Model Schematics. Left: CNN Encoder and Transformer Decoder. Right:
The Transformer Layer is identical to [28] except that Self Attention maybe
optionally Local. Teacher Forcing Sequence refers to the ground-truth shifted
right during training or the predicted tokens when predicting.

####  1.3.1 Image to Sequence Architecture

Our model (Figure 3) belongs to the lineage of attention-based sequence-to-
sequence and tensor-to-tensor architectures [33, 29]. It consists of a ResNet
[11] _encoder_ and a Transformer [28] _decoder_. It has a _Image to Sequence
architecture_ i.e., it learns to map an image to a sequence of tokens. The
model signals end of output sequence via a special <eos> token, so there is no
limit on sequence length222Except a limit set at prediction to prevent an
endless loop..

####  1.3.2 Formatting and Auxiliary Markup

We trained the model to extract text from text regions only, and to either
skip the the untranscribed regions or produce special markup tags ( <end-of-
region>, <math>, <deleted-text>, <table> and <drawing>) for them. This
requires the model to identify the untranscribed regions, which are almost
always _unique per sample_. It generalizes reasonably well in either of these
tasks. Similarly, because whitespace is preserved in our ground truth
transcription, the model learns to faithfully replicate multiple empty lines
and indentation spaces, which is important for computer source code for
example.

####  1.3.3 Token Vocabulary

In order to cater to a variety of subject matter terminology and proper nouns,
we chose to use character level token vocabularies in our base configuration.
That said, we experimented with different types of vocabularies, for example
character vs hybrid word/character, or lowercase vs mixed case. Model
performance did not differ significantly between these variations. Therefore,
we are confident that more complex transcription tasks – such as transcribing
math [6, 26] and complex layouts such as tables – should be possible given
appropriately transcribed training data and a character-level vocabulary.

####  1.3.4 Layout Agnostic Architecture

Since there are no assumptions regarding the layout of text, the model works
equally well on single-column vs two-column text, and easily learns to produce
a column separator tag (Table 2). The multistage HTR models described in
subsection 1.2 on the other hand, would concatenate adjacent lines from the
two columns, unless a separate code module was added to recognize columns and
modifications made to the rest of the code.

####  1.3.5 Performance

The model outperforms the best commercially available API on our proprietary
Free Form Answers dataset (Table 2) and state of art full page recognition [4]
and full paragraph recognition [3] models on the academic IAM dataset (Table
1).

##  2 Related Work

The first work to use Neural Networks for offline HTR was by Lecun et al. [15]
which recognized single digits. The next evolution was by Graves and
Schmidhuber [10] who used an MDLSTM model and CTC loss and decoding [9] to
recognize Tunisian town names in Arabic. Their model was designed to recognize
a single line of text, oriented horizontally. Pham et al. refined the
MDLSTM+CTC model by adding dropout, but it remained a single-line recognition
model dependent on prior line segmentation and normalization. Voigtlaender et
al. [30] further increased performance by adding deslanting preprocessing
(which is entirely dataset specific) and using a bigger network.

Bluche et al. [4] developed a model with the same vision as ours: full-page
recognition with no layout or size assumptions. However, they subsequently
abandoned this approach citing prohibitive memory requirements, unavailability
of GPU acceleration for training of MDLSTM and intractable inference time.
Follow-up work by the same authors [3, 2] saw a come-back to the encoder-only
+ CTC approach but with a scaled-back MDLSTM attention model that could
isolate individual lines of text, in effect performing automatic line
segmentation and enabling the model to recognize paragraphs. While far
superior to single-line recognition models, this approach still hard-codes the
assumption that lines of text stretch horizontally from left to right, and
fill the entire image. Therefore, it can’t handle arbitrary text layouts and
instead, relies on paragraph segmentation. This approach also does not output
a variable-length sequence rather a fixed number of lines $T$ (some of which
may be empty), $T$ being baked into the model during training. Finally, since
the predicted lines are joined together using a fixed separator it is unlikely
that the model can faithfully reproduce empty lines and indentation.

Further on, embracing the trend towards more parallel architectures, Bluche
and Messina [2] and Puigcerver [23] replaced the MDLSTM encoder with CNN, but
continued to rely on CTC.

Another recent trend in deep learning has been the move away from encoder-only
towards encoder-decoder sequence-to-sequence architectures. These
architectures decouple the sequence length of the output from the input
thereby eliminating the need for CTC loss and decoding. Cross-entropy loss and
greedy / beam-search decoding which are less compute-intensive are employed
instead. Ly et al. [16] applied such architecture to recognizing very short
passages of japanese text (a handful of short vertical lines). Similarly Wang
et al. [31] applied a bespoke sequence-to-sequence architecture for
recognizing single lines.

It is well established now that Transformers can completely replace LSTMs in
function and are more parameter efficient, accurate and enable longer
sequences. Embracing this trend Kang et al. [13] published the most ‘modern’
architecture to date employing CNN and Transformers somewhat like ours.
However since it collapses the vertical dimension of the image feature map,
this model is designed to recognize single lines only. It also employs a
transformer encoder thereby making it larger than our model. To our knowledge,
ours is the only work other than [4] that attempts full page handwriting
recognition.

##  3 Model Architecture

Our Neural Network architecture is shown in Figure 3. It is an encoder-decoder
architecture, using ResNet [11] for encoding the image, and Transformer [28]
for decoding the encoded representation into text. We refer you to [33, 26,
28, 27] for a background on neural image-to-sequence and sequence-to-sequence
models. This section will fill in the remaining details necessary to reproduce
our model architecture.

We use the term _base configuration_ to refer to our most frequently used
model configuration, and all configuration parameters we list hereafter
describe this configuration unless stated otherwise.

####  3.0.1 Encoder

The encoder uses a CNN to extract a 2D feature-map from the input image. It
uses the ResNet architecture without its last two layers: the average-pool and
linear projection. The feature-map is then projected to match the
Transformer’s hidden-size $d_{model}$, then a 2D positional encoding added and
finally flattened into a 1D sequence. 2D positional encoding is a fixed
sinusoidal encoding as in [28], but using the first $d_{model}/2$ channels to
encode the Y coordinate and the rest to encode the X coordinate (subsubsection
3.0.1) (similar to [20]). Output $\boldsymbol{I}$ of the _Flatten_ layer is
made available to all Transformer decoder layers, as is standard.

rCl PE(y, 2i) &= sin(y / 10000^2i/d_model)
PE(y, 2i+1) = cos(y / 10000^2i/d_model)
PE(x, d_model/2 + 2i) = sin(x / 10000^2i/d_model)
PE(x, d_model/2 + 2i+1) = cos(x / 10000^2i/d_model)
i ∈ [0, d_model/4)

####  3.0.2 Decoder

The decoder is a Transformer stack with non-causual attention to the encoder
output (its layers can attend to the encoder’s entire output) and causal self-
attention (it can only attend to past positions of its text input). As is
standard, training is done with teacher forcing, which means that the ground
truth text input is shifted one off from the output.

In total, the base configuration has 27.8 million parameters (6.3M decoder,
21.4M ResNet).

The input vectors are enhanced with 1D position encoding, as is standard.
Additionally, we concatenate a line number encoding ($lne$) - the scaled text
line number ($l$) that the token lies on - to it (Figure 3a). Assuming a
maximum of 100 text lines, $l\in[1,100]$ and $lne=l/100$. We added $lne$ to
the model in order to address line level errors i.e., missing or duplicated
lines but it was applied only in Table 1. We haven’t yet officially concluded
on its impact on model performance and mention it here only for completeness
sake.

In order to improve memory and computation requirements of our model, we
implemented a localized form of causal self-attention by limiting the
attention span to 50 (configurable) past positions. This is similar to Sliding
Window Attention of [1]) or a 1D version of Local Self Attention of [20]). We
hypothesized that a look back of 50 characters should be enough to satisfy the
language modeling needs of our task, while the limited attention span should
help training converge faster, both assumptions being validated by experiment.
Final model performance however, was not impacted by it. That said, a thorough
ablation study was not performed. Practically though, it allowed us to use
larger mini-batches by about 12%.

####  3.0.3 Objective Function

For each step $t$ the model outputs the token probability distribution
$\boldsymbol{p}_{t}$ over the vocabulary set $\\{1,\ldots,V\\}$ (subsubsection
3.0.3). This distribution is conditioned upon the tokens generated thus far
$\boldsymbol{y}_{<t}$ and $\boldsymbol{I}$ (subsubsection 3.0.3). Probability
of the entire token sequence is therefore given by subsubsection 3.0.3. rlll
p_t & : {1, …, V }→[0,1] ; Y_t ∼p_t
p_t(y_t) ≔ P(Y_t=y_t — y_¡t, I)
P(y—I) = ∏_t=1^τ p_t(y_t)

As is typical with sequence generators, the training objective here is to
maximize probability of the target sequence
$\boldsymbol{y}^{\scriptscriptstyle GT}$. We use the standard per-word cross-
entropy objective (subsubsection 3.0.3), modified slightly for the mini-batch
(subsubsection 3.0.3). We did not use any regularization objective, relying
instead on dropout, data-augmentations and synthetic data to provide
regularization. rCll L_seq & = -1τ ∑_t ln( p_t(y_t^GT)) ; τ≡sequence length
L_batch = -1n ∑_batch ∑_t ln( p_t(y_t^GT)) ; n ≡# of tokens in batch

The final Linear layer of the decoder (Figure. 3a) is a 1x1 convolution
function that produces logits which are then normalized by softmax to produce
$\boldsymbol{p}_{t}$.

####  3.0.4 Combination of Vision and NLP

One of the strengths of our architecture is in the combination of Vision and
Language models. CNNs such as ResNet are considered best for processing image
data. And Transformers are considered best for Language Modeling (LM) and
Natural Language Understanding (NLU) tasks [25, 24, 7], posessing properties
that are very useful in dealing with noisy and incomplete text that often
occurs in real handwriting. Having both the visual feature map and a language
model, the model can do a much better job than one relying on visual features
alone.

####  3.0.5 Inference

We use simple greedy decoding, which picks the highest probability token at
each step. Beam search decoding [8] did not yield any accuracy improvement
indicating that the model is quite opinionated / confident.

##  4 Training Configuration and Procedure

The base configuration uses grayscale images scaled down to 140-150 dots per
inch. Higher resolutions yielded slightly better accuracy at the cost of
compute and memory. We use the 34-layer configuration of ResNet, but have also
successfully trained the 18-layer and 50-layer configurations; larger models
tending to do better in general as expected.

The following is the base configuration of the Transformer stack:

  * •

$N$ (number of layers) = 6

  * •

$d_{model}$ = 260

  * •

$h$ (number of heads) = 4

  * •

$d_{ff}$ (inner-layer of positionwise feed-forward network) = 1024

  * •

Activation function inside feed-forward layer = GELU [12]

  * •

dropout = 0.5

The model was implemented in PyTorch [21], and training was carried out using
8 NVIDIA 2080Ti GPUs. For full page datasets a mini-batch size of 56 combined
with a gradient accumulation factor of 2 was used, yielding an effective
batch-size of 112. Single-line datasets had batch sizes as high as 200, but
were adjusted downwards when using higher angles of image rotation. ADAM
optimizer [14] was employed with a fixed learning rate ($\alpha$) of 0.0002,
$\beta_{1}$ = 0.9 and $\beta_{2}$ = 0.999.

While all images in a batch must have the same size; we also set all batches
to have the same image size, padding smaller images as needed. This helps
during training because any impending GPU OOM errors surface quickly at the
beginning of the run. It also makes the validation / test results agnostic of
the batching scheme since the images will always be the same size regardless
of how they are grouped. Smaller images within a batch are centered during
validation and testing. Padding color can be either the max of 4 corner pixels
or simply 0 (black), the choice having no impact on model accuracy.

The base configuration vocabulary consists of all lowercase ASCII printable
characters, including space and newline.

We observed that model performance increases with increased layer sizes and
also image resolution. It also tends to improve monotonically with training
time. Therefore, we trained our models for as long as possible the longest
being 11 days on full pages ($\scriptstyle\sim$47M samples) and 8 days
($\scriptstyle\sim$102M samples) on single lines. Typical training length
though is roughly 24M total training samples. Model state is saved
periodically during training. At the end of training, the checkpoint that
performs best on validation is selected for final testing.

##  5 Data

(a)

(b)

Figure 4: Synthetic samples generated from WikiText-103 Dataset [18]. For the
two column sample (b), the model – as per training – predicts the left column
first, then a column separator tag <col> and then the right column.

####  5.0.1 Data Sources

The following is a comprehensive set of all data sources, although each
experiment used only a subset:

  * •

IAM: The IAM dataset [17] with the RWTH Aachen University split [19], which is
a widely used benchmark for handwriting recognition.

  * •

WikiText: The WikiText-103 [18] dataset was used to generate synthetic images
(explained later).

  * •

Free Form Answers: Our proprietary dataset with about 13K samples described in
subsection 1.1.

  * •

Answers2: Another proprietary dataset of about 16K segmented words and lines
extracted from handwritten test answers. However since we train full pages, we
stitched back neighboring words and lines using heuristics.

  * •

Names: Yet another proprietary dataset of about 22K handwritten names and
identifiers.

Proprietary datasets were only used for results reported in Table 2.

####  5.0.2 Synthetic Data Augmentation

Since the IAM data set has word-level segmentation, we generate synthetic IAM
samples by stitching together images of random spans of words or lines. This
made it possible to significantly augment the IAM paragraph dataset beyond the
mere 747 training forms available in the RWTH Aachen split. Without this
augmentation the model would not generalize at all.

We also generate synthetic images on the fly from the WikiText data by picking
random text spans and rendering them into single-column and/or two-column
layout, using 34 fonts in multiple sizes for a total of 114 combinations
(Figure 4). The WikiText data is over 530 million characters long from which
over 583 billion unique strings of lengths ranging from 1 to 1100 may be
created. Multiplying this with 114 (fonts) yields an epoch size of 66.46
trillion which we would never get through in any of our runs. The dataset thus
provides us with a continuous stream of unique samples which builds the
implicit language model. Furthermore this trick can be used to ‘teach’ the
model new language and terminology by using an appropriate seed text file.
Addition of this dataset reduces the error rate on IAM paragraphs by only
about 0.4% (Table 1.) on the IAM dataset but significantly improves the
validation loss - about 30%.

Additionally, we generate empty images of varying backgrounds on the fly.
Without this data, the model generates text even on blank images – which
provides evidence of an underlying language model working a little
overzealously.

####  5.0.3 Image Augmentation

The following transformations were applied to individual training images: 1)
Image scale, 2) rotation, 3) brightness, 4) background color of synthetic
images, 5) contrast, 6) perspective and 7) Gaussian noise. At the batch level
the images were randomly placed within the larger batch image size during
training but centered during validation and testing.

####  5.0.4 Data Sampling

We found that the model generalized best when prior biases were eliminated
from the training data. Therefore the datasets are sampled on the fly for
every mini-batch with a configurable distribution. We also do not group images
by size or sequence length rather we sample them uniformly. Further,
parameters for synthetic data generation (e.g. the text span to render or the
font to render it in) and image augmentation (e.g. scale of image, angle of
rotation, background color) are also sampled uniformly.

Inspite of this sampling scheme, one bias does enter into the training data:
padding around text. This is so because most images of a batch are smaller
than the batch image size and therefore get padded. This causes the model to
perform poorly in inference unless the image was sufficient padding. The
optimal padding amount becomes a new hyperparameter that requires tuning after
training. We circumvented this problem by padding all images to the same
(large) size both during training and inference. This scheme though has the
downside of consuming the highest amount of encoder compute regardless of the
original image size. Therefore the first scheme is preferable for production.

##  6 Results

Table 1 shows character error rates (Levenstein edit distance divided by
length of ground truth) of paragraph and single line level tests on the IAM
dataset. FPHR refers to our model trained with only the IAM dataset whereas
FPHR+Aug refers to our model trained with the IAM plus WikiText based
synthetic dataset333We view synthetic WikiText based data as an augmentation
method since it does not rely on proprietary data or method.. FPHR trained at
$\scriptstyle\sim$145 DPI outperforms previous state of art in the paragraph
level test444Results from [2] are not included because it was trained on a lot
more than IAM data and 30% of it was proprietary. and FPHR+Aug improves error
rate by a further 0.4%. When trained on single lines only, performance is
similar to full page but short of the specialized state-of-the-art single-line
models. This is because the model does equally well on different text lengths
and number of lines provided they were uniformly distributed during training
i.e., performance tracks the training data distribution. Notice that our
corpus level CERs are lower than the averages indicating better performance on
longer text. This is not a characteristic of the model, rather that of the
training data which had fewer short sequences. We believe that the observed
performance (6+% test and 4+% validation CER) is the upper limit of this
system and task configuration i.e., the model architecture, its size, the
dataset and its split. Increasing the model size and / or image resolution
does improve performance slightly (albiet by less than 1%) as does increasing
the amount of training data.

When trained on all datasets and evaluated on Free Form Answers, our model
gets a error rate of 7.6% vs the best available cloud API’s 14.4% (Table 2).
Favoring the cloud models, we removed auxiliary markup, line indentations and
lower cased all predicted and ground truth text for this comparison555We
evaluated Microsoft, Google and Mathpix cloud APIs. Microsoft performed the
best and its results are reported here. This is not intended to be a
comparison of models, rather a practical data point that can be used to make
build-vs-buy decisions..

The model has no trouble transcribing two column layout; its performance on
two-column and single-column data are comparable (Table 2) providing evidence
of its adaptability to different text layouts. The Cloud API on the other hand
does well on single-column data, but falters with two-column text precisely
because it concatenates adjacent lines across columns.

Inferencing takes an average of 4.6 seconds on a single CPU thread for a set
of images averaging 2500x2200 pixels, 456 chars and 11.65 lines without model
compression i.e., model pruning, distillation or quantization.

Table 1: Comparison on the IAM dataset with and without closed lexicon
decoding (LM). Figures in brackets are corpus level scores. ★Model requires
paragraph segmentation. ‡FPHR trained with single lines only. SLS = Shredding
Line Segmentation.
Test Type | Model | Mean Test | Mean Test
---|---|---|---
CER w/o LM | CER w/ LM
Paragraph Level |  Bluche et al. [4] (150 dpi) | 16.2% |
FPHR ($\scriptstyle\sim$145 dpi) |  6.7 (6.5)% |
FPHR+Aug ($\scriptstyle\sim$145 dpi) |  6.3 (6.1)% |
Paragraph Level★ |  Bluche [3] (150 dpi) | 10.1% | 6.5%
Bluche [3] (300 dpi) | 7.9% |  5.5%
SLS + MDLSTM + CTC [4] (150 dpi) | 11.1% |
SLS + MDLSTM + CTC [4] (300 dpi) | 7.5% |
Single Line | Puigcerver [23] | 5.8% |  4.4%
Wigington et al. [32] | 6.4% |
Wang et al. [31] | 6.4% |
Kang et al. [13] |  4.7% |
FPHR+Aug‡ ($\scriptstyle\sim$145 dpi) | 6.5 (5.9)% |
Table 2: Character Error Rates on Free Form Answers and multi column synthetic
datasets. ⋆FPHR trained on one and two col. WikiText synthetic data.
Test Data set | Best Cloud API | FPHR
---|---|---
Free Form Answers | 14.4% |  7.6%
Wikitext (1 column) | 1.4% |  0.008%⋆
Wikitext (2 column) | 57% |  0.012%⋆

##  7 Conclusion

We have presented a “modern” neural network architecture that can be trained
to perform full page handwriting recognition without image segmentation,
delivers state of art accuracy and which is also small and fast enough to be
deployed into commercial production. It adapts reasonably to different
vocabularies, text layouts and auxiliary tasks and is therefore fit to serve a
variety of handwriting and printed text recognition scenarios. The model is
also quite easy to replicate using open source ‘off the shelf’ modules making
it all the more compelling. Although the overall Full Page HTR problem is not
solved yet, we believe it takes us one step forward from [4] and [3].

###  7.1 Limitations and Future Work

Although the presented framework encompasses multiple tasks, available
datasets are usually heavily biased towards one or two thereby masking the
model’s performance on outlier tasks. For e.g., there’s usually only one
transcribed text region per sample in the Free Form dataset which makes the
model tend to transcribe only one (main) text region while skipping others. On
the other hand when the dataset is balanced e.g., with one and two column
synthetic text, it performs well on both layouts. That said, this aspect needs
to be explored more thoroughly and hopefully with standardized datasets and
tasks so that the research community can iterate over it.

Second, we have only trained with text up to 1100 characters long and
averaging 360 characters. Should there be a need to transcribe longer lengths
of text say 10K characters, then some more work becomes necessary in order to
deal with longer sequence lengths. Possible solutions include the use of
multicharacter vocabularies and sparse Transformers such as [1, 5].

Other desirable improvements to the model include 1) reducing its sensitivity
to image padding. 2) Reducing the encoder’s size, which currently stands at
almost 22 million parameters. 3) Separating vision models for significantly
different visual data (e.g., Synthetic v/s Free Response Answers) so that they
may both contribute to the language model but not interfere with each other’s
vision models.

We believe that the Full Page HTR problem cannot be considered “solved” until
the error rate has been brought down to less than 1%. Therefore, more work
remains for the community.

####  7.1.1 Acknowledgements

We would like to thank Saurabh Bipin Chandra for implementing the fast
inference path ($O(N^{2})$) of the Transformer decoder, which was lacking in
PyTorch.

## References

  * Beltagy et al. [2020] Beltagy, I., Peters, M.E., and Cohan, A.  Longformer: The long-document transformer, 2020.
  * Bluche and Messina [2017] Bluche, T. and Messina, R.  Gated convolutional recurrent neural networks for multilingual handwriting recognition.  In _2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR)_ , volume 01, pages 646–651, 2017.
  * Bluche [2016] Bluche, T.  Joint line segmentation and transcription for end-to-end handwritten paragraph recognition.  _ArXiv_ , abs/1604.08352, 2016.
  * Bluche et al. [2016] Bluche, T., Louradour, J., and Messina, R.O.  Scan, attend and read: End-to-end handwritten paragraph recognition with MDLSTM attention.  _CoRR_ , abs/1604.03286, 2016.
  * Dai et al. [2019] Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q.V., and Salakhutdinov, R.  Transformer-xl: Attentive language models beyond a fixed-length context, 2019.
  * Deng et al. [2017] Deng, Y., Kanervisto, A., Ling, J., and Rush, A.M.  Image-to-markup generation with coarse-to-fine attention.  In _ICML_ , 2017.
  * Devlin et al. [2019] Devlin, J., Chang, M.W., Lee, K., and Toutanova, K.  Bert: Pre-training of deep bidirectional transformers for language understanding, 2019.
  * Graves [2008] Graves, A.  Supervised sequence labelling with recurrent neural networks.  In _Studies in Computational Intelligence_ , 2008.
  * Graves et al. [2006] Graves, A., Fernández, S., Gomez, F., and Schmidhuber, J.  Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks.  In _ICML ’06_ , 2006.
  * Graves and Schmidhuber [2008] Graves, A. and Schmidhuber, J.  Offline handwriting recognition with multidimensional recurrent neural networks.  In _NIPS_ , 2008.
  * He et al. [2015] He, K., Zhang, X., Ren, S., and Sun, J.  Deep residual learning for image recognition.  _CoRR_ , abs/1512.03385, 2015.
  * Hendrycks and Gimpel [2016] Hendrycks, D. and Gimpel, K.  Bridging nonlinearities and stochastic regularizers with gaussian error linear units.  _CoRR_ , abs/1606.08415, 2016.  URL http://arxiv.org/abs/1606.08415.
  * Kang et al. [2020] Kang, L., Riba, P., Rusiñol, M., Fornés, A., and Villegas, M.  Pay attention to what you read: Non-recurrent handwritten text-line recognition, 2020.
  * Kingma and Ba [2017] Kingma, D.P. and Ba, J.  Adam: A method for stochastic optimization, 2017.
  * Lecun et al. [1998] Lecun, Y., Bottou, L., Bengio, Y., and Haffner, P.  Gradient-based learning applied to document recognition.  _Proceedings of the IEEE_ , 86(11):2278–2324, 1998.
  * Ly et al. [2019] Ly, N.T., Nguyen, C.T., and Nakagawa, M.  An attention-based end-to-end model for multiple text lines recognition in japanese historical documents.  In _2019 International Conference on Document Analysis and Recognition (ICDAR)_ , pages 629–634, 2019.  doi: 10.1109/ICDAR.2019.00106.
  * Marti and Bunke [2002] Marti, U.V. and Bunke, H.  The iam-database: An english sentence database for offline handwriting recognition.  _International Journal on Document Analysis and Recognition_ , 5:39–46, 11 2002.  doi: 10.1007/s100320200071.
  * Merity et al. [2016] Merity, S., Xiong, C., Bradbury, J., and Socher, R.  Pointer sentinel mixture models.  _CoRR_ , abs/1609.07843, 2016.
  * [19] Open SLR.  Aachen data splits (train, test, val) for the iam dataset.  https://www.openslr.org/56/.  Identifier: SLR56.
  * Parmar et al. [2018] Parmar, N., Vaswani, A., Uszkoreit, J., Łukasz Kaiser, Shazeer, N., Ku, A., and Tran, D.  Image transformer, 2018.
  * Paszke et al. [2019] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S.  Pytorch: An imperative style, high-performance deep learning library.  In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., and Garnett, R., editors, _Advances in Neural Information Processing Systems 32_ , pages 8024–8035. Curran Associates, Inc., 2019.  URL http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf.
  * Pham et al. [2013] Pham, V., Kermorvant, C., and Louradour, J.  Dropout improves recurrent neural networks for handwriting recognition.  _CoRR_ , abs/1312.4569, 2013.  URL http://arxiv.org/abs/1312.4569.
  * Puigcerver [2017] Puigcerver, J.  Are multidimensional recurrent layers really necessary for handwritten text recognition?  In _2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR)_ , volume 01, pages 67–72, 2017.
  * Radford [2018] Radford, A.  Improving language understanding by generative pre-training.  2018\.
  * Raffel et al. [2020] Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P.J.  Exploring the limits of transfer learning with a unified text-to-text transformer, 2020.
  * Singh [2018] Singh, S.S.  Teaching machines to code: Neural markup generation with visual attention.  _CoRR_ , abs/1802.05415, 2018.
  * Sutskever et al. [2014] Sutskever, I., Vinyals, O., and Le, Q.V.  Sequence to sequence learning with neural networks.  _CoRR_ , abs/1409.3215, 2014.  URL http://arxiv.org/abs/1409.3215.
  * Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I.  Attention is all you need.  _CoRR_ , abs/1706.03762, 2017.
  * Vaswani et al. [2018] Vaswani, A., Bengio, S., Brevdo, E., Chollet, F., Gomez, A.N., Gouws, S., Jones, L., Kaiser, L., Kalchbrenner, N., Parmar, N., Sepassi, R., Shazeer, N., and Uszkoreit, J.  Tensor2tensor for neural machine translation.  _CoRR_ , abs/1803.07416, 2018.
  * Voigtlaender et al. [2016] Voigtlaender, P., Doetsch, P., and Ney, H.  Handwriting recognition with large multidimensional long short-term memory recurrent neural networks.  In _2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR)_ , pages 228–233, 2016.
  * Wang et al. [2019] Wang, T., Zhu, Y., Jin, L., Luo, C., Chen, X., Wu, Y., Wang, Q., and Cai, M.  Decoupled attention network for text recognition, 2019.
  * Wigington et al. [2018] Wigington, C., Tensmeyer, C., Davis, B., Barrett, W., Price, B., and Cohen, S.  Start, follow, read: End-to-end full-page handwriting recognition.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , September 2018.
  * Xu et al. [2015] Xu, K., Ba, J., Kiros, J.R., Cho, K., Courville, A.C., Salakhutdinov, R., Zemel, R.S., and Bengio, Y.  Show, attend and tell: Neural image caption generation with visual attention.  In _ICML_ , 2015.


# High-Resolution, Multi-Channel FPGA-Based Time-to-Digital Converter

Balázs Jákli1, Ádám Rák2 and György Cserey1  1Faculty of Information
Technology and Bionics, Pázmány Péter Catholic University, Budapest, Hungary
Email<EMAIL_ADDRESS>2MedioTrade Ltd., Budapest, Hungary

###### Abstract

In this paper we present a novel high-resolution multi-channel FPGA-based
time-to-digital converter (TDC). We designed and implemented a complex
electronic circuit on the FPGA, whose overall accuracy is several orders of
magnitude greater than the accuracy of the FPGA used in digital mode. Our
sensor device contains simple circuit elements that are cheap and easily
accessible (Xilinx Spartan 3 and Spartan 6). Using our design, many channels
(80-100 channels) can be implemented on a larger FPGA. The prototype of our
TDC has been implemented and functionally verified by experiments and
measurements. By a certified pulse generator 20 ps precision has been measured
over the range of 3 ns. Using more precise clock signal this range may be
extended. The achieved resolution is 5 ps. Its resolution, channel number and
range can be configured dynamically, which makes it suitable for effective use
in industrial purposes.

##  I Introduction

Integrated circuits which can measure time-intervals with high precision are
called time-to-digital converters (TDC). The majority of time measurement
tools on the market primarily use only a few channels (1-8 channels). A
channel is characterized by two inputs, it measures the time delay between
their signals. TDC has applications in various fields of industrial
engineering. It is used for LIDAR [1], medical imaging applications [2] and
time-of-flight mass spectrometry as well as it is used in logic analyzers,
where extremely high resolution and accuracy are needed. Multi-channel TDC can
be specifically apply to determine the 3D spatial localization of events with
cm precision in a particle physics detector. Until the recent years, most TDC
designs have been developed and implemented for specific applications, but the
latest trends shows that configurable circuit implementations can be
successful using field programmable gate arrays (FPGA)[6-13]. The advantages
of the FPGA are lower development time and lower cost in small series. Many
approaches have been tried to use the opportunities of the FPGA to measure
time [6-13]. One of the latest FPGA-based TDCs is documented in paper [3] that
is close in precision to our solution, but they used the Virtex II Pro FPGA,
which is more expensive, and it is not suitable for the realization of multi-
channel TDCs. [4] and [5] are presenting multi-channel TDCs (about 96
channels), but their resolution has the order of magnitude of 1 ns.

The paper is organized as follows: Section II describes our motivation.
Section III presents the proposed architecture. In Section IV experimental
results demonstrate the features of our TDC design. Finally, the conclusions
and future work are summarized in Section V.

##  II Motivation

The measurement of time is essential in electrical measurements, practicing
engineers use these techniques in many cases. An analog-digital conversion is
performed by a TDC. The analog input value is a relative time difference of
two events between each other, such as the delay between two signals. The
output is a value in a discrete range, which is the approximated result of the
converted input value by quantization. The unit of the conversion, the
resolution of the TDC is the shortest time which makes changes, either
increasing of decreasing of the output value. Currently, this magnitude is in
pico-second range.

The number of TDSs have reached a marketable level by the very large scale
integration of circuits, but the vast majority of commercially available
devices have only a few channels (channels 1-8). A channel is characterized by
two inputs, the relative temporal delay is measured between the inputs. These
sensors are implemented on ASICs (Application Specific Integrated Circuits).
Multi-channel TDCs are difficult and expensive, but there are tasks where they
can be used effectively: for example, using TDCs in particle detectors, 3D
spatial localization of events can be determined with precision in the order
of cm, and also their usage can be necessary in other particle physics
experiments. Nowadays these particle detectors are very expensive and have
only a relatively small accuracy. A cost-effective, high precision, multi-
channel TDC would be suitable to fulfill a currently empty market segment.
These TDCs would be used to complete measurements, which could not have been
done by physicists previously, as well as in multi-channel logic analyzer
devices, where high resolution and high accuracy is required.

##  III Proposed architecture

One of the fundamental innovations in our approach is that rather than only
compare the two input signals in a channel, we have included the many
calibrated reference signals as well in our comparison, and thus, we have been
able to achieve high precision and low-noise in our measurements. After
comparing the calibrated signals to the input signals, the resulting absolute
values render a measurement range that is theoretically infinite and
practically very large. In the event that we have multi-inputs, they are
simultaneously compared to the reference signals. Another innovation is the
design and measurement method of the internal structure of the FPGA, which
provides a stochastic bit-stream based time-to-digital converter (TDC). We can
reconfigure our system dynamically by changing the channel number or
measurement accuracy of each channel even during operation. The expected
achievable accuracy of the system is 1ps.

We designed and implemented an experimental prototype of our device (see Fig.
2). The schematic can be seen on Fig. 1. Besides the Xilinx Spartan 3 FPGA
chip, we placed a dsPic to do the control and communication to the FPGA chip.
This solution gives the opportunity to re-configurate the channel number and
the size of the actual measurement range of our system.

Figure 1:  Schematic of the prototype board. Figure 2: Implemented prototype.

##  IV Experimental Results

We measured the pulse delay between two signals generated by a two channel
function generator (Type: Agilent 81130A) A personal computer triggered the
signals, then read out the output values of the TDC conversion via USB
interface (see Fig. 3).

On the upper side of Fig. 5 we see the output of the TDC conversion depending
on the delay set on the function generator. At the bottom side of Fig. 5 , we
see the error between the theoretical and the measured delays. As we can see,
the accuracy of the measurement is about 20 pico-secundums, which is the
accuracy of the function generator. It is assumed that our method is capable
of even more accurate results.

The experimental prototypes of our device have been implemented and its
functionality is confirmed by experiments and measurements (see Fig. 6). We
have captured the raw stochastic bitstream output of our TDC. In our first
results we did not use any calibration on the bitstream, instead we have
simply summed the bits as integers. This summation decreased the stochastic
noise enough to allow us to see the working time measurements as seen in Fig.
6. We have also tested the shifting of our measurements range and it worked
exactly as precise as the clock source we have used as a reference source for
shifting by whole clock cycles therefore extending the range to practically
infinity (the precision is limited only by the frequency stability and the
jitter of the clock source). Our latest prototype was applied on the Xilinx
Spartan 6 FPGAs. The parameters of our innovative sensor can be configured
flexibly.

Figure 3:  We measured the pulse delay between two signals generated by a two
channel function generator (Type: Agilent 81130A) A personal computer
triggered the signals, then read out the output values of the TDC conversion
via USB interface.  Figure 4:  FPGA-A does the time-to-digital conversion.
FPGA-D is the control FPGA responsible for the temperature-stability, control
processes and measurements of the reference signals and their parameters.
Figure 5:  On the upper side we see the output of the TDC conversion depending
on the delay set on the function generator. At the bottom side, we see the
error between the theoretical and the measured delays. As we can see, the
accuracy of the measurement is about 20 pico-secundums, which is the accuracy
of the function generator. It is assumed that our method is capable of even
more accurate results.  Figure 6:  Measured raw uncalibrated data from our TDC
is depicted on the figure versus the input time difference of the impulses.

##  V Conclusions and future work

The experimental prototype of our device has been implemented (see Fig. 2) and
its functionality is confirmed by experiments and measurements (see Fig. 3).
We measured 20 ps accuracy rate by a certified pulse generator (20 ps, where
the range is 3 ns, see Fig. 5). The range of the measurement may be extended
by using a more accurate clock signal. Our prototypes were applied on the
smallest Xilinx Spartan 3 and Spartan 6 FPGAs. Using a greater FPGA, up to 100
high-precision channels can be implemented. The parameters of our innovative
sensor can be configured flexible. Using active cooling (Peltier-element) the
range of the sensor system can be extended, while operation range becomes more
stable.

## Acknowledgment

The invaluable help of the multidisciplinary doctoral school at the Faculty of
Information Technology of the Pázmány Péter Catholic University is gratefully
acknowledged. The authors acknowledge the support of the Hungarian Government
through grant no. VEKOP-2.1.7-15-2016-00550.

## References

  * [1] I. Nissinen and J. Kostamovaara, “On-chip voltage reference-based time-to-digital converter for pulsed time-of-flight laser radar measurements,” _Instrumentation and Measurement, IEEE Transactions on_ , vol. 58, no. 6, pp. 1938–1948, 2009.
  * [2] A. Yousif and J. Haslett, “A fine resolution tdc architecture for next generation pet imaging,” _Nuclear Science, IEEE Transactions on_ , vol. 54, no. 5, pp. 1574–1582, 2007.
  * [3] M. Daigneault and J. David, “A high-resolution time-to-digital converter on fpga using dynamic reconfiguration,” _Instrumentation and Measurement, IEEE Transactions on_ , vol. 60, no. 6, pp. 2070–2079, 2011.
  * [4] J. Wu, S. Hansen, and Z. Shi, “Adc and tdc implemented using fpga,” in _Nuclear Science Symposium Conference Record, 2007. NSS’07. IEEE_ , vol. 1. IEEE, 2007, pp. 281–286.
  * [5] V. Bocci, R. Nobrega _et al._ , “A multichannel tdc based on ring-delay time multiplexer: A prototype,” in _Nuclear Science Symposium Conference Record, 2007. NSS’07. IEEE_ , vol. 1. IEEE, 2007, pp. 720–724.
  * [6] C. Favi and E. Charbon, “A 17ps time-to-digital converter implemented in 65nm fpga technology,” in _Proceedings of the ACM/SIGDA international symposium on Field programmable gate arrays_. ACM, 2009, pp. 113–120.
  * [7] E. Bayer and M. Traxler, “A high-resolution ($\leq$ 10 ps rms) 32-channel time-to-digital converter (tdc) implemented in a field programmable gate array (fpga),” in _Real Time Conference (RT), 2010 17th IEEE-NPSS_. IEEE, 2010, pp. 1–5.
  * [8] M.-A. Daigneault and J. David, “A high-resolution time-to-digital converter on fpga using dynamic reconfiguration,” _Instrumentation and Measurement, IEEE Transactions on_ , vol. 60, no. 6, pp. 2070 –2079, june 2011.
  * [9] M. Daigneault and J. David, “A novel 10 ps resolution tdc architecture implemented in a 130nm process fpga,” in _NEWCAS Conference (NEWCAS), 2010 8th IEEE International_ , june 2010, pp. 281 –284.
  * [10] S. Korf, D. Cozzi, M. Koester, J. Hagemeyer, M. Porrmann, U. Ruckert, and M. Santambrogio, “Automatic hdl-based generation of homogeneous hard macros for fpgas,” in _Field-Programmable Custom Computing Machines (FCCM), 2011 IEEE 19th Annual International Symposium on_ , may 2011, pp. 125 –132.
  * [11] J. Imrek, G. Hegyesi, G. Kalinka, J. Molnar, F. Nagy, I. Valastyan, and Z. Szabo, “Fpga based tdc using virtex-4 iserdes blocks,” in _Nuclear Science Symposium Conference Record (NSS/MIC), 2010 IEEE_ , 30 2010-nov. 6 2010, pp. 1413 –1415.
  * [12] J. Zhang, B. Wang, Y. Cui, and B. Yang, “High-precision echo time of flight measurement method and its fpga implementation,” in _Remote Sensing, Environment and Transportation Engineering (RSETE), 2011 International Conference on_ , june 2011, pp. 6021 –6024.
  * [13] J. Wang, S. Liu, L. Zhao, X. Hu, and Q. An, “The 10-ps multitime measurements averaging tdc implemented in an fpga,” _Nuclear Science, IEEE Transactions on_ , vol. 58, no. 4, pp. 2011 –2018, aug. 2011.
  * [14] H. Orser, “High speed analog-to-digital conversion utilizing time quantization,” Ph.D. dissertation, FACULTY OF THE GRADUATE SCHOOL OF THE UNIVERSITY OF MINNESOTA, 2010.