,

# Data Incubation — Synthesizing Missing Data for
Handwriting Recognition

###### Abstract

In this paper, we demonstrate how a generative model can be used to build a
better recognizer through the control of content and style. We are building an
online handwriting recognizer from a modest amount of training samples. By
training our controllable handwriting synthesizer on _the same_ data, we can
synthesize handwriting with previously underrepresented content (_e.g_., URLs
and email addresses) and style (_e.g_., cursive and slanted). Moreover, we
propose a framework to analyze a recognizer that is trained with a mixture of
real and synthetic training data. We use the framework to optimize data
synthesis and demonstrate significant improvement on handwriting recognition
over a model trained on real data only. Overall, we achieve a 66% reduction in
Character Error Rate.

Index Terms—  Handwriting synthesis, handwriting recognition, synthetic data,
controllable generative models

##  1 Introduction

Fig. 1:  Data incubation starts with (a) collected data sparsely covering the
content and the style space. We then use a controllable generative model that
can (b) synthesize samples with unseen content, and (c) mix new style from
existing training samples. (d) Finally, we synthesize new samples that close
gaps in content and style, leading to a better coverage of the real data
distribution for handwriting recognition.

Data is the most valuable asset for machine learning applications, but are we
using it efficiently? The standard approach is relatively simple: (i) collect
and annotate data, and (ii) train a recognizer using the data. While this
approach has achieved success in many standard machine learning tasks, such as
image classification [1], we will show that it is sub-optimal for handwriting
recognition.

Handwriting data can be described (a) by its content (_i.e_., the text that
was written, a sequence of characters) and (b) by its style (_i.e_., all other
information about a handwriting sample except content — printed or cursive,
neat or messy, round or square corners, tight or loose spacing, _etc_.). The
content space is combinatorial and some handwriting styles are rare compared
to others. Therefore, it is difficult to collect sufficient data to cover the
entire content and style space well.

In this paper, we propose a method we refer to as data incubation to grow the
original collections and improve the generalization of recognizer training.
This capability is enabled by a controllable generative model that is used to
augment training with synthesized data covering underrepresented content and
rare styles (Fig. 1).

How does a controllable generative model improve data efficiency for a
downstream handwriting recognizer? After all, we learn the generative model
with the same dataset that we use to train the recognizer. Our approach is
based on the following observation: The total number of handwriting samples
that can be collected is limited and thus the diversity of contents and styles
in the collected samples is also limited (Fig. 1a). By utilizing our
controllable synthesizer that models the data distribution as a factored
distribution of style and content, we can combine information across collected
samples, complement the real training set with synthesized handwriting, and
fill both the content and the style gaps in the collected training data (Fig.
1b-d).

Effectively, we introduce prior knowledge about content and style being
factorizable into the creation of synthetic training data. This inductive bias
allows us to strategically increase the size of our training set to target a
more balanced distribution of writing styles and content as well as training
recognizers with better generalization.

Training with synthetic data, however, comes with its own challenges.
Synthetic data might introduce a shift from the empirical distribution that is
formed by the collected real data. While this shift can be due to the
artifacts produced by the generative model, it can also come from generating
content and styles that are underrepresented or missing in the collected data,
_i.e_., shifting the empirical distribution shown in Fig. 1a to that in Fig.
1d. In this aspect, this distribution shift can be seen both as a problem but
also as a feature. It is critical to make careful and deliberate choices about
which data to synthesize. If we only synthesize handwriting from well-formed
English sentences with highly legible, printed style, a recognizer trained on
this data will end up performing poorly on messy, cursive styles and content
including slang, uncommon words, or abbreviations. On the other hand, we can
carefully choose which data to synthesize and with that we will be able to
close the gaps in the empirical distribution.

In this paper, we demonstrate a systematic recipe to train a handwriting
recognizer with data synthesized by a controllable generative model. This
recipe enables us to examine and understand the problems associated with the
synthetic data. We build on the recently-proposed controllable generative
model [2], which separately controls the content and the handwriting style. As
we will demonstrate, utilizing data incubation significantly improves our
multilingual handwriting recognizer, achieving a 66% reduction in Character
Error Rate when compared to a recognizer trained with only original data.

##  2 Related work

Online handwriting recognition is a field of high relevance in mobile
computing because (a) an increasing number of devices are equipped with
styluses making handwritten input a natural interface, (b) many languages are
harder to enter with a keyboard than Latin scripts because the size of their
alphabet or the use of grapheme clusters makes building intuitive keyboards
difficult. While early work in online handwriting recognition used segment-
and-decode methods [3, 4, 5, 6], recent work trends toward end-to-end
approaches utilizing Convolutional Neural Networks (CNNs) and Recurrent Neural
Networks (RNNs), coupled with Connectionist Temporal Classification (CTC)
decoding [7, 8, 9, 10, 11].

Online handwriting synthesis models [2, 12, 13, 14, 15, 16] are sequence-to-
sequence models that take input text (a sequence of characters) and output
handwriting (a sequence of strokes). To improve downstream recognizers, it is
important for the synthesized handwriting to contain few artifacts and a wide
range of handwriting styles — from highly legible, printed-style handwriting
to less legible, cursive handwriting.

Synthetic data augmentation for training machine learning models is not new in
the literature. Most of the reference papers focus on image [17, 18] and
speech [19, 20, 21, 22, 23] synthesis, and obtain modest improvements. To the
best of our knowledge, this paper is the first to use a controllable
generative model to synthesize online handwriting and explicitly control the
content, style, and variation of the generation process to significantly
improve the accuracy of downstream handwriting recognizers.

##  3 Generative and recognizer models

Fig. 2:  (a) Online handwriting is a sequence of stylus movements. (b) Our
handwriting generative model takes an input content ${\bf c}$ and outputs
handwriting ${\bf x}$. The handwriting style is controlled using a latent
style variable $z_{t}$. The model can either mimic the style of an existing
style reference handwriting or it can sample a new style from a prior
distribution to generate unseen styles. (c) Our recognizer takes an input
handwriting ${\bf x}$ and outputs the probabilities of all symbols at each
time step, $p(\pi|{\bf x})$, which is converted to $p({\bf c}|{\bf x})$ by CTC
decoding.

###  3.1 Problem formulation and dataset

An online handwriting sample ${\bf x}=\left[({\Delta x}_{t},{\Delta
y}_{t},u_{t})\right]_{t=1\dots T}$ is a sequence of stylus movements on a
writing surface, where ${\Delta x}_{t}\,{\in}\,{\mathbb{R}}$ and ${\Delta
y}_{t}\,{\in}\,{\mathbb{R}}$ are the movement of the stylys in $x$ and $y$
directions respectively at time $t$, and $u_{t}\,{\in}\,\\{0,1\\}$ is a binary
variable indicating whether the stylus touches the surface during the
movement, as shown in Fig. 2a. The corresponding content of the sample ${\bf
c}=\left[c_{1},\dots c_{M}\right]$ is a sequence of characters, where $c_{m}$
is represented as a one-hot vector of dimension $Q$ (the total number of
characters). Note that in general ${\bf x}$ and ${\bf c}$ have different
lengths.

We use a subset of a proprietary dataset collected for research and
development. The subset consists of 600k online handwriting samples written by
1,500 people. Half of the samples (300k) is in English, and the rest is in
French, German, Italian, Spanish, and Portuguese.

###  3.2 Controllable generative model

We use the controllable generative model proposed in [2]. Given $N$ real
training samples, $\mathcal{X}=\left\\{({\bf x}^{i},{\bf
c}^{i})\right\\}_{i=1\dots N}$, the generative model learns a distribution of
handwriting samples, $p({\bf x}|{\bf c},{\bf z})$, conditioned on the content
${\bf c}$ and the style ${\bf z}$. In other words, given a target content
${\bf c}$ and a style ${\bf z}$, we can generate new handwriting by sampling
from the distribution. Note that the style ${\bf z}$ is modeled as a latent
variable with a (learnable) prior distribution. Fitting a prior distribution
for style is important, because it enables the model to create new styles
(_i.e_., ${\bf z}$ not associated with any training sample).

Fig. 2b shows an overview of the model architecture. We model in an auto-
regressive manner the output distribution $p({\Delta x}_{t},{\Delta
y}_{t},u_{t}|z_{t},{\bf x}_{1..{t{-}1}},{\bf c})$ as $p({\Delta x}_{t},{\Delta
y}_{t}|\cdot)p(u_{t}|\cdot)$, where $p(x_{t},y_{t}|\cdot)$ is a mixture of 20
bivariate Gaussian distributions modeling the movement of the stylus, and
$p(u_{t}|\cdot)$ is a Bernoulli distribution modeling whether the stylys
touches the surface. The prior distribution of $z_{t}$, $p(z_{t}|{\bf
x}_{1..{t{-}1}},{\bf c})$, is a multivariate Gaussian distribution
$\mathcal{N}(z_{t}|\mbox{\boldmath{$\mu$}}_{t},\Sigma_{t}))$, where
$\Sigma_{t}$ is a diagonal covariance, and $\mbox{\boldmath{$\mu$}}_{t}$ and
$\Sigma_{t}$ are functions of ${\bf x}_{1..{t{-}1}}$ and ${\bf c}$ (generated
by a linear layer). We refer to [2] for detailed description of the model. We
trained two generative models: an English model using only the English
training data and a multilingual model using training data from all six
languages.

One important property of the generative model is that we can control the
variation of the synthesized data by scaling the standard deviations of the
output distribution $p({\Delta x}_{t},{\Delta y}_{t},u_{t}|z_{t},{\bf
x}_{1..{t{-}1}},{\bf c})$. Intuitively, using a small standard deviation
generates handwriting close to the mean of the distribution, which results in
neat (average-style) handwriting; using a large standard deviation results in
more variations in the synthesized handwriting. When synthesizing, we use
sampling bias $b\geq 0$ to scale individual standard deviations and the
categorical distribution in the mixture of the Gaussian distributions,
following the rule in [14]. A small sampling bias produces large variations in
handwriting, and a large sampling bias results in small variations and
neat/typical handwriting.

###  3.3 Handwriting recognizer

A handwriting recognizer learns a distribution of the content, $p({\bf c}|{\bf
x})$, given an input handwriting sequence ${\bf x}$. As shown in Fig. 2c, our
handwriting recognizer is a 2-layer convolution network followed by a 2-layer
bidirectional LSTM and a linear layer. The input to the recognizer is a
sequence containing $u_{t},\sin(\theta_{t}),\cos(\theta_{t}),\ell_{t}$ for
$t=1,\dots,T$, where $\theta$ and $\ell$ are the angle and length of the
stylus movement at time $t$, _i.e_., they are features computed from ${\Delta
x}_{t}$ and ${\Delta y}_{t}$. The network outputs the probabilities of each
character (including a blank symbol) at every time $t$. We use CTC loss [7] to
train the model.

##  4 Synthesizing training data

If data is not synthesized carefully, the domain gap between real and
synthetic data can deteriorate the performance of the recognizer on real data.
Here, we present a systematic framework to identify problems with synthetic
data and optimize parameters of the generative model to minimize the gap. Our
recipe uses a similar procedure as [24] to evaluate the generation quality of
a generative adversarial network.

Given a synthetic dataset (randomly split into training, validation, and test
sets), we train three recognizers: one using only the real training data, one
using only the synthetic training data, and one using both. Each model is
evaluated using Character Error Rate (CER) on both the real and synthetic test
sets. We use the notation
$\mbox{CER}^{\mbox{}}_{\,\mbox{t}{\rightarrow}\mbox{v}}$ to indicate the CER
for a model trained on training set t and evaluated on set v, where
$\text{t},\text{v}\in\\{\text{r},\text{s},\text{b}\\}$ indicating real,
synthetic, and both data sets, respectively.

Note that the real training set (empirical distribution) is only a finite
sample approximation to the true data distribution and might be biased due to
the data collection procedure or lack certain content, style, or other
variations. Here, we refer to the empirical distribution as $\mathcal{S}_{r}$,
synthetic data distribution as $\mathcal{S}_{s}$ and the true data
distribution as $\mathcal{S}_{t}$. We assume that the real test set is
unbiased. The following are the _common_ cases:

#### Case 1: $\mathcal{S}_{s}\not\subseteq\mathcal{S}_{t}$.

Synthetic data contains artifacts or variations that do not exist in the true
distribution. These artifacts are equivalent to introducing noisy samples to
the training set. In an extreme case, the artifacts may correlate with the
content; therefore, the recognizer trained on the synthetic data can exploit
these artifacts and will not generalize. The extreme case can be diagnosed by
lower $\mbox{CER}^{\mbox{}}_{\,\mbox{s}{\rightarrow}\mbox{s}}$, but high
$\mbox{CER}^{\mbox{}}_{\,\mbox{s}{\rightarrow}\mbox{r}}$ and
$\mbox{CER}^{\mbox{}}_{\,\mbox{r}{\rightarrow}\mbox{s}}$.

#### Case 2: $\mathcal{S}_{s}\subseteq\mathcal{S}_{t}\ \ \&\ \
\mathcal{S}_{s}\subset\mathcal{S}_{r}$.

Synthetic data is a subset of the true distribution, but fails to capture all
the variations that exist in the real dataset (empirical distribution). The
model trained on synthetic data will fail to generalize well. For example,
when both real and synthetic data share the same corpus but the synthetic data
only contain printed-style handwriting, the recognizer trained on synthetic
data will perform poorly on the real data. Similarly, as we have discussed in
Sec. 3, when we use a high sampling bias to generate data, the synthetic
dataset contains mostly neat and typical handwriting. Thus, the recognizer
will perform poorly on real handwriting, which can be difficult to read
sometimes. This case can be diagnosed by lower
$\mbox{CER}^{\mbox{}}_{\,\mbox{s}{\rightarrow}\mbox{s}}$ and
$\mbox{CER}^{\mbox{}}_{\,\mbox{r}{\rightarrow}\mbox{s}}$, but high
$\mbox{CER}^{\mbox{}}_{\,\mbox{s}{\rightarrow}\mbox{r}}$.

#### Case 3: $\mathcal{S}_{s}\subseteq\mathcal{S}_{t}\ \ \&\ \
\mathcal{S}_{s}\not\subseteq\mathcal{S}_{r}$.

Synthetic data is a subset of the true distribution and contains variations
(_e.g_. missing style and content) that are not covered in the real dataset.
This is the scenario where the recognizer will benefit from the synthetic
data. By filling in missing data, synthetic data improves the performance of
the recognizer leading to lower
$\mbox{CER}^{\mbox{}}_{\,\mbox{s}{\rightarrow}\mbox{r}}$ (and
$\mbox{CER}^{\mbox{}}_{\,\mbox{s}{\rightarrow}\mbox{s}}$) than
$\mbox{CER}^{\mbox{}}_{\,\mbox{r}{\rightarrow}\mbox{r}}$ and
$\mbox{CER}^{\mbox{}}_{\,\mbox{r}{\rightarrow}\mbox{s}}$.

To mitigate the problems discussed in case 1 and case 2, either the generative
model or the sampling process has to be improved accordingly. When the
generative model exactly models the true distribution,
$\mbox{CER}^{\mbox{}}_{\,\mbox{s}{\rightarrow}\mbox{r}}$ and
$\mbox{CER}^{\mbox{}}_{\,\mbox{s}{\rightarrow}\mbox{s}}$ should be identical.
In practice, this is difficult to achieve; however, choosing the generative
model such that $\mbox{CER}^{\mbox{}}_{\,\mbox{s}{\rightarrow}\mbox{r}}$ and
$\mbox{CER}^{\mbox{}}_{\,\mbox{s}{\rightarrow}\mbox{s}}$ are close is a good
strategy to minimize the gap between synthetic and real data. Moreover, in
general, neither case 2 nor case 3 are exact (synthetic and real data contains
different variations of the true distribution); therefore, combining real data
with synthetic data ($\mbox{CER}^{\mbox{}}_{\,\mbox{b}{\rightarrow}\mbox{r}}$)
improves over using synthetic data
($\mbox{CER}^{\mbox{}}_{\,\mbox{s}{\rightarrow}\mbox{r}}$) or real data
($\mbox{CER}^{\mbox{}}_{\,\mbox{r}{\rightarrow}\mbox{r}}$) alone. In the
experiments section, we analyze these cases based on the sampling bias of the
generative model.

##  5 Experiments

We first use our proposed framework to evaluate the effect of style diversity.
As discussed, increasing the sampling bias reduces the style diversity of
synthetic data. Fig. 3 shows the CERs of the English recognizer when varying
the sampling bias and the amount of synthetic training data. To avoid
introducing new content, the data was synthesized using the text corpus of the
original training data. The results indicate that: (1) A large sampling bias
(bias $\geq$ 0.4; _i.e_. neat training data) leads to low
$\mbox{CER}^{\mbox{}}_{\,\mbox{r}{\rightarrow}\mbox{s}}$ and high
$\mbox{CER}^{\mbox{}}_{\,\mbox{s}{\rightarrow}\mbox{r}}$, indicating a
decrease in the diversity of the synthetic data (Fig. 3(a)), corresponding to
Case 2. (2) A small sampling bias (0.05 $\leq$ bias < 0.4) increases the
diversity of the data, covering a large distribution of styles, leads to lower
$\mbox{CER}^{\mbox{}}_{\,\mbox{s}{\rightarrow}\mbox{r}}$ and
$\mbox{CER}^{\mbox{}}_{\,\mbox{b}{\rightarrow}\mbox{r}}$ than
$\mbox{CER}^{\mbox{}}_{\,\mbox{r}{\rightarrow}\mbox{r}}$, corresponding to
Case 3. Due to the increase in data diversity, we also observe an increase in
$\mbox{CER}^{\mbox{}}_{\,\mbox{r}{\rightarrow}\mbox{s}}$ and
$\mbox{CER}^{\mbox{}}_{\,\mbox{s}{\rightarrow}\mbox{s}}$. (3) Further reducing
the sampling bias (bias < 0.05) leads to an increase in
$\mbox{CER}^{\mbox{}}_{\,\mbox{s}{\rightarrow}\mbox{r}}$ and
$\mbox{CER}^{\mbox{}}_{\,\mbox{b}{\rightarrow}\mbox{r}}$ due to the occasional
generation of out-of-distribution samples, corresponding to Case 1. When we
use a very small sampling bias, we sample from a mixture of Gaussian
distributions with large standard deviations. Therefore, the larger standard
deviations make it more likely to generate out-of-distribution samples.

These results demonstrate the effectiveness of data incubation to improve
recognizers and of the proposed recipe to analyze synthetic data and optimize
synthesis. By selecting the right sampling bias, we are able to reduce the CER
of the English recognizer by 28% compared to the model trained only using real
data (Table 1) because we are able to increase the diversity of writing
styles.

Further, for our multilingual recognizer, we show that using additional
content helps improve the CER even further in a scenario where we don’t have
good coverage of all relevant content. When training our system with the same
text corpus, we obtain an improvement of 46% on the normal multilingual test
corpus. However, in this scenario, we only have 60k training samples for each
of the non-English languages (compared to 300k for English). Therefore we
increase the content diversity by synthesizing from a large multilingual text
corpus. This leads to an improvement of 66% (Table 1).

Finally, we also show results on a special patterns (SP) evaluation set
containing content like URLs, emails, addresses, and hashtags which are
underrepresented in the collected training data. As shown in the last column
of Table 1, expanding the synthesis corpus enables our multilingual model to
also achieve 66% reduction in CER compared to the model trained using only the
real data.

Table 1:  CERs on real test data for the English and multi-language setup. We
report which training data was used, the size of the training data, the CER,
and relative improvement. For the multi-language setup, we also report the
results on a special patterns (SP) dataset containing URLs, made-up addresses,
and made-up email addresses.

Test SP Test Setup tr. data $|$tr. data$|$ CER impr. CER impr. English real
0.3 4.9 - n/a n/a synthetic 4.0 4.2 $\downarrow$ 16% n/a n/a both 4.3 3.6
$\downarrow$ 28% n/a n/a Multi-Lang real 0.6 8.2 - 18.5 - synthetic 3.0 5.2
$\downarrow$ 37% 18.9 $\uparrow$ 2% both 3.6 4.5 $\downarrow$ 46% 16.9
$\downarrow$ 9% both (ext) 8.1 2.8 $\downarrow$ 66% 6.3 $\downarrow$ 66%

(a) $\mbox{CER}^{\mbox{}}_{\,\mbox{r}{\rightarrow}\mbox{s}}$ on English

(b) $\mbox{CER}^{\mbox{}}_{\,\mbox{s}{\rightarrow}\mbox{r}}$ on English

(c) $\mbox{CER}^{\mbox{}}_{\,\mbox{s}{\rightarrow}\mbox{s}}$ on English

(d) $\mbox{CER}^{\mbox{}}_{\,\mbox{b}{\rightarrow}\mbox{r}}$ on English

Fig. 3: Effect of sampling bias and synthetic dataset size on English
recognizers.

##  6 Conclusion

We presented data incubation—a new framework to improve recognition by (i)
training a controllable generative model to synthesize missing data, (ii)
optimizing data synthesis, and (iii) training recognition models by
strategically combining synthetic and real data. We applied this framework to
handwriting recognition and achieved a $66\%$ reduction in CER.

## References

  * [1] Mingxing Tan and Quoc Le,  “EfficientNet: Rethinking model scaling for convolutional neural networks,”  in ICML, 2019, pp. 6105–6114.
  * [2] Jen-Hao Rick Chang, Ashish Shrivastava, Hema Swetha Koppula, Xiaoshuai Zhang, and Oncel Tuzel,  “Style equalization: Unsupervised learning of controllable generative sequence models,”  arXiv preprint arXiv:2110.02891, 2021.
  * [3] Larry S. Yaeger, Brandyn J. Webb, and Richard F. Lyon,  “Combining neural networks and context-driven search for on-line, printed handwriting recognition in the newton,”  AI Magazine, 1998.
  * [4] James A. Pittman,  “Handwriting recognition: Tablet PC text input,”  IEEE Computer, vol. 40, no. 9, pp. 49–54, 2007.
  * [5] Daniel Keysers, Thomas Deselaers, Henry A Rowley, Li-Lun Wang, and Victor Carbune,  “Multi-language online handwriting recognition,”  PAMI, vol. 39, no. 6, pp. 1180–1194, 2016.
  * [6] Stefan Jaeger, Stefan Manke, Jürgen Reichert, and Alex Waibel,  “Online handwriting recognition: the NPen++ recognizer,”  IJDAR, vol. 3, no. 3, pp. 169–180, 2001.
  * [7] Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber,  “Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks,”  in ICML, 2006, pp. 369–376.
  * [8] Réjean Plamondon and Sargur N Srihari,  “Online and off-line handwriting recognition: a comprehensive survey,”  PAMI, vol. 22, no. 1, pp. 63–84, 2000.
  * [9] Victor Carbune, Pedro Gonnet, Thomas Deselaers, Henry A Rowley, Alexander Daryin, Marcos Calvo, Li-Lun Wang, Daniel Keysers, Sandro Feuz, and Philippe Gervais,  “Fast multi-language LSTM-based online handwriting recognition,”  IJDAR, vol. 23, no. 2, pp. 89–102, 2020.
  * [10] A Graves, M Liwicko, H Bunke, J Schmidhuber, and S Ferandez,  “Unconstrained on-line handwriting recognition with recurrent neural networks,”  NeurIPS, 2008.
  * [11] Volkmar Frinken and Seiichi Uchida,  “Deep BLSTM neural networks for unconstrained continuous handwritten text recognition,”  in ICDAR, 2015, pp. 911–915.
  * [12] Yexun Zhang, Ya Zhang, and Wenbin Cai,  “Separating style and content for generalized style transfer,”  in CVPR, 2018, pp. 8447–8455.
  * [13] Ankan Kumar Bhunia, Salman Khan, Hisham Cholakkal, Rao Muhammad Anwer, Fahad Shahbaz Khan, and Mubarak Shah,  “Handwriting transformers,”  arXiv preprint arXiv:2104.03964, 2021.
  * [14] Alex Graves,  “Generating sequences with recurrent neural networks,”  arXiv preprint arXiv:1308.0850, 2013.
  * [15] Emre Aksan, Fabrizio Pece, and Otmar Hilliges,  “Deepwriting: Making digital ink editable via deep generative modeling,”  in CHI, 2018, pp. 1–14.
  * [16] Atsunobu Kotani, Stefanie Tellex, and James Tompkin,  “Generating handwriting via decoupled style descriptors,”  in ECCV. Springer, 2020, pp. 764–780.
  * [17] Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang, Alan L Yuille, and Quoc V Le,  “Adversarial examples improve image recognition,”  in CVPR, 2020, pp. 819–828.
  * [18] Hadi Pouransari, Mojan Javaheripi, Vinay Sharma, and Oncel Tuzel,  “Extracurricular learning: Knowledge transfer beyond empirical distribution,”  in CVPR, 2021, pp. 3032–3042.
  * [19] Aleksandr Laptev, Roman Korostik, Aleksey Svischev, Andrei Andrusenko, Ivan Medennikov, and Sergey Rybin,  “You do not need more data: Improving end-to-end speech recognition by text-to-speech data augmentation,”  in CISP-BMEI, 2020, pp. 439–444.
  * [20] Jason Li, Ravi Gadde, Boris Ginsburg, and Vitaly Lavrukhin,  “Training neural speech recognition systems with synthetic speech augmentation,”  arXiv preprint arXiv:1811.00707, 2018.
  * [21] Andrew Rosenberg, Yu Zhang, Bhuvana Ramabhadran, Ye Jia, Pedro Moreno, Yonghui Wu, and Zelin Wu,  “Speech recognition with augmented synthesized speech,”  in ASRU, 2019, pp. 996–1002.
  * [22] Xianrui Zheng, Yulan Liu, Deniz Gunceler, and Daniel Willett,  “Using synthetic audio to improve the recognition of out-of-vocabulary words in end-to-end asr systems,”  in ICASSP, 2021, pp. 5674–5678.
  * [23] Masato Mimura, Sei Ueno, Hirofumi Inaguma, Shinsuke Sakai, and Tatsuya Kawahara,  “Leveraging sequence-to-sequence speech synthesis for enhancing acoustic-to-word speech recognition,”  in SLT, 2018, pp. 477–484.
  * [24] Konstantin Shmelkov, Cordelia Schmid, and Karteek Alahari,  “How good is my GAN?,”  in ECCV, 2018, pp. 213–229.


# First Light Curve Study of the Low Mass Contact Binary V0610 Vir

Ailar Alizadehsabegh Binary Systems of South and North (BSN) Project,
https://bsnp.info/  Department of Photonics, University of Tabriz, Tabriz,
Iran  František Lomoz  Binary Systems of South and North (BSN) Project,
https://bsnp.info/  Variable Star and Exoplanet Section, Czech Astronomical
Society, Prague, Czech Republic  Atila Poro Binary Systems of South and North
(BSN) Project, https://bsnp.info/  Astronomy Department of the Raderon AI
Lab., BC., Burnaby, Canada  Ata Narimani  Binary Systems of South and North
(BSN) Project, https://bsnp.info/

###### Abstract

Photometric data were used to perform the first light curve analysis of the
V0610 Vir binary system. We extracted the minima from our observations and
compiled the literature, which was few in number. Therefore, we performed
computations using the reference ephemeris and presented a new ephemeris and
O-C diagram with a linear fit. Light curve analysis was performed using the
PHOEBE Python code and the Markov chain Monte Carlo (MCMC) approach. The
assumption of a cold spot was required due to the asymmetry in the light
curve’s maxima. The analysis shows that V0610 Vir is a contact binary system
with a fillout factor of 0.085, a mass ratio of 0.998, and an inclination of
$70.65^{\circ}$. The absolute parameters of the system were estimated based on
the Gaia DR3 parallax method. The results show that the system is a Low-Mass
Contact Binary (LMCB) with a total mass lower than $0.8(M_{\odot})$. The
location of the stars was shown in the $M-L$ and $M-R$ diagrams.

techniques: photometric, stars: binaries: eclipsing, stars: individual (V0610
Vir)

##  1 Introduction

The W Ursae Majoris (W UMa) binaries include two stars that are typically F,
G, or K spectral type stars whose Roche lobes have been fielded, and they
share a similar envelope (Kjurkchieva et al. 2016, Zhang, Han, & Liu 2016).
The orbital period of W UMa-type systems is less than one day. Also, the light
curves of these binary stars show two equal or almost equal minima,
demonstrating that their effective surface temperatures are close to each
other (Li et al. 2018). Further investigation of contact systems is important
since it can reveal many details about the evolution of stars.

According to the AAVSO International Variable Star Index
(VSX111https://www.aavso.org/vsx/) and ASAS-SN222http://asas-
sn.osu.edu/variables variable stars’ catalogs, V0619 Virgo is a W UMa-type
binary system with an orbital period of $0.3398754^{days}$ and
$0.3398768^{days}$, respectively. The coordinates of this system in the
Simbad333http://simbad.u-strasbg.fr/simbad/ database are RA: $11^{h}$
$47^{\prime}$ $05.887"$ and Dec: $+01^{\circ}$ $14^{\prime}$ $41.489"$
(J2000), with an apparent magnitude of $V=13.30^{mag}$.

The paper’s structure is as follows: Section 2 explains observation and data
redaction; Section 3 is about extracting minima and obtaining new ephemeris;
and Section 4 is related to light curve analysis. The technique used to
estimate the absolute parameters is described in Section 5, and the conclusion
is in Section 6.

##  2 Observation and Data Reduction

The photometric observations of V0610 Vir were carried out in March 2020 by a
Schmidt-Newton 254mm/1016mm telescope with the G2-8300 CCD camera at a private
observatory in the Czech Republic (49.65 N, 14.41 E). During observations, the
CCD average temperature was $-20^{\circ}C$.

A $V$-band filter was used, and a total of 279 images were obtained. Each
image has an exposure time of 90 seconds. Images were processed using MaxIm DL
software, which included dark, bias, and flat-field for basic data reduction.

Figure 1 displays the comparison and check stars that were selected that were
close to the target and had a suitable apparent magnitude in comparison to
V0610 Vir. So, we considered a comparison star named UCAC4 459-049136 (11 46
55.388, +01 44 51.534) with an apparent magnitude of $V=14.32$ and nine check
stars including UCAC4 457-049442 (11 47 17.678, +01 20 02.553) with a
$V=13.20$ magnitude, UCAC4 458-049431 (11 46 59.259, +01 21 57.295) with a
$V=13.72$ magnitude, UCAC4 457-049428 (11 46 52.883, +01 21 12.000) with a
$V=13.29$ magnitude, UCAC4 457-049456 (11 47 48.705, +01 19 04.569) with a
$V=13.90$ magnitude, UCAC4 457-049455 (11 47 46.001, +01 18 14.696) with a
$V=14.51$ magnitude, UCAC4 457-049448 (11 47 29.787, +01 22 38.372) with a
$V=14.13$ magnitude, UCAC4 458-051054 (11 46 49.701, +01 25 18.116) with a
$V=14.17$ magnitude, UCAC4 458-051065 (11 47 07.913, +01 32 12.092) with a
$V=13.65$ magnitude and UCAC4 458-051064 (11 47 07.423, +01 33 47.439) with a
$V=15.37$ magnitude. The coordinates and apparent magnitudes of all the
comparison and check stars were gathered from the ASAS-SN catalog.

Finally, we used the AstroImageJ program to normalize the flux of all the data
(Collins et al. 2017).

Figure 1: The V0610 Vir binary system, a comparison star, and check stars’
field-of-view.

##  3 New Ephemeris

In binary star systems, the O-C diagram is an important tool for finding new
ephemeris. The O-C value represents the difference between the predicted and
observed times of an eclipse in a binary system. The O-C diagram allows us to
visualize the changes in the timing of eclipses over a longer period of time.
By analyzing the trends of the O-C diagram, we can study various phenomena,
including orbital period variations, eclipse timing variations, and even the
presence of additional companions in the system. For this purpose, we
extracted one primary and one secondary from our observations and collected
them with ten other minima from the literature (Table 1). Also, we converted
all of the times of minima to the Barycentric Julian Date in Barycentric
Dynamical Time ($BJD_{TDB}$)444https://astroutils.astronomy.osu.edu/time/.

To compute Epoch and O-C, we used a reference ephemeris with a time of minima
of 2455291.81179(30) from Diethelm (2010) and an orbital period of
$0.3398768^{days}$ that we obtained from the ASAS-SN catalog. Therefore,
according to the O-C diagram and considering that the number of observations
for this system is limited and few minima are available for it, only a least-
squares linear fit can be considered (Figure 2). Based on this information,
the new ephemeris can be calculated as follows:

$Min.I(BJD_{TDB})=2455291.81304(13)+0.339875947(17)\times E$ (1)

Figure 2: O-C diagram with a linear fit for the V0610 Vir system. Table 1: Available times of minima for V0610 Vir. Min.($BJD_{TDB}$) | Error | Method | Epoch | O-C | Reference
---|---|---|---|---|---
2453499.64872 | 0.00030 | CCD | -5273 | 0.0073 | Diethelm (2010)
2455267.68109 | 0.00030 | $V$ | -71 | 0.0006 | Diethelm (2010)
2455291.81179 | 0.00030 | CCD | 0 | 0 | Diethelm (2010)
2455605.85759 | 0.00070 | $V$ | 924 | -0.0004 | Diethelm (2011)
2455978.87028 | 0.00070 | $V$ | 2021.5 | -0.0025 | Diethelm (2012)
2456038.68868 | 0.00070 | $V$ | 2197.5 | -0.0024 | Diethelm (2012)
2457811.65388 | 0.00010 | CCD | 7414 | -0.0045 | Lehký et al. (2021)
2457840.37499 | 0.00020 | CCD | 7498.5 | -0.0030 | Lehký et al. (2021)
2457840.54279 | 0.00020 | CCD | 7499 | -0.0051 | Lehký et al. (2021)
2458166.48289 | 0.00010 | CCD | 8458 | -0.0069 | Lehký et al. (2021)
2458932.39593 | 0.00047 | $V$ | 10711.5 | -0.0062 | This Study
2458932.56657 | 0.00024 | $V$ | 10712 | -0.0055 | This Study

##  4 Light Curve Solution

The PHOEBE 2.4.9 version and the MCMC method were used to analyze the light
curve of the V0610 Vir system (2020ApJS..250...34C).

We used the $P-T_{1}$ relationship from the Poro et al. (2022a) study to
calculate the effective temperature of the hotter star as the input (Equation
2).

$T_{1}=(6951.42_{\rm-112.68}^{+112.16})P+(3426.01_{\rm-43.90}^{+44.12})$ (2)

The gravity-darkening coefficients was determined $g_{1}=g_{2}=0.32$ (Lucy
1967) and the bolometric albedo was assumed to be $A_{1}=A_{2}=0.5$ (Ruciński
1969). Additionally, the stellar atmosphere was modeled using the Castelli &
Kurucz (2004) method, and the limb darkening coefficients were employed in the
PHOEBE as a free parameter.

Due to the availability of photometric data, we used $q$-search to estimate
the mass ratio. The obtained mass ratio was used as the MCMC process’s initial
parameter value.

The maxima of the light curve were asymmetric $(V_{max}1-V_{max}2)\neq 0$
(Tavakkoli et al. 2015). So, the light curve solution required the use of a
cold starspot on the hotter star (O’Connell 1951).

Then, the theoretical fit was improved using PHOEBE’s optimization tool.
Moreover, taking into account a normal Gaussian distribution in the range of
solutions for inclination, mass ratio, fillout factor, and effective
temperatures, we estimated the values of the parameters together with their
uncertainties using the MCMC approach based on the emcee package in PHOEBE
(Hogg & Foreman-Mackey 2018). We employed 96 walkers and 500 iterations for
each walker in the MCMC processing. Table 2 contains the results of the light
curve solution. The corner plots and final synthetic light curve are shown in
Figure 3 and Figure 4, respectively. The component positions for the four
phases of an orbital period are shown in Figure 5.

Table 2: Photometric solution of V0610 Vir. Parameter |  | Result
---|---|---
$T_{1}$ (K) |  | $5811_{\rm-(5)}^{+(7)}$
$T_{2}$ (K) |  | $5440_{\rm-(9)}^{+(4)}$
$q=M_{2}/M_{1}$ |  | $0.998_{\rm-(9)}^{+(15)}$
$\Omega_{1}=\Omega_{2}$ |  | $3.70(5)$
$i^{\circ}$ |  | $70.65_{\rm-(11)}^{+(12)}$
$f$ |  | $0.085_{\rm-(9)}^{+(9)}$
$l_{1}/l_{tot}$ |  | $0.580(2)$
$l_{2}/l_{tot}$ |  | $0.420(2)$
$r_{1(mean)}$ |  | $0.388(27)$
$r_{2(mean)}$ |  | $0.388(26)$
Phase shift |  | $-0.015(1)$
$Colatitude_{spot}(deg)$ |  | 74(1)
$Longitude_{spot}(deg)$ |  | 348(2)
$Radius_{spot}(deg)$ |  | 27(1)
$T_{spot}/T_{star}$ |  | 1.09(1)
Figure 3: The observed and synthetic light curves of the system in the $V$
filter. Figure 4: The corner plots of the system from the MCMC modeling.
Figure 5: Geometric structure of V0610 Vir with a cold spot on the hotter
component.

##  5 Absolute Parameters Estimation

The estimation of absolute parameters in contact binary stars typically
involves determining quantities such as mass, radius, luminosity, absolute
bolometric magnitude, and surface gravity. So, in order to estimate absolute
parameters, the Gaia DR3 parallax method was used (Poro et al. 2022a).
Therefore, we used $V_{max}=13.33\pm 0.09$ from our observations, the
extinction coefficient $A_{v}=0.063\pm 0.001$ from the 2011ApJ...737..103S
study, and the Gaia DR3 distance $d(pc)=479\pm 3$ for V0610 Vir.

So, we estimated the absolute magnitude $M_{v}$ of the system, subsequently
the absolute magnitude for the components, and the bolometric magnitude
$M_{bol}$ of each star, respectively (Equations 3 to 5).

$M_{v}=V-5log(d)+5-A_{v}$ (3)

$M_{v(1,2)}-M_{v(tot)}=-2.5log(\frac{l_{(1,2)}}{l_{(tot)}})$ (4)

$M_{bol}=M_{v}+BC$ (5)

The hotter and cooler components’ bolometric correction $BC_{1}=-0.073$ and
$BC_{2}=-0.153$ were derived as a function of the effective temperature of the
stars (Flower 1996).

The following equations were used to determine the luminosity and radius, and
separation between the center of mass of the components (6, 7 and 8):

$M_{bol}-M_{bol_{\odot}}=-2.5log(\frac{L}{L_{\odot}})$ (6)

$R=(\frac{L}{4\pi\sigma T^{4}})^{1/2}$ (7)

$a=\frac{R}{r_{mean}}$ (8)

Additionally, using the mass ratio determined by the outcomes of the light
curve analysis, each component’s mass was determined using the well-known
Kepler’s third law (Equation 9). Using Equation 11, the surface gravity was
estimated. The estimated parameters using the Gaia parallax method are shown
in Table 3.

$\frac{a^{3}}{G(M_{1}+M_{2})}=\frac{P^{2}}{4\pi^{2}}$ (9)

$g=G_{\odot}(M/R^{2})$ (10)

Table 3: The absolute parameters of the V0610 Vir binary system. Parameter |  | Hotter star |  | Cooler star
---|---|---|---|---
$M_{v}(mag.)$ |  | 5.457(72) |  | 5.807(70)
$M_{bol}(mag.)$ |  | 5.384(72) |  | 5.654(70)
$L(L_{\odot})$ |  | 0.553(35) |  | 0.431(27)
$R(R_{\odot})$ |  | 0.735(21) |  | 0.741(21)
$M(M_{\odot})$ |  | 0.400(44) |  | 0.399(49)
$log(g)(cgs)$ |  | 4.307(20) |  | 4.300(26)
$a(R_{\odot})$ |  | 1.902(71) |  |

##  6 Summery and Conclusion

We observed the V0610 Vir binary system at an observatory in the Czech
Republic. We extracted our observed minima in addition to collecting from the
literature. Then, we determined the epoch and O-C values using the reference
ephemeris. The O-C diagram shows that just a liner fit can be considered, and
that is descending.

Light curve analysis was performed using the latest available version of
PHOEBE Python code together with the MCMC approach. Moreover, the Gaia
parallax method was used to estimate the absolute parameters of the V0610 Vir
system.

According to the light curve analysis, the companion stars in this system have
a temperature difference of 371 K. In contact systems, the maximum temperature
difference between two stars is around 5%, which is consistent with our light
curve analysis in this regard (Poro et al. 2021). Based on the temperatures of
the stars, G3 and G8 are the spectral types of the hotter and cooler stars in
this system, respectively (Eker et al. 2018).

The evolution of V0610 Vir is depicted by the positions of each component on
the logarithmic scaled Mass-Luminosity ($M-L$) and Mass-Radius ($M-R$)
diagrams (Figure 6a and b). These diagrams show both the Terminal-Age Main
Sequence (TAMS) and the Zero-Age Main Sequence (ZAMS). Due to their very close
masses and radii, their position is next to each other and above TAMS.

The orbital angular momentum of the system is $51.173\pm 0.079$. This result
is based on the following equation from the Eker et al. (2006) study:

$J_{0}=\frac{q}{(1+q)^{2}}\sqrt[3]{\frac{G^{2}}{2\pi}M^{5}P}$ (11)

where $q$ is the mass ratio ($M2/M1$), $M$ is the total mass of the system
($M1+M2$), $P$ is the orbital period with the unit of the day, and $G$ is the
gravitational constant. The $logJ_{0}-logM_{tot}$ diagram (Figure 6c)
considers the V0610 Vir in a contact binary systems region.

According to the short orbital period, light curve solution, and estimation of
the absolute parameters of the V0610 Vir, it can be concluded that this system
is an LMCB contact binary system. The orbital period variation trend in LMCB
systems is usually on a decrease, which is consistent with the V0610 Vir
system. It should be noted that the LMCB systems have formed a disc that has
the potential to be a place for planet formation with an age considerably
shorter than the age of host stars (Stkepień & Gazeas 2012). So, based on the
position of this system’s stars in Figure 6 diagrams and the low mass of the
two stars, we suggest it for future observations and investigations.

Figure 6: a) $logM-logL$ diagram; b) $logM-logR$ diagram; c)
$logJ_{0}-logM_{tot}$ diagram.

## Acknowledgements

This manuscript was prepared by the Binary Systems of South and North (BSN)
project (https://bsnp.info/). We have made use of Gaia DR3 results. The Gaia
mission is from the European Space Agency (ESA) (http://cosmos.esa.int/gaia),
processed by the Gaia Data Processing and Analysis Consortium (DPAC).

## References

  * Castelli & Kurucz (2004) Castelli F., Kurucz R. L., 2004, A&A, 419, 725. doi:10.1051/0004-6361:20040079
  * Collins et al. (2017) Collins K. A., Kielkopf J. F., Stassun K. G., Hessman F. V., 2017, AJ, 153, 77. doi:10.3847/1538-3881/153/2/77
  * Diethelm (2010) Diethelm R., 2010, IBVS, 5945, 1
  * Diethelm (2011) Diethelm R., 2011, IBVS, 5992, 1
  * Diethelm (2012) Diethelm R., 2012, IBVS, 6029, 1
  * Eker et al. (2006) Eker Z., Demircan O., Bilir S., Karataş Y., 2006, MNRAS, 373, 1483. doi:10.1111/j.1365-2966.2006.11073.x
  * Eker et al. (2018) Eker Z., Bakış V., Bilir S., Soydugan F., Steer I., Soydugan E., Bakış H., et al., 2018, MNRAS, 479, 5491. doi:10.1093/mnras/sty1834
  * Flower (1996) Flower P. J., 1996, ApJ, 469, 355. doi:10.1086/177785
  * Hogg & Foreman-Mackey (2018) Hogg D. W., Foreman-Mackey D., 2018, ApJS, 236, 11. doi:10.3847/1538-4365/aab76e
  * Kjurkchieva et al. (2016) Kjurkchieva D. P., Popov V. A., Vasileva D. L., Petrov N. I., 2016, RAA, 16, 135. doi:10.1088/1674-4527/16/9/135
  * Lehký et al. (2021) Lehký M., Šmelcer L., Souza de Joode M., Jílek F., Mašek M., Urbaník M., Walter F., et al., 2021, OEJV, 211, 1. doi:10.5817/OEJV2021-0211
  * Li et al. (2018) Li K., Xia Q.-Q., Hu S.-M., Guo D.-F., Chen X., 2018, PASP, 130, 074201. doi:10.1088/1538-3873/aac067
  * Lucy (1967) Lucy L. B., 1967, ZA, 65, 89
  * O’Connell (1951) O’Connell D. J. K., 1951, PRCO, 2, 85
  * Poro et al. (2021) Poro A., Alicavus F., Fernández-Lajús E., et al., 2021, RAA, 21, 203. doi:10.1088/1674-4527/21/8/203
  * Poro et al. (2022a) Poro A., Sarabi S., Zamanpour S., et al., 2022, MNRAS, 510, 5315. doi:10.1093/mnras/stab3775
  * Ruciński (1969) Ruciński S. M., 1969, AcA, 19, 245
  * Stkepień & Gazeas (2012) Stkepień K., Gazeas K., 2012, AcA, 62, 153. doi:10.48550/arXiv.1207.3929
  * Tavakkoli et al. (2015) Tavakkoli F., Hasanzadeh A., Poro A., 2015, NewA, 37, 64. doi:10.1016/j.newast.2014.12.004
  * Zhang, Han, & Liu (2016) Zhang Y., Han Q. W., Liu J. Z., 2016, PASP, 128, 124201. doi:10.1088/1538-3873/128/970/124201


# Shape-based event pileup separation in Troitsk nu-mass experiment

Vasily Chernov  Institute for Nuclear Research RAS, 117312 Moscow, Russia
Alexander Nozik  Institute for Nuclear Research RAS, 117312 Moscow, Russia
Moscow Institute of Physics and Technology, Dolgoprudny, Moscow region 141700,
Russia

###### Abstract

Nowadays continuous signal digitization becomes a standard procedure in
experimental physics. Though, signal pileup separation at high count rate
remains a problem. The article presents algorithms for detecting and
extracting events based on shape of a single pulse. An example of applied use
of the described algorithms is also given on the example of the Troitsk nu-
mass experiment.

##  1 Introduction

One of the major problem at the Troitsk nu-mass experiment in search for
sterile neutrino with mass up to 5 keV ([1]) is the high count rate on the
main detector. According to proposal, it should handle up to 40-50 kHz count
rate with good discrimination of pile-up and very precise dead time estimation
(down to 100 ns precision or even lower). With signal shape width of about 4
$\mu s$, it is hard to implement hardware signal separation. So it was decided
to use continuous digitization and perform the signal separation in off-line
mode. The current setup includes very cheap RudShel Lan10-12PCI ADC board [2],
which have maximum precision of 20 ns. Due to specifics of board oscilloscope
mode it appeared that the single event mode could not be used on high count
rates. Instead, we acquired continuous fixed length blocks, performed fast
zero suppression in on-line mode and left everything else for off-line
analysis. This acquisition mode allows to partially bypass board hardware
limitations and eliminates hardware dead time, but requires a software to
perform off-line pulse separation.

There are several ways to detect event pileup. For example, restoring the
shape of a spectrum using a neural network. The algorithm described in [3]
uses modeled spectra as a training sample and requires knowledge of the form
of the original, undistorted spectrum. Another way is described in [4]. It is
based on analytical model that predicts dead time and spectrum distortions
caused by signal pileups. Despite good results, the model is rigidly tied to
the analytical pulse formula and therefore is not applicable in other systems
without significant modification.

Our proposed correction algorithm is also based single event shape. However,
instead of using information received from outside, such as shape of the
spectrum or the analytical signal formula, it relies only on the continuous
signal data collected from the ADC, and extracts all the necessary parameters
from the data itself. This makes the algorithm applicable for various signal
digitization problems without significant changes.

##  2 Hardware

Lan10-12PCI has a memory buffer of 512 kBit and can hold only one frame. The
restriction in the simultaneous storage of only one frame in the internal
memory makes it necessary to dump board memory to PC after each frame. With
Lan10-12PCI constant dump time of $55ms$ it leads to impermissible dead time
for trigger-based acquisition mode. However, board allows to acquire frames by
the program trigger which, with the maximum allowable frame size of 1048576
bins and the bin size equal $320ns$, allows to receive continuous signal data
with constant 87% live time. Considering the signal rising time of 2 $\mu$,
the mode is suitable for Troitsk nu-mass setup. Better hardware could produce
better sampling rate and live time.

###  2.1 Zero-suppression

To save hard disk space, continuous signal data frames are preprocess by the
zero-suppression algorithm. The algorithm truncates bins which have an
amplitude below the threshold and don’t have bins above the threshold in a
some neighborhood. Thus, a continuous signal frame is divided into variable-
length subframes, with one or more events contained each.

The size of the subframes should be chosen based on the event after-pulse
decay rate - it must be greater than length of the maximal possible after-
pulse in dataset. With this restriction, algorithm will safely separate frame
to independent subframes, since the impact of the closest event to the right
of the subframe will be obviously less than the minimum size of truncated
area.

Figure 1: Zero-suppression algorithm work example. Highlighted areas
correspond to algorithm output subframes. In this case they contain 1, 2 and 1
events respectively. The rest of the continuous signal frame is truncated.

The zero-suppression operation algorithm:

  1. 1.

Create frame binary mask

    1. (a)

Creation of threshold mask

    2. (b)

The application of morphological dilation with a core corresponding to the
desired boundaries of the event

  2. 2.

Extract subframes passed by mask

This implementation avoids duplication of events boundary areas due to merging
of close events into one subframe. Further in the work, for simplicity, we
will refer to the subframe, resulting from the zero-suppression preprocess as
a simple ”frame” and ”continuous signal frame” - for Lan10-12PCI input frame
before preprocess.

##  3 Continuous signal simulation

To develop and test extracting algorithms, a large labeled continuous signal
data sample is required. The only way to obtain it is simulation. During the
work, we developed Lan10-12PCI board data generator, that produces continuous
signal frames simulating real device output. To create each continuous signal
frame it uses set of event defining parameters such as it amplitude and time.
Since the parameters are defined explicitly the output becomes labeled
automatically.

Generator output is developed to be fully compatible with real board raw
output. This choice has its advantages and disadvantages. On the one hand raw
board output is the primary input for the entire processing system, the use of
such generated marked data allows to test all processing steps. In addition,
there is no need to develop excess testing only code, because all used data
manipulations will be required for working with real board. Another advantage
is the ability of developing without the board itself, since the generator can
also be used as a virtual device. The disadvantage is the need to perform all
dependent steps before testing a specific part of the processing. This greatly
reduces testing speed, especially for the final processing steps, compared to
the case of generating data for specific step. Also, a bottleneck could
appears in case when some intermediate steps works much longer than the tested
one. Weighing all the pros and cons the method of generating raw board input
was chosen as the optimal in terms of development costs.
Architecturally, the data generator consists of two modules:

  * •

Noise generator;

  * •

Clean event shape generator.

Continuous signal is obtained by overlaying clean event shapes with noise
background.

###  3.1 Noise simulation

We assume that noise consists of a large number of low amplitude events
uniformly distributed in time and having the same shape as the shape of useful
event. With this condition, the noise can be generated by explicit
superposition of the shapes created by clean event shape generator without
additional modules. However, to speed up simulation, it is possible to
optimize noise generation.

Taking into account the high frequency and peaks distribution of noise events,
it is possible to generate superimposed noise events final shape with method
based on applying a composition of smoothing operations on a random numbers
array.
Noise generation algorithm then works as follows:

  1. 1.

Create random numbers array with uniform distribution. Array size is equal to
output frame size.

  2. 2.

Convolve array with a large constant core to achieve average number of peaks
similar to real noise. Coarse tune amplitude distribution.

  3. 3.

Convolve array twice with small constant cores to achieve necessary blurring
of small peaks. Fine tune amplitude distribution.

  4. 4.

Round array values to integer and shift them by 2 bits to make them be similar
to 12 bit bin values acquired by board.

The method allows to generate superimposed shape of large number of uniformly
and densely distributed over time events at low costs. Unlike an events
superposition method, result won’t be labeled but it is acceptable for noise.

Comparison of parameters of real and simulated amplitude spectra was used for
noise generator validation. To obtain a real noise spectrum the frame slices
from left border to frame first event beginning was used. Due to zero-
suppression applied to data and while fulfilling the constraint from the
procedure explained in section 2.1 they will be the only source of undistorted
real noise data.

Convolution kernel sizes were tuned manually. During the tuning we watched
coincidence between real and generated noise for:

  * •

Correspondence of amplitudes spectrum of real and generated noise.

  * •

Visual similarity of the shapes of noise peaks and average distances between
neighboring peaks.

Figure 2: Noise spectrum comparison simulated vs real.

The spectra peaks at Fig. 2 show that the baseline for Troitsk setup
fluctuates slightly between series of measurements. Testing was conducted on
later sets of data and mean for noise has shifted by 12 channels. However, the
statistical parameters of the noise are close to each other.

Now we know the mean and the standard deviation for the distribution of the
noise amplitudes, which we need when analyzing the form of events.

###  3.2 Event shape modeling

To model an event, one need to know its clean form without noise. Forms of
events can be extracted from the data by grouping events by amplitude value
and then averaging the shapes by group. The frame size after trimming also
gives a preliminary condition which can filter out frames with overlapped
events.

Figure 3: Averaged event shapes for groups of amplitudes. Horizontal axis bin
size is 320 ns.

For the Troitsk nu-mass dataset averaged shapes shown on figure 3. The
following shape properties can be concluded from the figure:

  1. 1.

The peak of the event can be approximated by a Gaussian.

  2. 2.

Event after-pulse decays exponentially.

  3. 3.

The ratio between peak and after-pulse amplitudes is preserved.

  4. 4.

The distance between the positions of the peak maximum and the after-pulse
minimum is preserved.

To describe the obtained forms of real signals, an analytical formula has been
found:

$\begin{split}t_{f}=2.9604,p=2.2056,t_{a}=0.3701,\sigma=0.3416,f_{b}=3.125e+6,\\\
g(x)=exp(\frac{|\sigma
f_{b}log(x)|^{p}}{2}),g_{r}(y)=\frac{-2log(y)^{1/p}}{\sigma f_{b}},\\\
s(y)=\begin{cases}(((1+2xf_{b}s)^{t}_{f}-1)exp(-xf_{b}s))^{-1},x>0\\\ 0,x\leq
0\end{cases},\\\
signal(x,amp,pos)=g(x-pos)+s(x-g_{r}(0.1)-pos)t_{a}amp.\end{split}$ (1)

where:
$g(x)$ \- Gaussian function, $g_{r}(y)$ \- reverse Gaussian function, $s(x)$
\- after-pulse function, $signal(x,amp,pos)$ \- signal function.

The signal front is completely coincides with the Gaussian function. After-
pulse function starts at point where Gaussian amplitude reach value equal to
10% of the maximum. Parameters were chosen by fitting.

The analytic signal formula is used to increase the performance, however,
separation algorithm doesn’t require it - event shapes given numerically are
sufficient.

Fig. 4 show comparison of generated shapes with shapes obtained by averaging
real events. In this comparison, the real averaged pulse maximum and its
position was used as amplitude and time of corresponding generated event.

Figure 4: Real averaged vs generated signal shapes.

##  4 Events extraction algorithms

Several selection algorithms have been investigated and tested. Each
algorithm, has been tested on generated file with duration of 300 seconds and
a count rate of 40 kHz.

To define the correspondence between the original events and the events
obtained as a result of the algorithm operation, the following rules were
used:

  * •

Event corresponds to the original event if:

    * –

original event is the closest in time,

    * –

the distance between the original and the extracted event is in range of 3200
ns (10 time bins).

  * •

Original event is considered to be unrecognized there is no extracted event
within the range of 3200 ns.

  * •

Extracted event is considered to be false positive if there is no one original
event within the range of 3200 ns.

  * •

Extracted event is considered to be pileup if two or more original events
correspond to it.

The following metrics were used for testing:

  * •

Algorithm running time;

  * •

Number of false positives;

  * •

Number of unrecognized original events

The efficiency of the algorithms is compared by effective dead time.

###  4.1 Method 1. Simple

The first method is the simplest, and consists in isolating local extrema
above the threshold. To separate close events, the algorithm requires a clear
bend between the peaks. This method is the fastest, however, due to its
simplicity, it require events to be far from each other to successfully split
them. Also, the algorithm does not correct the amplitudes of subsequent events
by the tail value of previous event.

Figure 5: Method 1 testing results. The Upper left graph shows the extraction
results on single example frame. Top right graph contains amplitude histograms
for original, reconstructed, unrecognized, and pileup events. Bottom left and
bottom right graphs shows amplitude and time error distributions for
reconstructed events respectively.

The graphical results of tests are shown at Fig. 5. The algorithm did not
recognize the second and third events of the test sample: the second - because
of the lack of inflection between the peaks, the third - due to the imposition
of the event on the minimum of the tail of previous event. The position of the
fourth event has been successfully recognized, but it’s amplitude has an error
caused by overlapping previous events after-pulses. The contribution to errors
caused by the lack of amplitude correction can also be seen on the amplitude
errors histogram.

###  4.2 Method 2. Approximate After-pulses

The second method allows, with a relatively small increase in complexity
(about 30% ), to get rid of some of the disadvantages of the first method. It
works as follows:

  1. 1.

Locate frame local maxima above the threshold.

  2. 2.

Process found peaks as a sequence:

    1. (a)

Preserve amplitude and position of the current peak;

    2. (b)

Generate shape by amplitude and position;

    3. (c)

Subtract generated shape from frame (meaning all subsequent events);

    4. (d)

Go to the next peak;

Also, as in the first method, the detection of events depends on the bend of
the signal above the threshold. For optimization purposes, peaks locating
operation runs only once at the beginning of the frame processing. However,
this method corrects the amplitudes of events that have arrived at the tail of
previous events.

Figure 6: Method 2 testing results.

Fig. 6 shows that the algorithm also failed to recognize the second and third
events due to the overlapping with after-pulses. The fourth event amplitude is
now adjusted by earlier events after-pulse, but also has an error caused by
overlapping the tails of the second and third events which are not recognized.
The amplitude errors histogram shows that the method adjusts amplitudes of
reconstructed events affected by after-pulses.

###  4.3 Method 3. Front fit

The third method has two major differences. First one concerns peaks detection
- in this algorithm the peak search is repeated every time the event shape is
subtracted from the frame. After extracting the next event algorithm takes the
left-most one from re-searched peaks to process. Second difference is change
of peak position estimation - the position is fitted from several points in
the front of the event. The fitting range is chosen in resect to events
arrival time: the front of the event is not distorted by the subsequent events
and since each time we take left-most event - it will be adjusted by after-
pulses of earlier processed events from the frame. This approach allows to:

  * •

Resolve events that do not have a separation in the form of a bending.

  * •

Extract events that initially have an amplitude below the threshold

Figure 7: Method 3 testing results.

Testing results are shown at Fig. 7. The algorithm successfully resolves and
adjusts amplitudes for all events. Errors in extracted peaks could be caused
by noise and cumulative errors in recognition of previous events.

Also, it could be seen that peaks time extraction is less accurate than in
first and second methods. This can be caused by a greater sensitivity to noise
due to fitting by the front.

This method generates much greater number of false positives (2% vs. 0.005% in
first and second). However the most of false positive events are located near
the lower border of the amplitude spectrum and have amplitude values close to
noise. This makes it possible to almost completely filter them out without
large data loss by increasing threshold.

###  4.4 Comparison of the methods

The table 1 shows testing results of all three algorithms on the same
generated file. The acquisition time is 300 s, the counting rate is 40 kHz. In
this table, the ratios of false and pileup events are counted from the total
number of actual recognized events. The effective dead time is defined as a
dead time that causes the same amount of losses.

|  Simple |  Approximate After-pulses |  Front fit
---|---|---|---
Work time, $\%$ of acquisition time |  61.1 |  158.8 |  9771.5
Extracted, % |  89.65 |  89.65 |  98.34
Not extracted, % |  8.96 |  8.9 |  3.48
False positives,% |  0.01 |  0.01 |  2.37
Effective dead time, $\mu s$ |  3.195 |  3.167 |  0.632
Table 1: Methods testing summary.

The first and second methods give the same quantitative results and have the
same effective dead time of about 3 $\mu$s. The third method recognizes a lot
more events and has 5 times less dead time compared to the first and second,
but also has a much greater number of false positives and works 2 orders of
magnitude slower.

###  4.5 Discussion

Of the three algorithms considered, ”Front Fit” (number 3) shows the best dead
time - 5 times less than ”Simple” and ”Approximate After-pulses”. However, the
nature of the frequent number of false positives remains in question. One of
the reasons may be worse stability of the peak detection due to fitting on the
front of the event. If the position will be extracted with error of 1-2 bin
which is comparable to the size at which the after-pulse varies noticeably,
the algorithm can increase next events amplitude error. In this case, it can
be assumed that false events may have a different signal shape, which could be
possibly classified and filtered out by neural network.

Another cause of false positives may be inadequate values of the fitted event
parameters, which can also increase errors noise due to the after-pulse
shifting. In addition, the current framework used for fitting is rather slow
(it could be significantly optimized in future).

In the future, it is planned to analyze causes of false positives and improve
the operation of the algorithm in both directions.

###  4.6 Conclusions

In this work, we considered ways to extract events from a continuous signal,
taken from an ADC. We also considered an algorithm for creating a signal
generator that simulates the operation of an ADC. The described extraction
algorithms does not require additional knowledge about events and the spectrum
and relies only on the acquired data. It makes possible to use them without
significant changes for various signal digitization problems.

On the Lan10-12PCI board used, we could achieve a dead time of 0.6 $\mu$s,
with a bin size of 320 ns.

The described algorithms were used to analyze data at the Troitsk nu-mass
experiment on 2017_05 and 2017_11 data acquisition runs.

The implementation of the algorithms described in the article can be found in
the repository on github https://github.com/kapot65/signal-utils.

The work was supported with RFBR grant “17-02-00361 A”. We would like to thank
Vladislav Pantuev for manuscript revision and constructive criticism of the
work.

## References

  * [1] D. N. Abdurashitov, A. I. Belesev, A. I. Berlev, V. G. Chernov, E. V. Geraskin, A. A. Golubev, G. A. Koroteev, N. A. Likhovid, A. A. Lokhov, A. I. Markin, A. A. Nozik, V. I. Parfenov, A. K. Skasyrskaya, V. S. Pantuev, N. A. Titov, I. I. Tkachev, F. V. Tkachov, and S. V. Zadorozhny.  The current status of “Troitsk nu-mass” experiment in search for sterile neutrino.  Journal of Instrumentation, 10:T10005, October 2015.
  * [2] Rudnev-Shilyaev.  LAN10-12PCI/LAN10-12PCI-U board details.  https://translate.google.com/translate?js=n&sl=ru&tl=en&u=www.rudshel.ru/show.php?dev=13, 2014\.  [Online; accessed 20-December-2018].
  * [3] P. T. Komiske, E. M. Metodiev, B. Nachman, and M. D. Schwartz.  Pileup Mitigation with Machine Learning (PUMML).  Journal of High Energy Physics, 12:51, December 2017.
  * [4] V. Chaplin, N. Bhat, M. S. Briggs, and V. Connaughton.  Analytical modeling of pulse-pileup distortion using the true pulse shape; applications to Fermi-GBM.  Nuclear Instruments and Methods in Physics Research A, 717:21–36, July 2013.