11institutetext: University of Modena and Reggio Emilia, Modena, Italy
11email<EMAIL_ADDRESS>
# Binarizing Documents
by Leveraging both Space and Frequency

Fabio Quattrini  0009-0004-3244-6186 Vittorio Pippi  0009-0001-7365-6348
Silvia Cascianelli  0000-0001-7885-6050 Rita Cucchiara  0000-0002-2239-283X

###### Abstract

Document Image Binarization is a well-known problem in Document Analysis and
Computer Vision, although it is far from being solved. One of the main
challenges of this task is that documents generally exhibit degradations and
acquisition artifacts that can greatly vary throughout the page. Nonetheless,
even when dealing with a local patch of the document, taking into account the
overall appearance of a wide portion of the page can ease the prediction by
enriching it with semantic information on the ink and background conditions.
In this respect, approaches able to model both local and global information
have been proven suitable for this task. In particular, recent applications of
Vision Transformer (ViT)-based models, able to model short and long-range
dependencies via the attention mechanism, have demonstrated their superiority
over standard Convolution-based models, which instead struggle to model global
dependencies. In this work, we propose an alternative solution based on the
recently introduced Fast Fourier Convolutions, which overcomes the limitation
of standard convolutions in modeling global information while requiring fewer
parameters than ViTs. We validate the effectiveness of our approach via
extensive experimental analysis considering different types of degradations.

###### Keywords:

Document Enhancement Document Image Binarization Fast Fourier Convolution.

##  1 Introduction

Documents with good visual quality are easier to handle when performing
Document Analysis tasks, which is why the problems and approaches to Document
Enhancement have long been explored. However, this remains a challenging task,
especially for old and highly damaged documents, for which automatically
achieving good visual quality can benefit efficient content understanding. Due
to preservation conditions and time, these documents exhibit various types of
degradations, such as bleed-through ink, ink oxidation, faded ink, smears,
scratches, stains, and creases in the paper. Moreover, the digitalization
process can introduce artifacts due to image compression and unideal
acquisition conditions such as uneven illumination, contrast variation, and
blur [68]. A typical Document Enhancement task is Document Binarization, which
consists in determining whether each pixel in the document image pertains to
ink or to the background. This is a critical step both for facilitating
downstream Document Analysis tasks [63, 53, 9, 11, 24, 59, 75, 27, 43, 42] and
for enhancing the readability of the document by the end-users of digital
archives and libraries. One of the main challenges in Document Binarization is
the variability with which the aforementioned degradations appear within the
page. For this reason, early-proposed image processing approaches based on
estimating and applying a global threshold [41, 5] have been surpassed by
methods working on local document patches, on which an adaptive threshold is
calculated and applied. Learning-based solutions, generally based on
Convolutional Neural Networks, enhanced the performance by refining the
prediction on the single pixel based on its neighbors. Nonetheless, a
potential drawback of working on document patches is that, in case they
contain too different degradations, some artifacts can appear at their borders
when restitching them together to form the whole document image. For this
reason, considering both global and local information has been proven
beneficial by the promising performance of hybrid thresholding methods [70],
Image-to-Image translation approaches based on Generative Adversarial Networks
(GANs) [78, 61, 66], and multi-scale Convolutional approaches for Segmentation
[67, 69]. Note that global dependencies are difficult to handle with standard
Convolution-based models due to the fixed size and locality of their receptive
field. A strategy to integrate global information is to exploit attention
mechanisms over the pixels. This strategy has been successfully applied also
for Document Binarization [59, 60, 73]. However, this strategy requires a
large amount of training data, which is difficult to collect for document
images, and usually leads to large models, thus being inefficient.

To overcome these limitations, we devise an efficient, fully-convolutional
model able to integrate global information for Document Binarization by
exploiting the Fast Fourier Convolution (FFC) [8]. This operator combines
spatial and spectral information to expand the receptive field in the
frequency domain. The benefits of FFCs have been exploited for Inpainting
[65], Detection [52], and Super-Resolution [58] but, to the best of our
knowledge, FFCs have not been employed for Document Binarization. However, we
argue that their capability to encode both local and global information and
model pseudo-periodic patterns and textures is particularly suitable for
documents. In fact, text lines have a specific pseudo-periodic pattern, and
paper has a very peculiar texture, different from that of artifacts and
degradations. Moreover, due to their image-wide receptive field in the
spectral domain, FFCs are more robust to differences in resolution and scale
between training and test images compared to standard convolutions [65]. Thus,
we propose to train our model with $256{\times}256$ patches, as customary in
Document Binarization, but use larger patches at inference time. This allows
obtaining more training samples from the same annotated document and
seamlessly exploiting more global information at inference time for better
binarizing the patch. We demonstrate the effectiveness of our proposed
approach and its superiority with respect to both standard Convolution-based
and Transformer-based approaches via extensive experimental analysis on
multiple benchmark datasets, whose contained documents exhibit a number of
different degradations. We release code and checkpoints at
https://github.com/aimagelab/FourBi.

##  2 Related Work

Heuristics-based Approaches. Among the strategies that exploit a handcrafted
model of the image appearance, the most commonly applied is thresholding,
which consists in determining a global threshold [41, 5] or a local adaptive
threshold to deal with the non-uniform background typical of document images
[57]. Note that hybrid thresholding algorithms exist, such as Wolf [70], which
use both local and global statistics. Other binarization paradigms are based
on energy minimization [71, 72] or on edge detection for modeling strokes [10,
75].

Learning-based Approaches. Most learning-based approaches to Document
Binarizationment exploit Convolutional Neural Networks. These are employed as
denoising autoencoders [77, 21] or as encoder-decoders to perform Image
Processing [26, 13] or Segmentation [67, 69]. In this work, we treat Document
Binarization as a Segmentation task and devise a U-Net-like model [55] as
convolutional architecture, but with the novelty of featuring FFC layers.
Another successful research line entails treating the Document
Binarizationment task as a generative one by applying Conditional GANs to
obtain a binarized image starting from a degraded one in a style-transfer
fashion [78, 61, 66, 62]. In the context of Document Image Binarization, GANs
are also employed to alleviate the training data scarcity issue connoting the
task. In particular, these models are used for realistic training data
augmentation [6] or to achieve self-supervision via unpaired Image-to-Image
Translation with Cycle-GANs [80]. Recent preliminary approaches employ
Diffusion Models [20, 74, 19] in a similar way. To exploit long-range
dependencies in combination with local information, which is difficult to
achieve with standard convolutions, recent approaches [59, 60, 73] resort to
Vision Transformer (ViT)-based architectures [14]. The encouraging performance
of such methods suggests that combining global and local information is a
promising strategy for Document Binarization. In this work, we explore this
strategy by proposing to apply FFCs, which, contrary to standard convolutions,
can model both global and local dependencies and are less computationally
demanding than ViTs.

Regardless of the technique applied, the performance can be enhanced with some
post-processing steps. Typically, these consist in applying morphological
operators (_e.g._ , Connected Component Analysis, Erosion, Dilation, Shrink,
and Swell filtering) [33, 35] or ensembling the output of multiple
binarization algorithms [23, 36, 2]. In this work, we combine the output of
our model run on overlapping patches of the document of interest to reduce the
error on the borders, where artifacts due to discontinuity are most evident.

Frequency Information for Document Image Binarizationment. Previous approaches
[38, 1, 29] have successfully leveraged the spectral information of documents
images to address the difficulty of recognizing and recovering structures such
as edges, strokes, and corners. In recent works, the Discrete Wavelet
Transform [34], successfully applied to Super-Resolution [79], has also been
used for Document Binarization to represent the document image in different
frequency sub-bands either to perform Segmentation [1] or to obtain a ground
truth for a generator in an adversarial training setting [29]. The main
component of our architecture is the FFC, which exploits the Fast Fourier
Transform (FFT). Moreover, unlike previous works, we do not use spectral
information to represent the images but as an operator inside the
architecture.

##  3 Proposed Approach

Figure 1: Overview of our Document Binarization approach exploiting FFCs
(FourBi). The architecture consists in a U-Net-like model in which the outputs
of the down-scaling and the up-scaling blocks are combined via concatenation.
The central part of the model features FFC Residual Blocks (FFC-RB). The FFC
operator in these blocks combines local spatial information with global
frequency information, which results in semantic-preserving precise
binarization.

In this work, we treat the Document Binarization task as a two-class
segmentation problem, with a high imbalance towards the background class and
high intra-class variability. To tackle the task, we devise a novel approach
consisting of a U-Net-like model featuring FFC layers, which we call _FourBi_
(see Fig. 1).

###  3.1 FourBi Architecture

The proposed model takes as input square image patches, which are down-scaled
via three Down-scale Blocks. Each Down-scale Block contains three sub-blocks,
consisting of a Convolutional layer (Conv), a Batch Normalization (BN) layer,
and a Rectified Linear Unit activation function (ReLU). The output of the last
Down-scale Block is fed to a stack of FFC Residual Blocks (FFC-RB), which
apply the FFC operator described below. The feature map from the last FFC-RB
is then up-scaled via three Up-scale Blocks. Each of these blocks takes as
input the output of the preceding Up-scale Block concatenated with the output
of the corresponding Down-scale Block. The so-obtained tensor is fed to a
Transpose Convolution layer (TConv), followed by a BN layer, a ReLU, and two
other Conv-BN-ReLU blocks. The predicted tensor contains continuous values
ranging from 0 to 1, representing the probability of each pixel being
background or ink. From this, we obtain a binarized image by applying a
threshold set equal to 0.5.

Fast Fourier Convolution for Document Binarization. To obtain a high-quality
binarized version of a document, we need the global context of the whole image
in order to better model the patterns connoting it. In fact, the paper
texture, the ink intensity, the color artifacts of the image scan, and the
text orientation are aspects that change from document to document but are
roughly similar in different parts of the same document page. This must be
taken into account to distinguish the ink pixels from the background ones.
However, working with document images also requires precision when binarizing
fine parts like strokes and characters in order not to change the semantic
content of the text in the document. In light of this, the global context has
to be mixed with local information to classify the single pixel correctly.
Therefore, we propose to resort to the FFC operator proposed in [8], which
divides the network stream into a global and a local branch. The global branch
exploits the Fast Fourier Transform (FFT) to enlarge the receptive field,
while the local branch handles finer local patterns. In particular, the input
tensor to the FFC layer is split into two chunks along the channel dimension,
one for the local branch and one for the global branch. This way, the two
branches can focus on different aspects, and different information can be
stored in different parts of the tensor. The local branch consists of two
Convolutional layers, while the global branch contains a Convolutional layer
and a Spectral Transform (ST) component. The ST block exploits the two-
dimensional Real FFT operator (Real FFT2d) and applies a convolution in the
frequency domain. Note that the Real FFT is generally faster than the
classical FFT for real-valued signals because it takes advantage of the
Hermitian symmetry property. Specifically, the block works as follows. Given
an input tensor $\mathbf{X}$ with shape $[H{\times}W{\times}C]$, it is applied
the Real FFT2d:

$\text{Real~{}FFT2d}:\mathbb{R}^{H{\times}W{\times}C}{\rightarrow}\mathbb{C}^{H{\times}\frac{W}{2}{\times}C},$

to obtain a complex-valued tensor $\mathbf{Z}$, _i.e._ , a tensor containing
tuples that represent the real part $\mathbf{R}$ and the imaginary part
$\mathbf{I}$ of the input:

$\text{Real~{}FFT2d}(\mathbf{X})=\mathbf{Z}=\mathbf{R}+i\mathbf{I}.$

Then, the values in these tuples are stacked to form a real-valued tensor,
_i.e._ ,
$\mathbb{C}^{H{\times}\frac{W}{2}{\times}C}{\rightarrow}\mathbb{R}^{H{\times}\frac{W}{2}{\times}2C}$.
This tensor is fed to a Convolutional block featuring a Convolutional layer
with kernel size $1{\times}1$:

$\text{Conv}_{1{\times}1}\circ\text{BN}\circ\text{ReLU}:\mathbb{R}^{H{\times}\frac{W}{2}{\times}2C}{\rightarrow}\mathbb{R}^{H{\times}\frac{W}{2}{\times}2C}.$

Finally, the output of this block is reshaped again into a complex-valued
tensor, _i.e._ ,
$\mathbb{R}^{H{\times}\frac{W}{2}{\times}2C}{\rightarrow}\mathbb{C}^{H{\times}\frac{W}{2}{\times}C}$
and fed to a two-dimensional Inverse Real FFT operator (Inv Real FFT2d):

$\text{Inv~{}Real~{}FFT2d}:\mathbb{C}^{H{\times}\frac{W}{2}{\times}C}{\rightarrow}\mathbb{R}^{H{\times}W{\times}C}.$

The output of the Convolutional layer in the global branch is summed with the
output of one of the convolutions in the local branch, and the output of the
ST is summed to the output of the other. The resulting tensors are fed to
separate BN-ReLU blocks and then stacked to obtain the output of the FFC
block. Thanks to the combination of local spatial information and global
frequency information, the FFC allows handling and distinguishing global
pseudo-periodic patterns, such as those typical of the paper support and of
the text lines, without disregarding local patterns that need to be taken into
account when binarizing detailed parts of the image such as the strokes.

###  3.2 Training and Inference

Since obtaining the ground truth information for Document Binarizationment is
costly and time-consuming, publicly available datasets for the task usually
contain only a few annotated pages. Therefore, as customary for this task, we
enlarge the training set by training on 256${\times}$256 image patches. We
train our model with the Charbonnier loss [7], which approximates the Mean
Square Error (MSE) loss when the prediction error is smaller than the
hyperparameter $\epsilon$ and the Mean Absolute Error (MAE) loss when the
error is bigger [4]. Note that both the MSE and MAE loss are usually employed
in Document Binarizationment [77, 78, 6, 13, 59, 75]. For this reason, we
adopt this loss that has been found more beneficial than these two in tasks
such as image Super-Resolution [28]. The loss is:

$\mathcal{L}=\sqrt{(I-I^{\prime})^{2}+\epsilon^{2}},$

where $I$ and $I^{\prime}$ are the ground truth and the predicted image,
respectively, and the hyperparameter $\epsilon$ is set equal to 10-6.

At inference time, we exploit the robustness of the multi-scale
representations learned by FFCs when applied to test images larger than the
training ones and feed the model with 512$\times$512 patches from the test
images. The increased global context helps the model distinguish between
different degradation types and ink.

###  3.3 Post-Processing Strategy

In the commonly adopted setting, also applied in this work, the visual quality
enhancement of an entire document image is performed by working on separate
local patches and then recomposing them. This leads to most of the errors
occurring on the borders, causing discontinuities and artifacts in the
reconstructed document image. Therefore, to enhance the performance of our
approach, we introduce overlap between the patches used for prediction and
keep only the innermost predictions of the model. Specifically, at inference
time, we run our model over overlapping patches and recombine their prediction
of each pixel by assigning to that pixel the value that it assumes in the
patch to whose center it is closer. The process is shown in Fig. 2. In our
experiments, we set the overlap to 256 (half the patch size), which has been
determined empirically.

Figure 2: We combine the predictions of our model on overlapping patches at
each point based on the distance between the point of interest and the center
of each patch so that the artifacts at the patches borders are reduced.

##  4 Experimental Analysis

Implementation Details. Before training, we extract $384{\times}384$ image
patches from the entire document image, with stride $192$ pixels. Then, during
training, to further enlarge the training set, we apply a random rotation of
±10∘ to these patches, and random resize and crop to obtain the
$256{\times}256$ patches for training. Finally, randomly alter the image
brightness, contrast, saturation, and hue with factor 0.5. We train the model
by using the AdamW optmizer [31] with $\beta_{1}{=}0.9$, $\beta_{2}{=}0.95$,
batch size 8, and weight decay 0.05. Moreover, we set the learning rate to
$1.5{\cdot}10^{-4}$ and use a cosine scheduler with 10 linear warm-up steps.

Considered Datasets. Various datasets have been proposed in the literature for
the Document Binarization task. However, they all contain only a small number
of documents because of the complexity of producing the ground-truth
annotation masks. This poses a challenge when training deep learning networks,
as these require large amounts of data to model the several possible
degradations affecting the documents. Moreover, it is important that the set
of documents used in training features a number of diverse degradations, text
style, ink, and background. For this reason, in this work, we consider a
number of publicly available datasets containing both handwritten and
typewritten documents, from different periods and with different paper
supports and degradations. In particular, we use the ten public benchmark
datasets released over the years for the Document Image Binarization Contest
(DIBCO), namely DIBCO09 [17], H-DIBCO10 [44], DIBCO11 [45], H-DIBCO12 [46],
DIBCO13 [47], H-DIBCO14 [40], H-DIBCO16 [49], DIBCO17 [50], H-DIBCO18 [48],
and DIBCO19 [51], each containing from 10 to 20 document images, for a total
of 139. The documents feature several degradations representing real-world
cases, such as variable background intensity, shadows, smear, smudge, low-
contrast, bleed-through, or show-through. The documents in these datasets
contain machine-printed, typed, and handwritten text in the Latin alphabet,
except for DIBCO19, which also contains non-Latin characters. Note that these
datasets are from disjoint sets of documents and feature different types and
levels of degradation. Moreover, our training set also includes images from
the Bickley Diary [12], Synchromedia Multispectral Ancient Document Images
Dataset (SMADI) [22], and Nabuco [30] datasets. These datasets contain
documents in the Latin alphabet, both handwritten and typewritten, thus
enforcing the variability of our training set.

For evaluation, we consider three modern datasets (DIBCO11, DIBCO13, DIBCO17)
and five historical ones (H-DIBCO10, H-DIBCO12, H-DIBCO14, H-DIBCO16,
H-DIBCO18), and adopt the leave one-out strategy by using all other DIBCO
datasets for training. Note that we use DIBCO19 only for testing because the
patterns of the characters are too different from Latin scripts. Thus, we can
evaluate the generalization capabilities of our approach to the non-Latin
alphabet domain. In this respect, we also evaluate the performance of our
model and some competitors on the non-Latin scripts Persian Heritage Image
Binarization Dataset (PHIBD) [37, 3] and Irish Script on Screen Bleed-Through
Database (ISOS-BTD) [56] datasets.

Evaluation Scores. We evaluate our models by using the F-Measure (FM), the
pseudo-FMeasure (pFM) [39], the Peak Signal-to-Noise Ratio (PSNR), which
measures the similarity of two images, and the Distance Reciprocal Distortion
(DRD) [32], which measures the visual distortion of binary images.

###  4.1 Ablation Analysis

|
---|---
|

Figure 3: FM (top-left), pFM (top-right), PSNR (bottom-left), and DRD (bottom-
right) values obtained by our FFC-based model and a baseline variant
exploiting only standard convolutions when tested on patches of different
sizes from H-DIBCO18, with or without overlap.

Advantages of the FFC over the Standard Convolution. To assess the
effectiveness of FFCs over standard convolutions (Convs), we define a baseline
with the same architecture as FourBi that contains solely Convs. We swap the
ST module with a 1${\times}$1 convolution to maintain the global-local branch
division. The 1${\times}$1 convolution, in contrast to the ST module, focuses
on local aspects of the input. Therefore, the Conv-based baseline has the
global and local branches logically swapped, where the global branch has the
3${\times}$3 convolutions, and the local one has the 1${\times}$1 convolution.
Fig. 3 shows the performance comparison of the two models. As we can see, the
global receptive field of the FFC allows the model to outperform the Conv-
based baseline.

Role of the Test Patch Size. We ran an ablation analysis to assess the
capability of our model to learn robust representations and exploit the global
context provided by larger patches at inference time. In particular, we
evaluate our model and the corresponding Convolutional baseline on patches
whose size ranges from $128{\times}128$ to $704{\times}704$ (see Fig. 3). We
observe that the scores get better consistently, reaching the best value
around the patch size of 512.

Figure 4: Results of the Conv-based baseline and our FFC-based model on
$256{\times}256$ and $512{\times}512$ patches. Correct predictions are in
green, errors in red.

Figure 5: Average error distribution of our FFC-based model and the Conv-based
baseline on multiple $256{\times}256$ and $512{\times}512$ patches from
different datasets.

The same behavior emerges also in Fig. 5, which shows how the Conv-based
approach does not exploit the global information of the whole input patch and
processes the images locally due to a limited receptive field. On the other
hand, FourBi, thanks to the FFC and its infinite receptive field, increases
the performance on increasing the input patch size. From Fig. 5 it can be also
observed that the local patch contains blurred text, which both the FFC-based
and Convolution-based models incorrectly classify as foreground. In the lower
part of the figure, we can see that with a larger patch, the FFC-based network
leverages the surrounding context to identify that the innermost patch
contains bleed-through text and correctly classifies it as background.
Conversely, the Convolutional baseline does not benefit from the increased
patch size and still misclassifies the blurred text as foreground.

Finally, in Table 1, we compare the capabilities of FourBi and the
Transformer-based SotA DocEnTr [59] to model long-range dependencies. When
DocEnTr [59] is trained with larger patch sizes, the performance worsens,
showing that the model has difficulties capturing fine-grained details of the
documents. On the other hand, FourBi, which already outperforms DocEnTr on the
patch size 256${\times}$256, benefits from working on larger patches, even at
test time only.

|  Training patch |  Inference patch | FM$\uparrow$ | pFM$\uparrow$ | PSNR$\uparrow$ | DRD$\downarrow$
---|---|---|---|---|---|---
DocEnTr [59] | 256${\times}$256 | 256${\times}$256 | 92.53 | 95.15 | 19.11 | 2.37
FourBi | 92.94 | 97.29 | 19.26 | 2.32
DocEnTr [59] |  512${\times}$512 | 512${\times}$512 | 92.20 | 94.93 | 18.91 | 2.45
FourBi |  256${\times}$256 | 93.38 | 97.29 | 19.46 | 2.16
FourBi |  512${\times}$512 | 93.27 | 97.01 | 19.36 | 2.21

Table 1: Performance comparison of a Transformer-based model against our FFC-
based model on DIBCO17. Best results are in bold, second-best in italic.

Benefits of the Post-Processing Strategy. We perform a study of the most
common errors induced by working on single patches (see Fig. 5). In
particular, for a given patch size, we evaluate the FFC-based model and the
Convolutional baseline on several datasets and compute the average absolute
error for each pixel location. Then, we discretize the resulting image in
16${\times}$16 bins and compute the average error distribution. The values are
independently normalized for visualization purposes. From the figure, we
observe that the FFC-based model is more accurate on the borders with a larger
patch size. The global receptive field allows it to consider the whole
information of the patch. On the other hand, the Convolutional baseline is
less precise in the predictions when fed with a larger patch, and the
increased error compromises the potential benefit gained from a bigger
context. Fig. 3 also highlights that the applied overlapping procedure leads
to better performance in almost all patch sizes. In particular, we notice that
on the Conv-base baseline, the post-processing procedure alone saturates the
performance. This behavior suggests that the main weakness of the Conv-based
baseline is the predictions close to the edges. Therefore, with the overlap
procedure, the number of border pixels is minimized.

Alternative Loss Functions. In Table 2, we report a comparison of the
performance obtainable by training our model with other loss functions
commonly adopted for Document Binarizationment. In particular, we consider the
Binary Cross-Entropy (BCE), the MAE, and the MSE as alternatives to the
Charbonnier loss that we adopt. We recall that the Charbonnier loss behaves as
the MAE for high errors and as the MSE for low errors.

From the table, it emerges that the error-dependant behavior of the
Charbonnier loss benefits the training and leads to the best performance in
terms of all metrics apart from the FM, in terms of which it leads to second-
best results (comparable to those obtained when training with the BCE loss).
Note that, for Document Binarizationement, the pFM is more relevant than the
FM since it models the readability of the binarized document by penalizing
more the errors made near the ground truth ink strokes. In light of this, we
can conclude that the Charbonnier loss is a suitable objective function also
for this task.

|  | FM$\uparrow$ | pFM$\uparrow$ | PSNR$\uparrow$ | DRD$\downarrow$
---|---|---|---|---|---
BCE |  | 93.38 | 97.18 | 20.63 | 2.37
MSE |  | 92.68 | 97.40 | 20.56 | 2.56
MAE |  | 91.83 | 98.00 | 20.27 | 2.74
Charbonnier |  | 93.19 | 98.41 | 20.86 | 2.21

Table 2: Ablation analysis when training FourBi with different loss functions
on the H-DIBCO18 dataset. Best results in bold, second-best in italic.

Computational Complexity. Finally, we analyze the computational complexity of
our approach. Note that, given a square image of size $n$ and $C_{i}$
channels, the Real FFT2d and the Inv Real FFT2d operations have time
complexity $O(C_{i}{\cdot}n^{2}{\cdot}\text{log}n)$. The convolution has time
complexity $O(C_{i}{\cdot}C_{o}{\cdot}n^{2}{\cdot}k^{2})$, where $C_{o}$ is
the output channels and $k$ is the kernel size. The self-attention has time
complexity $O(n^{2}{\cdot}d)$, where $d$ is the representation dimension.
Nevertheless, the time complexity is not a complete indicator when comparing
completely different architectures. In fact, the inductive biases of each
operator, its characteristics, and how much its computation can be
parallelized influence how deep and big the network needs to be in order to
obtain a certain score. Therefore, to analyze the efficiency of our model, in
Table 3, we report runtimes obtained by processing a single image on a single
NVIDIA 2080Ti GPU and the number of parameters of the considered models fed
with $256{\times}256$ patches. We report this information for our approach,
the baseline exploiting standard convolution, and two SotA solutions, one
based on a fully-convolutional GAN (DE-GAN [61]) and the other on a ViT
(DocEnTr [59]).

| params | runtime
---|---|---
DE-GAN [62] | 31.0M | 11.2 ms
DocEnTr [59] | 68.8M | 3.8 ms
Conv. Baseline | 15.8M | 4.7 ms
FourBi | 17.6M | 9.1 ms

Table 3: Computational complexity and performance of SotA approaches and the
Conv-based baseline on the H-DIBCO18 dataset, all fed with $256{\times}256$
patches.

###  4.2 Comparison with the SotA

Quantitative comparison on Latin-characters documents. We evaluate our method
on eight DIBCO datasets. The results are reported in Tables 5, 5, 7, 7, 9, 9,
11 and 11. For space reasons, we sorted the competitors by PSNR, and we kept
for each table the first top ten methods. Moreover, to better highlight the
consistency of our method, we include for each table the rank of FourBi
compared to the competitors. It can be observed that our method achieves top
performance on most metrics. In the tables, we isolate Transformer-based
methods, which share with our approach the capability to model both long-range
and short-range dependencies. Further, our model is adaptable to different
image resolutions and has no strict constraints on the input size.

|  | FM$\uparrow$ | pFM$\uparrow$ | PSNR$\uparrow$ | DRD$\downarrow$
---|---|---|---|---|---
Winning Entry [44] |  | 91.50 | 93.58 | 19.78 | -
Guo _et al._ [19] |  | 86.75 | 89.74 | 17.93 | 3.62
Zhang _et al._ [76] |  | 85.70 | 90.61 | 17.96 | 3.60
Feng [16] |  | 88.16 | 90.96 | 18.25 | 3.41
Du & He [15] |  | 88.47 | 90.87 | 18.41 | 3.22
Guo _et al._ [18] |  | 88.56 | 93.50 | 18.56 | 3.06
Jia  _et al._ [25] |  | 91.75 | 95.36 | 19.84 | 2.17
Suh  _et al._ [64] |  | 93.92 | 96.53 | 21.18 | 1.50
cGANs [78] |  | 93.92 | 96.53 | 21.18 | 1.50
GDB [75] |  | 95.19 | 96.62 | 21.98 | 1.32
D2BFormer [73] |  | 96.23 | 97.49 | 23.10 | 0.95
FourBi |  | 96.43 | 98.60 | 23.37 | 0.92
Rank |  |  1st |  1st |  1st |  1st

Table 4: Quantitative comparison on the H-DIBCO10 dataset.

|  | FM$\uparrow$ | pFM$\uparrow$ | PSNR$\uparrow$ | DRD$\downarrow$
---|---|---|---|---|---
Winning Entry [45] |  | 88.74 | - | 17.97 | 5.36
Jia  _et al._ [25] |  | 91.65 | 95.56 | 18.88 | 2.66
HDSN [69] |  | 92.58 | 94.67 | 19.16 | 2.38
Suh  _et al._ [64] |  | 93.44 | 96.18 | 19.97 | 1.93
GDB [75] |  | 93.44 | 95.82 | 20.01 | 2.25
cGANs [78] |  | 93.81 | 95.70 | 20.26 | 1.81
Lin  _et al._ [29] |  | 94.08 | 97.08 | 20.51 | 1.75
Dang  _et al._ [10] |  | 95.61 | 97.34 | 22.09 | 1.48
DocEnTr [59] |  | 94.37 | 96.15 | 20.81 | 1.63
D2BFormer [73] |  | 94.82 | 96.62 | 21.27 | 1.56
Text-DIAE [60] |  | 95.01 | 96.86 | 21.29 | 1.48
FourBi |  | 96.03 | 98.70 | 22.26 | 1.19
Rank |  |  1st |  1st |  1st |  1st

Table 5: Quantitative comparison on the DIBCO11 dataset.

|  | FM$\uparrow$ | pFM$\uparrow$ | PSNR$\uparrow$ | DRD$\downarrow$
---|---|---|---|---|---
Winning Entry [46] |  | 92.85 | - | 21.80 | 2.66
Du & He [15] |  | 89.40 | 91.16 | 18.80 | 3.66
Jia  _et al._ [25] |  | 92.96 | 95.76 | 20.43 | 2.30
Suh  _et al._ [64] |  | 94.50 | 97.36 | 21.78 | 1.73
cGANs [78] |  | 94.96 | 96.15 | 21.91 | 1.55
Jemni  _et al._ [24] |  | 95.18 | 94.63 | 22.00 | 1.62
GDB [75] |  | 95.80 | 97.03 | 22.62 | 1.32
Rezabezhad  _et al._ [54] |  | 96.25 | 97.58 | 23.27 | 1.11
DocEnTr [59] |  | 95.31 | 96.29 | 22.29 | 1.60
D2BFormer [73] |  | 96.28 | 96.90 | 23.35 | 1.26
Text-DIAE [60] |  | 96.52 | 97.04 | 23.66 | 1.10
FourBi |  | 97.07 | 98.78 | 24.29 | 0.94
Rank |  |  1st |  1st |  1st |  1st

Table 6: Quantitative comparison on the H-DIBCO12 dataset.

|  | FM$\uparrow$ | pFM$\uparrow$ | PSNR$\uparrow$ | DRD$\downarrow$
---|---|---|---|---|---
Winning Entry [47] |  | 92.12 | 94.19 | 20.68 | 3.10
Du & He [15] |  | 89.94 | 93.97 | 19.26 | 3.24
Jia  _et al._ [25] |  | 93.28 | 96.58 | 20.76 | 2.01
HDSN [69] |  | 93.43 | 95.34 | 20.82 | 2.26
cGANS [78] |  | 93.86 | 96.47 | 21.53 | 2.32
Suh  _et al._ [64] |  | 94.75 | 97.20 | 21.79 | 1.66
Lin  _et al._ [29] |  | 95.24 | 97.51 | 22.27 | 1.59
GDB [75] |  | 95.19 | 96.37 | 22.58 | 1.78
Dang  _et al._ [10] |  | 95.96 | 98.13 | 23.14 | 1.43
DE-GAN [61] |  | 99.50 | 99.70 | 24.90 | 1.10
D2BFormer [73] |  | 95.96 | 96.96 | 22.80 | 1.30
FourBi |  | 96.71 | 98.27 | 24.17 | 1.13
Rank |  |  2nd |  2nd |  2nd |  2nd

Table 7: Quantitative comparison on the DIBCO13 dataset.

|  | FM$\uparrow$ | pFM$\uparrow$ | PSNR$\uparrow$ | DRD$\downarrow$
---|---|---|---|---|---
Winning Entry [40] |  | 96.88 | 97.65 | 22.66 | 0.90
Feng [16] |  | 92.99 | 95.54 | 19.42 | 2.26
Du & He [15] |  | 93.48 | 95.49 | 19.80 | 2.02
Jia  _et al._ [25] |  | 94.89 | 97.68 | 20.53 | 1.5
Suh  _et al._ [64] |  | 96.19 | 97.13 | 21.77 | 1.14
cGANs [78] |  | 96.41 | 97.55 | 22.12 | 1.07
Lin  _et al._ [29] |  | 86.65 | 98.19 | 22.27 | 0.96
GDB [75] |  | 97.66 | 97.26 | 23.13 | 1.21
HDSN [69] |  | 96.66 | 97.59 | 23.23 | 0.79
DocEnTr [59] |  | 97.16 | 98.28 | 22.99 | 0.16
D2BFormer [73] |  | 97.70 | 98.44 | 23.91 | 0.64
FourBi |  | 98.19 | 99.16 | 25.18 | 0.55
Rank |  |  1st |  1st |  1st |  2nd

Table 8: Quantitative comparison on the H-DIBCO14 dataset.

|  | FM$\uparrow$ | pFM$\uparrow$ | PSNR$\uparrow$ | DRD$\downarrow$
---|---|---|---|---|---
Winning Entry [49] |  | 88.72 | 91.84 | 18.45 | 3.86
Feng [16] |  | 88.88 | 90.30 | 18.45 | 4.99
Zhang _et al._ [76] |  | 88.04 | 90.89 | 18.58 | 4.00
HDSN [69] |  | 90.01 | 93.44 | 18.74 | 3.91
cGANs [78] |  | 89.77 | 94.85 | 18.80 | 3.85
Jia  _et al._ [25] |  | 90.01 | 93.72 | 19.00 | 4.03
GDB [75] |  | 90.41 | 94.70 | 19.00 | 3.34
Suh  _et al._ [64] |  | 91.11 | 95.22 | 19.34 | 3.25
Du & He [15] |  | 91.51 | 91.81 | 19.62 | 3.16
Jemni  _et al._ [24] |  | 94.95 | 94.55 | 21.85 | 1.56
D2BFormer [73] |  | 90.96 | 93.30 | 19.24 | 3.22
FourBi |  | 91.30 | 96.62 | 19.74 | 3.02
Rank |  |  2nd |  1st |  2nd |  2nd

Table 9: Quantitative comparison on the H-DIBCO16 dataset.

|  | FM$\uparrow$ | pFM$\uparrow$ | PSNR$\uparrow$ | DRD$\downarrow$
---|---|---|---|---|---
Winning Entry [50] |  | 91.04 | 92.86 | 18.28 | 3.40
cGANs [78] |  | 90.73 | 92.58 | 17.83 | 3.58
Suh  _et al._ [64] |  | 90.95 | 94.65 | 18.40 | 2.93
Lin  _et al._ [29] |  | 90.95 | 93.79 | 18.57 | 2.94
Dang  _et al._ [10] |  | 92.08 | 94.99 | 18.72 | 2.84
DE-GAN [61] |  | 97.71 | 98.23 | 18.74 | 3.01
Rezabezhad _et al._ [54] |  | 93.01 | 95.42 | 19.24 | 2.29
GDB [75] |  | 94.32 | 96.58 | 20.04 | 1.79
DocEnTr [59] |  | 92.53 | 95.15 | 19.11 | 2.37
D2BFormer [73] |  | 93.52 | 95.09 | 19.35 | 2.12
Text-DIAE [60] |  | 93.84 | 95.71 | 19.64 | 1.93
FourBi |  | 93.81 | 96.57 | 19.66 | 2.03
Rank |  |  4th |  3rd |  2nd |  3rd

Table 10: Quantitative comparison on the DIBCO17 dataset.

|  | FM$\uparrow$ | pFM$\uparrow$ | PSNR$\uparrow$ | DRD$\downarrow$
---|---|---|---|---|---
Winning Entry [48] |  | 88.34 | 90.24 | 19.11 | 4.92
DE-GAN [61] |  | 77.59 | 85.74 | 16.16 | 7.93
Dang  _et al._ [10] |  | 91.26 | 93.97 | 19.81 | 3.42
GDB [75] |  | 91.09 | 94.57 | 19.92 | 3.07
Lin  _et al._ [29] |  | 91.66 | 95.53 | 20.02 | 2.81
Suh  _et al._ [64] |  | 91.86 | 96.25 | 20.03 | 2.60
Jemni  _et al._ [24] |  | 92.41 | 94.35 | 20.18 | 2.60
Rezabezhad _et al._ [54] |  | 92.47 | 95.99 | 20.29 | 2.50
D2BFormer [73] |  | 88.84 | 93.42 | 18.91 | 3.99
DocEnTr [59] |  | 90.59 | 93.97 | 19.46 | 3.35
Text-DIAE [60] |  | 91.32 | 94.44 | 19.95 | 3.21
FourBi |  | 93.16 | 98.40 | 20.92 | 2.23
Rank |  |  1st |  1st |  1st |  1st

Table 11: Quantitative comparison on the H-DIBCO18 dataset.

Quantitative comparison on non-Latin-characters documents. In Table 12, we
report a quantitative performance analysis on the non-Latin DIBCO19, PHIBD,
and ISOS-BTD datasets, which were not included in any training set.
Specifically, we compare our approach, both alone and combined with the
overlap-based post-processing strategy, with two SotA approaches: the fully-
convolutional, generative DE-GAN [61] and the ViT-based DocEnTr [59]. It can
be observed the same trend as for the considered Latin-characters training
datasets. Specifically, FourBi achieves competitive or better performance
compared to DE-GAN and DocEnTr. The difference in the charset prevents the
networks from exploiting a priori knowledge possibly learned on the other
datasets, which might allow them to recognize _e.g._ , whether the text is
flipped (bleed-through) from the context. Therefore, the networks can only
exploit the image appearance to classify each pixel correctly. Nonetheless,
thanks to the FFC, FourBi can exploit global information, which helps it
better model the overall visual appearance.

|  |  | FM$\uparrow$ | pFM$\uparrow$ | PSNR$\uparrow$ | DRD$\downarrow$
---|---|---|---|---|---|---
PHIBD |  DE-GAN [61] |  | 69.83 | 71.58 | 12.33 | 28.30
DocEnTr [59] |  | 59.28 | 83.81 | 13.91 | 15.89
FourBi |  | 80.07 | 84.05 | 17.31 | 14.07
FourBi + Overlap |  | 80.36 | 90.66 | 17.41 | 13.22
ISOS-BTD |  DE-GAN [61] |  | 85.19 | 91.55 | 12.59 | 13.25
DocEnTr [59] |  | 79.34 | 97.48 | 12.44 | 14.12
FourBi |  | 78.23 | 98.75 | 12.46 | 14.12
FourBi + Overlap |  | 78.88 | 98.75 | 12.62 | 13.64
DIBCO19 |  DE-GAN [61] |  | 57.96 | 57.30 | 14.43 | 12.21
DocEnTr [59] |  | 67.20 | 77.13 | 14.12 | 10.20
FourBi |  | 63.15 | 84.99 | 14.94 | 9.89
FourBi + Overlap |  | 63.56 | 85.82 | 15.02 | 9.66

Table 12: Quantitative comparison on the considered non-Latin character
datasets. Best results in bold, second-best in italic for each of them.

Qualitative Comparison on with the SoTa. We qualitatively compare our
approach, DE-GAN [61], and DocEnTr [59], on Latin and non-Latin documents. We
show the results with one of the challenging images of H-DIBCO18 [48] in Fig.
6-Right. As we can see, both competitors produce images with speckles on the
background, while our model, thanks to its ability to manage the global
information, produces less noisy predictions and recognizes the page limit on
top. Nonetheless, this sample is heavily stained on the top, and all three
methods fail at accurately binarizing part of the head loop on the capital
letter “G” and the page number. Finally, we qualitatively compare our approach
on images from challenging ancient datasets. Different from those used in
training, these documents contain non-Latin text. Thus, our model would not be
able to exploit any potentially learned characters appearance information when
binarizing those images. In Fig. 6-Left, it can be observed that FourBi
accurately removes severe degradations such as faded ink and holes. Moreover,
it shows superior capabilities in handling bleed-through ink, exploiting the
global context.

##  5 Conclusion

In this work, we have proposed a fully-convolutional Document Binarization
approach that can handle both global and local information to predict more
precisely whether a pixel is background or ink. We achieve this capability by
exploiting the FFC operator in a U-Net-like architecture. The FFC layers allow
handling both local and global information, thus leading to results that are
comparable or superior to those achieved with Transformer-based models.
Moreover, we have observed that the common strategy to divide the document
image into patches and treat them separately leads to artifacts at the borders
of the patches. In light of this, we have applied a simple yet effective post-
processing strategy consisting in combining the predictions of our model on
overlapping patches. The results of deep experimental analysis on multiple
benchmark datasets demonstrate the suitability of our approach, together with
the possibility of applying the FFC operator to patches that are much wider
than those used in training without performance drop, further improving the
performance by reducing the artifacts at the borders.

Figure 6: Qualitative comparison between FourBi and other SotA approaches on
datasets containing ancient documents. Starting from the left column, we test
FourBi on the non-Latin datasets PHIBD [37, 3], ISOS-BTD [56], and DIBCO19
[51]. In the rightmost column, we test FourBi on the Latin dataset DIBCO18
[48]. All models were trained on Latin-script datasets.

## Acknowledgement

This work was supported by the “AI for Digital Humanities” project (Pratica
Sime n.2018.0390), funded by “Fondazione di Modena” and the PNRR project
Italian Strengthening of ESFRI RI Resilience (ITSERR) funded by the European
Union – NextGenerationEU (CUP: B53C22001770006).

## References

  * [1] Akbari, Y., Al-Maadeed, S., Adam, K.: Binarization of degraded document images using convolutional neural networks and wavelet-based multichannel images. IEEE Access 8, 153517–153534 (2020)
  * [2] Akbari, Y., Britto, A.S., Al-Maadeed, S., Oliveira, L.S.: Binarization of degraded document images using convolutional neural networks based on predicted two-channel images. In: ICDAR (2019)
  * [3] Ayatollahi, S.M., Nafchi, H.Z.: Persian heritage image binarization competition (PHIBC 2012). In: PRIA (2013)
  * [4] Barron, J.T.: A general and adaptive robust loss function. In: CVPR (2019)
  * [5] Barron, J.T.: A generalization of Otsu’s method and minimum error thresholding. In: ECCV (2020)
  * [6] Bhunia, A.K., Bhunia, A.K., Sain, A., Roy, P.P.: Improving document binarization via adversarial noise-texture augmentation. In: ICIP (2019)
  * [7] Charbonnier, P., Blanc-Feraud, L., Aubert, G., Barlaud, M.: Two deterministic half-quadratic regularization algorithms for computed imaging. In: ICIP (1994)
  * [8] Chi, L., Jiang, B., Mu, Y.: Fast fourier convolution. NeurIPS (2020)
  * [9] Cojocaru, I., Cascianelli, S., Baraldi, L., Corsini, M., Cucchiara, R.: Watch your strokes: improving handwritten text recognition with deformable convolutions. In: ICPR (2021)
  * [10] Dang, Q.V., Lee, G.S.: Document image binarization with stroke boundary feature guided network. IEEE Access 9, 36924–36936 (2021)
  * [11] De Gregorio, G., Biswas, S., Souibgui, M.A., Bensalah, A., Lladós, J., Fornés, A., Marcelli, A.: A Few Shot Multi-representation Approach for N-Gram Spotting in Historical Manuscripts. In: ICFHR (2022)
  * [12] Deng, F., Wu, Z., Lu, Z., Brown, M.S.: Binarizationshop: a user-assisted software suite for converting old documents to black-and-white. In: JCDL (2010)
  * [13] Dey, S., Jawanpuria, P.: Light-weight document image cleanup using perceptual loss. In: ICDAR (2021)
  * [14] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In: ICLR (2020)
  * [15] Du, Z., He, C.: Nonlinear diffusion equation with selective source for binarization of degraded document images. Appl. Math. Model. 99, 243–259 (2021)
  * [16] Feng, S.: Effective document image binarization via a convex variational level set model. Appl. Math. Comput. 419, 126861 (2022)
  * [17] Gatos, B., Ntirogiannis, K., Pratikakis, I.: ICDAR 2009 document image binarization contest (DIBCO 2009). In: ICDAR (2009)
  * [18] Guo, J., He, C., Wang, Y.: Fourth order indirect diffusion coupled with shock filter and source for text binarization. Signal Processing 171, 107478 (2020)
  * [19] Guo, J., He, C., Zhang, X.: Nonlinear edge-preserving diffusion with adaptive source for document images binarization. Appl. Math. Comput. 351, 8–22 (2019)
  * [20] Han, S., Ji, S., Rhee, J.: Diffusion-denoising process with gated u-net for high-quality document binarization. Applied Sciences 13(20), 11141 (2023)
  * [21] He, S., Schomaker, L.: DeepOtsu: Document enhancement and binarization using iterative deep learning. Pattern Recognit. 91, 379–390 (2019)
  * [22] Hedjam, R., Cheriet, M.: Ground-truth estimation in multispectral representation space: Application to degraded document image binarization. In: ICDAR (2013)
  * [23] Howe, N.R.: Document binarization with automatic parameter tuning. IJDAR 16, 247–258 (2013)
  * [24] Jemni, S.K., Souibgui, M.A., Kessentini, Y., Fornés, A.: Enhance to read better: A multi-task adversarial network for handwritten document image enhancement. Pattern Recognit. 123, 108370 (2022)
  * [25] Jia, F., Shi, C., He, K., Wang, C., Xiao, B.: Degraded document image binarization using structural symmetry of strokes. Pattern Recognit. 74, 225–240 (2018)
  * [26] Kang, S., Iwana, B.K., Uchida, S.: Complex image processing with less data—document image binarization by integrating multiple pre-trained u-net modules. Pattern Recognit. 109, 107577 (2021)
  * [27] Koloda, J., Wang, J.: Context Aware Document Binarization and Its Application to Information Extraction from Structured Documents. In: ICDAR (2023)
  * [28] Lai, W.S., Huang, J.B., Ahuja, N., Yang, M.H.: Fast and accurate image super-resolution with deep laplacian pyramid networks. IEEE Trans. PAMI (2018)
  * [29] Lin, Y.S., Ju, R.Y., Chen, C.C., Lin, T.Y., Chiang, J.S.: Three-stage binarization of color document images based on discrete wavelet transform and generative adversarial networks. arXiv preprint arXiv:2211.16098 (2022)
  * [30] Lins, R.: Nabuco—Two Decades of Processing Historical Documents in Latin America. J. Univers. Comput. Sci 17, 151–161 (2011)
  * [31] Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. ICLR (2017)
  * [32] Lu, H., Kot, A.C., Shi, Y.Q.: Distance-reciprocal distortion measure for binary document images. IEEE Signal Process. Lett. 11(2), 228–231 (2004)
  * [33] Lu, S., Su, B., Tan, C.L.: Document image binarization using background estimation and stroke edges. IJDAR 13, 303–314 (2010)
  * [34] Mallat, S.G.: A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans. PAMI 11(7), 674–693 (1989)
  * [35] Messaoud, I.B., Amiri, H., El Abed, H., Märgner, V.: Region based local binarization approach for handwritten ancient documents. In: ICFHR (2012)
  * [36] Moghaddam, R.F., Moghaddam, F.F., Cheriet, M.: Unsupervised ensemble of experts (EoE) framework for automatic binarization of document images. In: ICDAR (2013)
  * [37] Nafchi, H.Z., Ayatollahi, S.M., Moghaddam, R.F., Cheriet, M.: An efficient ground truthing tool for binarization of historical manuscripts. In: ICDAR (2013)
  * [38] Nafchi, H.Z., Moghaddam, R.F., Cheriet, M.: Phase-based binarization of ancient document images: Model and applications. IEEE Trans. Image Process. 23(7), 2916–2930 (2014)
  * [39] Ntirogiannis, K., Gatos, B., Pratikakis, I.: Performance evaluation methodology for historical document image binarization. IEEE Trans. Image Process. 22(2), 595–609 (2012)
  * [40] Ntirogiannis, K., Gatos, B., Pratikakis, I.: ICFHR2014 competition on handwritten document image binarization (H-DIBCO 2014). In: ICFHR (2014)
  * [41] Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979)
  * [42] Pippi, V., Cascianelli, S., Baraldi, L., Cucchiara, R.: Evaluating synthetic pre-training for handwriting processing tasks. Pattern Recognit. Lett. 172, 44–50 (2023)
  * [43] Pippi, V., Cascianelli, S., Kermorvant, C., Cucchiara, R.: How to Choose Pretrained Handwriting Recognition Models for Single Writer Fine-Tuning. In: ICDAR (2023)
  * [44] Pratikakis, I., Gatos, B., Ntirogiannis, K.: H-DIBCO 2010-handwritten document image binarization competition. In: ICFHR (2010)
  * [45] Pratikakis, I., Gatos, B., Ntirogiannis, K.: ICDAR 2011 Document Image Binarization Contest (DIBCO 2011). In: ICDAR (2011)
  * [46] Pratikakis, I., Gatos, B., Ntirogiannis, K.: ICFHR 2012 competition on handwritten document image binarization (H-DIBCO 2012). In: ICFHR (2012)
  * [47] Pratikakis, I., Gatos, B., Ntirogiannis, K.: ICDAR 2013 document image binarization contest (DIBCO 2013). In: ICDAR (2013)
  * [48] Pratikakis, I., Zagori, K., Kaddas, P., Gatos, B.: ICFHR 2018 Competition on Handwritten Document Image Binarization (H-DIBCO 2018). In: ICFHR (2018)
  * [49] Pratikakis, I., Zagoris, K., Barlas, G., Gatos, B.: ICFHR2016 handwritten document image binarization contest (H-DIBCO 2016). In: ICFHR (2016)
  * [50] Pratikakis, I., Zagoris, K., Barlas, G., Gatos, B.: ICDAR2017 competition on document image binarization (DIBCO 2017). In: ICDAR (2017)
  * [51] Pratikakis, I., Zagoris, K., Karagiannis, X., Tsochatzidis, L., Mondal, T., Marthot-Santaniello, I.: ICDAR 2019 Competition on Document Image Binarization (DIBCO 2019). In: ICDAR (2019)
  * [52] Quattrini, F., Pippi, V., Cascianelli, S., Cucchiara, R.: Volumetric fast fourier convolution for detecting ink on the carbonized herculaneum papyri. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (2023)
  * [53] Ray, A., Sharma, M., Upadhyay, A., Makwana, M., Chaudhury, S., Trivedi, A., Singh, A., Saini, A.: An end-to-end trainable framework for joint optimization of document enhancement and recognition. In: ICDAR (2019)
  * [54] Rezanezhad, V., Baierer, K., Neudecker, C.: A hybrid cnn-transformer model for historical document image binarization. In: HIP (2023)
  * [55] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: MICCAI. pp. 234–241 (2015)
  * [56] Rowley-Brooke, R., Pitié, F., Kokaram, A.: A ground truth bleed-through document image database. In: TPDL (2012)
  * [57] Sauvola, J., Seppanen, T., Haapakoski, S., Pietikainen, M.: Adaptive document binarization. In: ICDAR (1997)
  * [58] Sinha, A.K., Moorthi, S.M., Dhar, D.: NL-FFC: Non-Local Fast Fourier Convolution for Image Super Resolution. In: CVPR (2022)
  * [59] Souibgui, M.A., Biswas, S., Jemni, S.K., Kessentini, Y., Fornés, A., Lladós, J., Pal, U.: DocEnTr: an end-to-end document image enhancement transformer. In: ICPR (2022)
  * [60] Souibgui, M.A., Biswas, S., Mafla, A., Biten, A.F., Fornés, A., Kessentini, Y., Lladós, J., Gomez, L., Karatzas, D.: Text-DIAE: Degradation Invariant Autoencoders for Text Recognition and Document Enhancement. AAAI (2023)
  * [61] Souibgui, M.A., Kessentini, Y.: DE-GAN: A conditional generative adversarial network for document enhancement. IEEE Trans. PAMI 44(3), 1180–1191 (2020)
  * [62] Souibgui, M.A., Kessentini, Y., Fornés, A.: A conditional gan based approach for distorted camera captured documents recovery. In: MedPRAI (2021)
  * [63] Stauffer, M., Fischer, A., Riesen, K.: Keyword spotting in historical handwritten documents based on graph matching. Pattern Recognit. (2018)
  * [64] Suh, S., Kim, J., Lukowicz, P., Lee, Y.O.: Two-stage generative adversarial networks for document image binarization with color noise and background removal. arXiv preprint arXiv:2010.10103 (2020)
  * [65] Suvorov, R., Logacheva, E., Mashikhin, A., Remizova, A., Ashukha, A., Silvestrov, A., Kong, N., Goka, H., Park, K., Lempitsky, V.: Resolution-robust large mask inpainting with fourier convolutions. In: WACV (2022)
  * [66] Tamrin, M.O., El-Amine Ech-Cherif, M., Cheriet, M.: A two-stage unsupervised deep learning framework for degradation removal in ancient documents. In: Pattern Recognit. pp. 292–303 (2021)
  * [67] Tensmeyer, C., Martinez, T.: Document image binarization with fully convolutional neural networks. In: ICDAR (2017)
  * [68] Tensmeyer, C., Martinez, T.: Historical document image binarization: A review. SN comput. sci. 1(3), 173 (2020)
  * [69] Vo, Q.N., Kim, S.H., Yang, H.J., Lee, G.: Binarization of degraded document images based on hierarchical deep supervised network. Pattern Recognit. 74, 568–586 (2018)
  * [70] Wolf, C., Jolion, J.M., Chassaing, F.: Text localization, enhancement and binarization in multimedia documents. In: ICPR (2002)
  * [71] Xiong, W., Jia, X., Xu, J., Xiong, Z., Liu, M., Wang, J.: Historical document image binarization using background estimation and energy minimization. In: ICPR (2018)
  * [72] Xiong, W., Zhou, L., Yue, L., Li, L., Wang, S.: An enhanced binarization framework for degraded historical document images. EURASIP J Image Video Process 2021(1), 13 (2021)
  * [73] Yang, M., Xu, S.: A Novel Degraded Document Binarization Model through Vision Transformer Network. Information Fusion 93, 159–173 (2023)
  * [74] Yang, Z., Liu, B., Xxiong, Y., Yi, L., Wu, G., Tang, X., Liu, Z., Zhou, J., Zhang, X.: Docdiff: Document enhancement via residual diffusion models. In: ACM Multimedia (2023)
  * [75] Yang, Z., Xiong, Y., Wu, G.: GDB: Gated convolutions-based Document Binarization. Pattern Recognit. (2024)
  * [76] Zhang, X., He, C., Guo, J.: Selective diffusion involving reaction for binarization of bleed-through document images. Appl. Math. Comput. 81, 844–854 (2020)
  * [77] Zhao, G., Liu, J., Jiang, J., Guan, H., Wen, J.R.: Skip-connected deep convolutional autoencoder for restoration of document images. In: ICPR (2018)
  * [78] Zhao, J., Shi, C., Jia, F., Wang, Y., Xiao, B.: Document image binarization with cascaded generators of conditional generative adversarial networks. Pattern Recognit. 96, 106968 (2019)
  * [79] Zhong, Z., Shen, T., Yang, Y., Lin, Z., Zhang, C.: Joint sub-bands learning with clique structures for wavelet domain super-resolution. NeurIPS (2018)
  * [80] Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: ICCV (2017)