11institutetext: School of Computing, Dublin City University, Ireland
22institutetext: VNU-HCM, University of Science, Vietnam 33institutetext:
Vietnam National University Ho Chi Minh City, Vietnam 44institutetext: AISIA
Lab, Vietnam

# HADA: A Graph-based Amalgamation Framework in Image-text Retrieval

Manh-Duy Nguyen 11 0000-0001-6878-7039 Binh T. Nguyen 223344 Cathal Gurrin 11
0000-0003-2903-3968

###### Abstract

Many models have been proposed for vision and language tasks, especially the
image-text retrieval task. All state-of-the-art (SOTA) models in this
challenge contained hundreds of millions of parameters. They also were
pretrained on a large external dataset that has been proven to make a big
improvement in overall performance. It is not easy to propose a new model with
a novel architecture and intensively train it on a massive dataset with many
GPUs to surpass many SOTA models, which are already available to use on the
Internet. In this paper, we proposed a compact graph-based framework, named
HADA, which can combine pretrained models to produce a better result, rather
than building from scratch. First, we created a graph structure in which the
nodes were the features extracted from the pretrained models and the edges
connecting them. The graph structure was employed to capture and fuse the
information from every pretrained model with each other. Then a graph neural
network was applied to update the connection between the nodes to get the
representative embedding vector for an image and text. Finally, we used the
cosine similarity to match images with their relevant texts and vice versa to
ensure a low inference time. Our experiments showed that, although HADA
contained a tiny number of trainable parameters, it could increase baseline
performance by more than $3.6\%$ in terms of evaluation metrics in the
Flickr30k dataset. Additionally, the proposed model did not train on any
external dataset and did not require many GPUs but only 1 to train due to its
small number of parameters. The source code is available at
https://github.com/m2man/HADA.

###### Keywords:

image-text retrieval graph neural network fusion model.

##  1 Introduction

Image-text retrieval is one of the most popular challenges in vision and
language tasks, with many state-of-the-art models (SOTA) recently introduced
[19, 3, 18, 28, 10, 25, 17]. This challenge includes 2 subtasks, which are
image-to-text retrieval and text-to-image retrieval. The former subtask is
defined as an image query is given to retrieve relevant texts in a multimodal
dataset, while the latter is vice versa.

Most of the SOTA models in this research field shared 2 things in common: (1)
they were built on transformer-based cross-modality attention architectures
[3, 19] and (2) they were pretrained on the large-scale multimodal data
crawled from the Internet [28, 18, 19, 17, 13]. However, these things have
their own disadvantages. The attention structure between 2 modalities could
achieve an accurate result, but it cost a large amount of inference time due
to the massive computation. For instance, UNITER [3] contained roughly 303
millions parameters and it took a decent amount of time to perform the
retrieval in real-time [31]. Many recent work has resolved this model-related
problem by introducing joint-encoding learning methods. They can learn visual
and semantic information from both modalities without using any cross-
attention modules, which can be applied later to rerank the initial result
[18, 25, 31]. Figure 1 illustrated the architecture of these pipelines.
Regarding the data perspective, the large collected data usually come with
noisy annotation, and hence could be harmful to the models that are trained on
it. Several techniques have been proposed to mitigate this issue [19, 18, 17].
However, training on the massive dataset still creates a burden on the
computation facility, such as the number of GPUs, which are required to train
the model successfully and efficiently [28].

Figure 1: Two most popular pipelines of the SOTA for image-text retrieval
challenge. (a) A cross-modality transformer network is applied to measure the
similarity between an image and a text based on their features. (b) Each
modality used their own transformer network to get its global embedding.

It has motivated us to answer the question: Can we combine many SOTA models,
which are currently available to use, to get a better unified model without
intensively training with many GPUs? In this paper, we introduced a graph-
based amalgamation framework, called HADA, which formed a graph-based
structure to fuse the features produced by other pretrained models. We did not
use any time-consuming cross-modality attention network to ensure fast
retrieval speed. A graph neural network was employed to extract visual and
textual embedded vectors from fused graph-based structures of images and
texts, where we can measure their cosine similarity. To the best of our
knowledge, the graph structure has been widely applied in the image-text
retrieval challenge [26, 7, 27, 35, 21]. Nevertheless, it was utilized to
capture the interaction between objects or align local and global information
within images. HADA is the first approach that applies this data structure to
combine SOTA pretrained models by fusing their features in each modality. We
trained HADA only on the Flickr30k dataset without using any large-scale
datasets. We applied Momentum Distillation technique [18], which was shown
that can not only mitigate the harmful effect of noise annotation but also
improve the accuracy on a clean dataset. Our experiments showed that HADA,
with the tiny extra number of training parameters, could improve total recalls
by $3.64\%$ compared to the input SOTA without training with millions of
additional image-text pairs as other models. This is the most crucial part
since it is not easy to possess multiple GPUs to use, especially for small and
medium businesses or start-up companies. Therefore, we believe that HADA can
be applied not only in the academic field, but also in industry.

Our main contribution can be summarised as follow: (1) We introduced HADA, a
compact pipeline that can combine 2 or many SOTA pretrained models to address
the image-text retrieval challenge. (2) We proposed a way to fuse the
information between input pretrained models by using graph structures. (3) We
evaluated the performance of HADA on the well-known Flickr30k dataset [37] and
MSCOCO dataset [20] without using any other large-scale dataset but still
improved the accuracy compared to the baseline input models.

##  2 Related Work

A typical vision-and-language model, including image-text retrieval task, was
built with the usage of transformer-based encoders. In specific, OSCAR [19],
UNITER [3], and VILLA [10] firstly employed Faster-RCNN [29] and BERT [6] to
extract visual and text features from images and texts. These features were
then fed into a cross-modality transformer block to learn the contextualized
embedding that captured the relations between regional features from images
and word pieces from texts. An additional fully connected layer was used to
classify whether the images and texts were relevant to each other or not,
based on the embedding vectors. Although achieving superior results, these
approaches had a drawback of being applied to real-time use cases. It required
a huge amount of time to perform the retrieval online, since the models have
to process the intensive cross-attention transformer architecture many times
for every single query [31].

Recently, there have been some works proposing an approach to resolve that
problem by utilising 2 distinct encoders for images and text. The data from
each modality now can be embedded offline and hence improve the retrieval
speed [31, 18, 17, 25, 13, 28]. In terms of architecture, all approaches used
the similar BERT-based encoder for semantic data but different image encoders.
While LightningDOT [31] encoded images with detected objects extracted by the
Faster-RCNN model, FastnSlow [25] applied the conventional Resnet network to
embed images. On the other side, ALBEF [18] and BLIP [17] employed the Vision
Transformer backbone [8] to get the visual features corresponding to their
patches. Because these SOTA did not use the cross-attention structure, which
was a critical point to achieve high accuracy, they applied different
strategies to increase performance. Specifically, pretraining a model on a
large dataset can significantly improve the result [18, 19, 13]. For instance,
CLIP [28] and ALIGN [13] were pretrained on 400 millions and 1.8 billions
image-text pairs, respectively. Another way was that they ran another cross-
modality image-text retrieval model to rerank the initial output and get a
more accurate result [18, 31].

Regarding to graph structures, SGM [35] introduced a visual graph encoder and
a textual graph encoder to capture the interaction between objects appearing
in images and between the entities in text. LGSGM [26] proposed a graph
embedding network on top of SGM to learn both local and global information
about the graphs. Similarly, GSMN [21] presented a novel technique to assess
the correspondence of nodes and edges of graphs extracted from images and
texts separately. SGRAF [7] build a reasoning and filtration graph network to
refine and remove irrelevant interactions between objects in both modalities.

Although there are many SOTAs with different approaches for image-text
retrieval problems, there is no work that tries combining these models but
introducing a new architecture and pretrain on a massive dataset instead.
Training an entire new model from scratch on the dataset is not an easy
challenge since it will create a burden on the computation facilities such as
GPUs. In this paper, we introduced a simple method which combined the features
extracted from the pretrained SOTA by applying graph structures. Unlike other
methods that also used this data structure, we employed graphs to fuse the
information between the input features, which was then fed into a conventional
graph neural network to obtain the embedding for each modality. Our HADA
consisted of a small number of trainable parameters, hence can be easily
trained on a small dataset but still obtained higher results compared to the
input models.

##  3 Methodology

This section will describe how our HADA addressed the retrieval challenge by
combining any available pretrained models. Figure 2 depicted the workflow of
HADA. We started with only 2 models $(N_{models}=2)$ as illustrated in Figure
2 for simplicity. Nevertheless, HADA can be extended with a larger
$N_{models}$. HADA began with using some pretrained models to extract the
features from each modality. We then built a graph structure to connect the
extracted features together, which were fed into a graph neural network (GNN)
later to update them. The outputs of the GNN were concatenated with the
original global features produced by the pretrained models. Finally, simple
linear layers were employed at the end to get the final representation
embedding features for images and texts, which can be used to measure the
similarity to perform the retrieval. For evaluation, we could extract our
representation features offline to guarantee the high speed inference time.

Figure 2: The pipeline of the proposed HADA. The red borders indicated
trainable components. The ITM and ITC infered the training tasks which will be
discussed later.

###  3.1 Revisit State-of-the-art Models

We only used the pretrained models without using the cross-modality
transformer structure to extract the features as depicted in Figure 1 to
reduce the number of computations and ensure the high speed inference time.
Basically, they used a unimodal encoder to get the features of an image or a
text followed by a transformer network to embed them and obtain the [CLS]
embedding. This [CLS] token was updated by 1 or many fully connected layers to
become a representative global feature that can be compared with that of the
remaining modality to get the similarity score.

HADA began with the output of the transformer layer from the pretrained
models. In detail, for an input image I, we obtained the sequence of patch
tokens from each model $i$ denoted as
$\mathbf{v^{(i)}}=\\{v^{(i)}_{cls},v^{(i)}_{1},v^{(i)}_{2},...,v^{(i)}_{N_{i}}\\}$,
where $v^{(i)}_{j}\in\mathbb{R}^{d^{(i)}_{v}}$ and $N_{i}$ was the length of
the sequence. This length depended on the architecture of the image encoder
network employed in the pretrained model. For example, it could be the number
of patches if the image encoder was a Vision Transformer (ViT) network [8], or
the number of detected objects or regions of interest if the encoder was a
Faster-RCNN model [29]. Additionally, we also extracted the global visual
representation feature $h^{(i)}_{v}\in\mathbb{R}^{d^{(i)}_{h}}$ from
$v^{(i)}_{cls}$ as illustrated in Figure 1. Regarding the semantic modality,
we used the same process as that of the visual modality. Specifically, we
extracted the sequence of patch tokens
$\mathbf{w^{(i)}}=\\{w^{(i)}_{cls},w^{(i)}_{1},w^{(i)}_{2},...,w^{(i)}_{L}\\}$
where $w^{(i)}_{j}\in\mathbb{R}^{d^{(i)}_{w}}$ and $L$ was the length of the
text, and the global textual representation embedding
$h^{(i)}_{w}\in\mathbb{R}^{d^{(i)}_{h}}$ for an input text T using the
pretrained model $i$. The input model $i$ matched a pair of an image I and a
text T by calculating the dot product $\langle h^{(i)}_{v},h^{(i)}_{w}\rangle$
of their global features. However, HADA not only used the global embedding but
also the intermediate transformer tokens to make the prediction. We used our
learned [CLS] tokens to improve the global features. In contrast, using the
original global features could ensure high performance of the pretrained
models and mitigate the effect of unhelpful tokens.

###  3.2 Create Graph Structure

Each pretrained model i produced different [CLS] features $v^{(i)}_{cls}$ and
$w^{(i)}_{cls}$ for an image and text, respectively. Since our purpose was to
combine the models, we needed to fuse these [CLS] tokens to obtain the unified
ones for each modality separately. In each modality, for example, the visual
modality, HADA not only updated $v^{(i)}_{cls}$ based on $\mathbf{v^{(i)}}$
solely but also on those of the remaining pretrained models
$\\{\mathbf{v^{(j)}}\ |\ j\neq i\\}$. Because these $\mathbf{v}$ came from
different models, their dimensions could be not similar to each other.
Therefore, we applied a list of linear layers
$f^{(i)}_{v}:\mathbb{R}^{d^{(i)}_{v}}\rightarrow\mathbb{R}^{d_{p}}$ to map
them in the same dimensional space:

$\mathbf{p^{(i)}}=\\{f^{(i)}_{v}(x)|x\in\mathbf{v^{(i)}}\\}=\\{p^{(i)}_{cls},p^{(i)}_{1},p^{(i)}_{2},...,p^{(i)}_{N_{i}}\\}$

We performed a similar process for the textual modality to obtain:

$\mathbf{s^{(i)}}=\\{f^{(i)}_{w}(x)|x\in\mathbf{w^{(i)}}\\}=\\{s^{(i)}_{cls},s^{(i)}_{1},s^{(i)}_{2},...,s^{(i)}_{L}\\},\textnormal{where}\
f^{(i)}_{w}:\mathbb{R}^{d^{(i)}_{w}}\rightarrow\mathbb{R}^{d_{s}}$

We then used graph structures
$\mathcal{G}p=\\{\mathcal{V}_{p},\mathcal{E}_{p}\\}$ and
$\mathcal{G}_{s}=\\{\mathcal{V}_{s},\mathcal{E}_{s}\\}$ to connect these
mapped features together, where $\mathcal{V}$ and $\mathcal{E}$ denoted the
list of nodes and edges in the graph $\mathcal{G}$ accordingly. In our HADA,
nodes indicated the mapped features. Specifically,
$\mathcal{V}_{p}=\\{\mathbf{p^{(i)}}\\}$ and
$\mathcal{V}_{s}=\\{\mathbf{s^{(i)}}\\}$ for all $i\in[1,N_{models}]$.
Regarding edges, we symbolized $e_{a\rightarrow b}$ as a directed edge from
node $a$ to node $b$ in the graph, thus the set of edges of the visual graph
$\mathcal{E}_{p}$ and the textual graph $\mathcal{E}_{s}$ were:

$\mathcal{E}_{p}=\\{e_{{x}\rightarrow p^{(j)}_{cls}}\ |\ x\in\mathbf{p^{(i)}}\
\textnormal{and}\ i,j\in[1,N_{models}]\\}$
$\mathcal{E}_{s}=\\{e_{{x}\rightarrow s^{(j)}_{cls}}\ |\ x\in\mathbf{s^{(i)}}\
\textnormal{and}\ i,j\in[1,N_{models}]\\}$

To be more detailed, we created directed edges that went from every patch
features to the [CLS] feature, including from the [CLS] itself, for all
pretrained models but not in the reversed direction, as shown in Figure 2. The
reason was that [CLS] was originally introduced as a representation of all
input data, so it would summarize all patch tokens [8, 2, 6]. Therefore, it
would be the node that received information from other nodes in the graph.
This connection structure ensured that HADA can update the [CLS] tokens based
on the patch tokens from all pretrained models in a fine-grained manner.

###  3.3 Graph Neural Network

Graph neural networks (GNN) have witnessed an increase in its popularity over
the past few years, with many GNN structures having been introduced recently
[15, 5, 34, 11, 30, 1]. HADA applied the modified Graph Attention Network
(GATv2), which was recommended to be used as a baseline whenever employing GNN
[1], to fuse the patch features from different pretrained models together to
get the unified [CLS] features. Let $\mathcal{N}_{k}=\\{x\in\mathcal{V}\ |\
e_{{x}\rightarrow k}\in\mathcal{E}\\}$ be the set of neighbor nodes from which
there was an edge connecting to node $k$ in the graph $\mathcal{G}$. GATv2
used a scoring function $se$ to weight every edge indicating the importance of
the neighbor nodes $x$ in $\mathcal{N}_{k}$ before updating the node
$k\in\mathbb{R}^{d}$:

$se(e_{{x}\rightarrow
k})=\mathbf{A}^{\top}\textnormal{LeakyRELU}(\mathbf{W}_{1}x+\mathbf{W}_{2}k])$

where $\mathbf{A}\in\mathbb{R}^{d^{\prime}}$,
$\mathbf{W}_{1}\in\mathbb{R}^{d^{\prime}\times d}$, and
$\mathbf{W}_{2}\in\mathbb{R}^{d^{\prime}\times d}$ were learnable parameters.
These weights were then normalized across all neighbor nodes in
$\mathcal{N}_{k}$ by using a softmax function to get the attention scores:

$\alpha_{e_{{x}\rightarrow k}}=\frac{\textnormal{exp}(se(e_{{x}\rightarrow
k}))}{\sum_{y\in\mathcal{N}_{k}}\textnormal{exp}(se(e_{{y}\rightarrow k}))}$

The updated node $k^{\prime}\in\mathbb{R}^{d^{\prime}}$ was then calculated
based on its neighbors in $\mathcal{N}_{k}$, including $k$ if we add an edge
connect it to itself:

$k^{\prime}=\sigma(\sum_{x\in\mathcal{N}_{k}}\alpha_{e_{{x}\rightarrow
k}}\cdot\mathbf{W}_{1}x)$

where $\sigma$ was an nonlinearity activate function. Furthermore, this GATv2
network could be enlarged by applying a multi-head attention structure and
improved performance [34]. The output now was a concatenation of each head
output, which was similar to Transformer architecture [33]. An extra linear
layer was used at the end to convert these concatenated nodes to the desired
dimensions.

We used distinct GATv2 structures with $H$ attention heads for each modality
in this stage, as illustrated in Figure 2. HADA took the input graphs
$\mathcal{G}_{p}$ and $\mathcal{G}_{s}$ with nodes $\mathcal{V}_{p}$ and
$\mathcal{V}_{s}$ in the vector space of $d_{p}$ and $d_{s}$ dimensions and
updated them to $\mathcal{V^{\prime}}_{p}=\\{\mathbf{p^{\prime(i)}}\\}$ and
$\mathcal{V^{\prime}}_{s}=\\{\mathbf{s^{\prime(i)}}\\}$ with dimensions of
$d^{\prime}_{p}$ and $d^{\prime}_{s}$. We then concatenated the updated [CLS]
nodes $p^{\prime}_{cls}$ and $s^{\prime}_{cls}$ from all pretrained models
with their corresponding original global embedding $h_{v}$ and $h_{w}$.
Finally, we fed them into a list of linear layers to get our normalized global
representation $h_{p}\in\mathbb{R}^{d_{h}}$ and $h_{s}\in\mathbb{R}^{d_{h}}$.

###  3.4 Training Tasks

####  3.4.1 Image-Text Contrastive Learning.

HADA encoded the input image I and text T to $h_{p}$ and $h_{s}$, accordingly.
We used a similarity function that was a dot product
$S(\textnormal{{{I}}},\textnormal{{{T}}})=\langle
h_{p},h_{s}\rangle=h^{\top}_{p}h_{s}$ to ensure that a pair of relevant image-
text (positive pair) would have a higher similar representation compared to
irrelevant pairs (negative pairs). The contrastive loss for image-to-text
(i2t) retrieval and text-to-image (t2i) retrieval for the mini-batch of $M$
relevant pairs $(\textnormal{{{I}}}_{m},\textnormal{{{T}}}_{m})$ were:

$\mathcal{L}_{i2t}(\textnormal{{{I}}}_{m})=-\textnormal{log}\frac{\textnormal{exp}(S(\textnormal{{{I}}}_{m},\textnormal{{{T}}}_{m})/\tau)}{\sum^{M}_{i=1}\textnormal{exp}(S(\textnormal{{{I}}}_{m},\textnormal{{{T}}}_{i})/\tau)}$
$\mathcal{L}_{t2i}(\textnormal{{{T}}}_{m})=-\textnormal{log}\frac{\textnormal{exp}(S(\textnormal{{{T}}}_{m},\textnormal{{{I}}}_{m})/\tau)}{\sum^{M}_{i=1}\textnormal{exp}(S(\textnormal{{{T}}}_{m},\textnormal{{{I}}}_{i})/\tau)}$

where $\tau$ was a temperature parameter that could be learned during
training. This contrastive learning has been used in many vision-and-language
models and has been proven to be effective [18, 31, 17, 28]. In our
experiment, we trained HADA with the loss that optimized both subtasks:

$\mathcal{L}_{ITC}=\frac{1}{M}\sum^{M}_{m=1}(\mathcal{L}_{i2t}(\textnormal{{{I}}}_{m})+\mathcal{L}_{t2i}(\textnormal{{{T}}}_{m}))$

Inspired by ALBEF [18], we also applied momentum contrast (MoCo) [12] and
their momentum distillation strategy for this unsupervised representation
learning to cope with the problem of noisy information in the dataset and
improve accuracy.

####  3.4.2 Image-Text Matching

This objective was a binary classification task to distinguish irrelevant
image-text pairs, but were similar representations. This task would ensure
that they were different in fine-grained details. We implemented an additional
disciminator layer $dc:\mathbb{R}^{4d_{h}}\rightarrow\mathbb{R}$ on top of the
final embedding features $h_{p}$ and $h_{s}$ to classify whether the image I
and the text T is a positive pair or not:

$dc(h_{p},h_{s})=\textnormal{sigmoid}(\mathbf{W}^{\top}[h_{p}\|h_{s}\|\textnormal{abs}(h_{p}-h_{s})\|h_{p}\odot
h_{s}])$

where $\mathbf{W}\in\mathbb{R}^{4d_{h}}$ was trainable parameters, $\|$
indicated the concatenation, abs(.) was the absolute value, and $\odot$
denoted elementwise multiplication. We used binary cross-entropy loss for this
ordinary classification task:

$\mathcal{L}_{itm}(\textnormal{{{I}}},\textnormal{{{T}}})=y\textnormal{log}(dc(h_{p},d_{s}))+(1-y)\textnormal{log}(1-dc(h_{p},d_{s}))$

where $y$ was the one-hot vector representing the ground truth label of the
pair.

For each positive pair in the minibatch of $M$ positive pairs, we sampled 1
hard negative text for the image and 1 hard negative image for the text. These
negative samples were chosen from the current mini-batch in which they were
not relevant based on the ground-truth labels, but have the highest similarity
dot product score. Therefore, the objective for this task was:

$\mathcal{L}_{ITM}=\frac{1}{3M}\sum^{M}_{m=1}(\mathcal{L}_{itm}(\textnormal{{{I}}}_{m},\textnormal{{{T}}}_{m})+\mathcal{L}_{itm}(\textnormal{{{I}}}_{m},\textnormal{{{T}}}^{\prime}_{m})+\mathcal{L}_{itm}(\textnormal{{{I}}}^{\prime}_{m},\textnormal{{{T}}}_{m}))$

where $\textnormal{{{T}}}^{\prime}_{m}$ and $\textnormal{{{I}}}^{\prime}_{m}$
were the hard negative text and image samples in the mini-batch that were
corresponding with the $\textnormal{{{I}}}_{m}$ and $\textnormal{{{T}}}_{m}$,
respectively. The final loss function in HADA was:

$\mathcal{L}=\mathcal{L}_{ITC}+\mathcal{L}_{ITM}$

##  4 Experiment

###  4.1 Dataset and Evaluation Metrics

We trained and evaluated HADA on 2 different common datasets in the image-text
retrieval task which are Flickr30k [37] and MSCOCO [20]. Flickr30k dataset
consists of 31K images collected on the Flickr website, while MSCOCO comprises
123K images. Each image contains 5 relevant texts or captions that describe
the image. We used Karpathy’s split [14], which has been widely applied by all
models in the image-text retrieval task, to split each dataset into
train/evaluate/test on 29K/1K/1K and 113K/5K/5K images on Flickr30k and
MSCOCO, respectively.

The common evaluation metric in this task is the Recall at K ($R@K$) when many
SOTAs used this metric [18, 31, 17, 28, 13, 19, 3, 10]. This metric is defined
as the proportion of the number of queries that we found the correct relevant
output in the top K of the retrieved ranked list:

$R@K=\frac{1}{N_{q}}\sum^{N_{q}}_{q=1}\textbf{1}(q,K)$

where $N_{q}$ is the number of queries and $\textbf{1}(q,K)$ is a binary
function returning 1 if the model find the correct answer of the query $q$ in
the top $K$ of the retrieved output. In particular, for the image-to-text
subtask, $R@K$ is the percentage of the number of images where we found
relevant texts in the top K of the output result. In our experiment, we used
R@1, R@5, R@10, and RSum which was the sum of them.

###  4.2 Implementation Details

In our experiment, we combined 2 SOTA models that had available pretrained
weights fine-tuned on the Flickr30k dataset:
ALBEF111https://github.com/salesforce/ALBEF and
LightningDOT222https://github.com/intersun/LightningDOT. None of them used the
cross-modality transformer structure when retrieved to ensure the fast
inference speed333Indeed, these 2 models applied the cross-modality
transformer network to rerank the initial result in the subsequent step.
However, we did not focus on this stage.. Although they used the same BERT
architecture to encode a text, the former model employed the ViT network to
encode an image, while the latter model applied the Faster-RCNN model. We
chose these 2 models because we wanted to combine different models with
distinct embedding backbones to utilize the advantages of each of them.

Regarding ALBEF, their ViT network encoded an image to $577$ patch tokens
including the [CLS] one ($N_{ALB}=576$ and $d^{(ALB)}_{v}=768$). This [CLS]
was projected to the lower dimension to obtain the global feature
($d^{(ALB)}_{h}=256$). Because LightningDOT encoded an image based on the
detected objects produced by the Faster-RCNN model, its $N_{DOT}$ varied
depending on the number of objects in the image. The graph neural network,
unlike other conventional CNN, can address this inconsistent number of inputs
due to the flexible graph structure with nodes and edges. Unlike ALBEF, the
dimensions of image features and global features from LightningDOT were the
same with $d^{(DOT)}_{v}=d^{(DOT)}_{h}=768$. In terms of text encoder, the
output of both models was similar since they used the same BERT network:
$d^{(ALB)}_{w}=d^{(DOT)}_{w}=768$. We projected these features to a latent
space where $d_{p}=d_{s}=512$, which were the average of their original
dimensions. We used a 1-layer GATv2 network with $H=4$ multi-head attentions
to update the graph features while still keeping the input dimensions of
$d^{\prime}_{p}=d^{\prime}_{s}=512$. We also applied Dropout with $p=0.7$ in
linear layers and graph neural networks. In total, our HADA contained roughly
10M trainable parameters.

The input pretrained models were pretrained on several large external
datasets. For example, ALBEF was pretrained on 14M images compared to only 29K
images on Flickr30k that we used to train HADA. We used this advantage in our
prediction instead of train HADA in millions of samples. We modified the
similarity score to a weighted sum of our predictions and the original
prediction of the input models. Therefore, the weighted similarity score that
we used was:

$S(\textnormal{{{I}}},\textnormal{{{T}}})=(1-\alpha)\langle
h_{p},h_{s}\rangle+\alpha\langle h^{(ALB)}_{v},h^{(ALB)}_{w}\rangle$

where $\alpha$ was a trainable parameter. We did not include the original
result of the LightningDOT model, since its result was lower than ALBEF by a
large margin and therefore could have a negative impact on overall
performance444We tried including the LightningDOT in the weighted similarity
score, but the result was lower than using only ALBEF..

We trained HADA for 50 epochs (early stopping555In our experiment, it
converged after roughly 20 epochs. was implemented) using the batch size of 20
on 1 NVIDIA RTX3080Ti GPU. We used the AdamW [23] optimizer with a weight
decay of 0.02. The learning rate was set at $1e^{-4}$ and decayed to $5e^{-6}$
following cosine annealing [22]. Similarly to ALBEF, we also applied
RandAugment [4] for data augmentation. The initial temperature parameter was
$0.07$ [36] and we kept it in range of $[0.001,0.5]$ during training. To
mitigate the dominant effect of ALBEF global features on our weighted
similarity score, we first trained HADA with $\alpha=0$. After the model had
converged, we continued to train, but initially set $\alpha=0.5$ and kept it
in the range of $[0.1,0.9]$.

###  4.3 Baselines

We built 2 baselines that also integrated ALBEF and LightningDOT as an input
to show the advantages of using graph structures to fuse these input models.

####  4.3.1 Baseline B1.

We calculated the average of the original ranking results obtained from ALBEF
and LightningDOT and considered them as the distance between images and text.
This meant that the relevant pairs should be ranked at the top, whilst
irrelevant pairs would have lower places.

####  4.3.2 Baseline B2.

Instead of using a graph structure to fuse the features extracted from the
pretrained models, we only concatenated their global embedding and fed them
into the last linear layers to obtain the unified features. We trained this
baseline B2 following the same strategy as described in Section 4.2 using the
weighted similarity score.

###  4.4 Comparison to Baseline

Table 1 illustrated the evaluation metrics of the difference models in the
Flickr30k dataset. Similarly to LightningDOT, our main target was to introduce
an image-text retrieval model that did not implement a cross-modality
transformer module to ensure that it can perform in real-time without any
delay. Thus, we only reported the result from LightningDOT and ALBEF that did
not use the time-consuming compartment to rerank in the subsequent step. If
the model has a better initial result, it can have a better reranked result by
using the cross-modality transformer later. We also added UNITER [3] and VILLA
[10], which both used cross-modality transformer architecture to make the
prediction, to the comparison.

Table 1: Performance of models on Flickr30k Dataset. The symbol ✝ indicated the results were originally reported in their research, while others were from our re-implementation using their public pretrained checkpoints. The column $\vartriangle$R showed the difference compared to ALBEF. Methods | Image-to-Text | Text-to-Image | Total | $\vartriangle$R
---|---|---|---|---
R@1 | R@5 | R@10 | RSum | R@1 | R@5 | R@10 | RSum | RSum
$\textnormal{UNITER}^{\textnormal{\char 61}}$ | 87.3 | 98 | 99.2 | 284.5 | 75.56 | 94.08 | 96.76 | 266.4 | 550.9 |  $\shortdownarrow$13.68
$\textnormal{VILLA}^{\textnormal{\char 61}}$ | 87.9 | 97.2 | 98.8 | 283.9 | 76.26 | 94.24 | 96.84 | 267.34 | 551.24 |  $\shortdownarrow$13.34
LightningDOT | 83.6 | 96 | 98.2 | 277.8 | 69.2 | 90.72 | 94.54 | 254.46 | 532.26 |  $\shortdownarrow$32.32
$\textnormal{LightningDOT}^{\textnormal{\char 61}}$ | 83.9 | 97.2 | 98.6 | 279.7 | 69.9 | 91.1 | 95.2 | 256.2 | 535.9 |  $\shortdownarrow$28.68
ALBEF | 92.6 | 99.3 | 99.9 | 291.8 | 79.76 | 95.3 | 97.72 | 272.78 | 564.58 | 0
B1 | 90.7 | 99 | 99.6 | 289.3 | 79.08 | 94.5 | 96.94 | 270.52 | 559.82 |  $\shortdownarrow$4.76
B2 | 91.4 | 99.5 | 99.7 | 290.6 | 79.64 | 95.34 | 97.46 | 272.44 | 563.04 |  $\shortdownarrow$1.54
HADA | 93.3 | 99.6 | 100 | 292.9 | 81.36 | 95.94 | 98.02 | 275.32 | 568.22 |  $\shortuparrow$3.64

It was clearly that our HADA obtained the highest metrics at all recall values
compared to others. HADA achieved a slightly better R@5 and R@10 in Image-to-
Text (I2T) and Text-to-Image (T2I) subtasks than ALBEF. However, the gap
became more significant at R@1. We improved the R@1 of I2T by $0.7\%$
($92.96\to 93.3$) and the R@1 of T2I by $1.6\%$ ($79.76\to 81.36$). In total,
our RSum was $3.64\%$ higher than that of ALBEF ($564.58\to 568.22$).

The experiment also showed that LightningDOT, which encoded images using
Faster-RCNN, was much behind ALBEF when its total RSum was lower than that of
ALBEF by approximately $30\%$. The reason might be that the object detector
was not as powerful as the ViT network and LightningDOT was pretrained on 4M
images compared to 14M images used to train ALBEF. Although also using object
detectors as the backbone but applying a cross-modality network, UNITER and
VILLA surpassed LightningDOT by a large margin at $15\%$. It proved that this
intensive architecture made the large impact on the multimodal retrieval.

Regarding our 2 baselines B1 and B2, both of them were failed to get better
results than the input model ALBEF. Model B1, with the simple strategy of
taking the average ranking results and having no learnable parameters,
performed worse than model B2 which used a trainable linear layer to fuse the
pretrained features. Nevertheless, the RSum of B2 was lower than HADA by
$5.18\%$. It showed the advantages of using graph structure to fuse the
information between models to obtain the better result.

###  4.5 Ablation Study

To show the stable performance of HADA, we used it to combine 2 other
different pretrained models, including BLIP [17] and CLIP [28]. While CLIP is
well-known for its application in many retrieval challenges [24, 32, 9, 31],
BLIP is the enhanced version of ALBEF with the bootstrapping technique in the
training process. We used the same configuration as described in 4.2 to train
and evaluate HADA in Flickr30k and MSCOCO datasets. We used the pretrained
BLIP and CLIP from LAVIS library [16]. It was noted that the CLIP we used in
this experiment was the zero-shot model, since the fine-tuned CLIP for these
datasets is not available yet.

Table 2: Performance of models on the test set in Flickr30k and MSCOCO datasets. The column $\vartriangle$R showed the difference compared to BLIP in that dataset. Dataset | Methods | Image-to-Text | Text-to-Image | Total | $\vartriangle$R
---|---|---|---|---|---
R@1 | R@5 | R@10 | RSum | R@1 | R@5 | R@10 | RSum | RSum
Flickr30k | BLIP | 94.3 | 99.5 | 99.9 | 293.7 | 83.54 | 96.66 | 98.32 | 278.52 | 572.22 | 0
CLIP | 88 | 98.7 | 99.4 | 286.1 | 68.7 | 90.6 | 95.2 | 254.5 | 540.6 |  $\shortdownarrow$31.62
HADA | 95.2 | 99.7 | 100 | 294.9 | 85.3 | 97.24 | 98.72 | 281.26 | 576.16 |  $\shortuparrow$3.94
MSCOCO | BLIP | 75.76 | 93.8 | 96.62 | 266.18 | 57.32 | 81.84 | 88.92 | 228.08 | 494.26 | 0
CLIP | 57.84 | 81.22 | 87.78 | 226.84 | 37.02 | 61.66 | 71.5 | 170.18 | 397.02 |  $\shortdownarrow$97.24
HADA | 75.36 | 92.98 | 96.44 | 264.78 | 58.46 | 82.85 | 89.66 | 230.97 | 495.75 |  $\shortuparrow$1.49

Table 2 showed the comparison between HADA and the input models. CLIP
performed worst on both Flickr30k and MSCOCO with huge differences compared to
BLIP and HADA because CLIP was not fine-tuned for these datasets. Regarding
Flickr30k dataset, HADA managed to improve the RSum by more than $3.9\%$
compared to that of BLIP. Additionally, HADA obtained the highest scores in
all metrics for both subtasks. Our proposed framework also increased the RSum
of BLIP by $1.49\%$ in MSCOCO dataset. However, BLIP performed slightly better
HADA in the I2T subtask while HADA achieved higher performance in the T2I
subtask.

##  5 Conclusion

In this research, we proposed a simple graph-based framework, called HADA, to
combine 2 pretrained models to address the image-text retrieval problem. We
created a graph structure to fuse the extracted features obtained from the
pretrained models, followed by the GATv2 network to update them. Our proposed
HADA only contained roughly 10M learnable parameters, helping it become easy
to train using only 1 GPUs. Our experiments showed the promisingness of the
proposed method. Compared to input models, we managed to increase total recall
by more than $3.6\%$. Additionally, we implemented other 2 simple baselines to
show the advantage of using the graph structures. This result helped us
resolve 2 questions: (1) increase the performance of SOTA models in image-text
retrieval task and (2) not requiring many GPUs to train on any large-scale
external dataset. It has opened the possibility of applying HADA in industry
where many small and medium start-ups do not possess many GPUs.

Although we achieved the better result compared to the baselines, there are
still rooms to improve the performance of HADA. Firstly, it can be extended
not only by 2 pretrained models as proposed in this research, but can be used
with more than that number. Secondly, the use of different graph neural
networks, such as the graph transformer [30], can be investigated in future
work. Third, the edge feature in the graph is also considered. Currently, HADA
did not implement the edge feature in our experiment, but they can be
learnable parameters in graph neural networks. Last but not least, pretraining
HADA on a large-scale external dataset as other SOTA have done might enhance
its performance.

##  6 Acknowledgement

This publication has emanated from research supported in party by research
grants from Science Foundation Ireland under grant numbers SFI/12/RC/2289,
SFI/13/RC/2106, and 18/CRT/6223.

## References

  * [1] Brody, S., Alon, U., Yahav, E.: How attentive are graph attention networks? arXiv preprint arXiv:2105.14491 (2021)
  * [2] Chen, C.F.R., Fan, Q., Panda, R.: Crossvit: Cross-attention multi-scale vision transformer for image classification. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 357–366 (2021)
  * [3] Chen, Y.C., Li, L., Yu, L., El Kholy, A., Ahmed, F., Gan, Z., Cheng, Y., Liu, J.: Uniter: Universal image-text representation learning. In: European conference on computer vision. pp. 104–120. Springer (2020)
  * [4] Cubuk, E.D., Zoph, B., Shlens, J., Le, Q.V.: Randaugment: Practical automated data augmentation with a reduced search space. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. pp. 702–703 (2020)
  * [5] Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional neural networks on graphs with fast localized spectral filtering. Advances in neural information processing systems 29 (2016)
  * [6] Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
  * [7] Diao, H., Zhang, Y., Ma, L., Lu, H.: Similarity reasoning and filtration for image-text matching. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 35, pp. 1218–1226 (2021)
  * [8] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
  * [9] Dzabraev, M., Kalashnikov, M., Komkov, S., Petiushko, A.: Mdmmt: Multidomain multimodal transformer for video retrieval. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3354–3363 (2021)
  * [10] Gan, Z., Chen, Y.C., Li, L., Zhu, C., Cheng, Y., Liu, J.: Large-scale adversarial training for vision-and-language representation learning. Advances in Neural Information Processing Systems 33, 6616–6628 (2020)
  * [11] Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. Advances in neural information processing systems 30 (2017)
  * [12] He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 9729–9738 (2020)
  * [13] Jia, C., Yang, Y., Xia, Y., Chen, Y.T., Parekh, Z., Pham, H., Le, Q., Sung, Y.H., Li, Z., Duerig, T.: Scaling up visual and vision-language representation learning with noisy text supervision. In: International Conference on Machine Learning. pp. 4904–4916. PMLR (2021)
  * [14] Karpathy, A., Fei-Fei, L.: Deep visual-semantic alignments for generating image descriptions. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3128–3137 (2015)
  * [15] Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
  * [16] Li, D., Li, J., Le, H., Wang, G., Savarese, S., Hoi, S.C.H.: Lavis: A library for language-vision intelligence. arXiv preprint arXiv:2209.09019 (2022)
  * [17] Li, J., Li, D., Xiong, C., Hoi, S.: Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. arXiv preprint arXiv:2201.12086 (2022)
  * [18] Li, J., Selvaraju, R., Gotmare, A., Joty, S., Xiong, C., Hoi, S.C.H.: Align before fuse: Vision and language representation learning with momentum distillation. Advances in neural information processing systems 34, 9694–9705 (2021)
  * [19] Li, X., Yin, X., Li, C., Zhang, P., Hu, X., Zhang, L., Wang, L., Hu, H., Dong, L., Wei, F., et al.: Oscar: Object-semantics aligned pre-training for vision-language tasks. In: European Conference on Computer Vision. pp. 121–137. Springer (2020)
  * [20] Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: European conference on computer vision. pp. 740–755. Springer (2014)
  * [21] Liu, C., Mao, Z., Zhang, T., Xie, H., Wang, B., Zhang, Y.: Graph structured network for image-text matching. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 10921–10930 (2020)
  * [22] Loshchilov, I., Hutter, F.: Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983 (2016)
  * [23] Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)
  * [24] Luo, H., Ji, L., Zhong, M., Chen, Y., Lei, W., Duan, N., Li, T.: Clip4clip: An empirical study of clip for end to end video clip retrieval. arXiv preprint arXiv:2104.08860 (2021)
  * [25] Miech, A., Alayrac, J.B., Laptev, I., Sivic, J., Zisserman, A.: Thinking fast and slow: Efficient text-to-visual retrieval with transformers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9826–9836 (2021)
  * [26] Nguyen, M.D., Nguyen, B.T., Gurrin, C.: A deep local and global scene-graph matching for image-text retrieval. arXiv preprint arXiv:2106.02400 (2021)
  * [27] Nguyen, M.D., Nguyen, B.T., Gurrin, C.: Graph-based indexing and retrieval of lifelog data. In: International Conference on Multimedia Modeling. pp. 256–267. Springer (2021)
  * [28] Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International Conference on Machine Learning. pp. 8748–8763. PMLR (2021)
  * [29] Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems 28 (2015)
  * [30] Shi, Y., Huang, Z., Feng, S., Zhong, H., Wang, W., Sun, Y.: Masked label prediction: Unified message passing model for semi-supervised classification. arXiv preprint arXiv:2009.03509 (2020)
  * [31] Sun, S., Chen, Y.C., Li, L., Wang, S., Fang, Y., Liu, J.: Lightningdot: Pre-training visual-semantic embeddings for real-time image-text retrieval. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. pp. 982–997 (2021)
  * [32] Tran, L.D., Nguyen, M.D., Nguyen, B., Lee, H., Zhou, L., Gurrin, C.: E-myscéal: Embedding-based interactive lifelog retrieval system for lsc’22. In: Proceedings of the 5th Annual on Lifelog Search Challenge, pp. 32–37 (2022)
  * [33] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017)
  * [34] Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)
  * [35] Wang, S., Wang, R., Yao, Z., Shan, S., Chen, X.: Cross-modal scene graph matching for relationship-aware image-text retrieval. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision. pp. 1508–1517 (2020)
  * [36] Wu, Z., Xiong, Y., Yu, S.X., Lin, D.: Unsupervised feature learning via non-parametric instance discrimination. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3733–3742 (2018)
  * [37] Young, P., Lai, A., Hodosh, M., Hockenmaier, J.: From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics 2, 67–78 (2014)


# Infrared renormalon effects in color dipole TMD PDF at small-$x$

Nahid Vasim<EMAIL_ADDRESS>Raktim Abir<EMAIL_ADDRESS>Department of
Physics, Aligarh Muslim University, Aligarh - $202002$, India.

###### Abstract

The uncertainties from the infrared renormalons in the (color dipole) gluon
distribution is estimated. It is shown that non-linear saturation effects at
small-$x$ shift the first IR pole at the Borel plane from $2/\beta_{2}$ to
$1/\beta_{2}$. As a consequence the estimated uncertainty is found to be
${\cal O}\left(\Lambda_{\rm QCD}^{2}\right)$ instead of ${\cal
O}\left(\Lambda_{\rm QCD}^{4}\right)$.

###### pacs:

12.38.-t,12.38.Aw

##  I Introduction

Perturbation series in quantum field theory are usually divergent even after
term by term renormalization of mass and charge. This was first argued by
Freeman Dyson in early fifties on the eve of developement of quantum
electrodynamics (QED) Dyson:1952tj . In case of QED it was shown that
individual terms of the perturbation series $\sum_{n}{\cal
C}_{n}\alpha_{s}^{n}$ first decrease and gradually approaches to a minimum and
then start to increase without any limit. The index of the minimum term
estimated to be of the order $137$. The ever increasing terms make the
perturbation series divergent. However fortunately this no way restricts
predictions from the perturbation series for practical applications.
Nevertheless this anomaly raises important questions on very foundation upon
which the theory is built. B. Lautrap was first who point out that there could
be single gauge invariant diagram that contribute to amplitude to grow it like
$n!$ in the $n$-th order term of the perturbation series Lautrup:1977hs
making it a divergent series. Not just in QED, this subtlety also translates
to QCD, where strong coupling acts as the expansion parameter of the series.
Some non-perturbative effects in QCD are actually stems from the asymptotic
nature of the perturbation series and corroborated by the fact that the
coefficients ${\cal C}_{n}$ often have factorial-growth with $n$. There are
resummation procedure to regularize divergence series $e.g.$ Borel
resummation. For factorially divergent series Borel summation is mostly used.
The source of this divergence of perturbation series in QCD is then reflected
through the presence of pole singularities Mueller:1984vh  on the real axis of
the Borel plane leading to (a) Infrared Renormalons (IR), (b) Ultraviolet
Renormalons (UR) and the (c) Instantons Beneke:1998ui . If the Borel integral
has no singularity in the positive real axis and the terms in the series do
not increase faster than the factorial growth, the divergent series is Borel
summable. In QCD infrared renormalons are singularities on the positive
$b$-axis due to the integration over infrared regions and presence of them
spoil the Borel summability of the series. In fact it was argued that this
Borel non-summable contribution predicts a specific high-energy phenomenon
which calls for a growth of multi-jet production with high multiplicity.
Phenomenologically, the phenomena leads to apparently anomalous events that
would look as highly isotropic multi-jet events Zakharov:1992bx .

QCD renormalons in small-$x$ was first addressed by Levin Levin:1994di . In
QCD high energy scattering events generally involve cascade of gluons. This is
because unlike photon primary gluons with high virtuality themselves emit
further secondary gluons in high energy. At high enough energy this rapidly
growing cascade of gluons grow to an extent that fusion of multiple gluons to
single gluon begin. Over the time this fusion of gluons develops a statistical
detailed balance with the reverse phenomena of gluon cascade. This detail
balance of gluon number density leads to the origin of the phenomena of gluon
saturation with the emergence of a dynamically generated and energy dependent
momentum scale $Q_{s}$ known as saturation scale. Above this saturation scale
one may safely assumes independent stochastic multiple scattering
approximations but below this scale this is no longer valid. Highly correlated
non-linear gluon interactions prevail over normally distributed random
interactions below the saturation scale. In the small momenta regions, within
the window of momentum scale $\Lambda_{\rm QCD}$ and saturation scale $Q_{s}$
where the running coupling becomes large, infrared renormalons is believed to
be the source of the divergence in the perturbation series and gives estimates
to the uncertainty due to non-perturbative effects. In this article we revisit
the estimation of the uncertainties from the infrared renormalons in the
(color dipole) gluon distribution employing our recently derived expression
that is valid in small transverse momentum region Abir:2018hvk . We have shown
that non-linear saturation effects at small-$x$ shift the first IR pole at the
Borel plane towards zero from $2/\beta_{2}$ to $1/\beta_{2}$ where $\beta_{2}$
is beta function of QCD. This leads to enhanced non-perturbative uncertainty
${\cal O}\left(\Lambda_{\rm QCD}^{2}\right)$ instead of ${\cal
O}\left(\Lambda_{\rm QCD}^{4}\right)$ for the (color dipole) gluon
distribution.

##  II Infrared renormalon in (color dipole) gluon distribution

We start with the relation between dipole amplitude ${\cal N}$ and the
unintegrated dipole gluon distribution ${\cal F}$ Kovchegov_book

$\displaystyle\int d^{2}b~{}{\cal
N}(r_{\perp},b_{\perp},x)=\frac{2\pi}{N_{c}}\int
d^{2}k_{\perp}\left(1-e^{ik_{\perp}.r_{\perp}}\right)\alpha_{s}(k_{\perp}^{2})\frac{1}{k_{\perp}^{2}}{\cal
F}\left(x,k_{\perp}\right),$ (1)

Now at or around saturation scale dipole distribution ${\cal F}$ often taken
to be a $Gaussian\otimes quadratic$ distribution of transverse momentum

$\displaystyle{\cal F}(x,k_{\perp})$ $\displaystyle\propto$
$\displaystyle~{}\frac{k_{\perp}^{2}}{Q_{s}^{2}(x)}~{}\exp\left[{-\frac{k_{\perp}^{2}}{Q_{s}^{2}(x)}}\right],$
(2) $\displaystyle\approx$
$\displaystyle\frac{k_{\perp}^{2}}{Q_{s}^{2}(x)}~{}~{}~{}~{}~{}({\rm
when}~{}k_{\perp}^{2}\ll Q_{s}^{2}(x)).$ (3)

Eq.(2) can be readily derived from the dipole $S$-matrix that is Gaussian
($e.g$ in GGM, MV or in phenomenological GBW model) in scaling variable $t$
$(\equiv r_{\perp}Q_{s})$

$\displaystyle S(r_{\perp},x)=\exp\left(-\kappa
r_{\perp}^{2}Q_{s}^{2}(x)\right)$ (4)

where $\kappa\approx 1/4$ is a constant. Strictly speaking this is valid
around the saturation region and not deep inside where multiple scatterings
are highly correlated and can’t be taken just as collection of random
independent scatterings. Nonetheless the expression often taken to extrapolate
it in the low momentum limit and leads to the conclusion that the unintegrated
dipole gluon distribution behaves as $k_{\perp}^{2}/Q_{s}^{2}(x)$ in the low
momentum. The relation in Eq.(1) between dipole amplitude and the unintegrated
dipole gluon distribution follows from the two-gluon exchange. While it may
not be strictly valid in the highly nonlinear regime nonetheless we will take
Eq.(3) to estimate the contribution to the dipole amplitude from the
saturation region with $k_{\perp}<Q_{s}(x)$. In the kinematic region where the
gluon distribution varies as $\phi\propto k_{\perp}^{2}/Q_{s}^{2}$, the
contribution to dipole amplitude from the saturation region is proportional to

$\displaystyle\int\frac{d^{2}k_{\perp}}{k_{\perp}^{2}}\left(1-e^{ik_{\perp}.x_{\perp}}\right)\alpha_{s}(k_{\perp}^{2})\frac{k_{\perp}^{2}}{Q_{s}^{2}}~{}.$
(5)

We now introduce the renormalization scale $\mu$ writing,

$\displaystyle\alpha_{s}\left(k_{\perp}^{2}\right)=\frac{\alpha(\mu^{2})}{1+\alpha(\mu^{2})\beta_{2}\ln\left(k_{\perp}^{2}/\mu^{2}\right)}.$
(6)

We substitute this in Eq.(5) and expand in power of $\alpha_{s}(\mu)$, to
obtain,

$\displaystyle\frac{r_{\perp}^{2}}{Q_{s}^{2}}\alpha_{s}(\mu)\sum_{n=0}^{\infty}\left(-\alpha(\mu)\beta_{2}\right)^{2}\int_{0}^{Q_{s}^{2}}dk_{\perp}^{2}k_{\perp}^{2}\ln^{n}\frac{k_{\perp}^{2}}{\mu^{2}}$
(7)

Define $\zeta\equiv\ln\left(\mu^{2}/k_{\perp}^{2}\right)$ we rewrite

$\displaystyle\frac{r_{\perp}^{2}}{Q_{s}^{2}}\mu^{4}\alpha_{s}(\mu^{2})\sum_{n=0}^{\infty}\left(\alpha(\mu)\beta_{2}\right)^{2}\int_{\ln\left(\mu^{2}/Q_{s}^{2}\right)}^{\infty}d\zeta~{}\zeta^{n}~{}e^{-2\zeta}$
(8)

The expression contains an incomplete gamma integral for $\zeta$ and results
in a diverging series

$\displaystyle\frac{r_{\perp}^{2}}{2Q_{s}^{2}}\mu^{4}\alpha\left({\mu}\right)\sum_{n=0}^{\infty}\left(\frac{\alpha(\mu)\beta_{2}}{2}\right)^{n}~{}n!~{}{\cal
C}$ (9)

where ${\cal C}$ is just a correction factor mimics the deviation from
complete gamma function,

$\displaystyle{\cal
C}=1-\frac{Q_{s}}{\mu}\sum_{k=n+1}^{\infty}\frac{1}{k!}\ln^{k}\frac{\mu^{2}}{Q_{s}^{2}}.$
(10)

This clearly shows for large enough $n$ the integral is dominated by upper
limit so that its lower limit can be set equal to zero or otherwise ${\cal C}$
can be approximate to unity leaving only the rising factor proportional to
$n!$. This is the typical effect of infrared QCD renormalons. Divergent series
with rising factorial in the coefficients are best treated using Borel
summation techniques. Namely we rewrite the series as

$\displaystyle-\frac{2}{\beta_{2}}\frac{r_{\perp}^{2}}{2Q_{s}^{2}}\mu^{4}\int_{0}^{\infty}db~{}e^{-b/\alpha(\mu^{2})}\frac{1}{b-2/\beta_{2}}$
(11)

The pole at $b=2/\beta_{2}$ is known as the IR renormalon pole in the complex
$b$-plane. The $b$-integral is divergent as the renormalon pole is on the
positive real axis and hence the series is not Borel resummable. At best one
may estimate the size of the IR renormalon uncertainty by simply taking the
residue of the renormalon pole, which gives,

$\displaystyle\sim\frac{r_{\perp}^{2}}{Q_{s}^{2}}\mu^{4}\exp\left(-\frac{2}{\alpha_{s}(\mu^{2})\beta_{2}}\right)$
(12)

or in terms of $\Lambda_{QCD}$,

$\displaystyle\sim\frac{r_{\perp}^{2}}{Q_{s}^{2}}\mu^{4}\exp\left(-\frac{2}{\alpha_{s}(\mu^{2})\beta_{2}}\right)=\frac{r_{\perp}^{2}}{Q_{s}^{2}}\Lambda^{4}_{QCD}$
(13)

Non-perturbative origin of the uncertainty reflects from the fact that the
result is proportional to $\Lambda_{QCD}^{4}$.

##  III Infrared renormalon revisited

Recently we derive an analytical expression for unintegrated color dipole
gluon distribution at asymptotically small transverse momentum. As small
transverse momentum corresponds to large transverse separation, the new
expression is derived by performing the Fourier transform of $S$-matrix for
large dipoles with large transverse separations. In particular we take Levin-
Tuchin solution of Balitsky-Kovchegov equation to get the results. Though the
result found out to be in the form of a series of Bells polynomials, when
resumming in leading log accuracy, showing up Sudakov like soft factor
Abir:2018hvk

$\displaystyle{\cal
F}(x,k_{\perp})\propto\ln\left(\frac{k_{\perp}^{2}}{4Q_{s}^{2}}\right)\exp\left[-\tau\ln^{2}\left(\frac{k_{\perp}^{2}}{4Q_{s}^{2}}\right)\right]$
(14)

where $\tau\approx 0.2$ is a constant. Its interesting to note that the
small-$x$ evolution kills the $k_{\perp}^{2}/Q_{s}^{2}$ behaviour from Eq.(2)
to Eq.(3) in the small momentum limit and modifies it from $linear\otimes
normal$ to $logarithmic\otimes log~{}normal$ as given in Eq.(14). In the
following we take Eq.(14) to estimate the associated uncertainties stemming
from IR renormalon to have a comparative study with $e.g.$ Eq.(13). We start
by writing

$\displaystyle\int^{Q_{s}^{2}}\frac{d^{2}k_{\perp}}{k_{\perp}^{2}}\left(1-e^{ik_{\perp}.r_{\perp}}\right)\alpha_{s}(k_{\perp}^{2})~{}\ln\left(\frac{k_{\perp}^{2}}{4Q_{s}^{2}}\right)\exp\left[{-\tau\ln^{2}\left(\frac{k_{\perp}^{2}}{4Q_{s}^{2}}\right)}\right]$
(15)

We expand the exponentials (with $k_{\perp}x_{\perp}\ll 1$) and performing
angular integration to get

$\displaystyle
r_{\perp}^{2}\int^{Q_{s}^{2}}{dk_{\perp}^{2}}~{}\alpha_{s}(k_{\perp}^{2})~{}~{}\ln\left(\frac{k_{\perp}^{2}}{4Q_{s}^{2}}\right)\sum_{m=0}^{\infty}{\frac{(-\tau)^{m}}{m!}\ln^{2m}\left(\frac{k_{\perp}^{2}}{4Q_{s}^{2}}\right)}$
(16)

We now simplify by expressing the $m$-th term as $(2m+1)$-th derivative of
dummy variable $\epsilon$ in the limit $\epsilon\rightarrow 0$,

$\displaystyle
r_{\perp}^{2}\sum_{m=0}^{\infty}{\frac{(-\tau)^{m}}{m!}}\int^{Q_{s}^{2}}{dk_{\perp}^{2}}~{}\alpha_{s}(k_{\perp}^{2})~{}\lim_{\epsilon\rightarrow
0}\frac{\partial^{2m+1}}{\partial\epsilon^{2m+1}}{\left(\frac{k_{\perp}^{2}}{4Q_{s}^{2}}\right)^{\epsilon}}$
(17)

Substituting the effective QCD coupling,

$\displaystyle\alpha_{s}(k_{\perp}^{2})=\frac{\alpha\left({\mu^{2}}\right)}{1+\alpha(\mu^{2})\beta_{2}\ln(k_{\perp}^{2}/\mu^{2})}$
(18)

as an expansion in powers of $\alpha\left(\mu^{2}\right)$

$\displaystyle
r_{\perp}^{2}\sum_{m=0}^{\infty}{\frac{(-\tau)^{m}}{m!}}\lim_{\epsilon\rightarrow
0}\frac{\partial^{2m+1}}{\partial\epsilon^{2m+1}}{\left(\frac{\mu^{2}}{4Q_{s}^{2}}\right)^{\epsilon}}\sum_{n=0}^{\infty}(-1)^{n}\alpha^{n+1}(\mu)\beta^{n}_{2}\int^{Q_{s}^{2}}{dk_{\perp}^{2}}~{}\ln^{n}\left(\frac{k_{\perp}^{2}}{\mu^{2}}\right){\left(\frac{k_{\perp}^{2}}{\mu^{2}}\right)^{\epsilon}}$
(19)

We define $\xi=\ln\left(\mu^{2}/k_{\perp}^{2}\right)$ to perform the
$k_{\perp}$-integration in Eq.(19):

$\displaystyle\int^{Q_{s}^{2}}{dk_{\perp}^{2}}~{}\ln^{n}\left(\frac{k_{\perp}^{2}}{\mu^{2}}\right){\left(\frac{k_{\perp}^{2}}{\mu^{2}}\right)^{\epsilon}}$
$\displaystyle=$
$\displaystyle-\mu^{2}\int^{\ln\left({\mu^{2}}/{Q_{s}^{2}}\right)}_{\infty}{d\xi}~{}e^{-\xi}(-\xi)^{n}e^{-\xi\epsilon}$
$\displaystyle=$
$\displaystyle(-1)^{n}\mu^{2}\int_{\ln\left({\mu^{2}}/{Q_{s}^{2}}\right)}^{\infty}{d\xi}~{}\xi^{n}~{}e^{-\xi(1+\epsilon)}$
$\displaystyle=$
$\displaystyle(-1)^{n}\mu^{2}{\left(\frac{1}{1+\epsilon}\right)}^{n+1}\Gamma({n+1})$

which yields a divergent series

$\displaystyle
r_{\perp}^{2}\sum_{m=0}^{\infty}{\frac{(-\tau)^{m}}{m!}}\lim_{\epsilon\rightarrow
0}\frac{\partial^{2m+1}}{\partial\epsilon^{2m+1}}{\left(\frac{\mu^{2}}{4Q_{s}^{2}}\right)^{\epsilon}}\mu^{2}\sum_{n=0}^{\infty}{\beta^{n}_{2}}~{}\left(\frac{{\alpha(\mu)}}{1+\epsilon}\right)^{n+1}n!$
(20)

with rising factorials in the coefficients, similar to Eq.(9), due to the IR
renormalons. The inner series can be formally rewritten as

$\displaystyle\sum_{n=0}^{\infty}{\beta^{n}_{2}}~{}\left(\frac{{\alpha(\mu)}}{1+\epsilon}\right)^{n+1}n!=\sum_{n=0}^{\infty}\beta_{2}^{n}\int_{0}^{\infty}db~{}b^{n}~{}\exp\left(-\frac{(1+\epsilon)~{}b}{\alpha(\mu)}\right)$
(21)

where the integration variable $b$ can be taken as a complex dummy parameter
over which the integration is to be performed in the complex Borel $b$-plane.
We now introduce Borel summation technique to evaluate the factorial rising
expansion by formally switching the integration and summation

$\displaystyle\sum_{n=0}^{\infty}\beta_{2}^{n}\int_{0}^{\infty}db~{}b^{n}~{}\exp\left(-\frac{(1+\epsilon)~{}b}{\alpha(\mu)}\right)\rightarrow\int_{0}^{\infty}db~{}\exp\left(-\frac{(1+\epsilon)~{}b}{\alpha(\mu)}\right)\sum_{n=0}^{\infty}\beta_{2}^{n}b^{n}$
(22)

Interestingly now the pole of the integral is shifted to $b=1/\beta_{2}$
unlike the previous case where it was at $b=2/\beta_{2}$

$\displaystyle\int_{0}^{\infty}db~{}\exp\left(-\frac{(1+\epsilon)~{}b}{\alpha(\mu)}\right)\sum_{n=0}^{\infty}\beta_{2}^{n}b^{n}=\int_{0}^{\infty}db~{}\exp\left(-\frac{(1+\epsilon)~{}b}{\alpha(\mu)}\right)\frac{1}{1-\beta_{2}b}$
(23)

Taking the residue the $b$-integration in Eq.(23) found to be

$\displaystyle\sim$
$\displaystyle-\frac{1}{\beta_{2}}\exp\left(-\frac{1+\epsilon}{\beta_{2}~{}\alpha(\mu)}\right)$

The overall measure of the uncertainty from the IR pole at $1/\beta_{2}$ is
then

$\displaystyle\sim$
$\displaystyle-\frac{x_{\perp}^{2}\mu^{2}}{\beta_{2}}\sum_{m=0}^{\infty}{\frac{(-\tau)^{m}}{m!}}\lim_{\epsilon\rightarrow
0}\frac{\partial^{2m+1}}{\partial\epsilon^{2m+1}}{\left(\frac{\mu^{2}}{4Q_{s}^{2}}\right)^{\epsilon}}~{}\exp\left(-\frac{1+\epsilon}{\beta_{2}~{}\alpha(\mu)}\right)$
(24)

Next we perform $2m+1$ derivative taking

$\displaystyle t$ $\displaystyle=$
$\displaystyle{\left(\frac{\mu^{2}}{4Q_{s}^{2}}\right)^{\epsilon}}~{}\exp\left(-\frac{1+\epsilon}{\beta_{2}\alpha(\mu)}\right)$
$\displaystyle\ln t$ $\displaystyle=$
$\displaystyle\epsilon\ln\frac{\mu^{2}}{4Q_{s}^{2}}-\frac{1+\epsilon}{\beta_{2}\alpha\left(\mu\right)}$
$\displaystyle\frac{dt}{d\epsilon}$ $\displaystyle=$ $\displaystyle
t\left(\ln\frac{\mu^{2}}{4Q_{s}^{2}}-\frac{1}{\beta_{2}\alpha(\mu)}\right)$
$\displaystyle\frac{d^{2}t}{d\epsilon^{2}}$ $\displaystyle=$ $\displaystyle
t\left(\ln\frac{\mu^{2}}{4Q_{s}^{2}}-\frac{1}{\beta_{2}\alpha(\mu)}\right)^{2}$
$\displaystyle\Rightarrow\frac{d^{2m+1}t}{d\epsilon^{2m+1}}$ $\displaystyle=$
$\displaystyle
t\left(\ln\frac{\mu^{2}}{4Q_{s}^{2}}-\frac{1}{\beta_{2}\alpha(\mu)}\right)^{2m+1}$

After taking the limit $\epsilon\rightarrow 0$, the uncertainty is

$\displaystyle\sim$ $\displaystyle
r_{\perp}^{2}\mu^{2}~{}\exp\left(-\frac{1}{\beta_{2}\alpha(\mu)}\right)\sum_{m=0}^{\infty}{\frac{(-\tau)^{m}}{m!}}\left(\ln\left(\frac{\mu^{2}}{4Q_{s}^{2}}\right)-\frac{1}{\alpha(\mu)\beta_{2}}\right)^{2m+1}$
(25)

in terms of $\Lambda_{QCD}$

$\displaystyle\frac{1}{\alpha(\mu)\beta_{2}}=\ln{\frac{\mu^{2}}{\Lambda_{QCD}^{2}}}$
(26)

the non perturbative uncertainty from infrared renormalon found to be,

$\displaystyle\sim$ $\displaystyle
r_{\perp}^{2}~{}\Lambda_{QCD}^{2}~{}\ln{\frac{\Lambda_{QCD}^{2}}{4Q_{s}^{2}}}~{}\exp\left({-\tau\ln^{2}{\frac{\Lambda_{QCD}^{2}}{4Q_{s}^{2}}}}\right)$
(27)

Clearly the uncertainty is ${\cal O}\left(\Lambda_{QCD}^{2}\right)$ instead of
${\cal O}\left(\Lambda_{QCD}^{4}\right)$ that often taken as characteristic of
infrared renormalon. Presence of the sudakov factor indicates that the
saturation effect tend to suppress the IR renormalon effects at small-$x$.

##  IV Conclusion

Infrared renormalon contributions lead to non-perturbative effects similar to
those arising from vacuum condensation. The infrared renormalon contributions
presumably contain information about the hadron mass spectrum. Corresponding
Borel integral is dominated by the singularities closest to the origin. In
this work we find that small-$x$ effects push the infrared singularities
towards the origin and shifts it from $2/\beta_{2}$ to $1/\beta_{2}$. This
also makes the uncertainty proportional to ${\cal
O}\left(\Lambda_{QCD}^{2}\right)$ instead of ${\cal
O}\left(\Lambda_{QCD}^{4}\right)$. This very fact however anticipated earlier
by Mueller Mueller:1992xz  by citing the fact that perturbation series become
ambiguous at $b=1/\beta_{2}$ and was suggested that the $1/Q^{2}$ should be
pursued with vigor. In this study we find that ${\cal
O}\left(\Lambda_{QCD}^{2}\right)$ term corresponding to the pole at
$1/\beta_{2}$ exists in unintegrated dipole gluon distribution at small-$x$ as
the distribution as a $linear\otimes normal$ distribution in momentum evolved
to be a $logarithmic\otimes log~{}normal$ distribution at small momentum. The
result also shows that saturation effects indeed suppress the renormalon
effect however only through a Sudakov type form factor.

## References

  * (1) F. J. Dyson, “Divergence of perturbation theory in quantum electrodynamics,” Phys. Rev. 85, 631 (1952). doi:10.1103/PhysRev.85.631
  * (2) B. E. Lautrup, “On High Order Estimates in QED,” Phys. Lett. 69B (1977) 109. doi:10.1016/0370-2693(77)90145-9
  * (3) A. H. Mueller, “On the Structure of Infrared Renormalons in Physical Processes at High-Energies,” Nucl. Phys. B 250, 327 (1985). doi:10.1016/0550-3213(85)90485-7
  * (4) M. Beneke, “Renormalons,” Phys. Rept. 317, 1 (1999) doi:10.1016/S0370-1573(98)00130-6 [hep-ph/9807443].
  * (5) V. I. Zakharov, “QCD perturbative expansions in large orders,” Nucl. Phys. B 385, 452 (1992). doi:10.1016/0550-3213(92)90054-F
  * (6) E. Levin, “Renormalons at low x,” Nucl. Phys. B 453, 303 (1995) doi:10.1016/0550-3213(95)00416-P [hep-ph/9412345].
  * (7) M. Siddiqah, N. Vasim, K. Banu, R. Abir and T. Bhattacharyya, “Unintegrated dipole gluon distribution at small transverse momentum,” Phys. Rev. D 97, no. 5, 054009 (2018) doi:10.1103/PhysRevD.97.054009 [arXiv:1801.01637 [hep-ph]].
  * (8) Y. V. Kovchegov and E. Levin, “Quantum chromodynamics at high energy,” Camb. Monogr. Part. Phys. Nucl. Phys. Cosmol. 33, 1 (2012). doi:10.1017/CBO9781139022187
  * (9) A. H. Mueller, “The QCD perturbation series,” Talk given at Conference: QCD: 20 YEARS LATER (Aachen QCD Wkshp.1992, CU-TP-573), proceedings, edited by P.M. Zerwas and H.A. Kastrup, River Edge, N.J., World Scientific, 1993. 2 volumes p.162-171.