# Multi-Clue Reasoning with Memory Augmentation for
Knowledge-based Visual Question Answering

Chengxiang Yin1, Zhengping Che2, Kun Wu1, Zhiyuan Xu2, Jian Tang2
1Syracuse University, 2Midea Group

###### Abstract

Visual Question Answering (VQA) has emerged as one of the most challenging
tasks in artificial intelligence due to its multi-modal nature. However, most
existing VQA methods are incapable of handling Knowledge-based Visual Question
Answering (KB-VQA), which requires external knowledge beyond visible contents
to answer questions about a given image. To address this issue, we propose a
novel framework that endows the model with capabilities of answering more
general questions, and achieves a better exploitation of external knowledge
through generating Multiple Clues for Reasoning with Memory Neural Networks
(MCR-MemNN). Specifically, a well-defined detector is adopted to predict
image-question related relation phrases, each of which delivers two
complementary clues to retrieve the supporting facts from an external
knowledge base (KB), and these facts are further encoded into a continuous
embedding space using a content-addressable memory. Afterwards, mutual
interactions between visual-semantic representation and the supporting facts
stored in memory are captured to distill the most relevant information in
three modalities (i.e., image, question, and KB). Finally, the optimal answer
is predicted by choosing the supporting fact with the highest score. We
conduct extensive experiments on two widely-used benchmarks. The experimental
results well justify the effectiveness of MCR-MemNN as well as its superiority
over other KB-VQA methods.

##  1 Introduction

Recently, significant progress has been made for Visual Question Answering
(VQA) [3, 5, 8, 12, 36], which requires a joint comprehension of multi-modal
content from images and natural language. A VQA agent is expected to deliver
the correct answer to a text-based question according to a given image. Most
of the existing VQA models [4, 19, 22, 24, 37] and datasets [15, 23, 30, 41]
have focused on simple questions, which are answerable by solely analyzing the
question and image, i.e., no external knowledge is required. However, a truly
‘AI-complete’ VQA agent is required to combine both visual and semantic
observations with external knowledge for reasoning, which is effortless for
human but challenging for machine. Therefore, to bridge the gap between human-
level intelligence and algorithm design, Knowledge-based VQA (KB-VQA) [34] is
introduced to automatically find answers from the knowledge base (KB), given
image-question pairs.

Some efforts have been made in this direction. To begin with, Wang  _et al._
[34] presented a Fact-based VQA (FVQA) dataset to support a much deeper
reasoning. FVQA consists of questions that require external knowledge to
answer. Several classical solutions [34, 35] have been proposed to solve FVQA
by mapping each question to a query and retrieving supporting facts in KB
through a keyword-matching manner. These supporting facts are processed to
form the final answer. However, these query-mapping based approaches with
solely question parsing suffer from serious performance degradation when the
information hint is not captured in the external KB or the visual concept is
not exactly mentioned in the question. Moreover, special information (i.e.,
visual concept type and answer source) should be determined in advance during
querying and answering phases, which makes it hard to generalize to other
datasets. To address these issues, we introduce Multiple Clues for Reasoning,
a new KB retrieval method, where a relation phrase detector is proposed to
predict multiple complementary clues for supporting facts retrieval.

More recently, Out of the Box (OB) [26] and Straight to Facts (STTF) [27]
adopt cosine similarity technique to get the highest scoring fact for answer
prediction, where the whole visual information (i.e., object, scene, and
action) and the whole semantic information (i.e., all question words) are
indiscriminately applied to infer the final answer by implicit reasoning.
However, for an image-question pair, only part of the visual content in the
image and several specific words in the question are relevant to a given
supporting fact. Additionally, the direct concatenation of image-question-
entity embedding makes it hard to adaptively capture information among
different modalities. To exploit the inter-relationships among three
modalities (i.e., image, question and KB), we propose a two-way attention
mechanism with memory augmentation to model the interactions between visual-
semantic representation and the supporting facts. Then the most relevant
information in the three modalities can be distilled.

Figure 1: An illustration of Multi-Clue Reasoning.

In this paper, to deal with the KB-VQA task and address the aforementioned
issues, we present a novel framework that focuses on achieving a better
exploitation of external knowledge through generating Multiple Clues for
Reasoning with Memory Neural Networks. We name it as MCR-MemNN for short.
Specifically, as illustrated by Figure 1, the image-question pair is encoded
into a visual-semantic representation, which serves as the input to a well-
defined detector to obtain a three-term relation phrase (i.e., $\langle$
Subject, Relation, Object $\rangle$). In this case, either the subject (i.e.,
cat) or the object (i.e., tiger) acts a clue to retrieve supporting facts in
the external KB, and both deliver a fact set including the ground-truth fact
(i.e., cat RelatedTo tiger). In this manner, the ground-truth fact can be
successfully fetched as long as one of the two complementary clues is
predicted. Note that the predicted relation (i.e., RelatedTo) is adopted to
filter out some facts with different relations. The retained supporting facts,
including the ground-truth, are further encoded into a continuous embedding
space and stored in a content-addressable memory, where each memory slot
corresponds to a supporting fact. Afterwards, for reasoning procedure, we
assume that the visual and semantic content in the image-question pair can
contribute to a better exploitation of the external knowledge (i.e.,
supporting facts). Analogously, the external knowledge is helpful for a better
understanding of the image-question pair. Therefore, we employ a two-way
attention mechanism, which is not only intended to focus on the important
aspects of the memory in light of the image-question pair, but also the
important parts of image and question in light of the memory.

The main contributions of this paper are summarized as follows: (1) We
demonstrate a new KB retrieval method with two complementary clues (i.e.,
subject and object), which makes it a lot easier to fetch the ground-truth
supporting fact. (2) A two-way attention mechanism with memory augmentation is
employed to model the inter-relationships among image, question and KB to
distill the most relevant information in the three modalities. (3) We perform
extensive experiments on two widely-used benchmarks, which shows that the
proposed MCR-MemNN is an effective framework customized for KB-VQA.

##  2 Related Work

###  2.1 Visual Question Answering

Visual Question Answering (VQA) has emerged as a typical and popular
multimedia application, which has been studied by quite a few recent works [1,
21, 33, 38, 40, 39]. A typical CNN-RNN based approach, Neural-Image-QA [24],
learns an end-to-end trainable model by feeding both the image features and
questions into LSTM. Recently, Huang  _et al._ [11] proposes a multi-grained
attention mechanism, that addresses the failed cases on small objects or
uncommon concepts through learning word-object correspondence. To go a step
further, graph neural network is leveraged in [10] to support complex
relational reasoning among objects conditioned on the textual input. Even
though lots of methods and datasets have been proposed for VQA task, all of
them are targeting at the simple questions, and none of them can deal with the
general cases that external knowledge is required for a deep reasoning.

###  2.2 Knowledge-based Visual Question Answering

To deal with the knowledge-based visual question answering (KB-VQA) tasks,
combination of visual observations with the external knowledge is required,
which, however, is undoubtedly challenging. In [34], Wang  _et al._ introduces
a Fact-based VQA (FVQA) dataset, which requires and supports a much deeper
reasoning with the help of the external knowledge bases. It also provides a
query-mapping based approach, which maps each question to a query to retrieve
the supporting facts. Similarly, Wang  _et al._ [35] transforms the question
to an available query template and limits the question types. However, those
query-mapping based approaches suffers when the question does not focus on the
most obvious visual concept, since the visual representation is not adopted
for answer reasoning. In some existing learning based approaches [27, 26],
both visual and semantic information are wholly provided and may introduce
redundant information, which is harmful to the reasoning process. As a
solution, a heterogeneous graph neural network with multiple layers is
employed in [42] to collect the complementary evidence from three modalities
under the guidance of question. It achieves the state-of-the-art by adopting
dense captioning for semantic parsing. Analogously, this paper presents
another solution by leveraging multiple clues for KB retrieval and leveraging
a two-way attention mechanism with memory augmentation to model the inter-
relationships among the three modalities including image, question and KB,
which delivers a performance comparable with the state-of-the-art.

##  3 Methodology

We first describe the general definition and notations of Knowledge-based
Visual Question Answering (KB-VQA), followed by the modeling of relation
phrase detector. Finally, we present the details about the proposed framework.

###  3.1 Problem Statement

Given an image I and a related question Q, the KB-VQA task aims to predict an
answer A from the external knowledge base (KB), which consists of facts in the
form of triplet (i.e., $\langle$ Subject, Relation, Object $\rangle$), where
subject represents a visual concept, object represents an attribute or a
visual concept and relation denotes the relationship between the subject and
the object. Note that the answer A can be either the subject or object in the
triplet. The key of KB-VQA is to select the correct supporting fact and then
determine the answer. For convenience, in our notations, the fact $\langle$
Subject, Relation, Object $\rangle$ corresponds to the answer subject; while
its reversed form $\langle$ Object, Relation, Subject $\rangle$ corresponds to
the answer object. For instance, as for the ground-truth supporting fact
(i.e., cat RelatedTo tiger) in Figure 3, the corresponding answer is cat, and
the answer of its reversed form (i.e., tiger RelatedTo cat) is tiger. In this
manner, during inference stage, the optimal answer is the first term of the
predicted fact.

Figure 2: Relation Phrase Detector. Figure 3: Architecture of MCR-MemNN.

###  3.2 Relation Phrase Detector

Same as the fact in KB, the relation phrase is depicted in a form of triplet
(i.e., $\langle$ Subject, Relation, Object $\rangle$), which contains two
complementary clues (i.e., subject and object) for KB retrieval. As mentioned,
the two complementary clues make it a lot easier to fetch the ground-truth
supporting fact, and the relation predicted can be leveraged to filter out the
facts with wrong relations. Therefore, we develop a detector to obtain a
relation phrase based on the visual and semantic content of the image-question
pair.

As shown in Figure 2, given an image-question pair
$\\{\textbf{I},\textbf{Q}\\}$, the relation phrase prediction is formulated as
a multi-task classification problem following [20]. For image embedding, we
use a Faster R-CNN [31] in conjunction with the ResNet-101 [9] pretrained on
Visual Genome dataset [15] to generate an output set of image features v as
the visual representation of the input image.

$\textbf{v}=\mbox{Faster R-CNN}(\textbf{I})$ (1)

where $\textbf{v}\in{\mathcal{R}}^{2048\times K}$ is based on the bottom-up
attention [1], which represents the ResNet features centered on the top-$K$
objects in the image. $K$ is set to 36 in our experiments.

For question embedding, we first transfer each word in Q into a feature vector
using the pre-trained 300D GloVe [29] vectors, and use randomly initialized
vectors for the words which are out of GloVe’s vocabulary. Here we denote the
resulting sequence of word embeddings as ${\textbf{H}}^{q}$. Then the Gated
Recurrent Unit (GRU) [7] with hidden state dimension 512 is adopted to encode
the ${\textbf{H}}^{q}$ as semantic representation
$\textbf{q}\in{\mathcal{R}}^{512}$.

$\textbf{q}=\mbox{GRU}({\textbf{H}}^{q})$ (2)

To encode the image and question in a shared embedding space, top-down
attention [1] is employed to fuse the visual representation v and semantic
representation q. Specifically, for each object from $i=1...K$, its feature
${\textbf{v}}_{i}$ is concatenated with the semantic representation q, and
then passed through a non-linear layer $f_{a}$ and a linear layer to obtain
its corresponding attention weight $a_{i}$.

$\displaystyle a_{i}$
$\displaystyle={\textbf{W}}_{a}{f_{a}}([{\textbf{v}}_{i},\textbf{q}])$ (3)
$\displaystyle\hat{\textbf{a}}$ $\displaystyle=\mbox{softmax}(\textbf{a})$ (4)
$\displaystyle\hat{\textbf{v}}$
$\displaystyle={\sum}_{i=1}^{K}\hat{a}_{i}{\textbf{v}}_{i}$ (5)

where ${\textbf{W}}_{a}$ is a learnable weight vector and the attention
weights a are normalized with a softmax function to $\hat{\textbf{a}}$. The
image features are weighted by the normalized attention values to get the
weighed visual representation $\hat{\textbf{v}}\in{\mathcal{R}}^{2048}$.
Following [33], the non-linear layer
$f_{a}:\textbf{x}\in{\mathcal{R}}^{m}\to\textbf{y}\in{\mathcal{R}}^{n}$ with
parameters $a$ is defined as follows:

$\displaystyle\widetilde{\textbf{y}}$
$\displaystyle=\mbox{tanh}(\textbf{W}\textbf{x}+\textbf{b})$ (6) g
$\displaystyle=\sigma({\textbf{W}}^{{}^{\prime}}\textbf{x}+{\textbf{b}}^{{}^{\prime}})$
(7) y $\displaystyle=\widetilde{\textbf{y}}\circ\textbf{g}$ (8)

where $\sigma$ is the sigmoid activation function,
$\textbf{W},{\textbf{W}}^{{}^{\prime}}\in{\mathcal{R}}^{n\times m}$ and
$\textbf{b},{\textbf{b}}^{{}^{\prime}}\in{\mathcal{R}}^{n}$ are learnable
parameters, and $\circ$ denotes the element-wise multiplication.

A joint embedding h of the question and the image is obtained by the fusion of
the weighed visual representation $\hat{\textbf{v}}$ and the semantic
representation q.

$\textbf{h}=f_{v}(\hat{\textbf{v}})\circ f_{q}(\textbf{q})$ (9)

where $\textbf{h}\in{\mathcal{R}}^{512}$. Both $f_{v}$ and $f_{q}$ are the
non-linear layers with the same form as $f_{a}$. The joint embedding h is then
fed into a group of linear classifiers for the prediction of subject, relation
and object in a relation phrase.

$\displaystyle\hat{\textbf{s}}$
$\displaystyle=\mbox{softmax}({\textbf{W}}_{s}\textbf{h}+{\textbf{b}}_{s})$
(10) $\displaystyle\hat{\textbf{r}}$
$\displaystyle=\mbox{softmax}({\textbf{W}}_{r}\textbf{h}+{\textbf{b}}_{r})$
(11) $\displaystyle\hat{\textbf{o}}$
$\displaystyle=\mbox{softmax}({\textbf{W}}_{o}\textbf{h}+{\textbf{b}}_{o})$
(12)

where ${\textbf{W}}_{s},{\textbf{W}}_{r},{\textbf{W}}_{o}$ and
${\textbf{b}}_{s},{\textbf{b}}_{r},{\textbf{b}}_{o}$ are learnable parameters,
and $\hat{\textbf{s}},\hat{\textbf{r}},\hat{\textbf{o}}$ denote the predicted
probability for subject, relation and object over each candidate,
respectively. The loss function for the relation phrase detector is defined as

${\mathcal{L}}_{d}={\lambda}_{s}{\mathcal{L}}_{c}(\textbf{s},\hat{\textbf{s}})+{\lambda}_{r}{\mathcal{L}}_{c}(\textbf{r},\hat{\textbf{r}})+{\lambda}_{o}{\mathcal{L}}_{c}(\textbf{o},\hat{\textbf{o}})$
(13)

where $\textbf{s},\textbf{r},\textbf{o}$ are the ground-truth labels for
subject, object and relation, respectively. ${\mathcal{L}}_{c}$ represents the
cross-entropy loss, and the weights of the loss terms (i.e.,
${\lambda}_{s},{\lambda}_{r},{\lambda}_{o}$) are set to 1.0 in our
experiments.

###  3.3 MCR-MemNN

As shown in Figure 3, besides the Relation Phrase Detector, our proposed MCR-
MemNN framework consists of six components, which are Memory Module, Visual-
Semantic Embedding Module, Visual-Semantic Aware Attention Module, Memory
Aware Attention Module, Generalization Module and Answer Module.

####  3.3.1 Memory Module

For an image-question pair $\\{\textbf{I},\textbf{Q}\\}$, after the three-term
phrase (i.e., Subject, Relation, Object) is predicted by the relation phrase
detector, all the facts in the external KB with entities pointed by the
subject or pointed to the object within $h$ hops are collected as candidate
supporting facts. $h$ is set to 1 in our experiments. The relation predicted
is leveraged to further filter out the facts with wrong relations. Then a set
of candidate supporting facts $\\{{\textbf{F}}_{i}\\}_{i=1}^{N}$ is collected,
where $N$ denotes the number of facts in the fact set, and each fact
${\textbf{F}}_{i}$ consists of a sequence of words.

Afterwards, similar to question embedding, each fact ${\textbf{F}}_{i}$ is
transformed to a sequence of word embeddings ${\textbf{H}}^{f}_{i}$ based on
GloVe’s vocabulary, and encoded using a BiLSTM to get its representation
${\textbf{f}}_{i}\in{\mathcal{R}}^{128}$.

${\textbf{f}}_{i}=\mbox{BiLSTM}({\textbf{H}}^{f}_{i})$ (14)

To store the candidate supporting facts, a key-value structured memory network
[25] is leveraged. The key memory is used in the addressing stage, while the
value memory is used in the reading stage. The representation
${\textbf{f}}_{i}$ of each fact is passed through two separate linear layers
to obtain its key embedding ${\textbf{M}}^{k}_{i}\in{\mathcal{R}}^{128}$ and
value embedding ${\textbf{M}}^{v}_{i}\in{\mathcal{R}}^{128}$ respectively.

$\displaystyle{\textbf{M}}^{k}_{i}$
$\displaystyle={\textbf{W}}_{k}{\textbf{f}}_{i}$ (15)
$\displaystyle{\textbf{M}}^{v}_{i}$
$\displaystyle={\textbf{W}}_{v}{\textbf{f}}_{i}$ (16)

where ${\textbf{W}}_{k}\in{\mathcal{R}}^{128\times 128}$ and
${\textbf{W}}_{v}\in{\mathcal{R}}^{128\times 128}$ are learnable parameters.
For the set of candidate supporting facts $\\{{\textbf{F}}_{i}\\}_{i=1}^{N}$,
we have a set of key embeddings
${\textbf{M}}^{k}=\\{{\textbf{M}}^{k}_{i}\\}_{i=1}^{N}$ and a set of value
embeddings ${\textbf{M}}^{v}=\\{{\textbf{M}}^{v}_{i}\\}_{i=1}^{N}$. Note that
one memory slot is defined as a pair of key embedding and value embedding
(i.e., $\\{{\textbf{M}}^{k}_{i},{\textbf{M}}^{v}_{i}\\}$) of one candidate
supporting fact.

####  3.3.2 Visual-Semantic Embedding Module

The visual-semantic embedding module has basically the same architecture as
the relation phrase detector and the only difference is about the question
embedding before top-down attention, where self-attention are applied over the
sequence of word embeddings ${\textbf{H}}^{q}$ to get the semantic
representation $\textbf{q}\in{\mathcal{R}}^{128}$.

$\displaystyle{\hat{\textbf{a}}^{qq}}$
$\displaystyle=\mbox{softmax}({({\textbf{H}}^{q})}^{\intercal}{\textbf{H}}^{q})$
(17) q
$\displaystyle=\mbox{BiLSTM}([{\textbf{H}}^{q}{({\hat{\textbf{a}}}^{qq})}^{\intercal},{\textbf{H}}^{q}])$
(18)

where the ${\textbf{H}}^{q}$ are weighted by the normalized values,
concatenated with itself and fed into a BiLSTM. Afterwards, same procedures
are conducted to obtain the visual-semantic representation
$\textbf{h}\in{\mathcal{R}}^{128}$.

####  3.3.3 Visual-Semantic Aware Attention Module

Given visual-semantic representation h, we apply an attention over all the
memory slots $\\{{\textbf{M}}^{k}_{i},{\textbf{M}}^{v}_{i}\\}_{i=1}^{N}$ to
obtain the memory summary $\textbf{m}\in{\mathcal{R}}^{128}$ in light of the
visual and semantic content of the image-question pair.

$\displaystyle a_{i}^{m}$
$\displaystyle={\textbf{W}}_{3}\mbox{tanh}({\textbf{W}}_{1}\textbf{h}+{\textbf{W}}_{2}{\textbf{M}}_{i}^{k})$
(19) $\displaystyle{\hat{\textbf{a}}}^{m}$
$\displaystyle=\mbox{softmax}({\textbf{a}}^{m})$ (20) m
$\displaystyle={\sum}_{i=1}^{N}\hat{a}_{i}^{m}{\textbf{M}}_{i}^{v}$ (21)

where ${\textbf{W}}_{1}\in{\mathcal{R}}^{128\times
128},{\textbf{W}}_{2}\in{\mathcal{R}}^{128\times
128},{\textbf{W}}_{3}\in{\mathcal{R}}^{1\times 128}$ are learnable parameters.
The attention for each memory slot is calculated and normalized based on h and
the corresponding key embedding ${\textbf{M}}^{k}_{i}$. Then the set of value
embeddings ${\textbf{M}}^{v}=\\{{\textbf{M}}^{v}_{i}\\}_{i=1}^{N}$ are
weighted to get the memory summary m.

####  3.3.4 Memory Aware Attention Module

As we have obtained the memory summary m, we proceed to compute the attentions
over all the question words and all the image features in light of the memory.

Given memory summary m, the sequence of word embeddings ${\textbf{H}}^{q}$ and
the set of image features v, the memory-aware question embedding
${\textbf{q}}^{m}\in{\mathcal{R}}^{128}$ and the memory-aware image embedding
${\textbf{v}}^{m}\in{\mathcal{R}}^{2048}$ are derived as follows:

$\displaystyle{\hat{\textbf{a}}}^{q}$
$\displaystyle=\mbox{softmax}({({\textbf{H}}^{q})}^{\intercal}\textbf{m})$
(22) $\displaystyle{\textbf{q}}^{m}$
$\displaystyle={\textbf{H}}^{q}{\hat{\textbf{a}}}^{q}$ (23)
$\displaystyle{\hat{\textbf{a}}}^{v}$
$\displaystyle=\mbox{softmax}({({\textbf{W}}_{v}\textbf{v})}^{\intercal}\textbf{m})$
(24) $\displaystyle{\textbf{v}}^{m}$
$\displaystyle=\textbf{v}{\hat{\textbf{a}}}^{v}$ (25)

where ${\hat{\textbf{a}}}^{q}$ represents the normalized memory aware
attention over all the question words, ${\hat{\textbf{a}}}^{v}$ represents the
normalized memory aware attention over all the image features and
${\textbf{W}}_{v}\in{\mathcal{R}}^{128\times 2048}$ are learnable parameters.

The visual-semantic representation ${\textbf{h}}^{m}$ in light of the memory
is obtained by the fusion of the memory-aware question embedding
${\textbf{q}}^{m}$ and the memory-aware image embedding ${\textbf{v}}^{m}$.

${\textbf{h}}^{m}=f_{v}^{m}({\textbf{v}}^{m})\circ
f_{q}^{m}({\textbf{q}}^{m})$ (26)

where ${\textbf{h}}^{m}\in{\mathcal{R}}^{128}$. Both $f_{v}^{m}$ and
$f_{q}^{m}$ are the non-linear layers with the same form as $f_{a}$. Note that
the aforementioned two-way mechanism corresponds to Sections 3.3.3 and 3.3.4.

####  3.3.5 Generalization Module

Inspired by [6], another hop of the attention process is conducted over the
memory before answer prediction. Attention mechanism of Section 3.3.3 is
leveraged here, given memory aware visual-semantic representation
${\textbf{h}}^{m}$, to fetch the most relevant information
${\textbf{m}}^{h}\in{\mathcal{R}}^{128}$ from the memory to obtain the final
visual-semantic representation $\hat{\textbf{h}}\in{\mathcal{R}}^{128}$. To be
more specific, the fetched information ${\textbf{m}}^{h}$ is concatenated with
${\textbf{h}}^{m}$ and fed into a GRU to update the visual-semantic
representation. To the end, a batch normalization (BN) layer is used.

$\displaystyle{a}_{i}^{h}$
$\displaystyle={\textbf{W}}_{6}\mbox{tanh}({\textbf{W}}_{4}{\textbf{h}}^{m}+{\textbf{W}}_{5}{\textbf{M}}_{i}^{k})$
(27) $\displaystyle\hat{\textbf{a}}^{h}$
$\displaystyle=\mbox{softmax}({\textbf{a}}^{h})$ (28)
$\displaystyle{\textbf{m}}^{h}$
$\displaystyle={\sum}_{i=1}^{N}\hat{a}_{i}^{h}{\textbf{M}}_{i}^{v}$ (29)
$\displaystyle\hat{\textbf{h}}$
$\displaystyle=\mbox{BN}({\textbf{h}}^{m}+\mbox{GRU}([{\textbf{h}}^{m},{\textbf{m}}^{h}]))$
(30)

where ${\textbf{W}}_{4}\in{\mathcal{R}}^{128\times
128},{\textbf{W}}_{5}\in{\mathcal{R}}^{128\times
128},{\textbf{W}}_{6}\in{\mathcal{R}}^{1\times 128}$ are learnable parameters.

####  3.3.6 Answer Module

Given the final visual-semantic representation $\hat{\textbf{h}}$, and the set
of key embeddings ${\textbf{M}}^{k}\in{\mathcal{R}}^{128\times N}$, the key
embedding ${\textbf{M}}_{i}^{k}$ of each candidate supporting fact is
concatenated with the $\hat{\textbf{h}}$ to compute the probability of whether
the fact is correct. The predicted supporting fact is

$\mbox{argmax}_{i}\mbox{softmax}({\textbf{W}}_{7}[\hat{\textbf{h}},{\textbf{M}}^{k}]+{\textbf{b}}^{7})$
(31)

where $i=1,...,N$, ${\textbf{W}}_{7}\in{\mathcal{R}}^{1\times 256}$ and
${\textbf{b}}^{7}\in{\mathcal{R}}^{N}$ are learnable parameters. As we have
stated in Section 3.1, the optimal answer is the first term of the predicted
fact.

####  3.3.7 Loss Function

Once we have the candidate supporting facts retrieved from the external KB,
all the learnable parameters of the proposed MCR-MemNN (besides the Relation
Phrase Detector) are trained in an end-to-end manner by minimizing the
following loss function over the training set.

$\mathcal{L}=\frac{1}{D}\sum_{k=1}^{D}{\mathcal{L}}_{c}({\textbf{Y}}_{k},\hat{\textbf{Y}}_{k})$
(32)

where ${\mathcal{L}}_{c}$ is defined as the cross-entopy loss, $D$ is the
number of training samples, ${\textbf{Y}}_{k}$ and $\hat{\textbf{Y}}_{k}$
represent the ground-truth label and the predicted probability over each
candidate supporting fact.

##  4 Performance Evaluation

We employed two compelling benchmarks, FVQA [34] and Visual7W+ConceptNet [16]
to evaluate the proposed MCR-MemNN on Knowledge-based Visual Question
Answering (KB-VQA) task.

###  4.1 Benchmark Datasets

FVQA. The Factual Visual Question Answering (FVQA) dataset consists of $2,190$
images, $5,286$ questions and $4,126$ unique facts corresponding to the
questions. The external KB of FVQA, consisting of $193,449$ facts, are
constructed based on three structured KBs, including DBpedia [2], ConceptNet
[18] and WebChild [32]. Following [34], the top-1 and top-3 accuracies are
averaged over five test splits.

Visual7W+ConceptNet. The Visual7W+ConceptNet dataset, built by [16], is a
collection of knowledge-based questions with images sampled from the test
split of Visual7W [41] dataset. It consists of $16,850$ open domain question-
answer pairs based on $8,425$ images. Note that the supporting facts of each
question-answer pair are retrieved directly from the ConceptNet, which serves
as the external KB. Following [34], the top-1 and top-3 accuracies are
calculated over the test set.

###  4.2 Experimental Setup

####  4.2.1 Implementation Details

For the training of the relation phrase detector, the model was trained with
Adam optimizer [14] with an initial learning rate of $1\times 10^{-4}$ and
weight decay of $1\times 10^{-6}$, and the batch size is set to 32.

For the training of the proposed MCR-MemNN (besides the relation phrase
detector), the model was trained with Adam optimizer with an initial learning
rate of $1\times 10^{-3}$ and weight decay of $1\times 10^{-6}$, and the batch
size is set to 64. Top 40 predicted subjects and objects were adopted as clues
to retrieve candidate supporting facts from the external KB. The memory size
$N_{mem}$ was set to 96. Note that no matter the number of candidate
supporting facts $N$ is larger than, equal to or smaller than $N_{mem}$, the
ground-truth fact is preserved and the negative facts are randomly selected to
fill up the remaining memory. Our code was implemented in PyTorch [28] and run
with NVIDIA Tesla P100 GPUs.

####  4.2.2 Baselines

For the FVQA dataset, CNN-RNN based approaches including LSTM-
Question+Image+Pre-VQA [34] and Hie-Question+Image+Pre-VQA [34], semantic
parsing based approaches including FVQA (top-3-QQmaping) [34] and FVQA
(Ensemble) [34], learning-based approaches including Straight to the Facts
(STTF) [27], Out of the Box (OB) [26], Reading Comprehension [17], and Multi-
Layer Cross-Modal Knowledge Reasoning (Mucko) [42] are compared with the
proposed MCR-MemNN.

For the Visual7W+ConceptNet dataset, learning based approaches including KDMN
[16], Out of the Box (OB) are compared with the proposed MCR-MemNN. Note that
both KDMN and MCR-MemNN adopted memory augmentation technique for storing the
retrieved external knowledge.

Table 1: Results for the relation phrase detector on FVQA.

Case | Classification Acc. | Recall | QA Acc.
---|---|---|---
Sub | Obj | Union | w/ Rel | w/o Rel | w/ Rel | w/o Rel
Top-20 | 64.31 | 39.43 | 69.80 | 78.56 | 82.08 | 60.36 | 62.76
Top-30 | 69.46 | 43.30 | 74.24 | 83.61 | 86.90 | 66.34 | 65.28
Top-40 | 72.65 | 45.71 | 76.22 | 85.76 | 88.85 | 70.92 | 68.85
Top-60 | 73.59 | 46.23 | 77.10 | 86.58 | 89.81 | 68.52 | 67.30
Top-100 | 75.60 | 47.61 | 79.37 | 88.21 | 90.33 | 68.02 | 65.57

###  4.3 Experimental Results

####  4.3.1 Relation Phrase Detector

As shown in Table 1, to evaluate the performance of the relation phrase
detector, five different cases are considered, including Top-20, Top-30,
Top-40, Top-60 and Top-100. For each case, the classification accuracies of
subject (i.e., ‘Sub’), object (i.e, ‘Obj’) and both union (i.e., ‘Union’) are
calculated. In addition, the answer recall is reported with the top-3 relation
limitation (i.e., ‘w/ Relation’) or not (i.e., ‘w/o Relation’). Finally, the
downstream question answering accuracy (‘QA Acc.’) is also calculated.

For more clarity, the ‘Sub’ in the case of Top-40 represents the fraction of
test samples that the ground-truth subject is included in the top 40 predicted
subjects, and these subjects are further adopted as the clues for KB
retrieval. The Recall w/ Relation in the case of Top-40 represents the
fraction of test samples that the correct answer is included in the candidate
answer set corresponding to the supporting facts retrieved by the top-40
subject or object clues and filtered by the top 3 predicted relations. Note
that the top-3 classification accuracy for relation prediction using the
relation phrase detector is $93.20\%$.

Results in Table 1 show that both union accuracy and answer recall of the
Top-40 case are higher than those of the Top-20 case, and improvements of
$10.56\%$ and $6.09\%$ on downstream QA accuracies (with and without relation
filtering) are caused. Even though much more relevant supporting facts are
retrieved from the external KB in the Top-100 case, which delivers higher both
union accuracy and answer recall, the downstream QA accuracies are lower than
those of the Top-40 case. This observation clearly shows that excessive
retrieved facts can lead to more redundant information, which is harmful to
the reasoning process. We choose the top 40 subjects and objects as clues for
KB retrieval as this gives the best downstream QA accuracies.

Table 2: Accuracy comparison on FVQA.

Method | Overall Accuracy
---|---
top-1 | top-3
LSTM-Question+Image+Pre-VQA [34] | 24.98 | 40.40
Hie-Question+Image+Pre-VQA [34] | 43.14 | 59.44
FVQA (top-3-QQmaping) [34] | 56.91 | 64.65
FVQA (Ensemble) [34] | 58.76 | -
Straight to the Facts (STTF) [27] | 62.20 | 75.60
Reading Comprehension [17] | 62.94 | 70.08
Out of the Box (OB) [26] | 69.35 | 80.25
Mucko (w/o Semantic Graph) [42] | 71.28 | 82.76
MCR-MemNN (Ours) | 70.92 | 81.83

Table 3: Accuracy comparison on Visual7W+ConceptNet.

Method | Overall Accuracy
---|---
top-1 | top-3
KDMN-NoKnowledge [16] | 45.1 | -
KDMN-NoMemory [16] | 51.9 | -
KDMN [16] | 57.9 | -
KDMN-Ensemble [16] | 60.9 | -
Out of the Box (OB) [26] | 57.32 | 71.61
MCR-MemNN (Ours) | 64.23 | 79.18

Figure 4: Success and failure cases. (Terms in yellow/blue indicates the
correctly predicted subjects/objects (in top-40), terms in green denotes the
correctly predicted relations (in top-3) and terms underlined represents the
predicted answer.)

####  4.3.2 MCR-MemNN

Tables 2 and 3 show the comparison of the proposed MCR-MemNN with the above-
mentioned baselines on FVQA and Visual7W+ConceptNet, respectively, and the
following observations can be made:

1) On FVQA dataset, MCR-MemNN outperforms almost all the baselines in terms of
top-1 and top-3 accuracies. To be specific, MCR-MemNN outperforms the semantic
parsing based approaches including FVQA (top-3-QQmaping) and FVQA (Ensemble),
and achieves more than $12\%$ boost on top-1 accuracy and more than $15\%$
boost on top-3 accuracy. In addition, compared with the typical learning based
approaches including STTF and OB, which wholly introduce the visual and
semantic information without selection, MCR-MemNN gains an improvement by
leveraging a two-way attention mechanism (i.e., Sections 3.3.3 and 3.3.4) to
exploit the inter-relationships among image, question and KB and distill the
most relevant information in each of the three modalities.

2) Even though a heterogeneous graph neural network with high complexity is
employed in Mucko, the proposed MCR-MemNN can still deliver a comparable
performance. Note that the full model of Mucko leveraged dense captions [13]
as input for performance improvement. For fair comparison, the semantic graph
for dense caption parsing is removed.

3) On Visual7W+ConceptNet dataset, MCR-MemNN consistently outperforms a series
of models based on KDMN, which leverages a dynamic memory network to preserve
the retrieved external knowledge. Since both MCR-MemNN and KDMN adopted the
memory augmentation technique, this observation further evidences the
effectiveness of modeling inter-relationships among image, question and KB for
KB-VQA task. Note that Mucko does not provide the result without Semantic
Graph.

Table 4: Ablation studies on FVQA. (Sub: Subject as Clue; Obj: Obj as Clue;
Rel: Relation Filtering; Att: Two-way Attention)

Case | Sub | Obj | Rel | Att | Overall Accuracy
---|---|---|---|---|---
top-1 | top-3
1 | ✓ |  |  |  | 61.60 | 68.18
2 |  | ✓ |  |  | 44.07 | 52.72
3 | ✓ | ✓ |  |  | 65.04 | 73.58
4 | ✓ | ✓ | ✓ |  | 67.52 | 76.48
5 | ✓ | ✓ |  | ✓ | 68.85 | 78.37
6 | ✓ | ✓ | ✓ | ✓ | 70.92 | 81.83

####  4.3.3 Ablation Studies

To validate the superiority of the proposed MCR-MemNN on KB-VQA, several
ablation experiments were conducted based on FVQA dataset, and we can make the
following observations based on Table 4:

1) MCR-MemNN adopts both subject and object as clues for KB retrieval, which
leads to better performance. Specifically, compared with Case-1, where only
subject is adopted as clues for KB retrieval, there exists a jump on top-1
(i.e., $3.44\%$) and top-3 (i.e., $5.40\%$) accuracies when both of the
subject and object are leveraged as clues in Case-3. A similar gain can be
observed when Case-3 is compared with Case-2.

2) To validate the effectiveness of the relation filtering, the experiment is
conducted in Case-4, which achieves $2.48\%$ improvement over Case-3 on top-1
accuracy. This observation clearly implies that the predicted relation can
successfully remove the redundant supporting facts retrieved from the external
KB. Similarly, compared with Case-5 without relation filtering, Case-6 brings
up an improvement of $2.07\%$.

3) The two-way attention mechanism (i.e., Sections 3.3.3 and 3.3.4) can
deliver an additional performance gain on KB-VQA task. For instance, compared
with Case-4, where the inter-relationships are not exploited among image,
question and KB, Case-6 brings up improvements of $3.40\%$ on top-1 accuracy.
This indicates that the redundant information of the three modalities (i.e.,
image, question and KB) is removed during the reasoning process, and the most
relevant information is collected by modeling inter-relationships among the
three modalities.

####  4.3.4 Qualitative Results

Figure 4 shows several success and failure cases using MCR-MemNN. Two steps
are indispensable to predict the correct answer: (1) Either the correct
subject or object is included in the top-40 predicted subject set or object
set. (2) The correct relation is predicted as the top-3 relations. The ground-
truth fact will be retrieved as one of the candidate supporting facts, if both
the two steps are successfully accomplished. For instance, all five samples in
the first row have their corresponding ground-truth facts successfully
predicted and the correct answers are delivered. Specifically, the first two
samples have both of their subjects and objects correctly predicted. The 3rd
sample have its subject correctly predicted while the last two have their
objects correctly predicted. This clearly verifies the advantage of multiple
clues reasoning, retrieval using two complementary clues (i.e., subject and
object) makes it a lot easier to fetch the ground-truth fact and deliver the
correct answer. Some other cases are presented in the second row. Generally,
if a wrong fact is predicted (e.g., the 2nd, 3rd and 5th samples), the correct
answer cannot be given. However, in some special cases, even if the ground-
truth fact is not successfully predicted (e.g., the 1st sample), the correct
answer can still be delivered. If the correct relation is not included in the
top-3 (e.g., the 3rd sample), the correct answer cannot be given even if the
subject or object is correctly predicted. The MCR-MemNN always fails if the
question is intend to know the relationship between subject and object (e.g.,
4th sample).

##  5 Conclusions

This paper, by introducing Multiple Clues for Reasoning with Memory Neural
Network (MCR-MemNN), presents a novel framework for knowledge-based visual
question answering (KB-VQA). Comprehensive experiments have been conducted on
two widely-used benchmarks and the extensive experimental results have shown
that 1) Retrieval using two complementary clues (i.e., subject and object)
makes it a lot easier to fetch the ground-truth fact and deliver the correct
answer; 2) Two-way attention mechanism with memory augmentation can
successfully model the inter-relationships among the three modalities
including image, question and KB, and brings up a remarkable performance gain
on KB-VQA; 3) MCR-MemNN outperforms most of the KB-VQA methods and achieves a
comparable performance with the state-of-the-art.

## References

  * [1] Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang.  Bottom-up and top-down attention for image captioning and visual question answering.  In Proceedings of the IEEE CVPR, pages 6077–6086, 2018.
  * [2] Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives.  Dbpedia: A nucleus for a web of open data.  In The semantic web, pages 722–735. Springer, 2007.
  * [3] Ali Furkan Biten, Ruben Tito, Andres Mafla, Lluis Gomez, Marçal Rusinol, Ernest Valveny, CV Jawahar, and Dimosthenis Karatzas.  Scene text visual question answering.  In Proceedings of the IEEE ICCV, pages 4291–4301, 2019.
  * [4] Kan Chen, Jiang Wang, Liang-Chieh Chen, Haoyuan Gao, Wei Xu, and Ram Nevatia.  Abc-cnn: An attention based convolutional neural network for visual question answering.  arXiv preprint arXiv:1511.05960, 2015.
  * [5] Long Chen, Xin Yan, Jun Xiao, Hanwang Zhang, Shiliang Pu, and Yueting Zhuang.  Counterfactual samples synthesizing for robust visual question answering.  In Proceedings of the IEEE/CVF CVPR, pages 10800–10809, 2020.
  * [6] Yu Chen, Lingfei Wu, and Mohammed J Zaki.  Bidirectional attentive memory networks for question answering over knowledge bases.  arXiv preprint arXiv:1903.02188, 2019.
  * [7] Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio.  Empirical evaluation of gated recurrent neural networks on sequence modeling.  arXiv preprint arXiv:1412.3555, 2014.
  * [8] Difei Gao, Ke Li, Ruiping Wang, Shiguang Shan, and Xilin Chen.  Multi-modal graph neural network for joint reasoning on vision and scene text.  In Proceedings of the IEEE/CVF CVPR, pages 12746–12756, 2020.
  * [9] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE CVPR, pages 770–778, 2016.
  * [10] Ronghang Hu, Anna Rohrbach, Trevor Darrell, and Kate Saenko.  Language-conditioned graph networks for relational reasoning.  In Proceedings of the IEEE International Conference on Computer Vision, pages 10294–10303, 2019.
  * [11] Pingping Huang, Jianhui Huang, Yuqing Guo, Min Qiao, and Yong Zhu.  Multi-grained attention with object-level grounding for visual question answering.  In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3595–3600, 2019.
  * [12] Huaizu Jiang, Ishan Misra, Marcus Rohrbach, Erik Learned-Miller, and Xinlei Chen.  In defense of grid features for visual question answering.  In Proceedings of the IEEE/CVF CVPR, pages 10267–10276, 2020.
  * [13] Justin Johnson, Andrej Karpathy, and Li Fei-Fei.  Densecap: Fully convolutional localization networks for dense captioning.  In Proceedings of the IEEE CVPR, pages 4565–4574, 2016.
  * [14] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  In ICLR, 2015.
  * [15] Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al.  Visual genome: Connecting language and vision using crowdsourced dense image annotations.  IJCV, 123(1):32–73, 2017.
  * [16] Guohao Li, Hang Su, and Wenwu Zhu.  Incorporating external knowledge to answer open-domain visual questions with dynamic memory networks.  arXiv preprint arXiv:1712.00733, 2017.
  * [17] Hui Li, Peng Wang, Chunhua Shen, and Anton van den Hengel.  Visual question answering as reading comprehension.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6319–6328, 2019.
  * [18] Hugo Liu and Push Singh.  Conceptnet—a practical commonsense reasoning tool-kit.  BT technology journal, 22(4):211–226, 2004.
  * [19] Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh.  Hierarchical question-image co-attention for visual question answering.  In NeurIPS, pages 289–297, 2016.
  * [20] Pan Lu, Lei Ji, Wei Zhang, Nan Duan, Ming Zhou, and Jianyong Wang.  R-vqa: learning visual relation facts with semantic attention for visual question answering.  In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1880–1889, 2018.
  * [21] Chao Ma, Chunhua Shen, Anthony Dick, Qi Wu, Peng Wang, Anton van den Hengel, and Ian Reid.  Visual question answering with memory-augmented networks.  In Proceedings of IEEE CVPR, pages 6975–6984, 2018.
  * [22] Lin Ma, Zhengdong Lu, and Hang Li.  Learning to answer questions from image using convolutional neural network.  In Proceedings of AAAI, volume 3, page 16, 2016.
  * [23] Mateusz Malinowski and Mario Fritz.  A multi-world approach to question answering about real-world scenes based on uncertain input.  In NeurIPS, pages 1682–1690, 2014.
  * [24] Mateusz Malinowski, Marcus Rohrbach, and Mario Fritz.  Ask your neurons: A neural-based approach to answering questions about images.  In Proceedings of IEEE ICCV, pages 1–9, 2015.
  * [25] Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bordes, and Jason Weston.  Key-value memory networks for directly reading documents.  arXiv preprint arXiv:1606.03126, 2016.
  * [26] Medhini Narasimhan, Svetlana Lazebnik, and Alexander Schwing.  Out of the box: Reasoning with graph convolution nets for factual visual question answering.  In NeurIPS, pages 2654–2665, 2018.
  * [27] Medhini Narasimhan and Alexander G Schwing.  Straight to the facts: Learning knowledge base retrieval for factual visual question answering.  In Proceedings of ECCV, pages 451–468, 2018.
  * [28] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer.  Automatic differentiation in pytorch.  2017\.
  * [29] Jeffrey Pennington, Richard Socher, and Christopher D Manning.  Glove: Global vectors for word representation.  In Proceedings of EMNLP, pages 1532–1543, 2014.
  * [30] Mengye Ren, Ryan Kiros, and Richard Zemel.  Image question answering: A visual semantic embedding model and a new dataset.  NeurIPS, 1(2):5, 2015.
  * [31] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks.  In NeurIPS, pages 91–99, 2015.
  * [32] Niket Tandon, Gerard De Melo, and Gerhard Weikum.  Acquiring comparative commonsense knowledge from the web.  In AAAI, pages 166–172, 2014.
  * [33] Damien Teney, Peter Anderson, Xiaodong He, and Anton Van Den Hengel.  Tips and tricks for visual question answering: Learnings from the 2017 challenge.  In Proceedings of the IEEE CVPR, pages 4223–4232, 2018.
  * [34] Peng Wang, Qi Wu, Chunhua Shen, Anthony Dick, and Anton Van Den Hengel.  Fvqa: Fact-based visual question answering.  IEEE TPAMI, 40(10):2413–2427, 2017.
  * [35] Peng Wang, Qi Wu, Chunhua Shen, Anton van den Hengel, and Anthony Dick.  Explicit knowledge-based reasoning for visual question answering.  page 1290–1296, 2017.
  * [36] Xinyu Wang, Yuliang Liu, Chunhua Shen, Chun Chet Ng, Canjie Luo, Lianwen Jin, Chee Seng Chan, Anton van den Hengel, and Liangwei Wang.  On the general value of evidence, and bilingual scene-text visual question answering.  In Proceedings of the IEEE/CVF CVPR, pages 10126–10135, 2020.
  * [37] Caiming Xiong, Stephen Merity, and Richard Socher.  Dynamic memory networks for visual and textual question answering.  In Proceedings of ICML, pages 2397–2406, 2016.
  * [38] Chengxiang Yin, Jian Tang, Zhiyuan Xu, and Yanzhi Wang.  Memory augmented deep recurrent neural network for video question answering.  IEEE transactions on neural networks and learning systems, 31(9):3159–3167, 2020.
  * [39] Chengxiang Yin, Kun Wu, Zhengping Che, Bo Jiang, Zhiyuan Xu, and Jian Tang.  Hierarchical graph attention network for few-shot visual-semantic learning.  In Proceedings of the IEEE ICCV, pages 2177–2186, 2021.
  * [40] Amir Zadeh, Michael Chan, Paul Pu Liang, Edmund Tong, and Louis-Philippe Morency.  Social-iq: A question answering benchmark for artificial social intelligence.  In Proceedings of IEEE CVPR, pages 8807–8817, 2019.
  * [41] Yuke Zhu, Oliver Groth, Michael Bernstein, and Li Fei-Fei.  Visual7w: Grounded question answering in images.  In Proceedings of the IEEE CVPR, pages 4995–5004, 2016.
  * [42] Zihao Zhu, Jing Yu, Yujing Wang, Yajing Sun, Yue Hu, and Qi Wu.  Mucko: Multi-layer cross-modal knowledge reasoning for fact-based visualquestion answering.  arXiv preprint arXiv:2006.09073, 2020.


# LSTM-based Video Quality Prediction Accounting for Temporal Distortions in
Videoconferencing Calls

###### Abstract

Current state-of-the-art video quality models, such as VMAF, give excellent
prediction results by comparing the degraded video with its reference video.
However, they do not consider temporal distortions (e.g., frame freezes or
skips) that occur during videoconferencing calls. In this paper, we present a
data-driven approach for modeling such distortions automatically by training
an LSTM with subjective quality ratings labeled via crowdsourcing. The videos
were collected from live videoconferencing calls in 83 different network
conditions. We applied QR codes as markers on the source videos to create
aligned references and compute temporal features based on the alignment
vectors. Using these features together with VMAF core features, our proposed
model achieves a PCC of 0.99 on the validation set. Furthermore, our model
outputs per-frame quality that gives detailed insight into the cause of video
quality impairments. The VCM model and dataset are open-sourced at
https://github.com/microsoft/Video_Call_MOS.

Index Terms—  Video quality, machine learning, QoE

##  1 Introduction

One of the most important performance metrics for evaluating videoconferencing
(VC) system is the perceived video quality of a call. The ground truth quality
can only be assessed in subjective experiments, for example, according to
ITU-T (International Telecommunication Union) Recommendation P.910 [1], in
which participants are asked to view sample videos and rate their quality. The
average rating across all participants gives the so-called Mean Opinion Score
(MOS). Recently, crowdsourcing has become increasingly popular for conducting
video-quality experiments. Implementations, such as P.910 Crowd [2], allow for
conducting accurate video-quality experiments with high reproducibility.

Because subjective experiments are time-consuming and costly, signal-based
prediction models have been developed that are able to predict the quality of
a given video. The most commonly used video quality metrics are PSNR (Peak
Signal to Noise Ratio), SSIM [3], MS-SSIM [4], and VMAF [5], which are
computed on a frame basis and with subsequent averaging across all frame
scores to give the overall quality. Although other pooling mechanisms have
been studied in order to take recency effects into account, in most studies
mean pooling achieved the best performance [6, 7]. VMAF is the only of those
metrics that include a temporal component, which considers temporal masking
effects. However, video transmitted during VC calls can be affected by a
number of temporal distortions that are perceived as frame freezes, frame
skips, frame rate variations (e.g., video is played back faster after delayed
packets arrive), or generally low frame rate. These kinds of distortions
cannot be covered by the aforementioned quality metrics as they are mostly
frame-based and do not take time dependencies into account.

More recently, deep learning-based video quality models such as VSFA [8],
DeepVQA [9], CompressedVQA [10], and C3DVQA [11] have been introduced.
Although some of them take temporal-memory effects or temporal masking into
account and apply more advanced pooling mechanisms, they are neither designed
for nor trained on videos with temporal distortions as they appear in VC
calls. In general, most of the temporal video quality modeling has been
focused around streaming applications [12] that are subject to different
degradation types than VC systems. The MOVIE [13], ST-MAD [14], and STRRED
[15] models consider visual perception of motion artifacts. VQM_VFD [16] takes
the impact of variable frame delays (including freezes) into account by
combining 8 different parameters with a shallow neural network. ITU-T Rec.
P.1203 [17] is a parametric bitstream-based audiovisual quality prediction
model for adaptive streaming services. ATLAS [18] includes rebuffering-aware
and memory-driven effects. The model in [19] also considers rebuffering in
video streams by applying recurrent and dynamic neural networks. ST-GREED [20]
applies entropy variations between reference and distorted videos to consider
quality variations caused by frame rate changes.

Another challenge for full reference VC quality prediction is the alignment of
the received video with the original video. In the VQM_VFD model, a heuristic
algorithm based on pixel-by-pixel comparisons is applied, which is prone to
alignment errors. Instead, we apply markers to the source videos, which allows
for reading the corresponding reference frame index from the degraded frames.
To the best of our knowledge, this is the first work in which the alignment
vector is leveraged to model the impact of temporal distortions as they occur
during video calls with a data-driven approach. To this end, we use a
recurrent neural network and directly input features based on the alignment
vector as a sequence, fused together with image quality metrics.

##  2 Dataset

To train and evaluate the presented model, a new dataset with recordings from
live Microsoft Teams calls was created. The recorded videos are based on ten
source videos. Eight of the videos contain one person speaking into the camera
and two of the videos contained two people talking to each other. The video
resolution of the source videos is 1920x1080 with a frame rate of 30 FPS. In a
preprocessing step, two QR codes, containing the original frame index, were
drawn onto the source videos to make an alignment between degraded and
reference videos possible (see Subsection 3.1).

To create the dataset, calls between two machines and various emulated real-
call network scenarios were conducted. The scenarios comprised of different
fixed bandwidths, burst loss, random loss, fluctuating bandwidth, cross
traffic, and combinations of these conditions. During the call, the video
bitrate and resolution are adapted depending on the network condition. As a
consequence, the quality of the received videos changes over time and may
contain different bitrates, switches between video resolutions, frame rate
variation, and frozen or skipped frames. The degraded videos were captured on
the receiving side with 30 FPS and cropped from the call duration of 2 minutes
to clips of 6 seconds, where a random segment of the call was chosen as the
clip. In that way, 1628 videos with 83 different network profiles were
collected. The videos were annotated for their overall quality on an ACR
(Absolute Category Rating) scale according to P.910 [1] via crowdsourcing
using the P.910 Toolkit111https://github.com/microsoft/P.910 [2]. On average,
17 valid votes were collected per clip. To split the dataset, the videos that
are based on two of the ten source videos were selected for validation,
resulting in 1299 training set videos and 329 validation set videos, where the
training and validation sets are sharing the same network profiles but having
different source videos.

##  3 Method

###  3.1 Alignment

Most video quality models compare reference video frames with degraded frames,
but this is challenging for live video calls as recordings are not aligned
with the original reference video. We added QR codes to the source videos to
enable alignment, but reading codes in poor network conditions can be
difficult. To improve detection, we added two identical QR codes to each
video, one in the top-left and one in the bottom-right corner.

To create an aligned reference video, the QR codes from the degraded videos
are read for each frame. This gives an index vector $r(i)$ with the reference
frame indices for each degraded frame. In the case of frozen frames the vector
contains consecutive repetitions of the same reference frame index, in the
case of frame skips the vector is missing certain reference frame indices.
Then we create a new reference video with the frame order given by $r(i)$.

###  3.2 Features

Because VMAF is widely used and has been shown to give robust quality
predictions for videos, we use the same input features as VMAF to cover the
image quality of the individual frames. However, these features cannot cover
temporal quality degradation caused by freezes/skips. For this reason, we add
two additional frame freeze features that are based on the reference frame
indices $r(i)$.

#### VMAF core features

  * •

VIF [21] – Image quality metric that measures the information of fidelity loss
between degraded and reference images. VMAF uses a modified version where the
loss of fidelity in each scale is included as an elementary metric [5].

  * •

ADM [22] – Image quality metric that measures the loss of details that affect
the content visibility and additionally the redundant impairment that
distracts the viewer’s attention.

  * •

Motion2 [5] – Temporal metric that measures the temporal difference between
adjacent frames by calculating the average absolute pixel difference for the
luminance component.

#### Frame freeze feature

  * •

Frame skips – Measures the reference frame indices difference between
consecutive frames as follows: $s_{i}=r_{i}-r_{i-1}$ for $i\in[1,N-1]$ with
$s_{0}=0$, where $r(i)$ is the reference indices vector and $N$ is the number
of frames of the degraded video.

  * •

Frame freezes – Simple measure to capture the number of consecutive frozen
frames according to Algorithm 1.

Algorithm 1 Pseudo code for computing freeze feature

Input: Reference frame index $r_{i}$ for each degraded frame index
$i\in[0,N-1]$, where $N$ is the number of frames of the degraded video.
Output: Freeze value $f_{i}$.

1: $f_{0}=0$

2: for $i=1$; $i<N$; $i=i+1$ do

3: if $r_{i}==r_{i-1}$ then

4: $f_{i}=f_{i-1}+1$

5: else

6: $f_{i}=0$

7: end if

8: end for

###  3.3 Model

As a model, we apply a simple LSTM (Long Short-Term Memory) network, as it is
well-suited for modeling sequences and time dependencies. We use 6 layers with
each 256 hidden units, although in our experiments smaller models (e.g., with
1 or 2 layers and 128 hidden units) reached similar performances. The model
architecture is shown in Figure 1. Besides predicting the overall video
quality, another goal of this work is to output the quality per frame,
including the impact of temporal effects, such as frozen frames. In order to
force the model to output a single quality value per frame, instead of
applying time pooling first and then using a linear layer for the final
output, we apply the linear layer directly after the LSTM to reduce the
feature size from 256 to 1. Then we apply an average pooling layer to compute
the final overall video quality. In this way, also the common problem of
choosing a suitable time pooling method is solved, as the LSTM learns to weigh
the impact of the individual frames on the overall quality automatically.

Fig. 1: Block diagram of the proposed neural network.

##  4 Results and Discussion

In this section, the results of the model on the validation set are presented.
At first, we present the results compared to the default VMAF model. Then, we
present the results of an ablation study, where we compare the results for
different input features and a retrained VMAF model. All of the presented
scores were mapped to the subjective MOS with a Sigmoid function. The mapped
values were then used for the plots and computing the figures of metrics PCC
(Pearson’s correlation coefficient) and RMSE (root-mean-square error). The
mapping is particularly necessary for VMAF, SSIM, and PSNR. For example, the
scores of the default VMAF model range between 0–100, while the videos in our
dataset are rated in MOS between 1–5. All of the presented results are
computed on a per-file base.

###  4.1 Validation set results

(a) VMAF

(b) Proposed

Fig. 2: Scatterplot showing the ground truth and predicted MOS values per file
on the validation set.

Figure 2 shows the scatter plots of the predicted values for the default VMAF
model (a) and the proposed LSTM with VMAF and temporal features as input (b).
The proposed model outperforms the default VMAF model with a PCC increase from
0.94 to 0.99. The plots also show that most of the VMAF outliers are caused by
overestimating the video quality, presumably because these video samples
contain video freezes that are not captured by VMAF.

Fig. 3: Per-frame quality over time for two example videos.

To further analyze this presumption, we plot the per-frame video quality over
time for two example videos in Figure 3. The black line represents the overall
MOS rating from the crowdsourcing experiment, the orange line shows the per-
frame VMAF score, and the blue line shows the per-frame quality of the
proposed LSTM model. In addition to the quality predictions, we also plot the
frame freeze and skip features, which are shown in frames divided by 10 (i.e.,
in the plot a “freeze” value of 1 represents 10 consecutive frozen frames
which corresponds to 0.33 seconds). In the top graph, a video without any
major freezes or frame skips is shown. Both VMAF and the LSTM predict the
quality to be close to the ground truth MOS. The bottom graph shows a video
with two video freezes. VMAF clearly overestimates the quality of this video.
The LSTM begins with a similar prediction as VMAF but lowers the quality
prediction once the first frame freeze starts. After the freeze has ended and
the video continues playing, the LSTM increases its quality prediction again.
By considering the freezes the model reaches an average video quality
prediction close to the ground truth MOS. These results show that the per-
frame quality prediction of the proposed model works very well and can be used
to analyze the root cause of videos with poor quality.

###  4.2 Ablation study

In this subsection, we study the influence of the different input features on
the results and compare the model to a retrained VMAF model. For the
retraining of VMAF, we used the default SVR (Support Vector Regression)
settings with $\gamma=0.05$, $C=4.0$, and $\nu=0.9$ and the 11 VMAF core
features VIF (vif_scale0, vif_scale1, vif_scale2, vif_scale3), ADM (adm2,
adm_scale0, adm_scale1, adm_scale2, adm_scale3), and Motion (motion, motion2).
Table 1 shows the validation set results for the default and retrained VMAF.
We also show the results for the image quality metrics PSNR, SSIM, and MS-
SSIM, which are commonly used as baseline metrics. It should be noted that the
results for the LSTM show the average PCC and RMSE over three training runs
for each feature configuration, where the best checkpoint of each training run
was selected.

Table 1: Validation set results for baseline and proposed model using different input features. Model | PCC | RMSE
---|---|---
PSNR | 0.928 | 0.415
SSIM | 0.951 | 0.342
MS-SSIM | 0.946 | 0.360
VMAF | 0.937 | 0.388
VMAF (Retrained) | 0.965 | 0.289
LSTM - VIF / ADM | 0.956 | 0.335
LSTM - VIF / ADM / Motion | 0.983 | 0.218
LSTM - VIF / ADM / Skip / Freeze | 0.984 | 0.213
LSTM - VIF / ADM / Skip | 0.979 | 0.253
LSTM - VIF / ADM / Freeze | 0.983 | 0.213
LSTM - VIF / ADM / Motion / Skip / Freeze | 0.985 | 0.206
LSTM - ” / ” / ” / ” / ” / PSNR / SSIM | 0.983 | 0.211

Interestingly, SSIM outperforms MS-SSIM and the default VMAF model with a PCC
of 0.951. Retraining VMAF notably improves the performance and increases the
PCC from 0.937 to 0.965. The LSTM with the same VMAF core input features
(VIF/ADM/Motion) outperforms the retrained VMAF model with a PCC of 0.983,
showing that LSTM models are better suited to model the temporal distortions
in VC calls than the default SVR used by VMAF. It can further be noted that
adding either of the temporal features (Motion, Skip, or Freeze) increases the
results from a PCC of 0.956, when only the image quality metrics VIF and ADM
are applied, to a PCC of around 0.98.

The LSTM seems to be able to consider frame freezes or skips with any of the
temporal features. The purpose of the VMAF Motion feature is to capture the
temporal masking of coding artifacts and is calculated solely on the basis of
the reference video. In the case of high motion values, VMAF will judge
distortions less severely as the human eye does not perceive some distortions
during high motion between frames. However, since we aligned our reference
videos to the degraded video, the reference also contains the frame freezes of
the degraded video and thus leads to a value of zero for frozen frames as the
frames are exactly identical. Therefore, the LSTM can use the motion feature
to detect video freezes instead of only using it to capture the temporal
masking effect. That being said, the Motion feature is not able to capture
frame rate variations. However, in practice, such changes are usually preceded
by frame freezes which already impact the perceived quality. Therefore no
significant performance improvements could be observed on our dataset when the
skip or freeze features were added. Furthermore, the LSTM could confuse a
video with still sequences with frozen frames when Motion is used as the only
temporal feature (in cases where the reference alignment is not done via video
markers as they will lead to non-zero Motion values). Although the performance
difference between the temporal features is only marginal, overall, the best
performance can be achieved by including all of them, in particular when
considering the RMSE. Adding the Skip and Freeze features decreases the RMSE
from 0.218 to 0.206, compared to when only Motion is used. Adding PSNR and
SSIM to the LSTM model did not further improve the results.

##  5 Conclusions

We proposed the open-sourced VCM model based on a simple but highly efficient
data-driven approach for modeling temporal distortions in VC videos. An open-
source dataset with live video calls in 83 different network conditions was
created in which we preprocessed the source videos with QR code markers for
the alignment between degraded and reference videos. We show that by using
VMAF core features to cover the non-temporal visual quality of the video
frames together with temporal features based on the alignment vector, the LSTM
learns to model temporal distortions automatically and achieves very high
accuracy with a PCC of 0.99 on the validation set. Our model further outputs
per-frame MOS that gives detailed insight into the cause of degraded quality.
In future work, the VMAF core features could be replaced with an end-to-end
deep learning approach to further improve the results.

## References

  * [1] ITU-T Rec. P.910,  “Subjective video quality assessment methods for multimedia applications,” 2022.
  * [2] Babak Naderi and Ross Cutler,  “A crowdsourcing approach to video quality assessment,”  arXiv preprint arXiv:2204.06784, 2022.
  * [3] Zhou Wang, Alan Conrad Bovik, Hamid R. Sheikh, and Eero P. Simoncelli,  “Image quality assessment: from error visibility to structural similarity,”  IEEE Transactions on Image Processing, 2004.
  * [4] Zhou Wang, Eero P. Simoncelli, and Alan Conrad Bovik,  “Multiscale structural similarity for image quality assessment,”  The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003, 2003.
  * [5] Zhi Li, Anne Aaron, Ioannis Katsavounidis, Anush Moorthy, and Megha Manohara,  “Toward a practical perceptual video quality metric,”  Tech. Rep., Netflix, 2016.
  * [6] Michael Seufert, Martin Slanina, Sebastian Egger, and Meik Kottkamp,  ““To pool or not to pool”: A comparison of temporal pooling methods for HTTP adaptive video streaming,”  2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX), 2013.
  * [7] Zhi Li, Christos Bampis, Julie Novak, Anne Aaron, Kyle Swanson, Anush Moorthy, and Jan De Cock,  “VMAF: The journey continues,”  Tech. Rep., Netflix, 2018.
  * [8] Dingquan Li, Tingting Jiang, and Ming Jiang,  “Quality assessment of in-the-wild videos,”  Proceedings of the 27th ACM International Conference on Multimedia, 2019.
  * [9] Woojae Kim, Jongyoo Kim, Sewoong Ahn, Jinwoo Kim, and Sanghoon Lee,  “Deep video quality assessor: From spatio-temporal visual sensitivity to a convolutional neural aggregation network,”  in ECCV, 2018.
  * [10] Wei Sun, Tao Wang, Xiongkuo Min, Fuwang Yi, and Guangtao Zhai,  “Deep learning based full-reference and no-reference quality assessment models for compressed UGC videos,”  2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 2021.
  * [11] Munan Xu, Junming Chen, Haiqiang Wang, Shan Liu, Ge Li, and Zhiqiang Bai,  “C3DVQA: Full-reference video quality assessment with 3D convolutional neural network,”  ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020.
  * [12] Christos George Bampis, Zhi Li, Anush K. Moorthy, Ioannis Katsavounidis, Anne Aaron, and Alan Conrad Bovik,  “Study of temporal effects on subjective video quality of experience,”  IEEE Transactions on Image Processing, 2017.
  * [13] Kalpana Seshadrinathan and Alan Conrad Bovik,  “Motion tuned spatio-temporal quality assessment of natural videos,”  IEEE Transactions on Image Processing, 2010.
  * [14] Phong V. Vu, Cuong T. Vu, and Damon M. Chandler,  “A spatiotemporal most-apparent-distortion model for video quality assessment,”  2011 18th IEEE International Conference on Image Processing, 2011\.
  * [15] Rajiv Soundararajan and Alan Conrad Bovik,  “Video quality assessment by reduced reference spatio-temporal entropic differencing,”  IEEE Transactions on Circuits and Systems for Video Technology, 2013\.
  * [16] Margaret H. Pinson, Lark Kwon Choi, and Alan Conrad Bovik,  “Temporal video quality model accounting for variable frame delay distortions,”  IEEE Transactions on Broadcasting, 2014.
  * [17] Werner Robitza, Steve Goering, Alexander Raake, David Lindero, Gunnar Heikkilä, Jorgen Gustafsson, Peter List, Bernhard Feiten, Ulf Wüstenhagen, Marie-Neige Garcia, Kazuhisa Yamagishi, and Simon Broom,  “Http adaptive streaming qoe estimation with itu-t rec. p. 1203: open databases and software,”  Proceedings of the 9th ACM Multimedia Systems Conference, 2018.
  * [18] Christos George Bampis and Alan Conrad Bovik,  “Learning to predict streaming video QoE: Distortions, rebuffering and memory,”  IEEE Signal Processing Letters, 2017.
  * [19] Christos George Bampis, Zhi Li, Ioannis Katsavounidis, and Alan Conrad Bovik,  “Recurrent and dynamic models for predicting streaming video quality of experience,”  IEEE Transactions on Image Processing, 2018.
  * [20] Pavan C. Madhusudana, Neil Birkbeck, Yilin Wang, Balu Adsumilli, and Alan Conrad Bovik,  “ST-GREED: Space-time generalized entropic differences for frame rate dependent video quality prediction,”  IEEE Transactions on Image Processing, 2021.
  * [21] Hamid R. Sheikh and Alan Conrad Bovik,  “Image information and visual quality,”  2004 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2004.
  * [22] Songnan Li, Fan Zhang, Lin Ma, and King Ngi Ngan,  “Image quality assessment by separately evaluating detail losses and additive impairments,”  IEEE Transactions on Multimedia, 2011.


Analysis of the $\overline{\Xi}_{cc}\Xi_{cc}$ hexaquark molecular state with
the QCD sum rules

Zhi-Gang Wang 111E-mail<EMAIL_ADDRESS>
Department of Physics, North China Electric Power University, Baoding 071003,
P. R. China

###### Abstract

In this work, we construct the color-singlet-color-singlet type six-quark
pseudoscalar current to investigate the $\overline{\Xi}_{cc}\Xi_{cc}$
hexaquark molecular state with the QCD sum rules, the predicted mass
$M_{X}\sim 7.2\,\rm{GeV}$ supports assigning the $X(7200)$ to be the
$\overline{\Xi}_{cc}\Xi_{cc}$ hexaquark molecular state with the quantum
numbers $J^{PC}=0^{-+}$. The $X(7200)$ can decay through fusions of the
$\bar{c}c$ and $\bar{q}q$ pairs, we can search for the $X(7200)$ and explore
its properties in the $J/\psi J/\psi$, $\chi_{c1}\chi_{c1}$,
$D^{*}\bar{D}^{*}$ and $D_{1}\bar{D}_{1}$ invariant mass spectrum in the
future.

PACS number: 12.39.Mk, 12.38.Lg

Key words: Hexaquark molecular state, QCD sum rules

##  1 Introduction

In 2017, the LHCb collaboration observed the doubly charmed baryon state
$\Xi_{cc}^{++}$ in the $\Lambda_{c}^{+}K^{-}\pi^{+}\pi^{+}$ invariant mass
distribution [1]. The observation of the $\Xi_{cc}^{++}$ makes a great
progress on the spectroscopy of the doubly charmed baryon states, tetraquark
states and pentaquark states.

In 2020, the LHCb collaboration investigated the $J/\psi J/\psi$ invariant
mass distributions and observed a narrow structure about $6.9\,\rm{GeV}$ and a
broad structure just above the $J/\psi J/\psi$ threshold at
$p_{T}>5.2\,\rm{GeV}$ with the global significances larger than $5\sigma$ [2].
The Breit-Wigner mass and width of the $X(6900)$ are $M_{X}=6905\pm 11\pm
7\,\rm{MeV}$ and $\Gamma_{X}=80\pm 19\pm 33\,\rm{MeV}$, respectively.
Furthermore, they also observed some vague structures around $7.2\,\rm{GeV}$,
which coincides with the $\overline{\Xi}_{cc}\Xi_{cc}$ threshold
$7242.4\,\rm{MeV}$ [2]. The energy is sufficient to create a baryon-antibaryon
pair $\overline{\Xi}_{cc}\Xi_{cc}$ containing the valence quarks
$ccq\,\bar{c}\bar{c}\bar{q}$.

In the dynamical diquark model, the first radial excited states of the D-wave
tetraquark states with the valence quarks $cc\bar{c}\bar{c}$ have the masses
about $7.2\,\rm{GeV}$, the insignificant enhancement $X(7200)$ may be a
combination of some 2P and (or) 2D diquark-antidiquark type $cc\bar{c}\bar{c}$
states with threshold effects of the transitions
$\Xi_{cc}\overline{\Xi}_{cc}\to J/\psi J/\psi$ [3]. The assignment of the
$Y(4630)$ serves as a benchmark, in this model, the $Y(4630)$ can be assigned
to be a diquark-antidiquark type tetraquark state, fragmentation of the color
flux-tube connecting the diquark-antidiquark pair $cq\bar{c}\bar{q}$ leads to
the lowest-lying baryon-antibaryon pair $\Lambda_{c}^{+}\Lambda_{c}^{-}$ [4].
Just like the $Y(4630)$, fragmentation of the color flux-tube connecting the
diquark-antidiquark pair $cc\bar{c}\bar{c}$ in the $X(7200)$ leads to the
lowest-lying baryon-antibaryon pair $\Xi_{cc}\overline{\Xi}_{cc}$, then
translates to the $J/\psi J/\psi$ pair.

In the V-baryonium tetraquark scenario and the string-junction scenario, which
share the same feature, the exotic $X$, $Y$ and $Z$ states are genuine
tetraquarks rather than molecular states, they have a baryonic vertex (or
string-junction) attaching a $cc$-diquark in color antitriplet, which is
connected by a string to an anti-baryonic vertex (or string-junction)
attaching a $\bar{c}\bar{c}$-antidiquark in color triplet. In the V-baryonium
tetraquark scenario, the un-conformed structure $X(7200)$ is assigned to be
the first radially excited state of the $X(6900)$, the decays to the baryon-
antibaryon pair $\Xi_{cc}\overline{\Xi}_{cc}$ can occur via breaking the
string and creating a quark-antiquark pair [5]. In the string junction
scenario, the $X(6900)$ and $X(7200)$ are assigned to be the 2S tetraquark
states with the quantum numbers $J^{PC}=0^{++}$ and $2^{++}$, respectively
[6].

If the $\overline{\Xi}_{cc}^{(\ast)}\Xi_{cc}^{(\ast)}$ system and
$D^{(\ast)}\bar{D}^{(\ast)}$ system are related to each other via heavy
antiquark-diquark symmetry, several molecular states can be predicted based on
the contact-range effective field theory, the $\overline{\Xi}_{cc}\Xi_{cc}$
molecular states which have the quantum numbers $J^{PC}=0^{-+}$ and (or)
$1^{--}$ maybe contribute to the $X(7200)$ [7].

The exotic $X$, $Y$, $Z$ and $P$ states always lie near two-particle
thresholds, such as

$\displaystyle D\bar{D}^{*}/\bar{D}D^{*}$ $\displaystyle:$ $\displaystyle
X(3872)\,,\,Z_{c}(3885/3900)\,,$ $\displaystyle D^{*}\bar{D}^{*}$
$\displaystyle:$ $\displaystyle Z_{c}(4020/4025)\,,$ $\displaystyle
D\bar{D}_{s}^{*}/D^{*}\bar{D}_{s}$ $\displaystyle:$ $\displaystyle
Z_{cs}(3985/4000)\,,$ $\displaystyle D_{s}^{*}\bar{D}_{s}^{*}$
$\displaystyle:$ $\displaystyle X(4140)\,,$ $\displaystyle
D\bar{D}_{1}/\bar{D}D_{1}$ $\displaystyle:$ $\displaystyle
Y(4260/4220)\,,\,Z_{c}(4250)\,,$ $\displaystyle
D^{*}\bar{D}_{0}/\bar{D}^{*}D_{0}$ $\displaystyle:$ $\displaystyle
Y(4360/4320)\,,$ $\displaystyle\bar{D}\Sigma_{c}$ $\displaystyle:$
$\displaystyle P_{c}(4312)\,,$
$\displaystyle\bar{D}\Xi^{\prime}_{c}/\bar{D}^{*}\Xi_{c}$ $\displaystyle:$
$\displaystyle P_{cs}(4459)\,,$ $\displaystyle\bar{D}\Sigma^{*}_{c}$
$\displaystyle:$ $\displaystyle P_{c}(4380)\,,$
$\displaystyle\bar{D}^{*}\Sigma_{c}$ $\displaystyle:$ $\displaystyle
P_{c}(4440/4457)\,,$
$\displaystyle\Lambda_{c}^{+}\Lambda_{c}^{-}/f_{0}(980)\psi^{\prime}$
$\displaystyle:$ $\displaystyle Y(4630/4660)\,,$ $\displaystyle
B\bar{B}^{*}/\bar{B}B^{*}$ $\displaystyle:$ $\displaystyle Z_{b}(10610)\,,$
$\displaystyle B^{*}\bar{B}^{*}$ $\displaystyle:$ $\displaystyle
Z_{b}(10650)\,,$ (1)

it is natural to assume that they are molecular states composed of two color-
singlet constituents and investigate their properties with the QCD sum rules
[8, 9, 10, 12, 13, 14]. The QCD sum rules approach is powerful theoretical
tool in exploring the hadron properties, and has given many successful
descriptions of the masses and decay widths of the tetraquark and pentaquark
(molecular) states [8, 9, 10, 12, 13, 14], and has been successfully applied
to investigate the dibaryon states and baryonium states [15, 16].

In the present work, we explore the color-singlet-color-singlet type hexaquark
molecular state $\overline{\Xi}_{cc}\Xi_{cc}$ with the quantum numbers
$J^{PC}=0^{-+}$, and make possible assignment of the $X(7200)$.

The article is arranged as follows: we derive the QCD sum rules for the mass
and pole residue of the $\overline{\Xi}_{cc}\Xi_{cc}$ molecular state in
section 2; in section 3, we present the numerical results and discussions;
section 4 is reserved for our conclusion.

##  2 QCD sum rules for the $\overline{\Xi}_{cc}\Xi_{cc}$ hexaquark molecular
state

Let us write down the two-point correlation function $\Pi(p^{2})$ firstly,

$\displaystyle\Pi(p^{2})$ $\displaystyle=$ $\displaystyle i\int
d^{4}xe^{ip\cdot x}\langle 0|T\left\\{J(x)J^{\dagger}(0)\right\\}|0\rangle\,,$
(2)

where

$\displaystyle J(x)$ $\displaystyle=$
$\displaystyle\bar{J}_{cc}(x)i\gamma_{5}J_{cc}(x)\,,$ $\displaystyle
J_{cc}(x)$ $\displaystyle=$
$\displaystyle\varepsilon^{ijk}c^{T}_{i}(x)C\gamma_{\alpha}c_{j}(x)\gamma^{\alpha}\gamma^{5}q(x)\,,$
(3)

$q=u$, $d$, the $i$, $j$, $k$ are color indexes, the current $J_{cc}(x)$ has
the same quantum numbers as the doubly charmed baryon state $\Xi_{cc}$, and
couples potentially to the color-singlet clusters with the same quantum
numbers as the $\Xi_{cc}$ [17].

At the hadron side, we isolate the ground state contribution of the
pseudoscalar hexaquark molecular state, we obtain the result,

$\displaystyle\Pi(p^{2})$ $\displaystyle=$
$\displaystyle\frac{\lambda_{X}^{2}}{M^{2}_{X}-p^{2}}+\cdots\,\,,$ (4)

where the pole residue $\lambda_{X}$ is defined by

$\displaystyle\langle 0|J(0)|X(p)\rangle$ $\displaystyle=$
$\displaystyle\lambda_{X}\,.$ (5)

In Eq.(4), we have neglected the two-particle scattering state contributions,
just like in the QCD sum rules for the tetraquark (molecular) states. In
Ref.[18], Lucha, Melikhov and Sazdjian obtain the conclusion that ”A possible
exotic tetraquark state may appear only in $N_{c}$-subleading contributions to
the QCD Green functions” based on the naive large-$N_{c}$ counting. An
explicit analysis by S. Narison et al at the real word $N_{c}=3$ shows the
opposite due to the induced loop factor missed in a naive $N_{c}$ counting
rule [19]. In fact, as pointed in Ref.[20], without excluding the
contributions of factorizable Feynman diagrams in the color space to the QCD
sum rules by hand, we cannot obtain the conclusion that the factorizable parts
of the operator product expansion series cannot have any relationship to the
possible tetraquark bound states. For the baryon, tetraquark, pentaquark
states with string junctions [21], the standard large-$N_{c}$ counting rules
for the ordinary mesons cannot be trivially extrapolated to the exotic hadrons
but should be modified for being properly applied at the real word $N_{c}=3$.
All in all, we can examine the predictions of the multiquark states based on
the QCD sum rules in different channels in the future.

In the QCD side, we carry out the operator product expansion up to the vacuum
condensates of dimension 10 in a consistent way. There are four heavy quark
propagators and two light quark propagators in the correlation function
$\Pi(p^{2})$ after accomplishing the Wick’s contractions. If each heavy quark
line emits a gluon, and each light quark line contributes a quark-antiquark
pair, we obtain a quark-gluon operator
$g_{s}G_{\mu\nu}g_{s}G_{\alpha\beta}g_{s}G_{\rho\sigma}g_{s}G_{\lambda\tau}\bar{q}q\bar{q}q$,
which is of dimension $14$, we should take account of the vacuum condensates
of dimensions up to dimension 14. In the QCD sum rules for the hidden-charm or
hidden-bottom tetraquark (molecular) states, pentaquark (molecular) states, we
usually take the truncation $\mathcal{O}(\alpha_{s}^{k})$ with $k\leq 1$ [10,
13, 22, 23]. If we also take the truncation $k\leq 1$ to discard the quark-
gluon operators of the orders $\mathcal{O}(\alpha_{s}^{>1})$ in the present
work, the operator product expansion is terminated at the vacuum condensates
of dimension $10$.

In the QCD sum rules for the triply-charmed diquark-diquark-diquark type
hexaquark states and triply-charmed dibaryon states, there are three light
quark propagators and three heavy quark propagators in the correlation
functions after accomplishing the Wick’s contractions [15, 24]. Again, if each
heavy quark line emits a gluon and each light quark line contributes quark-
antiquark pair, we obtain a quark-gluon operator
$g_{s}G_{\mu\nu}g_{s}G_{\alpha\beta}g_{s}G_{\lambda\tau}\bar{q}q\bar{q}q\bar{q}q$,
which is of dimension 15. If we take the truncations $k\leq 1$, the operator
product expansion is terminated at the vacuum condensates of dimension $13$.
The operator
$g_{s}G_{\mu\nu}g_{s}G_{\alpha\beta}g_{s}G_{\lambda\tau}\bar{q}q\bar{q}q\bar{q}q$
leads to the vacuum condensates
$\langle\frac{\alpha_{s}GG}{\pi}\rangle\langle\bar{q}q\rangle^{2}\langle\bar{q}g_{s}\sigma
Gq\rangle$, $\langle g_{s}^{3}GGG\rangle\langle\bar{q}q\rangle^{3}$ and
$\langle\bar{q}g_{s}\sigma Gq\rangle^{3}$, we calculate the vacuum condensate
$\langle\bar{q}g_{s}\sigma Gq\rangle^{3}$, and observe that its small
contribution can be neglected safely [15, 24]. The vacuum condensates $\langle
g_{s}^{3}GGG\rangle\langle\bar{q}q\rangle^{3}$ and
$\langle\frac{\alpha_{s}GG}{\pi}\rangle\langle\bar{q}q\rangle^{2}\langle\bar{q}g_{s}\sigma
Gq\rangle$ receive additional suppressions due to the small contributions of
the gluon condensate and three-gluon condensate. In Ref.[25], we re-explore
the mass spectrum of the ground state triply-heavy baryon states with the QCD
sum rules by taking account of the three-gluon condensate $\langle
g_{s}^{3}GGG\rangle$ for the first time, and observe that the contributions of
the three-gluon condensate $\langle g_{s}^{3}GGG\rangle$ are tiny indeed. In
summary, the truncations $k\leq 1$ work very good.

In the present wok, we take account of the vacuum condensates
$\langle\bar{q}q\rangle$, $\langle\bar{q}g_{s}\sigma Gq\rangle$,
$\langle\bar{q}q\rangle^{2}$, $\langle\bar{q}q\rangle\langle\bar{q}g_{s}\sigma
Gq\rangle$, $\langle\frac{\alpha_{s}GG}{\pi}\rangle$,
$\langle\bar{q}q\rangle\langle\frac{\alpha_{s}GG}{\pi}\rangle$,
$\langle\bar{q}q\rangle^{2}\langle\frac{\alpha_{s}GG}{\pi}\rangle$ and
$\langle\bar{q}g_{s}\sigma Gq\rangle^{2}$, which are the vacuum expectation
values of the quark-gluon operators of the orders
$\mathcal{O}(\alpha_{s}^{k})$ with $k\leq 1$. The vacuum condensates $\langle
g_{s}^{3}GGG\rangle$, $\langle\frac{\alpha_{s}GG}{\pi}\rangle^{2}$,
$\langle\frac{\alpha_{s}GG}{\pi}\rangle\langle\bar{q}g_{s}\sigma Gq\rangle$
and $\langle\bar{q}q\rangle\langle g_{s}^{3}GGG\rangle$ have the dimensions
$6$, $8$, $9$ and $9$, respectively, however, they are the vacuum expectation
values of the quark-gluon operators of the orders
$\mathcal{O}(\alpha_{s}^{\frac{3}{2}})$, $\mathcal{O}(\alpha_{s}^{2})$,
$\mathcal{O}(\alpha_{s}^{\frac{3}{2}})$ and
$\mathcal{O}(\alpha_{s}^{\frac{3}{2}})$, respectively, and are neglected,
direct calculations indicate that those contributions are tiny indeed [26].

We obtain the spectral density at the quark level through dispersion relation,
take the quark-hadron duality below the continuum threshold $s_{0}$, and
perform Borel transform in regard to the variable $P^{2}=-p^{2}$ to obtain the
QCD sum rules:

$\displaystyle\lambda^{2}_{X}\,\exp\left(-\frac{M^{2}_{X}}{T^{2}}\right)$
$\displaystyle=$
$\displaystyle\int_{16m_{c}^{2}}^{s_{0}}ds\,\rho_{QCD}(s)\exp\left(-\frac{s}{T^{2}}\right)\,,$
(6)

where the $\rho_{QCD}(s)$ is the spectral density at the quark level.

We derive Eq.(6) with respect to $\tau=\frac{1}{T^{2}}$, then eliminate the
pole residue $\lambda_{X}$, and obtain the QCD sum rules for the mass of the
pseudoscalar hexaquark molecular state $\overline{\Xi}_{cc}\Xi_{cc}$,

$\displaystyle M^{2}_{X}$ $\displaystyle=$
$\displaystyle-\frac{\frac{d}{d\tau}\int_{16m_{c}^{2}}^{s_{0}}ds\,\rho_{QCD}(s)\exp\left(-\frac{s}{T^{2}}\right)}{\int_{16m_{c}^{2}}^{s_{0}}ds\,\rho_{QCD}(s)\exp\left(-\frac{s}{T^{2}}\right)}\,.$
(7)

##  3 Numerical results and discussions

We adopt the standard values of the vacuum condensates
$\langle\bar{q}q\rangle=-(0.24\pm 0.01\,\rm{GeV})^{3}$,
$\langle\bar{q}g_{s}\sigma Gq\rangle=m_{0}^{2}\langle\bar{q}q\rangle$,
$m_{0}^{2}=(0.8\pm 0.1)\,\rm{GeV}^{2}$,
$\langle\frac{\alpha_{s}GG}{\pi}\rangle=0.012\pm 0.004\,\rm{GeV}^{4}$ at the
energy scale $\mu=1\,\rm{GeV}$ [27, 28, 29, 30, 31], and take the
$\overline{MS}$ mass of the charm quark, $m_{c}(m_{c})=(1.275\pm
0.025)\,\rm{GeV}$, from the Particle Data Group [32]. In addition, we take
account of the energy-scale dependence of all the input parameters [33],

$\displaystyle\langle\bar{q}q\rangle(\mu)$ $\displaystyle=$
$\displaystyle\langle\bar{q}q\rangle({\rm 1GeV})\left[\frac{\alpha_{s}({\rm
1GeV})}{\alpha_{s}(\mu)}\right]^{\frac{12}{33-2n_{f}}}\,,$
$\displaystyle\langle\bar{q}g_{s}\sigma Gq\rangle(\mu)$ $\displaystyle=$
$\displaystyle\langle\bar{q}g_{s}\sigma Gq\rangle({\rm
1GeV})\left[\frac{\alpha_{s}({\rm
1GeV})}{\alpha_{s}(\mu)}\right]^{\frac{2}{33-2n_{f}}}\,,$ $\displaystyle
m_{c}(\mu)$ $\displaystyle=$ $\displaystyle
m_{c}(m_{c})\left[\frac{\alpha_{s}(\mu)}{\alpha_{s}(m_{c})}\right]^{\frac{12}{33-2n_{f}}}\,,$
$\displaystyle\alpha_{s}(\mu)$ $\displaystyle=$
$\displaystyle\frac{1}{b_{0}t}\left[1-\frac{b_{1}}{b_{0}^{2}}\frac{\log
t}{t}+\frac{b_{1}^{2}(\log^{2}{t}-\log{t}-1)+b_{0}b_{2}}{b_{0}^{4}t^{2}}\right]\,,$
(8)

where $t=\log\frac{\mu^{2}}{\Lambda^{2}}$, $b_{0}=\frac{33-2n_{f}}{12\pi}$,
$b_{1}=\frac{153-19n_{f}}{24\pi^{2}}$,
$b_{2}=\frac{2857-\frac{5033}{9}n_{f}+\frac{325}{27}n_{f}^{2}}{128\pi^{3}}$,
$\Lambda=213\,\rm{MeV}$, $296\,\rm{MeV}$ and $339\,\rm{MeV}$ for the quark
flavor numbers $n_{f}=5$, $4$ and $3$, respectively [32]. In the present work,
we explore the hidden-charm hexaquark molecular state
$\overline{\Xi}_{cc}\Xi_{cc}$ and choose $n_{f}=4$, then evolve all the input
parameters to a typical energy scale $\mu$ to extract the hexaquark molecule
mass. Furthermore, we present the predictions based on the updated parameters
obtained by S. Narison, $m_{c}(m_{c})=(1.266\pm 0.006)\,\rm{GeV}$ and
$\langle\frac{\alpha_{s}GG}{\pi}\rangle=0.021\pm 0.001\,\rm{GeV}^{4}$ [34],
and in this case the energy scales of other vacuum condensates are taken at
$\mu=1\,\rm{GeV}$.

$J^{PC}$ | 1S | 2S | energy gaps | References
---|---|---|---|---
$1^{+-}$ | $Z_{c}(3900)$ | $Z_{c}(4430)$ | 591 MeV | [35, 36, 37]
$0^{++}$ | $X(3915)$ | $X(4500)$ | 588 MeV | [38, 39, 40]
$1^{+-}$ | $Z_{c}(4020)$ | $Z_{c}(4600)$ | 576 MeV | [41, 42]
$1^{++}$ | $X(4140)$ | $X(4685)$ | 566 MeV | [43]
Table 1:  The energy gaps between the ground states (1S) and first radial
excited states (2S) of the hidden-charm tetraquark candidates with the
possible assignments.

We should choose suitable continuum threshold $s_{0}$ to avoid contamination
from the first radial excited state. In the scenario of the tetraquark states,
the possible assignments of the exotic states $Z_{c}(3900)$, $Z_{c}(4430)$,
$X(3915)$, $X(4500)$, $Z_{c}(4020)$, $Z_{c}(4600)$, $X(4140)$ and $X(4685)$
are presented plainly in Table 1 according to the (possible) quantum numbers,
decay modes and energy gaps. From the table, we can obtain the conclusion
tentatively that the energy gaps between the ground states and first radial
excited states of the hidden-charm tetraquark states are about
$0.58\,\rm{GeV}$. We can choose the continuum threshold parameter as
$\sqrt{s_{0}}=M_{X}+0.6\pm 0.1\,\rm{GeV}$. If the mass of the
$\overline{\Xi}_{cc}\Xi_{cc}$ hexaquark molecular state and the energy scale
of the spectral density at the quark level satisfies energy scale formula
$M_{X}=\sqrt{\mu^{2}+(4\mathbb{M}_{c})^{2}}$, the lower bound of the mass
$M_{X}\geq\sqrt{(1\,\rm{GeV})^{2}+(4\times
1.85\,\rm{GeV})^{2}}=7.47\,\rm{GeV}>2M_{\Xi_{cc}}$.

Now let us suppose that the multiquark states $X$, $Y$, $Z$ and $P$ have
$N_{Q}+N_{q}$ valence quarks, where the $N_{Q}$ and $N_{q}$ are the numbers of
the heavy quarks and light quarks, respectively. Generally speaking, if
$N_{Q}\leq N_{q}$, we can apply the energy scale formula
$\mu=\sqrt{M_{X/Y/Z/P}^{2}-(N_{Q}\mathbb{M}_{Q})^{2}}$ to enhance the pole
contributions and improve the convergent behavior of the operator product
expansion [10, 13, 15, 22, 23, 24]. In the present case, $N_{Q}=4>N_{q}=2$,
the energy scale formula is not applicable.

After trial and error, we obtain the continuum threshold parameter
$\sqrt{s_{0}}=7.8\pm 0.1\,\rm{GeV}$, and Borel parameters which are shown in
Table 2 for four typical energy scales $\mu=1.0\,\rm{GeV}$, $m_{c}(m_{c})$,
$1.5\,\rm{GeV}$, $2.0\,\rm{GeV}$ and $1.0\,\rm{GeV}^{*}$, where the
superscript $*$ denotes the $c$-quark mass $m_{c}(m_{c})=(1.266\pm
0.006)\,\rm{GeV}$ and gluon condensate
$\langle\frac{\alpha_{s}GG}{\pi}\rangle=0.021\pm 0.001\,\rm{GeV}^{4}$ are
taken for Ref.[34]. In the Borel windows, the ground state contributions are
about $(18-37)\%$ or $(22-41)\%$ and the pole contribution cannot reach
$50\%$, the contributions of the vacuum condensates of dimension 10 are about
$(2-6)\%$, $<1\%$, $<1\%$, $<1\%$ and $(1-2)\%$, respectively, the operator
product expansion converges very well.

Then we take account of all uncertainties of the parameters, and obtain the
values of the mass and pole residue of the pseudoscalar hexaquark molecular
state $\overline{\Xi}_{cc}\Xi_{cc}$, which are shown explicitly in Table 2 and
Fig.1. From Fig.1, we can see that the predicted mass is rather stable with
variation of the Borel parameter, the uncertainty comes from the Borel
parameter is rather small. From Table 2, we can see that the predicted mass
$M_{X}$ is almost independent on the energy scale of the QCD spectral density,
while the pole residue depends heavily on the energy scale of the QCD spectral
density, which is qualitatively consistent with the evolution behavior of the
current operator from the re-normalization group equation,
$J(x,\mu)=L^{\gamma_{J}}J(x,\mu_{0})$ and
$\lambda_{X}(\mu)=L^{\gamma_{J}}\lambda_{X}(\mu_{0})$, where
$L=\frac{\alpha_{s}(\mu_{0})}{\alpha_{s}(\mu)}$, and the $\gamma_{J}$ is the
anomalous dimension of the current operator $J(x)$. The predicted mass
$M_{X}\sim 7.2\,\rm{GeV}$ is compatible with the vague structure around
$7.2\,\rm{GeV}$ in the $J/\psi J/\psi$ invariant mass spectrum observed by the
LHCb collaboration [2], and supports assigning the $X(7200)$ as the
$\overline{\Xi}_{cc}\Xi_{cc}$ hexaquark molecular state with the quantum
numbers $J^{PC}=0^{-+}$. On the other hand, direct calculations based on the
QCD sum rules [44] do not support assigning the $X(7200)$ as the first radial
excited state of the diquark-antidiquark-type (${\bf\bar{3}}_{c}{\bf 3}_{c}$\-
type) $cc\bar{c}\bar{c}$ tetraquark state claimed in Refs.[3, 5, 6].

Figure 1:  The mass of the $\overline{\Xi}_{cc}\Xi_{cc}$ hexaquark molecular state with variation of the Borel parameter $T^{2}$, where the $m_{c}(m_{c})=1.266\,\rm{GeV}$ denotes the $c$-quark mass and gluon condensate are taken from Ref.[34].  $\mu(\rm GeV)$ | $T^{2}(\rm{GeV}^{2})$ | pole | $M_{X/Y}(\rm{GeV})$ | $\lambda(10^{-3}\rm{GeV}^{8})$
---|---|---|---|---
$1$ | $4.7-5.3$ | $(18-37)\%$ | $7.21\pm 0.13$ | $6.35\pm 2.19$
$1.275$ | $5.3-5.9$ | $(18-35)\%$ | $7.21\pm 0.11$ | $13.1\pm 3.5$
$1.5$ | $5.7-6.3$ | $(17-32)\%$ | $7.21\pm 0.12$ | $18.4\pm 4.6$
$2$ | $6.1-6.7$ | $(18-32)\%$ | $7.18\pm 0.11$ | $27.9\pm 6.2$
$1^{*}$ | $5.0-5.6$ | $(22-41)\%$ | $7.21\pm 0.12$ | $13.3\pm 2.8$
Table 2:  The energy scale of the QCD spectral density, Borel parameter, pole
contribution, mass and pole residue of the hexaquark molecular state
$\overline{\Xi}_{cc}\Xi_{cc}$, where the superscript $*$ denotes the $c$-quark
mass and gluon condensate are taken from Ref.[34].

The decays of the $\overline{\Xi}_{cc}\Xi_{cc}$ hexaquark molecule candidate
$X(7200)$ can take place through fusions of the $\bar{c}c$ and $\bar{q}q$
pairs,

$\displaystyle X(7200)$ $\displaystyle\to$
$\displaystyle\overline{\Xi}_{cc}\Xi_{cc}\to J/\psi
J/\psi\,,\,\chi_{c1}\chi_{c1}\,,\,D^{*}\bar{D}^{*}\,,\,D_{1}\bar{D}_{1}\,,$
(9)

we can search for the $\overline{\Xi}_{cc}\Xi_{cc}$ hexaquark molecular state
in the $J/\psi J/\psi$, $\chi_{c1}\chi_{c1}$, $D^{*}\bar{D}^{*}$ and
$D_{1}\bar{D}_{1}$ invariant mass spectrum at the BESIII, LHCb, Belle II,
CEPC, FCC and ILC in the future.

##  4 Conclusion

In the present work, we construct the color-singlet-color-singlet type six-
quark pseudoscalar current to interpolate the $\overline{\Xi}_{cc}\Xi_{cc}$
hexaquark molecular state, then accomplish the operator product expansion by
calculating the vacuum condensates up to dimension 10, which are vacuum
expectation values of the quark-gluon operators of the orders
$\mathcal{O}(\alpha_{s}^{k})$ with $k\leq 1$, and obtain the QCD sum rules for
the mass and pole residue. The predicted mass $M_{X}\sim 7.2\,\rm{GeV}$
supports assigning the $X(7200)$ to be the $\overline{\Xi}_{cc}\Xi_{cc}$
hexaquark molecular state with the quantum numbers $J^{PC}=0^{-+}$. Moreover,
direct calculations based on the QCD sum rules do not support assigning the
$X(7200)$ as the first radial excited state of the diquark-antidiquark-type
(${\bf\bar{3}}_{c}{\bf 3}_{c}$\- type) $cc\bar{c}\bar{c}$ tetraquark state.
The decays of the $X(7200)$ can take place through fusions of the $\bar{c}c$
and $\bar{q}q$ pairs, we can search for the $\overline{\Xi}_{cc}\Xi_{cc}$
hexaquark molecular state and explore its properties in the $J/\psi J/\psi$,
$\chi_{c1}\chi_{c1}$, $D^{*}\bar{D}^{*}$ and $D_{1}\bar{D}_{1}$ invariant mass
spectrum at the BESIII, LHCb, Belle II, CEPC, FCC and ILC in the future.

## Acknowledgements

This work is supported by National Natural Science Foundation, Grant Number
11775079.

## References

  * [1] R. Aaij et al, Phys. Rev. Lett. 119 (2017) 112001.
  * [2] R. Aaij et al, Sci. Bull. 65 (2020) 1983.
  * [3] J. F. Giron and R. F. Lebed, Phys. Rev. D102 (2020) 074003.
  * [4] S. J. Brodsky, D. S. Hwang and R. F. Lebed, Phys. Rev. Lett. 113 (2014) 112001.
  * [5] J. Sonnenschein and D. Weissman, Eur. Phys. J. C81 (2021) 25.
  * [6] M. Karliner and J. L. Rosner, Phys. Rev. D102 (2020) 114039.
  * [7] M. Z. Liu and L. S. Geng, Eur. Phys. J. C81 (2021) 179.
  * [8] R. M. Albuquerque, F. Fanomezana, S. Narison and A. Rabemananjara, Phys. Lett. B715 (2012) 129; R. Albuquerque, S. Narison, F. Fanomezana, A. Rabemananjara, D. Rabetiarivony and G. Randriamanatrika, Int. J. Mod. Phys. A31 (2016) 1650196.
  * [9] S. H. Lee, M. Nielsen and U. Wiedner, J. Korean Phys. Soc. 55 (2009) 424; J. M. Dias, F. S. Navarra, M. Nielsen and C. M. Zanetti, Phys. Rev. D88 (2013) 016004.
  * [10] Z. G. Wang, Int. J. Mod. Phys. A34 (2019) 1950097; Z. G. Wang and X. Wang, Chin. Phys. C44 (2020) 103102.
  * [11] J. R. Zhang and M. Q. Huang, J. Phys. G37 (2010) 025005; J. R. Zhang, M. Zhong and M. Q. Huang, Phys. Lett. B704 (2011) 312.
  * [12] K. Azizi, Y. Sarac and H. Sundu, Phys. Rev. D95 (2017) 094016; K. Azizi, Y. Sarac and H. Sundu, Phys. Lett. B782 (2018) 694.
  * [13] Z. G. Wang and T. Huang, Eur. Phys. J. C74 (2014) 2891; Z. G. Wang, Eur. Phys. J. C74 (2014) 2963.
  * [14] H. X. Chen, W. Chen, X. Liu, T. G. Steele and S. L. Zhu, Phys. Rev. Lett. 115 (2015) 172001; W. Chen, T. G. Steele, H. X. Chen and S. L. Zhu, Phys. Rev. D92 (2015) 054002.
  * [15] Z. G. Wang, Phys. Rev. D102 (2020) 034008.
  * [16] B. D. Wan, L. Tang and C. F. Qiao, Eur. Phys. J. C80 (2020) 121.
  * [17] Z. G. Wang, Eur. Phys. J. C78 (2018) 826.
  * [18] W. Lucha, D. Melikhov and H. Sazdjian, Phys. Rev. D103 (2021) 014012.
  * [19] R. M. Albuquerque, S. Narison and D. Rabetiarivony, Phys. Rev. D103 (2021) 074015.
  * [20] Z. G. Wang, arXiv:2102.07520 [hep-ph].
  * [21] G. C. Rossi, T. Vergata and G. Veneziano, arXiv:2011.09774 [hep-ph].
  * [22] Z. G. Wang and T. Huang, Phys. Rev. D89 (2014) 054019; Z. G. Wang, Phys. Rev. D102 (2020) 014018.
  * [23] Z. G. Wang, Int. J. Mod. Phys. A35 (2020) 2050003; Z. G. Wang, Int. J. Mod. Phys. A36 (2021) 2150071.
  * [24] Z. G. Wang, Int. J. Mod. Phys. A35 (2020) 2050073.
  * [25] Z. G. Wang, AAPPS Bull. 31 (2021) 5.
  * [26] X. W. Wang and Z. G. Wang, in preparation.
  * [27] M. A. Shifman, A. I. Vainshtein and V. I. Zakharov, Nucl. Phys. B147 (1979) 385, 448.
  * [28] L. J. Reinders, H. Rubinstein and S. Yazaki, Phys. Rept. 127 (1985) 1.
  * [29] B. L. Ioffe, Nucl. Phys. B188 (1981) 317; Erratum: Nucl.Phys. B191 (1981) 591.
  * [30] V. M. Belyaev and B. L. Ioffe, Sov. Phys. JETP 56 (1982) 493.
  * [31] P. Colangelo and A. Khodjamirian, hep-ph/0010175.
  * [32] P. A. Zyla et al, Prog. Theor. Exp. Phys. 2020 (2020) 083C01.
  * [33] S. Narison and R. Tarrach, Phys. Lett. 125 B (1983) 217.
  * [34] S. Narison, arXiv:2101.12579.
  * [35] L. Maiani, F. Piccinini, A. D. Polosa and V. Riquer, Phys. Rev. D89 (2014) 114010.
  * [36] M. Nielsen and F. S. Navarra, Mod. Phys. Lett. A29 (2014) 1430005.
  * [37] Z. G. Wang, Commun. Theor. Phys. 63 (2015) 325.
  * [38] R. F. Lebed and A. D. Polosa, Phys. Rev. D93 (2016) 094024.
  * [39] Z. G. Wang, Eur. Phys. J. C77 (2017) 78.
  * [40] Z. G. Wang, Eur. Phys. J. A53 (2017) 19.
  * [41] H. X. Chen and W. Chen, Phys. Rev. D99 (2019) 074022.
  * [42] Z. G. Wang, Chin. Phys. C44 (2020) 063105.
  * [43] Z. G. Wang, arXiv:2103.04236 [hep-ph].
  * [44] Z. G. Wang and Z. Y. Di, Acta Phys. Polon. B50 (2019) 1335; Z. G. Wang, Chin. Phys. C44 (2020) 113106; Z. G. Wang, Int. J. Mod. Phys. A36 (2021) 2150014.