# Ask Questions with Double Hints: Visual Question Generation with Answer-
awareness and Region-reference

Kai Shen, Lingfei Wu, Siliang Tang, Fangli Xu, Bo Long, Yueting Zhuang, Jian
Pei  Kai Shen, Siliang Tang and Yueting Zhuang are with Zhejiang University,
Hang Zhou, China. Email: {shenkai, siliang<EMAIL_ADDRESS>Corresponding
author: Siliang Tang.
Lingfei Wu and Fangli Xu are with Anytime AI. Email: {teddy.lfwu,
<EMAIL_ADDRESS>
Bo Long is with Meta AI. Email<EMAIL_ADDRESS>
Jian Pei is with Duke University. Email<EMAIL_ADDRESS>

###### Abstract

The visual question generation (VQG) task aims to generate human-like
questions from an image and potentially other side information (e.g. answer
type). Previous works on VQG fall in two aspects: i) They suffer from one
image to many questions mapping problem, which leads to the failure of
generating referential and meaningful questions from an image. ii) They fail
to model complex implicit relations among the visual objects in an image and
also overlook potential interactions between the side information and image.
To address these limitations, we first propose a novel learning paradigm to
generate visual questions with answer-awareness and region-reference.
Concretely, we aim to ask the right visual questions with _Double Hints -
textual answers and visual regions of interests_ , which could effectively
mitigate the existing one-to-many mapping issue. Particularly, we develop a
simple methodology to self-learn the visual hints without introducing any
additional human annotations. Furthermore, to capture these sophisticated
relationships, we propose a new double-hints guided Graph-to-Sequence learning
framework, which first models them as a dynamic graph and learns the implicit
topology end-to-end, and then utilizes a graph-to-sequence model to generate
the questions with double hints. Experimental results demonstrate the priority
of our proposed method.

###### Index Terms:

Semi-supervised Learning, graph neural network, vision and language, question
generation

##  1 Introduction

Visual Question Generation (VQG) is an emerging task in both computer vision
(CV) and natural language processing (NLP), which aims to generate human-like
questions from an image and potentially other side information (e.g. answer
type or answer itself). Recently, there has been a surge of interests in VQG
because it is particularly useful for providing high-quality synthetic
training data for visual question answering (VQA) [1, 2, 3] and visual dialog
system [4]. Conceptually, it is a challenging task because the generated
questions are not only required to be consistent with the image content but
also meaningful and answerable by humans.

Despite promising results that have been achieved, the one-to-many mapping and
complex implicit relation modelling problems hinder the development of visual
question generation.

The one-to-many mapping problem occurs when many potential questions can be
mapped to certain inputs, which betrays the supervised objective which is
usually one certain question. This phenomenon leads to severe ambiguity
preventing the model from producing the referential and meaningful questions
from an image. Conceptually, the existing VQG methods can be generally
categorized into three classes with respect to what hints are used for
generating visual questions: 1) the whole image as the only context input [5];
2) the whole image and the desired answers [1]; 3) the whole image with the
desired answer types [6]. Since a picture is worth a thousand words, it can be
potentially mapped to many different questions, leading to the generation of
diverse non-informative questions with poor quality. Even with the answer type
or desired answer information, the similar one-to-many mapping issue remains,
partially because the answer hints are often very short or too broad.

Figure 1: An example of our new setting for learning VQG with _Double Hints:
textual answers and visual regions of interests_.

As Figure 1 shows, one person is facing the camera while another person is
away from the camera, but both of them wear black clothes. The answer here is
black color and we need to ask a right question given this answer. In this
case, asking question about either of them could be acceptable when the answer
information is the only hint. As a result, existing side information is often
not informative enough for guiding question generation process, which causes
the failure of generating referential and meaningful questions from an image.
Therefore, the visual regions of interest hints are often crucial for helping
reduce the ambiguities in order to ask a right question. Based on both the
textual answer hints and visual regions of interest hints, we can
significantly mitigate the existing one-to-many mapping issue of VQG.

Figure 2: An example of the correlations among the visual objects.

The second severe issue is due to complex implicit relation modelling problem
in VQG. This is partially because that the existing VQG methods often ignore
the rich correlations among the visual objects in an image and potential
interactions between the side information and image [6]. Conceptually, the
implicit relations among the visual objects (e.g., spatial, semantic) could be
the key to generating meaningful and high-quality questions. For example, as
Figure 2 shows, the girl has multiple relations (hold, besides) with the other
men, which provide vital clues for generating high-quality questions. In
addition, another important factor for producing informative and referential
questions is how to make full use of the side information to align with the
targeted image. In this example, given the target answer girl, the relation
hold between the man in the front and the target answer girl greatly
contributes to the question generation process. In consequence, modeling such
potential interactions between the side information and an image becomes a
critical component for generating referential and meaningful questions.

To address the first issue, we first propose a novel learning paradigm for the
visual question generation task, which generates the visual questions with
_Double Hints - textural answer and visual regions of interests_. Thanks to
the proposed double hints, the ambiguities are largely reduced since the
questions can be clearly mapped to the specific answer and visual hints.
Concretely, we aim to utilize the referential visual regions of interest hints
(denoted as visual hints for simplicity) of the images and the textual answers
(denoted as answer hints) to faithfully guide question generation. As
illustrated in Figure 1, by giving an image with visual hints (the region
enclosed by the orange rectangle) and answer hints (the answer), the model is
able to generate the right question with key entities that reflects the visual
hints and is answerable to the answer hints. To this end, in order to learn
these visual hints, we develop a multi-task auto-encoder to self-learn the
visual hints and the unique attributes automatically without introducing any
additional human annotations.

Furthermore, to address the second issue, to capture the rich interactions
between double hints and the image, as well as the sophisticated relationships
among the visual objects, we propose a new Double-Hints guided Graph-to-
Sequence learning framework (DH-Graph2Seq). The proposed model first models
these interactions as a dynamic graph and learns the implicit topology end-to-
end. Then it utilizes a Graph2Seq model to generate the questions with double
hints. In addition, in the decoder side, we also design a visual-hint guided
separate attention mechanism to attend image and object graph separately and
overlook the non-visual-hints particularly.

In summary, we highlight our main contributions as follows:

  * •

We propose a novel learning paradigm to generate visual questions with _Double
Hints - textual answer and visual regions of interests_ , which could
effectively mitigate the one-to-many mapping issue. To the best of our
knowledge, this is the first time both visual hints and answers hints are used
for the VQG task.

  * •

We explicitly cast the VQG task as a Graph-to-Sequence (Graph2Seq) learning
problem. We employ graph learning technique to learn the implicit graph
topology to capture various rich interactions between and within an image, and
then utilize a Graph2Seq model to guide question generation with double hints.

  * •

Our extensive experiments on VQA2.0 and COCO-QA datasets demonstrate that our
proposed model can significantly outperform existing state-of-the-art by a
large margin. Further experiments show that our VQG can help VQA as a data
augmentation method when the training data is limited.

Figure 3: The overall framework of our proposed model with double hints to
guide VQG.

##  2 Related Work

###  2.1 Visual Question Generation

Visual question generation is an emerging task in the visual-language domain
[5, 7, 8, 9, 10]. Mora et al. [5] firstly made an attempt to generate
question-answer pairs based on an image to support the visual question
answering (VQA) task. They simply employed the VGG network as the encoder and
use the long short-term memory networks (LSTM) to produce both the questions
and answers outputs. Mostafazadeh et al. [7] formally defined the VQG task.
Different from straightforward questions, they focused on questions that can
potentially engage a human in starting a conversation. In order to generate
more diverse questions, Jain et al.[11] combined the advantage of variational
autoencoders with LSTMs to entrust the model with creativity. Compared with
conventional methods, it can generate a large set of varying questions given a
single input image. Zhang et al.[12] proposed to generate questions that can
be visually grounded in the given images. They firstly generated the textural
captions from the images and then predict the specific question types.
Secondly, the final question was generated with the guidance of both the
question type and the caption. These works simply mapped the visual images
into the textural questions, which lead to imprecise and generic questions.

Liu et.al.[13] viewed this task the inverse of VQA, which regarded the target
answers as the guidance to generate high-quality questions. They posed the
question generation as a multi-modal dynamic inference process to help improve
the questions’ quality. What’s more, the dual learning mechanism is applied in
the VQG task. In [1], the VQA and VQG are considered as dual tasks and are
trained jointly in a unified end-to-end framework. Furthermore, the parameter
sharing and regular techniques are proposed as constraints to leverage the
question-answer dependency. Shah et al.[14] employ the VQG to boost the VQA
models’ robustness. Different from conventional methods, Krishna et al.[6]
introduce the fine-grained answer type as the guidance to the variational
method, which generates goal-driven questions. Xu et al.[15] adapt the graph
method to generate meaningful questions with the target of answers. Different
from these works, we will generate the questions under the guidance of both
visual regions of interest and textual answers, which can generate more
referential and answerable questions.

###  2.2 Graph-to-Sequence Learning

Graph neural networks (GNN) [16, 17] has drawn a significant amount of
attention in recent years. In the NLP domain, graph-to-sequence learning is to
generate sequential results from graph-structured data, which maps the
structural data to sequence output [18, 19, 20]. When coming to the non-graph
structured data just like the regions of an image, researchers explore some
methods to construct objects’ [21] or words’ [22, 19] topology.

Unlike these previous methods, we propose a novel cross-modal graph2seq model
which models the relations upon the visual and textural hints. With the
guidance of these two modalities, the model is able to learn the appropriate
embeddings which are crucial to generate high-quality questions.

##  3 Method

We first introduce our problem formulation and then the proposed learning
paradigm that generates visual questions with double hints. Next, we discuss
each key component of our overall framework of proposed model in Figure 3.

###  3.1 Problem Formulation

The goal of the visual question generation task is to generate human-like and
answerable natural language questions based on the given images and
potentially other side information, such as textural answer and answer type.
The generated questions need to be consistent with the given images and the
hints semantically. In this part, we will first discuss the traditional
answer-based hints and then present our new learning paradigm.

We assume that the raw image is $I$, and the target answer (answer hint) is a
collection of word tokens represented by $A=\\{a_{1},a_{2},...,a_{m}\\}$,
where $m$ denotes the total number of the answer words. The task of
traditional answer-guided VQG is to generate the natural language question
consisting of a word tokens sequence $Q=\\{q_{1},q_{2},...,q_{n}\\}$ which
maximizes the conditional likelihood:

$\hat{Q}=\text{argmax}_{Q}P(Q|I,A),$ (1)

where $n$ is the total number of question words. To address the existing one-
to-many mapping issue, we introduce a new setting that focuses not only on the
answer-awareness but also on region-reference. Specifically, we cast it as a
joint optimization problem over visual hints finding tasks and double-hints
guided graph2seq learning task. The visual hint is a collection of visual
object regions of interest in the image which are served as direct visual
clues for visual question generation. We denote it as
$\mathcal{V}=\\{v_{1},v_{2},...,v_{N}\\}$, where $v_{i}$ is the bounding box
in the image and $N$ is the number of visual hints. Under this setting, the
likelihood can be cast as:

$P(Q|I,A)=P(Q|\mathcal{V},I,A)P(\mathcal{V}|I,A).$ (2)

In particular, the visual hints will be generated in the data pre-processing
phase (see Sec. 4.1) and the model can automatically learn them without any
human annotations. Thus there is no additional visual hints annotation during
inference.

###  3.2 Image and Answer Encoder

Image Encoder. Given a raw image $I$, we first extract the image feature by a
pretrained convolutional neural network [23] given by
$\mathbf{I}\in\mathbb{R}^{c\times F_{v}}$ where $c$ is the channel number and
$F_{v}$ is the visual feature dimension. Secondly, we assume that the image is
represented by a collection of object regions denoted by
$\mathcal{R}=\\{v_{1},v_{2},..,v_{T}\\}$. The object regions are associated
with visual feature
$\mathbf{V}=[\mathbf{v}_{1},\mathbf{v}_{2},...,\mathbf{v}_{T}]$, where
$\mathbf{v}_{i}\in\mathbb{R}^{F_{v}}$ and $F_{v}$ is the feature dimension. In
addition, each object $v_{i}$ has addition bounding box position vector
$p_{i}=(x_{0},y_{0},x_{1},y_{1})$, category attribute (eg. it belongs to
animal class) $c_{i}$. $(x_{0},y_{0}),(x_{1},y_{1})$ are the normalized up-
left and bottom-right coordinates. It is worth noting that the visual hints
$\mathcal{V}$ is the subset of visual regions collection $\mathcal{R}$
($\mathcal{V}\subseteq\mathcal{R}$).

Answer Encoder. As for the textural answers $A=\\{a_{1},a_{2},...,a_{m}\\}$,
we first employ the pretrained GloVe weights to initialize the embeddings
which are represented by
$\mathbf{A}_{initial}=[\mathbf{a}_{1},\mathbf{a}_{2},...,\mathbf{a}_{m}]$.
Then we adopt a GRU module to learn the word semantic features given by:

$\mathbf{A}=\text{GRU}(\mathbf{A}_{initial}),$ (3)

where $\mathbf{A}\in\mathbb{R}^{m\times F_{a}}$ and $F_{a}$ denotes the answer
vector dimension.

###  3.3 Double-Hints Guided Multi-Task Auto-Encoder

How to effectively find the visual hints and how to combine the visual
features with double hints are particularly important for addressing the first
issue. In this section, we first introduce a cross-modal alignment that aligns
the cross-modal visual objects and textural answers into the latent space.
Then we propose a multi-task decoder that can effectively predict the visual
hints and the answer attributes from the learned latent space.

####  3.3.1 Cross-modal Alignments

How to infer which regions of interest are suitable for asking questions from
the answer clues and visual object features is a critical task. Intuitively,
exploiting the rich fine-grained interactions between visual clues and textual
answer hints is beneficial to find out the vital visual hints. To this end, we
explicitly model the global correlations between them by the cross-modal
alignment technique in the embedding space.

Firstly, we observe that the position and category attributes of objects are
indispensable during fine-grained object relation modeling. Thus, for object
$v_{i}$, we incorporate them by projecting the position vector $p_{i}$ and
category attribute $c_{i}$ into two embedding spaces denoted as
$\mathbf{p}_{i}\in\mathbb{R}^{d}$ and $\mathbf{c}_{i}\in\mathbb{R}^{d}$. Then
we concatenate them with visual feature $\mathbf{v}_{i}$ given by:

$\mathbf{v}_{i}=\text{F}(\text{CONCAT}(\mathbf{v}_{i},\mathbf{p}_{i},\mathbf{c}_{i})),$
(4)

where $\text{F}(\cdot):\mathbb{R}^{2\times
d+F_{v}}\rightarrow\mathbb{R}^{F_{v}}$ is the linear projection with ReLU
activation function and $\text{CONCAT}(\cdot,...,\cdot)$ denotes the vector
concatenation operation. We slightly overload the notation $\mathbf{v}_{i}$
here for simplicity in the subsequent sections.

For each object $v_{i}$ ($\mathbf{v}_{i}$) and answer word $a_{j}$
($\mathbf{a}_{j}$), we will calculate the alignment score $\mathbf{S}_{ij}$ as
follows:

$\mathbf{S}_{ij}=\sigma(\mathbf{v}_{i}\mathbf{W}_{r}+\mathbf{a}_{j}\mathbf{W}_{a})\mathbf{W},$
(5)

where $\sigma(\cdot)$ denotes the Tanh function,
$\mathbf{W}_{r}\in\mathbb{R}^{F_{v}\times F_{align}}$,
$\mathbf{W}_{a}\in\mathbb{R}^{F_{a}\times F_{align}}$ and
$\mathbf{W}\in\mathbb{R}^{F_{align}\times 1}$, where $F_{align}$ is the hidden
dimension size. The alignment score matrix $\mathbf{S}\in\mathbb{R}^{N\times
m}$ represents the correlations’ weight between the objects and the answer
words. Following this correlation matrix, we can aggregate the answer words
($\\{a_{1},a_{2},...,a_{m}\\}$) by visual object ($v_{j}$) given by:

$\begin{split}\alpha_{ij}&=\frac{exp(\mathbf{S}_{ij})}{\sum_{k=0}^{k=m}{exp(\mathbf{S}_{ik})}},\\\
\mathbf{a}^{\prime}_{i}&=\sum^{m}_{j=1}\alpha_{ij}\mathbf{a}_{j}.\end{split}$
(6)

By this operation, the answer words having strong correlations with the
specific objects are assigned with high alignment scores. Then we fuse the
visual features with the aggregated textural answer features as follows:

$\mathbf{\overline{v}}_{i}=\text{Align}(\mathbf{v}_{i},\mathbf{a}^{\prime}_{i})=\text{F}(\text{CONCAT}(\mathbf{v}_{i},\mathbf{a}^{\prime}_{i})),$
(7)

where $\mathbf{\overline{v}}_{i}$ is the aligned object representation,
$\text{CONCAT}(\cdot,\cdot)$ is the vector concatenation operation and
$\text{F}(\cdot):\mathbb{R}^{F_{v}+F_{a}}\rightarrow\mathbb{R}^{F_{v}}$ is the
linear projection with ReLU nonlinearity.

####  3.3.2 Multi-Task Decoder

After aligning the visual features with the answers, we introduce the multi-
task decoder which can infer the visual hints as well as the unique attributes
(i.e the objects’ position and category attributes and the target answers).
Our hope is that the latent embedding could absorb the double hints
information while retaining the robust features like objects’ position clues.

Firstly, we apply MLP with ReLU activation to project the aligned visual
objects’ features to a $F_{h}$ dimension latent space denoted as
$\mathbf{\overline{V}}=[\mathbf{\overline{v}}_{1},\mathbf{\overline{v}}_{2},...,\mathbf{\overline{v}}_{T}],\mathbf{\overline{V}}\in\mathbb{R}^{T\times
F_{h}}$. Empirically, we find it is beneficial to fuse the coarse-grained
image features (i.e., $\mathbf{I}\in\mathbb{R}^{c\times F_{v}}$ in Sec. 3.2)
with the fine-grained object features (i.e.,
$\mathbf{\overline{V}}\in\mathbb{R}^{T\times F_{h}}$). For each object feature
$\mathbf{\overline{v}_{i}}$, we capture the top-down signals [24] by attending
to the image features as follows:

$\centering\begin{split}\mathbf{M}_{i,j}&=\sigma(\mathbf{\overline{v}}_{i}\mathbf{W}_{1}+\mathbf{I}(j)\mathbf{W}_{2})\mathbf{W}_{3},\\\
\alpha_{i,j}&=\frac{exp(\mathbf{M}_{ij})}{\sum_{k=1}^{c}{exp(\mathbf{M}_{i,k})}},\\\
\mathbf{o}_{i}&=\sum^{c}_{j=1}\alpha_{i,j}\mathbf{I}(j)\mathbf{W},\end{split}\@add@centering$
(8)

where $\mathbf{I}(j)\in\mathbb{R}^{F_{v}}$ denotes the $j$-th channel’s
feature vector, $\sigma(\cdot)$ is the Tanh function, and
$\mathbf{W}_{1}\in\mathbb{R}^{F_{h}\times F_{hid}}$,
$\mathbf{W}_{2}\in\mathbb{R}^{F_{v}\times F_{hid}}$,
$\mathbf{W}_{3}\in\mathbb{R}^{F_{hid}}\times 1$,
$\mathbf{W}\in\mathbb{R}^{F_{v}\times F_{h}}$ are learnable parameters. Then
we use residual connection to combine the aligned feature
$\mathbf{\overline{v}}_{i}$ (Eq. 7) and top-down feature $\mathbf{o}_{i}$
given by:

$\mathbf{\widetilde{v}}_{i}=\mathbf{\overline{v}}_{i}+\mathbf{o}_{i}.$ (9)

In what follows, we will discuss how the multi-task decoder could generate the
visual hints while keeping the unique attributes in the latent embedding.

##### Visual hint prediction

We first predict the probability of the object $v_{i}\in\mathcal{R}$ being a
visual hint given by:

$\begin{split}s_{i}&=\text{MLP}(\mathbf{\widetilde{v}})_{i},\\\
P(i)&=\frac{exp(s_{i})}{\sum^{T}_{j=1}exp(s_{j})},\end{split}$ (10)

where $\text{MLP}(\cdot):\mathbb{R}^{F_{h}}\rightarrow\mathbb{R}$ is the
multi-layer projection with ReLU nonlinearity and $\widetilde{v}_{i}$ is the
visual object features mentioned in Eq. 9. In practice, the number of visual-
hint objects is much smaller than non-visual-hint objects, thus we combine
balanced-cross-entropy loss and focal loss [25] together as follows:

$\begin{split}L_{vh}=-\frac{\eta}{N_{pos}}\sum_{i}\hat{y}_{i}P(i)^{\lambda}log(P(i))\\\
-\frac{\eta}{N_{neg}}\sum_{i}(1-\hat{y}_{i})(1-P(i))^{\lambda}log(1-P(i)),\end{split}$
(11)

where $\hat{y}_{i}$ is 1 iff $v_{i}\in\mathcal{V}$ (0 otherwise), $N_{neg}$
denotes the number of non-visual-hints and $N_{pos}$ is the number of visual
hints.

##### Object position prediction

For high-quality question generation, the relative spatial relations among
objects are important clues. To keep these essential features in the latent
space, we employ the task by predicting the coordinates of the objects. Just
like the previous visual hints prediction, we apply the feed-forward layer on
$\mathbf{\overline{v}_{i}}$ and predict the absolute normalized position
coordinates $p_{i}^{\prime}$. The loss $L_{pos}$ is the mean square loss
function of $p_{i}^{\prime}$ and ground-truth $p_{i}$.

##### Target answer prediction

Since the answer hint is important in guiding how to ask high-quality
questions, we ensure that the latent embedding indeed retains this information
via regenerating the answer. Technically, we formulate it as a classification
problem. We denote the answer is $t_{i}\in\\{t_{0},t_{1},...,t_{c}\\}$, where
$i$ is the $i$-th sample, and $c$ is the amount of all answers in the training
dataset. Firstly, we apply a dilated CNN with a max-pooling layer to get the
representation as follows:

$\begin{split}\mathbf{w}=\text{MaxPool}(\text{ReLU}(\text{DilatedCNN}(\mathbf{\widetilde{V}}))).\end{split}$
(12)

Then we apply the feed-forward layer on the pooled vector $\mathbf{w}$ and
softmax function to calculate the probability. Finally, we adopt the cross-
entropy loss to calculate the answer prediction loss denoted as $L_{ans}$.

###  3.4 Double-Hints Guided Graph Construction and Embedding Learning

To capture the complex correlations among visual objects in the image, we
regard objects as nodes in the object graph $\mathcal{G}$ and adopt the
paradigm of graph learning. Typically, all GNN algorithms are operated
directly on graph-structured data and then compute the corresponding graph
node embeddings. However, there is no prior graph-structured data in VQG since
the relations among different objects are not explicitly given. Therefore, we
will first discuss how to construct an object graph $\mathcal{G}$’s topology
with the guidance of double hints and then employ a GNN model to encode them.

The topology of a graph represents the relations of the nodes. In our setting,
the graph edges are weighted and learnable, since $\mathcal{G}$ is essentially
a dynamic graph. We take the aligned objected embedding
$\mathbf{\overline{v}}_{i}$ of multi-task auto-encoder (in Eq. 7) to exploit
this graph topology since the objects’ visual features have incorporated
double hints information in the auto-encoder.

Inspired by [26], a good similarity metric function has been proved be
learnable and expressively powerful for learning graph structure. As a result,
we first calculate the dense similarity matrix $\mathbf{S}$ by multi-head
weighted cosine similarity function as follows:

$\begin{split}\mathbf{s}_{ij}^{p}&=cos(\mathbf{w}_{p}\odot\mathbf{\overline{v}}_{i},\mathbf{w}_{p}\odot\mathbf{\overline{v}}_{j}),\\\
\mathbf{S}_{i,j}&=\frac{\sum_{p=1}^{K}{\mathbf{s}_{i,j}^{p}}}{K},\end{split}$
(13)

where $\mathbf{w}_{p}\in\mathbb{R}^{F_{h}}$ is the learnable weight, $\odot$
is the Hadamard product, and $\mathbf{s}_{i,j}^{p}$ denotes the similarity
between node $i$ and $j$ of head $p$. The model learns to highlight specific
dimensions of the latent embedding space. $K$ is the number of heads. The
final similarity results is computed as the mean of the similarity scores from
different subspaces. Because the learned graph similarity matrix $\mathbf{S}$
is dense and ranges between [-1, 1], we adopt $\epsilon-$sparsing to make it
non-negative and sparse. Specifically, we mask off $\mathbf{S}_{i,j}$ (i.e.,
set to zero) if it is smaller than the threshold $\epsilon$, which leads to a
final sparse adjacency matrix $\mathbf{A}$.

It is desirable to further align the objects’ features with the learned double
hints embeddings. Therefore, the final node features will be represented as
follows:

$\begin{split}\mathbf{X}&=\text{F}(\mathbf{V};\mathbf{A})=\text{CONCAT}(\mathbf{V},\text{Latent}(\mathbf{V},\mathbf{A}))\\\
&=\text{CONCAT}(\mathbf{V},\mathbf{\widetilde{V}})\end{split}$ (14)

where $\text{Latent}(\cdot,\cdot)$ is the latent space representation
generated from the auto-encoder described in Sec. 3.3. And
$\text{CONCAT}(\cdot,\cdot)$ is the node-wise concatenation.

Next, we apply a multi-layer graph convolution network (GCN) with residual
connection [23] to effectively learn the node embedding for each object from
the constructed object graph. The basic layer is shown as follows:

$\mathbf{X}^{out}=\frac{\sigma(\mathbf{D}^{-1/2}\mathbf{\hat{A}}\mathbf{D}^{-1/2}\mathbf{X}^{in}\mathbf{W})+\mathbf{X}^{in}}{\sqrt{2}}$
(15)

where $\mathbf{X}^{in}$ is the node feature ($\mathbf{\overline{V}}$ in Eq.7),
$\mathbf{\hat{A}}$ is the adjacency matrix ($\mathbf{A}+\mathbf{I}$),
$\mathbf{D}$ is the degree matrix of $\mathbf{\hat{A}}$, $\mathbf{W}$ is the
trainable weights and $\sigma(\cdot)$ is the ReLU activation function. Then we
stack $k$ layers of classic spectral GCN [16] together with residual
architecture to aggregate the fine-grained object features.

###  3.5 Double-Hints Guided Question Generation

In this section, we employ the attention-based hierarchical sequence decoder
from [27] for the double-hints guided question generation step with two
alternative implementations: 1) LSTM based decoder and 2) transformer based
decoder. We will discuss them respectively.

####  3.5.1 LSTM based Question Generation Decoder

This module consists of two LSTMs: 1) vision LSTM and 2) language LSTM. The
vision LSTM is used to encode the visual features and the language LSTM is
used to generate words. Note that the starting states are initialized by
answer features. Between the two LSTMs, the model attends on the image inputs
and graph inputs separately guided by visual hints. We refer to this procedure
as visual-hint-guided separate attention.

Vision LSTM. Technically, at time step $t$, we first adopt the vision LSTM to
fuse the previous step’s hidden state $\mathbf{h}^{t-1}_{1}$ with the global
image features $\mathbf{I}$ and the word embedding $\mathbf{q}_{t}$ to create
a current step’s hidden state $\mathbf{h}^{t}_{1}$ as follows:

$\mathbf{h}^{t}_{1}=\text{LSTM}(\overline{\mathbf{I}},\mathbf{q}_{t};\mathbf{h}^{t-1}_{1}),$
(16)

where $\mathbf{\overline{I}}\in\mathbb{R}^{F_{v}}$ is the mean pooling result
of image feature $\mathbf{I}$.

Visual-hint-guided Separate Attention. The visual-hint-guided separate
attention module then attends to the coarse-grained image feature and fine-
grained graph features, respectively. For image attention, we apply the
classic attention mechanism on the image feature given by:

$\mathbf{h}_{image}=\text{Attention}(\mathbf{I},\mathbf{h}_{1}^{t}),$ (17)

where $\text{Attention}(\cdot,\cdot)$ is the classic attention mechanism [28].
For graph attention, we ignore the nodes which are predicted to be non-visual-
hint. Therefore, we can define our graph attention mechanism as follows:

$\begin{split}\mathbf{X}_{vh}&=\text{VisualHintMask}(\mathbf{X})\\\
\mathbf{h}_{graph}&=\text{Attention}(\mathbf{X}_{vh},\mathbf{h}_{1}^{t})\end{split}$
(18)

where $\text{VisualHintMask}(\cdot)$ is to mask off the non-visual-hint
objects, $\mathbf{X}$ is the graph node embedding and
$\text{Attention}(\cdot,\cdot)$ is also the attention function.

Language LSTM. Finally, the language LSTM will absorb the image attention
features $\mathbf{h}_{image}$ and graph attention result $\mathbf{h}_{graph}$
as follows:

$\mathbf{h}^{t}_{2}=\text{LSTM}(\text{CONCAT}(\mathbf{h}_{image},\mathbf{h}_{graph},\mathbf{h}^{t}_{1});\mathbf{h}^{t-1}_{2}),$
(19)

where the $\mathbf{h}_{image},\mathbf{h}_{graph}$ and the vision LSTM’s hidden
state $\mathbf{h}^{t}_{1}$ are concatenated, $\mathbf{h}^{t-1}_{2}$ is the
language LSTM’s previous step hidden state and $\mathbf{h}^{t}_{2}$ is the
generated hidden state. We project the $\mathbf{h}^{t}_{2}$ to the vocabulary
space to predict the word. We train it with the cross-entropy loss $L_{ques}$.

####  3.5.2 Transformer based Question Generation Decoder

Another widely-adopted architecture for language generation is the
transformer. Similar to the LSTM-based architecture, we apply visual-hint-
guided separate attention by the multi-head attention mechanism. The
transformer decoder consists of $n$ basic layers.

Formally, we define the input word embeddings as
$\mathbf{Q}=\\{\mathbf{q}_{1},\mathbf{q}_{2},...,\mathbf{q}_{n}\\}\in\mathbb{R}^{n\times
d}$, where $n$ is the number of answer words. In order to fuse the answer
hints into the decoder, we additionally add the the mean pooling of answer
features (denoted as $\mathbf{\overline{a}}$) to the front of the $\mathbf{Q}$
as
$\mathbf{Q}=\\{\mathbf{\overline{a}},\mathbf{q}_{1},\mathbf{q}_{2},...,\mathbf{q}_{n}\\}$
(we overwrite $\mathbf{Q}$ in this section to make it clear to illustrate).
The framework of the basic decoder layer is shown as follows:

$\begin{split}\mathbf{Q}&=\text{Norm}(\text{MSA}(\mathbf{Q},\mathbf{Q},\mathbf{Q})+\mathbf{Q}),\\\
\mathbf{Q}&=\text{Norm}(\text{VHSA}(\mathbf{Q},\mathbf{I},\mathbf{X}_{vh})+\mathbf{Q}),\\\
\mathbf{Q}&=\text{Norm}(\text{FeedForward}(\mathbf{Q})),\end{split}$ (20)

where $\mathbf{Q}$ is the word embedding matrix, $\mathbf{I}$ is the image
feature, $\mathbf{X}_{vh}$ is the visual hints feature (the same as Sec.
3.5.1), $\text{MSA}(\cdot,\cdot,\cdot)$ denotes the multi-head self-attention,
$\text{VHSA}(\cdot,\cdot,\cdot)$ denotes the visual-hint-guided separate
attention and $\text{FeedForward}(\cdot)$ denotes the feedforward layer. In
detail, the $\text{VHSA}(\cdot,\cdot,\cdot)$ which denotes the visual-hint-
guided separate attention module can be formulated as follows:

$\begin{split}\mathbf{Q}_{img}&=\text{Norm}(\text{MHA}(\mathbf{Q},\mathbf{I},\mathbf{I})+\mathbf{Q}),\\\
\mathbf{Q}_{graph}&=\text{Norm}(\text{MHA}(\mathbf{Q},\mathbf{X}_{vh},\mathbf{X}_{vh})+\mathbf{Q}),\\\
\mathbf{Q}_{all}&=\mathbf{Q}+\mathbf{Q}_{graph}+\mathbf{Q}_{img},\\\
\mathbf{Q}_{out}&=\text{Norm}(\text{MSA}(\mathbf{Q}_{all},\mathbf{Q}_{all},\mathbf{Q}_{all})+\mathbf{Q}_{all}),\end{split}$
(21)

where the $\text{MHA}(\cdot,\cdot,\cdot)$ denotes the multi-head attention.

After decoding, we simply drop the first row of $\mathbf{Q}_{out}$ which is
corresponding to the answer hints, and use the rest to calculate the cross-
entropy loss $L_{ques}$.

The final overall loss is thus the combination of all the previous key
components,

$L=L_{ques}+\alpha L_{vh}+\beta L_{pos}+\gamma L_{ans}.$ (22)

TABLE I: Results on VQA2.0 and COCO-QA val set. Ours-LSTM denotes the DH-Graph2Seq with LSTM decoder. Ours-transformer denotes the DH-Graph2Seq with transformer decoder. All accuracies are in %.  Dataset | VQA 2.0 | COCO-QA
---|---|---
Method | BLEU@4 | CIDEr | METEOR | ROUGE | Q-BLEU@1 | BLEU@4 | CIDEr | METEOR | ROUGE | Q-BLEU@1
I2Q | 9.02 | 63.21 | 13.89 | 35.33 | 26.32 | 13.53 | 95.90 | 12.61 | 36.23 | 31.37
IT2Q | 18.41 | 134.88 | 19.90 | 45.71 | 40.27 | 17.80 | 128.64 | 16.17 | 43.22 | 38.88
IMVQG | 19.72 | 149.28 | 20.43 | 46.76 | 40.40 | 18.43 | 127.18 | 17.21 | 44.07 | 39.22
Dual | 19.90 | 151.60 | 20.60 | 47.00 | 41.90 | 18.80 | 131.10 | 17.73 | 44.19 | 39.92
Radial | 20.70 | 161.90 | 21.40 | 48.10 | 43.50 | 19.24 | 139.55 | 18.19 | 44.21 | 40.98
Ours-LSTM | 22.43 | 180.55 | 22.57 | 49.36 | 45.61 | 20.84 | 166.78 | 19.81 | 46.80 | 43.57
Ours-transformer | 22.43 | 180.66 | 22.54 | 49.31 | 45.66 | 20.87 | 166.82 | 19.89 | 46.76 | 43.61
TABLE II: Human Evaluation Results on VQA2.0. Method | Syntax | Semantics | Relevance
---|---|---|---
Radial | 4.25 (0.36) | 4.04 (0.45) | 3.09 (0.24)
Dual | 4.53 (0.40) | 4.38 (0.58) | 3.24 (0.26)
Ours | 4.73 (0.35) | 4.61 (0.40) | 3.58 (0.39)
GT | 4.52 (0.34) | 4.63 (0.41) | 4.14 (0.50)
TABLE III: Ablation Results on VQA2.0. All accuracies are in %. Method | BLEU@4 | METEOR | Q-BLEU@1
---|---|---|---
Full Model(Double hints) | 22.43 | 22.57 | 45.61
\- visual hints | 21.67 | 21.93 | 44.52
\- visual/answer hints | 18.53 | 20.06 | 40.32
\+ answer type |  |  |
\- gnn | 22.17 | 22.31 | 45.34
\- gnn + transformer | 22.30 | 22.40 | 45.53
\- visual attn | 21.98 | 22.26 | 45.11
\- gnn,visual attn | 21.92 | 22.21 | 45.04
\- pos/ans predict | 22.34 | 22.51 | 45.57

##  4 Experiments

In this section, we evaluate the effectiveness of our proposed model. The code
and data for our model are provided for research purposes111Please refer to
the github repo..

###  4.1 Datasets and Pre-processing

Datasets. We conduct the experiments on the VQA2.0 [29] and COCO-QA [30]
datasets. Both of them use images from MS-COCO [31]. And to fit our setting,
we introduce a simple method to generate the visual hints of the original
pairs (image, question, answer) without human annotations: (1) we use the Mask
R-CNN to generate objects (with category attributes) $\mathcal{R}$ in the
image. (2) we use Standford CoreNLP to find the noun-words in both questions
and answers. For each object attributes and noun-words we use the GloVe model
to initialize them and take the average to get the vector representation
denoted by $\mathbf{g}_{obj}$ and $\mathbf{g}_{noun}$. The object is
recognized as visual hints iff its’ l2 distance with any $\mathbf{g}_{noun}$
is smaller than the threshold $\mu$. But there are two special cases that can
lead to no aligned objects: (1) there are exactly no visual hints (eg. Q: Is
there any book? A: No) (2) the error caused by the detection model or the NLP
tools leads to no visual hints. For the first case, we will keep them. For the
second case, we will drop them due to the technical drawback. Moreover,
although one image could have multiple answer hints (one image-answer pair
generally corresponds to one question), there are a small portion of image-
answer pair being linked to multiple questions. In this case, we will randomly
reserve one. After pre-processing, the VQA 2.0 contains 239973 examples for
training and 116942 for validation, respectively. And the COCO-QA dataset
contains 53440 for training and 26405 for validation, respectively.

It is worth noting that the comparison with other baselines under this setting
is quite fair since we just use data pre-processing to generate the visual
hints without any human annotations. Furthermore, although the generated
visual hints are fairly noisy since we simply align them by GloVe l2-distance
metric, we will show later that the generated visual hints are still very
useful, providing important additional information beyond the corresponding
textual answer information.

Pre-processing. We employ pre-trained ResNet-101 to extract visual image
features. And for each image, we use a Masked-RCNN detector with ResNeXt-101
backbone to detect 100 object regions (selected by confidence score) and
extract features (fc6). The ResNet-101 is from torchvision and the Mask RCNN
is from Detectron2 [32]. After truncating questions longer than 20 words, we
build vocabulary on words with at least 3 occurrences. Since the test split is
not open for the public, we divide the train set to 90% train split and 10%
validation split.

###  4.2 Hyper-parameter settings

Overall setting. The image feature dimension is 2048 and the object feature
dimension is 1024. The words’ dimension is 512 and their weights are randomly
initialized. The hidden size in Eq. 3 is 128 to save CUDA memory. All the
hidden size is 1024 if not otherwise specified. As for the visual hints
prediction, the $\eta$ is 4 and $\lambda$ is 2. We optimize hyper-parameters
with random search [33]. The overall loss function’s $\alpha$ is 0.005,
$\beta$ is 0.001 and $\gamma$ is 0.001 for most cases. But for ablation model
’-vh’, $\alpha$ is 0, $\beta$ is 0.001 and $\gamma$ is 0.002. For ablation
model ’-position’, $\alpha$ is 0.005, $\beta$ is 0 and $\gamma$ is 0.001. And
for ablation model ’-answer’, $\alpha$ is 0.005, $\beta$ is 0.001 and $\gamma$
is 0. When obtaining visual hints, $\mu$ is set 5.7.We adopt adam [34]
optimizer with 0.0002 learning rate. The batch size is 120.

Graph embedding module setting. The multi-head cosine similarity metric’s $k$
is 3. The layer of GCN is 2. The sparsing hyper-parameter $\epsilon$ is 0.75.

###  4.3 Baseline Methods and Evaluation Metrics

####  4.3.1 Baseline Methods

We compare against the following baselines in our experiments:

##### I2Q

It means generating the questions without any hints. We adopt the classic
image caption show attend and tell method [35].

##### IT2Q

It means generating questions with answer types. We modify the show attend and
tell [35] method to take input from the joint embedding of the image and
answer type. And since there are no answer type annotations in the original
datasets, we adopt the same answer-type information as IMVQG which is a
baseline we will discuss later.

##### IMVQG [6]

This is a baseline that maximizes the mutual information among the generated
questions, the input images, and the expected answers. They use the answer
category (type) as a hint. For VQA 2.0 dataset, since they just annotate 80%
of the original dataset, so we annotate the rest of them as ”other”. And for
the COCO-QA dataset, we find that there are only 430 answers, so we annotate
their type attribute by ourselves just like they do in VQA 2.0.

##### Dual [1]

This is another competitive baseline that views the VQG task as the dual task
of VQA based on MUTAN architecture. They train the VQG task along with VQA to
enhance both VQG and VQA’s performance.

##### Radial [15]

This is the latest strong baseline for VQG. They use answers to build an
answer-radial object graph and learn the graph embedding. Then they use the
graph2seq paradigm to generate the questions.

####  4.3.2 Evaluation Metrics

Following previous works [6, 15], we adopt the standard linguistic measures
including BLEU [36], CIDEr [37], METEOR [38], ROUGE-L [39] and Q-BLEU [40].
These scores are calculated by officially released evaluation scripts.

Figure 4: Case study (left) and error analysis (right) results. Figure 5: The
details of case study examples. The red rectangles mean the visual-hint
regions.

###  4.4 Result Analysis and Human Evaluation

Table I shows the automatic evaluation results comparing our proposed method
against other state-of-art baselines. We can see that our method consistently
outperforms previous methods by a significant margin on both datasets. It
highlights that with double hints our proposed method makes a solid step
towards addressing the two identified issues described in Sec. 1. In addition,
we can find that the DH-Graph2Seq with LSTM decoder (i.e., Ours-LSTM) achieves
comparable performance compared with DH-Graph2Seq with transformer decoder
(i.e., Ours-transformer) on both datasets.

Furthermore, we conduct a small-scale human evaluation study on VQA2.0 val set
to assess the quality of the questions generated by our method, the ground
truth (GT) and the baselines: Radial and Dual in terms of syntax, semantics,
and relevance metrics. Concretely, we randomly select 50 examples for each
system: 1) the ground-truth results (denoted as GT), 2) our results (denoted
as Ours), 3) the ’Radial’ baseline’s results (denoted as Radial), 4) the
’dual’ baseline’s results (denoted as Dual).

We ask 5 human evaluators to give feedback on the quality of questions
randomly selected in the results of 4 systems. In each example, given a triple
containing a raw image, a target answer, and an anonymized system’s output,
they are asked to rate the quality of the output by answering the three
questions: a) is the question syntactically correct? b) is the question
semantically correct? c) is the question relevant to the image and the answer
pair? For each question, the rating scale is from 1 to 5. The standard is 1.
Pool (not acceptable), 2. Marginal, 3. Acceptable, 4. Good, 5. Excellent. We
develop software to automatically collect the evaluation results. The software
will feed the examples and calculate the scores.

The results are shown in Table II. We report the mean and standard deviation
scores. It is clear to see that our model significantly outperforms all strong
baseline methods on all metrics, making it the closest one to the ground-
truth.

###  4.5 Ablation Study

In this section, we conduct the ablation studies to demonstrate the importance
of each key component including double hints, double-hints guided graph,
visual attention, and multi-task auto-encoder on the VQA 2.0. Without loss of
generality, we use the DH-Graph2Seq with LSTM decoder as the full model (abbr:
Full Model(Double hints)).

Specifically, we will remove one or more components at one time to generate
the following ablation models: (1) w/o. visual hints (abbr: -visual hints): we
remove the visual-hints prediction module. (2) w/o. double hints (abbr:
-visual/answer hints + answer type): we remove both the visual and answer
hints but use the answer-type to further assess the effect of double hints.
(3) w/o. double-hints guided graph (abbr: -gnn): we remove the implicit graph
construction and gcn encoding modules. (4) replace gnn with transformer
encoder (abbr: - gnn + transformer): we remove the gnn encoder and use the
transformer encoder to encode the object features. (5) w/o. visual attention
(abbr: -visual attention): we remove the non-visual-hint mask during attention
when decoding. (6) w/o. double-hints guided graph and visual attention (abbr:
gnn, visual attn): We combine the ablation (3) and (4) together to further
assess the effect of them. (7) w/o. position and answer auto-encoder (abbr:
-pos/ans predict): we set $\beta$ and $\gamma$ to zero for reducing multi-task
auto-encoder to visual hints only based auto-encoder.

The ablation study results on VQA2.0 val set’s results are shown in Table III.
There are several interesting observations worth noting here. Specifically, by
turning off the visual hints (-visual hints), the model’s performance drops
nearly 2.8% (METEOR). It confirms that the visual hints are indeed helpful for
high-quality question generation. In addition, it’s interesting to find that
it still outperforms the baseline methods. We think the fine-grained
interaction of visual objects and answers helps. And when we further discard
the answer hints (-visual/answer hints + answer type), the performance
continues to drop rapidly by 18.2% (BLEU@4), which further demonstrates that
the answer hints are a more helpful signal compared to the answer type. These
results indicate the importance of double hints.

Next, by turning off the double-hints guided graph (i.e., - gnn), we observe
that the performance drops nearly 1.2% (METEOR). Furthermore, when we replace
the gnn with transformer (i.e., - gnn + transformer), the performance still
drops nearly 0.8% (METEOR). It demonstrates that it is beneficial to exploit
the hidden relations by learning a dynamic graph. Furthermore, when we discard
the visual attention during decoding (-visual attn), the performance drops by
1.3% (METEOR). This result illustrates that if we force the model to focus on
only the predicted visual-hint object regions, it can generate higher quality
questions, which further confirms the impact of visual hints. Additionally, if
we turn off both the graph and the visual attention (-gnn,visual attn), the
performance continues to drop, which further confirms that both these two
components play an indispensable role in the framework. Finally, we observe
that if we discard the multi-task auto-encoder (-pos/ans predict), the
performance drops slightly. It shows by predicting the object position and
target answer, the model can learn more robust visual hint features.

Figure 6: (a) The influence of $\epsilon$ for different graph sparsity and (b)
The influence of $\mu$ in visual hints annotating procedure.

###  4.6 The analysis of hyper-parameters

####  4.6.1 The $\epsilon$ analysis

To study the effect of the $\epsilon$ in Sec. 3.3, we conduct experiments on
VQA 2.0 with the $\epsilon$ varying in $[0,1]$. The results are shown in
Figure 6 (a). We find that the model achieves the best performance when
$\epsilon$ ranging in [0.5, 0.8]. It drops rapidly when $\epsilon$ is close to
1 which means the graph is too sparse.

####  4.6.2 The $\mu$ analysis

To study the effect of the $\mu$ in Sec. 4.1, we conduct experiments on VQA
2.0 with the $\mu$ varying in $\\{3,5.7,7\\}$. When $\mu$ is larger, the
visual hint will be more accurate, but the reserved questions will be much
less. So in order to compare the effect fairly, we use the same test set and
evaluate both the DH-Graph2Seq model and DH-Graph2Seq w/o visual hint (abbr:
DH-Graph2Seq w/o vh). The results are shown in Figure 6 (b). We can find that
the DH-Graph2Seq consistently outperforms the DH-Graph2Seq w/o vh. When $\mu$
is smaller, although the visual hints are more accurate, the processed dataset
has less questions which lead to performance drop. And when $\mu$ is larger,
although we can reserve more questions, the visual hints’ quality is lower. So
we think a reasonable choice of $\mu$ is near 5.7.

###  4.7 Case Study and Error Analysis

Figure 7: The visualization of the learned graph structure. Figure 8: The
details of error examples.

In this section, we will further perform the case study to illustrate the
superiority of our method compared with other baselines. What’s more, we also
dive into the failures of our model by pinpointing different error cases. The
results are shown in Figure 4. We further visualize the learned graph in this
example in Figure 7. By learning the object graph and generating with the
guidance of double hints, we can find that our model indeed generates more
answer-aware, region-referential, and high-quality questions. We also provide
more qualitative cases in Figure 5. We can find that our model can generate
more complete and vivid questions compared with baseline methods. We think
that with the help of double hints, our graph2seq model can find the proper
image regions and exploit the rich structure relations better.

For error cases, since it is very difficult to classify a particular example
into the certain error type, we are only able to qualitatively divide them
into five different error categories. See Figure 8 for error cases of our
results. We present one example of each error reason.

  * •

1) detection error: It means our model recognizes the objects incorrectly. In
the picture, the ’teddy bear’ is next to the refrigerator but our model
recognizes it as ’microwave’.

  * •

2) visual hint prediction error: It means our model predicts the visual hints
incorrectly, which largely misleads the generation. Indeed, the answer ’3’
refers to the number of humans, but the model picks the horses out and
overlooks the men.

  * •

3) reasoning error: It means our model infers the relations among the objects
incorrectly. In the image, the batter wears the helmet but the expected answer
is ’no’.

  * •

4) semantic error: It means our model generates questions which are
semantically incorrect. In the image, there are at least two cars, but in the
generated question, the car is in the singular form.

  * •

5) syntactic error: It means our model generates questions which are
syntactically incorrect. The ’How many of’ is incorrect.

Figure 9: Performance of VQA model when varying the proportions of training
data via data augmentation.

###  4.8 Data Augmentation

One of the most important applications of VQG is to provide more training data
for VQA. Here, we use the proposed method as data augmentation model to
generate more questions for training VQA methods. In particular, we employ
three existing strong VQA models: i) Bottom-Up Top-Down (abbr: BUTD) [24], ii)
the Bilinear Attention Network (abbr: BAN) [41] and iii) the Modular Co-
Attention Networks (abbr: MCAN) [42]. We train these three VQA models on VQA
2.0 dataset and use top-1 accuracy as the evaluation metric. And we choose the
best VQG baseline (Radial) and our proposed method as two data augmentation
methods. In order to examine the effect of QG-driven data augmentation on the
VQA, we compare the performance of the VQA model (i.e. BUTD) with two data
augmentation variants, namely, BUTD w/Radial, BUTD w/DH-Graph2Seq.
Specifically, we split the train-set (the same as the VQG dataset) to
$x\in\\{0.3,0.5,0.7\\}$. The VQA models are trained only on the $x$ part,
while the other VQA variants are trained on the combination of the golden $x$
part and the questions generated by VQG models. What’s more, to train the VQG
model, we further split $x$ part to $80\%/20\%$(train/dev).

As shown in Figure 9, we observe that both VQG models are able to consistently
help improve the VQA performance and the performance boost is the most
significant when training data is scarce (i.e., 30%). Notably, our DH-
Graph2Seq model outperforms the Radial baseline consistently. Surprisingly, we
observe that the Radial augmented VQA models perform inferior to the single
VQA models slightly. This is because the generated questions may contain
noise, which may mislead the VQA model in some ways [15].

###  4.9 Zero-shot VQA

Furthermore, we introduce VQG as an effective solution for zero-shot VQA (ZS-
VQA) problems. Here, we employ VQG as a natural way to generate VQA questions,
which can be used as valuable data supplementary to address the zero-shot VQA
problem. To further evaluate the VQG models under the zero-shot VQA downstream
task, we design various combinations from the state-of-art VQA and VQG methods
on the VQA2.0 dataset.

Technically, following [15, 43], we firstly obtain the zero-shot words the
then construct the zero-shot VQA dataset. In particular, we mix the original
train and test samples together and find the same words between questions and
answers by filtering out stop words. Then, after randomly sampling 10% words
from them (denoted as ZS-VQA words), we create the ZS-VQA test data set from
all samples with the ZS-VQA words in answer. Then we can obtain the ZS-VQA
training set which has no overlap with the ZS-VQA testing set.

After that, we employ two VQG methods: 1) the best baseline Radial and 2) our
DH-Graph2Seq method to produce the questions for the ZS-VQA problem. In
particular, we employ two SOTA VQA methods: 1) BUTD and 2) BAN and use top-1
accuracy as the evaluation metric. Specifically, following [15], since ZS-VQA
words appear in the question, they can definitely be found in the images, and
we treat them as the target answers to generate questions to help the VQA
model to predict the zero-shot answer. The results are shown in Table IV.

From Table IV, we can observe that both BUTD and BAN VQA methods achieve 0%
accuracy in the ZS-VQA test set. The reason is that during training, the zero-
shot answers are not available, thus the models cannot recognize them
correctly in the inference. However, when we combine the VQG methods, both the
VQA models gain large improvements, which proves that the VQG is potentially
helpful for zero-shot VQA. Specifically, we find that the variants with DH-
Graph2Seq consistently outperform those with Radial, which verifies the
superiority of our proposed method.

TABLE IV: Zero-shot VQA results on zero-shot VQA2.0 test set.
All accuracies are in %. Method | w/o. VQG | Radial | DH-Graph2Seq
---|---|---|---
BUTD | 0.00 | 18.15 | 18.83
BAN | 0.00 | 18.37 | 18.91

##  5 Conclusion

Despite promising results that have been achieved in the visual question
generation task, we show in this paper that VQG still suffers from _one-to-
many mapping_ and _complex implicit relation modelling_ problems. To address
those issues, we propose a novel setting with double hints for the visual
question generation task, which could effectively mitigate the one-to-many
mapping issue. Under the new setting, we explicitly cast the VQG task as a
Graph-to-Sequence learning problem and design a novel DH-Graph2Seq model. This
model learns an implicit graph topology to capture the rich relationships
within an image and then utilizes the Graph2Seq model to generate the
questions with answer-awareness and region-reference. Our extensive
experiments on VQA2.0 and COCO-QA datasets demonstrate that our proposed model
can significantly outperform existing state-of-the-art by a large margin.
Furthermore, the data augmentation and zero-shot VQA experiments demonstrate
that the DH-Graph2Seq can effectively help real-world applications. We hope
that our work will encourage further exploration of VQG to generate more vivid
and meaningful questions.

##  6 Limitation and Future Works

Despite the significant progress achieved by DH-Graph2Seq, several challenges
and areas for improvement remain:

  * •

Visual Hint Quality. The quality of visual hints plays a pivotal role in
question generation quality. While we have introduced a rule-based method for
generating visual hints without requiring human annotation, this approach is
susceptible to noise and inaccuracies, which can adversely affect generation
performance. In future work, we aim to explore more effective methods to
enhance the quality of visual hints, thereby improving overall question
generation accuracy.

  * •

Visual-Language Pretraining. Although DH-Graph2Seq has advanced the quality of
visual question generation with the double hints guided learning paradigm and
graph-to-sequence framework, challenges persist in visual-language reasoning.
We think that current limitations in data and model scale may impede effective
multi-modal reasoning. In the future, we will extend our approach through
large-scale pretraining, involving the collection of more extensive datasets
to bolster the quality of question generation and enhance visual-language
reasoning capabilities.

## Acknowledgments

This work was supported by the Key Research and Development Projects in
Zhejiang Province (No. 2024C01106), the NSFC (No. 62272411), the National Key
Research and Development Project of China (2018AAA0101900), the Tencent WeChat
Rhino-Bird Special Research Program (Tencent WXG-FR-2023-10), and Research
funding from FinVolution Group.

## References

  * [1] Y. Li, N. Duan, B. Zhou, X. Chu, W. Ouyang, X. Wang, and M. Zhou, “Visual question generation as dual task of visual question answering,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 6116–6124.
  * [2] P. Wang, Q. Wu, C. Shen, A. Dick, and A. van den Hengel, “Fvqa: Fact-based visual question answering,” _IEEE Trans. Pattern Anal. Mach. Intell._ , vol. 40, no. 10, p. 2413–2427, Oct. 2018. [Online]. Available: https://doi.org/10.1109/TPAMI.2017.2754246
  * [3] Q. Wu, C. Shen, P. Wang, A. Dick, and A. v. d. Hengel, “Image captioning and visual question answering based on attributes and external knowledge,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 40, no. 6, pp. 1367–1381, 2018.
  * [4] U. Jain, S. Lazebnik, and A. G. Schwing, “Two can play this game: Visual dialog with discriminative question generation and answering,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2018.
  * [5] I. M. Mora, S. P. de la Puente, and X. G.-i. Nieto, “Towards automatic generation of question answer pairs from images,” _CVPRW_ , 2016.
  * [6] R. Krishna, M. Bernstein, and L. Fei-Fei, “Information maximizing visual question generation,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 2008–2018.
  * [7] N. Mostafazadeh, I. Misra, J. Devlin, M. Mitchell, X. He, and L. Vanderwende, “Generating natural questions about an image,” in _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. Berlin, Germany: Association for Computational Linguistics, Aug. 2016, pp. 1802–1813. [Online]. Available: https://aclanthology.org/P16-1170
  * [8] L. Peng, Y. Yang, Z. Wang, Z. Huang, and H. T. Shen, “Mra-net: Improving vqa via multi-modal relation attention network,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , pp. 1–1, 2020.
  * [9] Q. Cao, X. Liang, B. Li, and L. Lin, “Interpretable visual question answering by reasoning on dependency trees,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 43, no. 3, pp. 887–901, 2021.
  * [10] F. Liu, T. Xiang, T. M. Hospedales, W. Yang, and C. Sun, “Inverse visual question answering: A new benchmark and vqa diagnosis tool,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 42, no. 2, pp. 460–474, 2020.
  * [11] U. Jain, Z. Zhang, and A. G. Schwing, “Creativity: Generating diverse questions using variational autoencoders,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2017, pp. 6485–6494.
  * [12] S. Zhang, L. Qu, S. You, Z. Yang, and J. Zhang, “Automatic generation of grounded visual questions,” in _Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17_ , 2017, pp. 4235–4243. [Online]. Available: https://doi.org/10.24963/ijcai.2017/592
  * [13] F. Liu, T. Xiang, T. M. Hospedales, W. Yang, and C. Sun, “ivqa: Inverse visual question answering,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 8611–8619.
  * [14] M. Shah, X. Chen, M. Rohrbach, and D. Parikh, “Cycle-consistency for robust visual question answering,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 6649–6658.
  * [15] X. Xu, T. Wang, Y. Yang, A. Hanjalic, and H. T. Shen, “Radial graph convolutional network for visual question generation,” _IEEE Transactions on Neural Networks and Learning Systems_ , 2020.
  * [16] T. N. Kipf and M. Welling, “Semi-Supervised Classification with Graph Convolutional Networks,” in _Proceedings of the 5th International Conference on Learning Representations_ , ser. ICLR ’17, 2017. [Online]. Available: https://openreview.net/forum?id=SJU4ayYgl
  * [17] J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, “Neural message passing for quantum chemistry,” in _Proceedings of the 34th International Conference on Machine Learning-Volume 70_. JMLR. org, 2017, pp. 1263–1272.
  * [18] K. Xu, L. Wu, Z. Wang, Y. Feng, M. Witbrock, and V. Sheinin, “Graph2seq: Graph to sequence learning with attention-based neural networks,” _arXiv preprint arXiv:1804.00823_ , 2018.
  * [19] Y. Chen, L. Wu, and M. J. Zaki, “Reinforcement learning based graph-to-sequence model for natural question generation,” in _International Conference on Learning Representations_ , 2020. [Online]. Available: https://openreview.net/forum?id=HygnDhEtvr
  * [20] Y. Gao, L. Wu, H. Homayoun, and L. Zhao, “Dyngraph2seq: Dynamic-graph-to-sequence interpretable learning for health stage prediction in online health forums,” _arXiv preprint arXiv:1908.08497_ , 2019.
  * [21] W. Norcliffe-Brown, S. Vafeias, and S. Parisot, “Learning conditioned graph structures for interpretable visual question answering,” in _Advances in Neural Information Processing Systems_ , 2018, pp. 8334–8343.
  * [22] Y. Chen, L. Wu, and M. J. Zaki, “Graphflow: Exploiting conversation flow with graph neural networks for conversational machine comprehension,” in _Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20_ , C. Bessiere, Ed. International Joint Conferences on Artificial Intelligence Organization, 7 2020, pp. 1230–1236, main track. [Online]. Available: https://doi.org/10.24963/ijcai.2020/171
  * [23] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 770–778.
  * [24] P. Anderson, X. He, C. Buehler, D. Teney, M. Johnson, S. Gould, and L. Zhang, “Bottom-up and top-down attention for image captioning and visual question answering,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 6077–6086.
  * [25] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” in _Proceedings of the IEEE international conference on computer vision_ , 2017, pp. 2980–2988.
  * [26] Y. Chen, L. Wu, and M. Zaki, “Iterative deep graph learning for graph neural networks: Better and robust node embeddings,” in _Advances in Neural Information Processing Systems_ , H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2020, pp. 19 314–19 326. [Online]. Available: https://proceedings.neurips.cc/paper/2020/file/e05c7ba4e087beea9410929698dc41a6-Paper.pdf
  * [27] J. Lu, J. Yang, D. Batra, and D. Parikh, “Neural baby talk,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 7219–7228.
  * [28] D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” _arXiv preprint arXiv:1409.0473_ , 2014.
  * [29] S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. L. Zitnick, and D. Parikh, “Vqa: Visual question answering,” in _The IEEE International Conference on Computer Vision (ICCV)_ , December 2015.
  * [30] M. Ren, R. Kiros, and R. Zemel, “Exploring models and data for image question answering,” in _Advances in neural information processing systems_ , 2015, pp. 2953–2961.
  * [31] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in _European conference on computer vision_. Springer, 2014, pp. 740–755.
  * [32] Y. Wu, A. Kirillov, F. Massa, W.-Y. Lo, and R. Girshick, “Detectron2,” https://github.com/facebookresearch/detectron2, 2019.
  * [33] J. Bergstra and Y. Bengio, “Random search for hyper-parameter optimization.” _Journal of machine learning research_ , vol. 13, no. 2, 2012.
  * [34] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” _arXiv preprint arXiv:1412.6980_ , 2014.
  * [35] K. Xu, J. Ba, R. Kiros, K. Cho, A. Courville, R. Salakhudinov, R. Zemel, and Y. Bengio, “Show, attend and tell: Neural image caption generation with visual attention,” in _International conference on machine learning_ , 2015, pp. 2048–2057.
  * [36] K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in _Proceedings of the 40th annual meeting on association for computational linguistics_. Association for Computational Linguistics, 2002, pp. 311–318.
  * [37] R. Vedantam, C. Lawrence Zitnick, and D. Parikh, “Cider: Consensus-based image description evaluation,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2015, pp. 4566–4575.
  * [38] S. Banerjee and A. Lavie, “Meteor: An automatic metric for mt evaluation with improved correlation with human judgments,” in _Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization_ , 2005, pp. 65–72.
  * [39] C.-Y. Lin, “ROUGE: A package for automatic evaluation of summaries,” in _Text Summarization Branches Out_. Barcelona, Spain: Association for Computational Linguistics, Jul. 2004, pp. 74–81. [Online]. Available: https://www.aclweb.org/anthology/W04-1013
  * [40] P. Nema and M. M. Khapra, “Towards a better metric for evaluating question generation systems,” in _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_. Brussels, Belgium: Association for Computational Linguistics, Oct.-Nov. 2018, pp. 3950–3959. [Online]. Available: https://aclanthology.org/D18-1429
  * [41] J.-H. Kim, J. Jun, and B.-T. Zhang, “Bilinear attention networks,” _arXiv preprint arXiv:1805.07932_ , 2018.
  * [42] Z. Yu, J. Yu, Y. Cui, D. Tao, and Q. Tian, “Deep modular co-attention networks for visual question answering,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2019.
  * [43] Y. Li, Y. Yang, J. Wang, and W. Xu, “Zero-shot transfer vqa dataset,” _arXiv preprint arXiv:1811.00692_ , 2018.


# On t-extensions of abelian groups

H.Sahleh
Department of Mathematics, University of Guilan, P.O.BOX 1914
Rasht-Iran
e-mail<EMAIL_ADDRESS>
A.A. Alijani
Department of Mathematics, University of Guilan
e-mail<EMAIL_ADDRESS>
###### Abstract

Let $\Re$ be the category of all discrete abelian groups and $\pounds$ the
category of all locally compact abelian (LCA) groups. For a group
$G\in\pounds$, the maximal torsion subgroup of $G$ is denoted by $tG$. A short
exact sequence $0\to
A\stackrel{{\scriptstyle\phi}}{{\to}}B\stackrel{{\scriptstyle\psi}}{{\to}}C\to
0$ in $\Re$ is said to be a t-extension if $0\to
tA\stackrel{{\scriptstyle\phi}}{{\to}}tB\stackrel{{\scriptstyle\psi}}{{\to}}tC\to
0$ is a short exact sequence. We show that the set of all t-extensions of $A$
by $C$ is a subgroup of $Ext(C,A)$ which contains $Pext(C,A)$ for discrete
abelian groups $A$ and $C$. We establish conditions under which the
t-extensions split and determine those groups in $\Re$ which are t-injective
or t-projective in $\Re$. Finally we determine the compact groups $G$ in
$\pounds$ such that every pure extension $0\to G\to Y\to C\to 0$ splits where
$C$ is a compact connected group in $\pounds$.

Introduction

Throughout, all groups are Hausdorff topological abelian groups and will be
written additively. Let $\pounds$ denote the category of locally compact
abelian (LCA) groups with continuous homomorphisms as morphisms and $\Re$ the
category of discrete abelian groups. The Pontrjagin dual and the maximal
torsion subgroup of a group $G\in\pounds$ are denoted by $\hat{G}$ and
$tG$,respectively. A morphism is called proper if it is open onto its image
and a short exact sequence $0\to
A\stackrel{{\scriptstyle\phi}}{{\to}}B\stackrel{{\scriptstyle\psi}}{{\to}}C\to
0$ in $\pounds$ is said to be proper exact if $\phi$ and $\psi$ are proper
morphisms. In this case the sequence is called an extension of $A$ by $C$ ( in
$\pounds$ ). Following [4], we let $Ext(C,A)$ denote the (discrete) group of
extensions of $A$ by $C$. A subgroup $H$ of a group $C$ is called pure if
$nH=H\cap nC$ for all positive integers $n$. An extension $0\to
A\stackrel{{\scriptstyle\phi}}{{\to}}B\stackrel{{\scriptstyle\psi}}{{\to}}C\to
0$ is called a pure extension if $\phi(A)$ is pure in $B$. The elements
represented by pure extensions of $A$ by $C$ form a subgroup of $Ext(C,A)$
which is denoted by $Pext(C,A)$. In section 1 and 2, all groups are discrete
abelian groups. An extension $0\to
A\stackrel{{\scriptstyle\phi}}{{\longrightarrow}}B\stackrel{{\scriptstyle\psi}}{{\longrightarrow}}C\to
0$ in $\Re$ will be called a t-extension if $0\to
tA\stackrel{{\scriptstyle\phi}}{{\longrightarrow}}tB\stackrel{{\scriptstyle\psi}}{{\longrightarrow}}tC\to
0$ is an extension. Let $Ext_{t}(C,A)$ denote the set of all elements in
$Ext(C,A)$ represented by t-extensions. In section 1, we show that
$Ext_{t}(C,A)$ is a subgroup of $Ext(C,A)$ which contains $Pext(C,A)$ (see
Theorem 1.6 and Lemma 1.7). In section 2, we establish some results on
splitting of t-extensions (see Lemma 2.1, Theorem 2.11 and Theorem 2.13).
Assume that $\Im$ is any subcategory of $\pounds$. The section four is a part
of an investigation which answers the following question:
Under what conditions on $G\in\pounds$, $Ext(X,G)=0$ or $Pext(X,G)=0$ for all
$X\in\Im$?. In [2], [3], [4], [5], [7] and [8] the question is answered in
some subcategories of $\pounds$ such as the category of divisible locally
compact abelian groups. In [5, Corollary 3.4], Fulp and Griffith proved that a
compact group $G$ satisfies $Ext(C,G)=0$ for all compact connected groups $C$
if and only if $G\cong(R/Z)^{\sigma}$ where $\sigma$ is a cardinal. It may
happen that $Ext(X,G)\neq 0$ but $Pext(X,G)=0$. For example, $Ext(Z_{n},Z)\neq
0$ but $Pext(Z_{n},Z)=0$ where $Z$ is the group of integers and $Z_{n}$ is the
cyclic group of order $n$. In this paper, we show that a compact group $G$
satisfies $Pext(C,G)=0$ for all compact connected groups $C$ if and only if
$G\cong(R/Z)^{\sigma}\bigoplus H$ where $H$ is a compact totally disconnected
group (see Theorem 3.2). For the characterization of compact groups $G$ which
$Pext(C,G)=0$ for all compact connected groups $C$, we need to show that
$Pext(X,A)=0$ for a discrete torsion group $X$ and a discrete torsion-free
group $A$ (see Corollary 2.2). As a result,[5, Corollary 3.4] is a consequence
of Theorem 3.2 (see Remark 3.3).

The additive topological group of real numbers is denoted by $R$ and $Q$ is
the group of rationals with the discrete topology. We denote by $G_{0}$, the
identity component of a group $G\in\pounds$. For more on locally compact
abelian groups, see [6].

##  1 t-extension

In this section, we define the concept of a t-extension of $A$ by $C$. We show
that the set of all t-extensions of $A$ by $C$ form a subgroup of $Ext(C,A)$
which contains $Pext(C,A)$.

Definition 1.1. An extension $0\to
A\stackrel{{\scriptstyle\phi}}{{\longrightarrow}}B\stackrel{{\scriptstyle\psi}}{{\longrightarrow}}C\to
0$ is called a t-extension if $0\to
tA\stackrel{{\scriptstyle\phi}}{{\longrightarrow}}tB\stackrel{{\scriptstyle\psi}}{{\longrightarrow}}tC\to
0$ is an extension.

Lemma 1.2. A pushout or a pullback of a t-extension is a t-extension.

Proof. Suppose $0\to
A\stackrel{{\scriptstyle\phi}}{{\longrightarrow}}B\stackrel{{\scriptstyle\psi}}{{\longrightarrow}}C\to
0$ is a t-extension and

$\textstyle{0\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{A\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\phi}$$\scriptstyle{\mu}$$\textstyle{B\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\psi}$$\textstyle{C\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{1_{C}}$$\textstyle{0}$$\textstyle{0\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{A^{\prime}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\phi^{\prime}}$$\textstyle{(A^{\prime}\bigoplus
B)/H\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\psi^{\prime}}$$\textstyle{C\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{0}$

is a standard pushout diagram (see [1]). Then

$H=\\{(\mu(a),-\phi(a)),a\in A\\}$

and

$\phi^{\prime}:a^{\prime}\longmapsto(a^{\prime},0)+H,~{}~{}~{}~{}~{}\psi^{\prime}:(a^{\prime},b)+H\longmapsto\psi(b)$

We show that $0\to
tA^{\prime}\stackrel{{\scriptstyle\phi^{\prime}}}{{\longrightarrow}}t((A^{\prime}\bigoplus
B)/H)\stackrel{{\scriptstyle\psi^{\prime}}}{{\longrightarrow}}tC\to 0$ is
exact. First, we show that $\psi^{\prime}:t((A^{\prime}\bigoplus B)/H)\to tC$
is surjective. Let $c\in tC$. Since $0\to
tA\stackrel{{\scriptstyle\phi}}{{\longrightarrow}}tB\stackrel{{\scriptstyle\psi}}{{\longrightarrow}}tC\to
0$ is exact, so there exists $b\in tB$ such that $\psi(b)=c$. Clearly,
$(0,b)+H\in t((A^{\prime}\bigoplus B)/H)$. On the other hand,
$\psi^{\prime}((0,b)+H)=\psi(b)=c$. Hence $\psi^{\prime}$ is surjective. Now,
we show that $Ker\psi^{\prime}\mid_{X}\subseteq
Im\phi^{\prime}\mid_{tA^{\prime}}$ where $X=t((A^{\prime}\bigoplus B)/H)$. Let
$(a^{\prime},b)+H\in X$ and $\psi^{\prime}((a^{\prime},b)+H)=0$. So,
$\psi(b)=0$. Hence, there exists $a\in A$ such that $\phi(a)=-b$. On the other
hand, there exists a positive integer $n$ such that $(na^{\prime},nb)\in H$.
So, there exists $a_{1}\in A$ such that $\mu(a_{1})=na^{\prime}$ and
$-\phi(a_{1})=nb$. Now, we have

$\phi(a_{1}-na)=\phi(a_{1})-n\phi(a)=0$

So $a_{1}=na$ and $n(a^{\prime}-\mu(a))=0$. It follows that
$a^{\prime}-\mu(a)\in tA^{\prime}$ and
$\phi^{\prime}(a^{\prime}-\mu(a))=(a^{\prime}-\mu(a),0)+H=(a^{\prime},b)+H$
(since
$(a^{\prime}-\mu(a),0)-(a^{\prime},b)=(-\mu(a),-b)=(\mu(-a),-\phi(-a))\in H$).
Now, suppose that

$\textstyle{0\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{A\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\phi^{\prime}}$$\scriptstyle{1_{A}}$$\textstyle{B^{\prime}\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\psi^{\prime}}$$\textstyle{C^{\prime}\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\gamma}$$\textstyle{0}$$\textstyle{0\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{A\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\phi}$$\textstyle{B\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\psi}$$\textstyle{C\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{0}$

is a standard pullback diagram. Then

$B^{\prime}=\\{(b,c^{\prime});\psi(b)=\gamma(c^{\prime})\\}$

and

$\phi^{\prime}:a\longmapsto(\phi(a),0),~{}~{}~{}~{}~{}~{}\psi^{\prime}:(b,c^{\prime})\longmapsto
c^{\prime}$

We show that $0\to
tA\stackrel{{\scriptstyle\phi^{\prime}}}{{\longrightarrow}}tB^{\prime}\stackrel{{\scriptstyle\psi^{\prime}}}{{\longrightarrow}}tC^{\prime}\to
0$ is exact. Let $c^{\prime}\in tC^{\prime}$. Then, there exists a positive
integer $n$ such that $nc^{\prime}=0$. Since $\psi$ is surjective,
$\psi(b)=\gamma(c^{\prime})$ for some $b\in B$ . Now,
$n\psi(b)=\gamma(nc^{\prime})=0$. Hence, $\psi(b)\in tC$. Since $0\to
tA\stackrel{{\scriptstyle\phi}}{{\longrightarrow}}tB\stackrel{{\scriptstyle\psi}}{{\longrightarrow}}tC\to
0$ is exact, so $\psi(b_{1})=\psi(b)$ for some $b_{1}\in tB$. Hence,
$(b_{1},c^{\prime})\in tB^{\prime}$ and
$\psi^{\prime}(b_{1},c^{\prime})=c^{\prime}$. Therefore,
$\psi^{\prime}:tB^{\prime}\to tC^{\prime}$ is surjective. Now, suppose that
$(b,c^{\prime})\in tB^{\prime}$ and $\psi^{\prime}(b,c^{\prime})=0$. Then
$c^{\prime}=0$ and $nb=0$ for some positive integer $n$. So $b\in tB$. Since
$\psi(b)=\gamma(c^{\prime})=0$ and $0\to
tA\stackrel{{\scriptstyle\phi}}{{\longrightarrow}}tB\stackrel{{\scriptstyle\psi}}{{\longrightarrow}}tC\to
0$ is exact, there exists $a\in tA$ such that $\phi(a)=b$. Now, we have

$\phi^{\prime}(a)=(\phi(a),0)=(b,0)=(b,c^{\prime})$

It follows that $Ker\psi^{\prime}\mid_{tB^{\prime}}\subseteq
Im\phi^{\prime}\mid_{tA}$.

Remark 1.3. Let $\beta:B\to X$ be an isomorphism and $x\in tX$. Then $nx=0$
for some positive integer $n$. Since $\beta$ is surjective, so there exist
$b\in B$ such that $\beta(b)=x$. Hence, $\beta(nb)=0$. Since $\beta$ is
injective, so $nb=0$. Therefore, $\beta\mid_{tB}:tB\to tX$ is an isomorphism.

Recall that two extensions $0\to
A\stackrel{{\scriptstyle\phi_{1}}}{{\longrightarrow}}B\stackrel{{\scriptstyle\psi_{1}}}{{\longrightarrow}}C\to
0$ and $0\to
A\stackrel{{\scriptstyle\phi_{2}}}{{\longrightarrow}}X\stackrel{{\scriptstyle\psi_{2}}}{{\longrightarrow}}C\to
0$ are said to be equivalent if there is an isomorphism $\beta:B\to X$ such
that the following diagram

$\textstyle{0\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{A\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\phi_{1}}$$\scriptstyle{1_{A}}$$\textstyle{B\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\psi_{1}}$$\scriptstyle{\beta}$$\textstyle{C\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{1_{C}}$$\textstyle{0}$$\textstyle{0\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{A\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\phi_{2}}$$\textstyle{X\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\psi_{2}}$$\textstyle{C\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{0}$

is commutative.

Lemma 1.4. An extension equivalent to a t-extension is a t-extension.

Proof. Let

$E_{1}:0\to
A\stackrel{{\scriptstyle\phi_{1}}}{{\longrightarrow}}B\stackrel{{\scriptstyle\psi_{1}}}{{\longrightarrow}}C\to
0$

and

$E_{2}:0\to
A\stackrel{{\scriptstyle\phi_{2}}}{{\longrightarrow}}X\stackrel{{\scriptstyle\psi_{2}}}{{\longrightarrow}}C\to
0$

be two equivalent extensions such that $E_{1}$ is a t-extension. Then, there
is an isomorphism $\beta:B\to X$ such that $\beta\phi_{1}=\phi_{2}$ and
$\psi_{2}\beta=\psi_{1}$. Let $x\in tC$. Since $E_{1}$ is t-extension, so
$\psi_{1}(b)=x$ for some $b\in tB$. Hence, $\psi_{2}(\beta(b))=\psi_{1}(b)=x$.
So, $\psi_{2}:tX\to tC$ is surjective. Now, let $\psi_{2}(x)=0$ for some $x\in
tX$. By Remark 1.3, there exist $b\in tB$ such that $\beta(b)=x$. Hence,
$\psi_{1}(b)=\psi_{2}(\beta(b))=0$. Since $E_{1}$ is t-extension, so
$\phi_{1}(a)=b$ for some $a\in tA$. Consequently,
$\phi_{2}(a)=\beta(\phi_{1}(a))=x$.

Remark 1.5. Let $C$ and $A$ be two groups and $0\to
A\stackrel{{\scriptstyle\phi_{1}}}{{\longrightarrow}}B_{1}\stackrel{{\scriptstyle\psi_{1}}}{{\longrightarrow}}C\to
0$ and $0\to
A\stackrel{{\scriptstyle\phi_{2}}}{{\longrightarrow}}B_{2}\stackrel{{\scriptstyle\psi_{2}}}{{\longrightarrow}}C\to
0$ two t-extensions of $A$ by $C$. An easy calculation shows that $0\to
A\bigoplus
A\stackrel{{\scriptstyle(\phi_{1}\bigoplus\phi_{2})}}{{\longrightarrow}}B_{1}\bigoplus
B_{2}\stackrel{{\scriptstyle(\psi_{1}\bigoplus\psi_{2})}}{{\longrightarrow}}C\bigoplus
C\to 0$ is a t-extension where
$(\phi_{1}\bigoplus\phi_{2})(a_{1},a_{2})=(\phi_{1}(a_{1}),\phi_{2}(a_{2}))$
and
$(\psi_{1}\bigoplus\psi_{2})(b_{1},b_{2})=(\psi_{1}(b_{1}),\psi_{2}(b_{2}))$.

Theorem 1.6. Let $A$ and $C$ be two groups. Then, the class $Ext_{t}(C,A)$ of
all equivalence classes of t-extensions of $A$ by $C$ is an subgroup of
$Ext(C,A)$ with respect to the operation defined by

$[E_{1}]+[E_{2}]=[\nabla_{A}(E_{1}\bigoplus E_{2})\triangle_{C}]$

where $E_{1}$ and $E_{2}$ are t-extensions of $A$ by $C$ and $\nabla_{A}$ and
$\triangle_{C}$ are the diagonal and codiagonal homomorphism.

Proof. Clearly, $0\to A\to A\bigoplus C\to C\to 0$ is a t-extension. By Lemma
1.2 and Remark 1.5, $[E_{1}]+[E_{2}]\in Ext_{t}(C,A)$ for two t-extensions
$E_{1}$ and $E_{2}$ of $A$ by $C$. So, $Ext_{t}(C,A)$ is a subgroup of
$Ext(C,A)$.

Lemma 1.7. Let $A$ and $C$ be two groups. Then, $Pext(C,A)\subseteq
Ext_{t}(C,A)$.

Proof. Let $0\to
A\stackrel{{\scriptstyle\phi}}{{\longrightarrow}}B\stackrel{{\scriptstyle\psi}}{{\longrightarrow}}C\to
0$ be an element of $Pext(C,A)$. It is sufficient to show that
$tB/t\phi(A)\cong t(B/\phi(A))$. Consider the map $\varphi:tB\to t(B/\phi(A))$
given by $b\longmapsto b+\phi(A)$. Clearly, $\varphi$ is a homomorphism. We
show that $\varphi$ is surjective. Let $b+\phi(A)\in t(B/\phi(A))$. Then,
there exists a positive integer $n$ such that $nb\in\phi(A)$. Since $\phi(A)$
is pure in $B$, so $nb=n\phi(a)$ for some $a\in A$. Hence, $n(b-\phi(a))=0$.
This shows that $b-\phi(a)\in tB$ and $\varphi(b-\phi(a))=b+\phi(A)$. So
$\varphi$ is surjective. We have

$Ker\varphi=\\{b\in tB;b\in\phi(A)\\}=\phi(A)\bigcap tB=t\phi(A)$

Hence $tB/t\phi(A)\cong t(B/\phi(A))$.

Corollary 1.8. If $A$ is a divisible group or $C$ a torsion-free group, then
$Pext(C,A)=Ext_{t}(C,A)=Ext(C,A)$.

Proof. It is clear.

##  2 Splitting of t-extensions

In this section, we establish some conditions on $A$ and $C$ such that
$Ext_{t}(C,A)=0$. We also determine the t-injective and t-projective groups in
$\Re$.

Lemma 2.1. Let $A$ be a torsion-free group and $C$ a torsion group. Then,
$Ext_{t}(C,A)=0$.

Proof. Let $E:0\to
A\stackrel{{\scriptstyle\phi}}{{\longrightarrow}}B\stackrel{{\scriptstyle\psi}}{{\longrightarrow}}C\to
0$ be a t-extension. Then $\psi_{|tB}:tB\to C$ is an isomorphism. Let $b\in
B$. Then, $\psi(b)\in C$. So $\psi(b)=\psi(b^{\prime})$ for some
$b^{\prime}\in tB$. Hence, $b-b^{\prime}=\phi(a)$ for some $a\in A$. This
follows that $B=\phi(A)+tB$. Since $\phi(A)$ is torsion-free and $tB$ torsion,
so $\phi(A)\bigcap tB=0$. Hence, $B=\phi(A)\bigoplus tB$ and $E$ splits.

Corollary 2.2. Let $A$ be a torsion-free group and $C$ a torsion group. Then,
$Pext(C,A)=0$.

Proof. It is clear by Lemma 1.7 and Lemma 2.1.

Lemma 2.3. Let $A$ and $C$ be two torsion groups. Then
$Ext(C,A)=Ext_{t}(C,A)$.

Proof. Let $A$ and $C$ be two torsion groups. It is clear that
$Ext_{t}(C,A)\subseteq Ext(C,A)$. Suppose that $E:0\to A\to B\to C\to 0$ is an
extension. Then, $B$ is a torsion group. Hence, $E$ is a t-extension.

Lemma 2.4. Let $C$ be a torsion group. Then, $Ext(C,tA)\cong Ext_{t}(C,A)$ for
every group $A$.

Proof. The exact sequence $0\to tA\stackrel{{\scriptstyle
i}}{{\longrightarrow}}A\stackrel{{\scriptstyle\pi}}{{\longrightarrow}}A/tA\to
0$ induces the following exact sequence

$Hom(C,A/tA)\to Ext(C,tA)\stackrel{{\scriptstyle
i_{\ast}}}{{\longrightarrow}}Ext(C,A)\stackrel{{\scriptstyle\pi_{\ast}}}{{\longrightarrow}}Ext(C,A/tA)\to
0$

Note that $Hom(C,A/tA)=0$. By Lemma 2.1, $Ext_{t}(C,A/tA)=0$. Since
$\pi_{\ast}(Ext_{t}(C,A))\subseteq Ext_{t}(C,A/tA)$, so
$\pi_{\ast}(Ext_{t}(C,A))=0$. Hence, $Ext_{t}(C,A)\subseteq
Ker\pi_{\ast}=i_{\ast}(Ext(C,tA))$. By Lemma 2.3, $Ext(C,tA)=Ext_{t}(C,tA)$.
So, $i_{\ast}(Ext(C,tA))\subseteq Ext_{t}(C,A)$. Hence, $Ext(C,tA)\cong
Ext_{t}(C,A)$.

Corollary 2.5. Let $A$ be a group. Then, $Ext_{t}(Z(m),A)\cong tA/m(tA)$ for
every positive integer $m$.

Proof. It is clear by Lemma 2.4 and [1,page 222].

Corollary 2.6. Let $C$ be a torsion group and $\\{A_{i}:i\in I\\}$ a
collection of groups. If $I$ is finite, then $Ext_{t}(C,\Pi_{i\in
I}A_{i})\cong\Pi_{i\in I}Ext_{t}(C,A_{i})$.

Proof. Let $I=\\{1,...,n\\}$ for some positive integer $n$. By Lemma 2.4,
$Ext_{t}(C,\Pi_{i=1}^{n}A_{i})\cong Ext(C,t(\Pi_{i=1}^{n}A_{i}))$. Since
$t(\Pi_{i=1}^{n}A_{i})=\Pi_{i=1}^{n}t(A_{i})$, so
$Ext_{t}(C,\Pi_{i=1}^{n}A_{i})\cong\Pi_{i\in I}Ext(C,tA_{i})\cong\Pi_{i\in
I}Ext_{t}(C,A_{i})$.

Remark 2.7. In general, $Ext_{t}(C,\Pi_{i\in I}A_{i})\not\cong\Pi_{i\in
I}Ext_{t}(C,A_{i})$.

Example 2.8. Let $p$ be a prime and $H=\prod_{n=1}^{\infty}Z(p^{n})$. By Lemma
2.4, $Ext_{t}(Q/Z,H)\cong Ext(Q/Z,tH)$. Consider the following exact sequence

$(2.1)~{}~{}~{}~{}~{}0\to Ext(Q/Z,tH)\to Ext(Q/Z,H)\to Ext(Q/Z,H/tH)\to 0$

By [1, Theorem 52.2] and Lemma 2.3,

$Ext(Q/Z,H)\cong\prod_{n=1}^{\infty}Ext(Q/Z,Z(p^{n}))\cong\prod_{n=1}^{\infty}Ext_{t}(Q/Z,Z(p^{n}))$

If $Ext_{t}(Q/Z,H)\cong\prod_{n=1}^{\infty}Ext_{t}(Q/Z,Z(p^{n}))$,then
$Ext(Q/Z,H)\cong Ext(Q/Z,tH)$. It follows from (2.1) that, $Ext(Q/Z,H/tH)=0$
which is a contradiction, since $H/tH$ is not divisible.

Lemma 2.9. Let $A$ be a torsion-free group. Then, $Ext(C/tC,A)\cong
Ext_{t}(C,A)$ for every group $C$.

Proof. The exact sequence $0\to tC\stackrel{{\scriptstyle
i}}{{\longrightarrow}}C\stackrel{{\scriptstyle\pi}}{{\longrightarrow}}C/tC\to
0$ induces the following exact sequence

$Hom(tC,A)\to
Ext(C/tC,A)\stackrel{{\scriptstyle\pi_{\ast}}}{{\longrightarrow}}Ext(C,A)\stackrel{{\scriptstyle
i_{\ast}}}{{\longrightarrow}}Ext(tC,A)\to 0$

Note that $Hom(tC,A)=0$. By Lemma 2.1, $Ext_{t}(tC,A)=0$. So,
$i_{\ast}(Ext_{t}(C,A))\subseteq Ext_{t}(tC,A)=0$. Hence,
$Ext_{t}(C,A)\subseteq Keri_{\ast}=\pi_{\ast}(Ext(C/tC,A))$. By Corollary 1.8,
$Ext(C/tC,A)=Ext_{t}(C/tC,A)$. So, $\pi_{\ast}(Ext(C/tC,A))\subseteq
Ext_{t}(C,A)$. Hence, $Ext(C/tC,tA)\cong Ext_{t}(C,A)$.

Definition 2.10. Let $G$ be a group. We call $G$ a t-injective group in $\Re$
if for every t-extension

$0\to A\stackrel{{\scriptstyle\phi}}{{\longrightarrow}}B\to C\to 0$

and a homomorphism $f:A\to G$, there is a homomorphism
$\bar{f}:B\longrightarrow G$ such that $\bar{f}\phi=f$.

We call $G$ a t-projective group in $\Re$ if for every t-extension

$0\to A\to B\stackrel{{\scriptstyle\psi}}{{\longrightarrow}}C\to 0$

and a homomorphism $f:G\to C$, there is a homomorphism $\bar{f}:G\to B$ such
that $\psi\bar{f}=f$ .

Recall that a group $A$ is said to be cotorsion if $Ext(Q,A)=0$.

Theorem 2.11. Let $A$ be a group. The following statements are equivalent:

  1. 1.

$A$ is t-injective in $\Re$.

  2. 2.

$Ext_{t}(C,A)=0$ for all $C\in\Re$.

  3. 3.

$A\cong B\bigoplus D$ where $B$ is a torsion divisible group and $D$ a
torsion-free cotorsion group.

Proof. $1\Longrightarrow 2$: Let $A$ be a t-injective in $\Re$ and $E:0\to
A\stackrel{{\scriptstyle\phi}}{{\longrightarrow}}B\stackrel{{\scriptstyle}}{{\longrightarrow}}C\to
0$ a t-extension of $A$ by $C$. Then, there is a homomorphism $\bar{\phi}:B\to
G$ such that $\bar{\phi}\phi=1_{G}$. Consequently, $E$ splits.

$2\Longrightarrow 3$: Let $Ext_{t}(C,A)=0$ for every group $C$. So
$Ext_{t}(Z(m),A)=0$ for every positive integer $m$. By Corollary 2.5,
$m(tA)=tA$ for every positive integer $m$. So, $tA$ is divisible. Hence,
$A\cong tA\bigoplus A/tA$. Therefore, $Ext(Q,A)\cong Ext(Q,tA)\bigoplus
Ext(Q,A/tA)$. Since $Ext(Q,A)=Ext_{t}(Q,A)=0$, then $Ext(Q,A/tA)=0$. Hence,
$A/tA$ is cotorsion. Now, we set $tA=B$ and $A/tA=D$.

$3\Longrightarrow 2$: Suppose that $A\cong B\bigoplus D$ where $B$ is a
torsion divisible group and $D$ a torsion-free cotorsion group. Let $C$ be a
group. Since $Ext(C,B)=0$, so $\pi_{2}^{*}:Ext(C,A)\to Ext(C,D)$ is an
isomorphism. By Lemma 2.9, $Ext_{t}(C,D)\cong Ext(C/tC,D)$. Since $D$ is a
cotorsion group, so $Ext(C/tC,D)=0$. Hence, $Ext_{t}(C,D)=0$. So,
$\pi_{2}^{*}(Ext_{t}(C,A))\subseteq Ext_{t}(C,D)=0$. Since $\pi_{2}^{*}$ is an
isomorphism, so $Ext_{t}(C,A)=0$.

$2\Longrightarrow 1$: Let $E:0\to
G\stackrel{{\scriptstyle\phi}}{{\longrightarrow}}B\to C\to 0$ be a t-extension
and $f:G\to A$ a homomorphism. Then $f$ induces a pushout diagram

$\textstyle{E:0\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{G\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\phi}$$\scriptstyle{f}$$\textstyle{B\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{C\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{0}$$\textstyle{fE:0\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{A\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\mu}$$\textstyle{(A\bigoplus
B)/H\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{C\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{0}$

Where $H=\\{(-f(a),\phi(a));a\in A\\}$ and $\mu:a\longmapsto(a,0)+H$. By Lemma
1.2, $fE$ is a t-extension and by assumption it splits. Hence $A$ is
t-injective.

Recall that a group $A$ is called algebraically compact if and only if
$Pext(X,A)=0$ for every group $X$ [1].

Corollary 2.12. A torsion-free, cotorsion group is algebraically compact.

Proof. Let $A$ be a torsion-free, cotorsion group. By Theorem 2.11,
$Ext_{t}(C,A)=0$ for every group $C$. Hence, $Pext(C,A)=0$ for every group
$C$.

Theorem 2.13. Let $C$ be a group. The following statements are equivalent:

  1. 1.

$C$ is t-projective in $\Re$.

  2. 2.

$Ext_{t}(C,A)=0$ for all $A\in\Re$.

  3. 3.

$C$ is a free group.

Proof. $1\Longrightarrow 2$: Let $C$ be a t-projective in $\Re$ and $E:0\to
A\stackrel{{\scriptstyle\phi}}{{\longrightarrow}}B\stackrel{{\scriptstyle}}{{\longrightarrow}}C\to
0$ a t-extension of $A$ by $C$. Then there is a homomorphism $\bar{\phi}:C\to
B$ such that $\bar{\phi}\phi=1_{C}$. Consequently, $E$ splits.

$2\Longrightarrow 3$: Let $Ext_{t}(C,A)=0$ for every group $A$. By Lemma 1.7,
$Pext(C,A)=0$ for every group $A$. Hence, $C$ is free group.

$3\Longrightarrow 1$: It is clear.

##  3 Splitting of pure extensions in the category of locally compact abelian
groups

In this section, we determine the structure of a compact group $G$ such that
$Pext(C,G)=0$ for all compact connected groups $C$.

Lemma 3.1. If $A$ is a compact totally disconnected group and $C$ a compact
connected group, then $Pext(C,A)=0$.

Proof. By [7, Lemma 2.3], $Pext(C,A)\cong Pext(\hat{A},\hat{C})$. On the other
hand, $\hat{A}$ and $\hat{C}$ are a discrete, torsion group and a discrete,
torsion-free respectively [6, Theorem 24.25 and 24.26]. Hence, by Corollary
2.2,$Pext(\hat{A},\hat{C})=0$. So $Pext(C,A)=0$.

Let $G$ be a compact group. Then, $Ext(C,G)=0$ for every compact connected
group $C$ if and only if $G\cong(R/Z)^{\sigma}$ [5, Corollary 3.4]. In the
next Theorem, we show that $Pext(C,G)=0$ for every compact connected group $C$
if and only if $G\cong(R/Z)^{\sigma}\bigoplus H$ where $H$ is a compact
totally disconnected group.

Theorem 3.2. Let $G$ be a compact group. Then, $Pext(C,G)=0$ for all compact
connected groups $C$ if and only if $G\cong(R/Z)^{\sigma}\bigoplus H$ where
$H$ is a compact totally disconnected group.

Proof. Let $G$ be a compact group and $C$ a compact connected group. Consider
the exact sequence $0\to G_{0}\to G\to G/G_{0}$. By Proposition 4 of [2], we
have the exact sequence

$Hom(C,G/G_{0})\to Ext(C,G_{0})\to Pext(C,G)\to Pext(C,G/G_{0})$

By Lemma 3.1, $Pext(C,G/G_{0})=0$. Since $G/G_{0}$ is totally disconnected, so
$Hom(C,G/G_{0})=0$. It follows that $Pext(C,G)\cong Ext(C,G_{0})$. Now, Let
$Pext(C,G)=0$ for all compact connected groups $C$. Then $Ext(C,G_{0})=0$ for
all compact connected groups $C$. By [5, Corollary 3.4],
$G_{0}\cong(R/Z)^{\sigma}$. So, the extension $0\to G_{0}\to G\to G/G_{0}\to
0$ splits. This shows that $G\cong(R/Z)^{\sigma}\bigoplus G/G_{0}$. The
converse is clear.

Remark 3.3. Let $G$ be a compact group such that $Ext(C,G)=0$ for every
compact connected group $C$. Then $Pext(C,G)=0$ for every compact connected
group $C$. So by Theorem 3.2, $G\cong(R/Z)^{\sigma}\bigoplus H$ where $H$ is a
compact totally disconnected group. Since $R/Z$ is a compact connected group,
so $Ext(R/Z,H)=0$. Consider the following exact sequence

$Hom(R,H)\to Hom(Z,H)\to Ext(R/Z,H)=0$

Since $R$ is a connected group and $H$ a totally disconnected group, so
$Hom(R,H)=0$. Hence $H=0$ and $G\cong R/Z$.

## References

  * [1] L.Fuchs, Infinite Abelian Groups, Vol. I, Academic Press, New York, 1970.
  * [2] R. O. Fulp, Homological study of purity in locally compact groups, Proc. London Math. Soc. 21 (1970), 501-512.
  * [3] R.O.Fulp, Splitting locally compact abelian groups, Michigan Math. J. 19 ( 1972), 47-55.
  * [4] R.O.Fulp and P. Griffith, Extensions of locally compact abelian groups I,Trans. Amer. Math. Soc. 154 (1971), 341-356.
  * [5] R.O.Fulp and P.Griffith, Extensions of locally compact abelian groups II, Trans Math. Soc. 154 (1971), 357-363.
  * [6] E. Hewitt and K. Ross, Abstract Harmonic Analysis, Vol I, Second Edition, Springer Verlag, Berlin, 1979.
  * [7] P. Loth, Pure extensions of locally compact abelian groups, Rend.Sem.Mat.Univ.Padova . 116 (2006), 31-40.
  * [8] H. Sahleh and A. A. Alijani, Splitting of extensions in the category of locally compact abelian groups, International Journal of Group Theory, Vol. 3, No. 3(2014), 39-45.