# Consecutive Question Generation via Dynamic Multitask Learning

Yunji Li1,2, Sujian Li1, Xing Shi2
1MOE Key Lab of Computational Linguistics, Peking University, China
2ByteDance Lark Search
{liyunji.0529<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
###### Abstract

In this paper, we propose the task of consecutive question generation (CQG),
which generates a set of logically related question-answer pairs to understand
a whole passage, with a comprehensive consideration of the aspects including
accuracy, coverage, and informativeness. To achieve this, we first examine the
four key elements of CQG, i.e., question, answer, rationale111The sentence
based on which a question is generated., and context history222The coverage of
all previous rationales, representing the background information of the
current question series., and propose a novel dynamic multitask framework with
one main task generating a question-answer pair, and four auxiliary tasks
generating other elements. It directly helps the model generate good questions
through both joint training and self-reranking. At the same time, to fully
explore the worth-asking information in a given passage, we make use of the
reranking losses to sample the rationales and search for the best question
series globally. Finally, we measure our strategy by QA data augmentation and
manual evaluation, as well as a novel application of generated question-answer
pairs on DocNLI. We prove that our strategy can improve question generation
significantly and benefit multiple related NLP tasks.

##  1 Introduction

Question Generation (QG) is an important and promising task in natural
language generation (NLG). It has long served as an effective way to improve
other NLP tasks. The applications of synthetic questions have expanded from QA
data augmentation (Duan et al., 2017; Lewis et al., 2021) to building tutoring
or dialogue systems (Lindberg et al., 2013; Bordes and Weston, 2017), self-
assessing the ability of language models (Sun et al., 2019), and checking the
faithfulness of an abstract summary (Durmus et al., 2020), etc.

Today is Jessica’s 80th birthday. Her daughter Mela
---
and Mela’s husband Josh is coming over to the birth-
day party…
Q1: Who is her daughter? A1: Mela.
Q2: Who is Josh? A2: Mela’s husband.
Q3: Who has a birthday party? A3: Mela.
Table 1: Example QG results using a two-step inconsecutive method based on
extractive answers.

Traditionally, syntax-based methods such as semantic parsing are commonly
adopted to synthesize questions (Berant et al., 2013; Khullar et al., 2018).
Recently, transformer-based pre-trained language models (Vaswani et al., 2017;
Devlin et al., 2019) are widely used to generate questions. Most of these
works are two-step QG methods (Sun et al., 2018; Rennie et al., 2020), which
rely on ground-truth or pre-extracted answers (Wang et al., 2019; Jia et al.,
2020) and generate questions independently (Puri et al., 2020; Bartolo et al.,
2021). However, in real scenarios such as daily conversations or reading
comprehension, we usually raise several questions consecutively to understand
a whole story. Current QG methods are inadequate to generate such questions,
as Table 1 shows. We can see that there are no logical connections between the
questions (e.g., Q3 and Q1) and pre-extracted answers also lead to simplicity
(e.g., Q1) and inconsistency (e.g., Q3).

In such cases, we propose the task of consecutive question generation (CQG),
which automatically produces a set of well-ordered and logically related
question-answer (Q-A) pairs to help understand a given passage (or story).
Table 2 shows several “ideal” questions which are mutually connected and cover
diverse information in the text. To achieve this, unlike traditional QG
methods, which mainly focus on “what are good questions given an answer”, our
CQG also requires a model to automatically find “which information in a text
is worth-asking”. Additionally, since we pose questions not only to get
separate information, but to understand a whole story, we propose three key
qualities simultaneously to evaluate consecutive questions, i.e., accuracy,
coverage, and informativeness.

With these demands, we propose an integrated dynamic multitask framework, with
five unified Seq2Seq generation tasks. One main task generates Q-A pairs and
four auxiliary tasks make full use of the generation of four key CQG elements
(i.e., question, answer, rationale, and context history). We link the
qualities of key aspects with the inference losses of four auxiliary tasks
respectively. Based on it, we then design four distinct methods to improve the
model performance from all aspects and from all stages during training and
inference.

The five tasks are jointly trained in one model to help it learn from
different views. In inference, the main task generates candidates and then the
auxiliary tasks self-rerank them, improving Q-A accuracy, coverage, and
informativeness all-roundly. To fully exploit the worth-asking information in
each sentence and generate questions properly and dynamically, we propose a
novel rationale sampling method and sentence-level beam-search. We recompose
the context history reranking losses to measure the information in each
rationale, and then design a sample probability to guarantee that the more
information a rationale leaves, the more likely it is asked once again. To
relieve the error cascade and guide the direction of a Q-A flow, we reinvent
beam-search to sentence-level, which rearranges the total reranking results
and seeks the global optimum Q-A series for a whole passage.

$S$: [Once upon a time in Greece, there lived a young man called
Narcissus.]${}^{stc_{1}}$ [He lived in a small
---
village on the sea and was famous in the land because he was quite
handsome.]${}^{stc_{2}}$ …
$Q_{1}$: What was the name of the young man? $A_{1}$: Narcissus. $R_{1}$:
$stc_{1}$
$Q_{2}$: Where did he live? $A_{2}$: A small village on the sea. $R_{2}$:
$stc_{2}$
$Q_{3}$: Was he famous in the land? $A_{3}$: Yes. $R_{3}$: $stc_{2}$
$Q_{4}$: Why? $A_{4}$: Because he was quite handsome. $R_{4}$: $stc_{2}$
Task Input Output
$a$ $Q_{1}A_{1}\cdots Q_{n-1}A_{n-1}$$\ <sep>\ answer\ this:\ $$Q_{n}$ $<sep>\
$ $S$ $A_{n}$
$q$ $Q_{1}A_{1}\cdots Q_{n-1}A_{n-1}$$\ <sep>\ question\ it:\ $$A_{n}$ $<sep>\
$ $S$ $Q_{n}$
$main$ $Q_{1}A_{1}\cdots Q_{n-1}A_{n-1}$$\ <sep>\ pose\ pair:\ $$R_{n}$ $\
<sep>\ $ $S$ $Q_{n}?A_{n}$
$r$ $Q_{1}A_{1}\cdots Q_{n-1}A_{n-1}$$\ <sep>\ find\ rationale:\ $
$Q_{n}A_{n}$$<sep>\ $ $S$ $R_{n}$
$h$ $Q_{1}A_{1}\cdots$ $Q_{n}A_{n}$$\ <sep>\ generate\ history<sep>$
$\bigcup_{i=1}^{n}R_{i}$
Table 2: An ideal CQG example, where the questions are mutually connected and
can cover diverse information to help understand the whole story. Also an
example of data composition of our multitask generation framework, as well as
the input and output in the $n^{th}$ generation step. In this example, the
output of Task $h$ is $stc_{1}$ when $n=1$, and is $stc_{1}stc_{2}$ when
$n\geq 2$. “$\bigcup$” means coverage, or union set, with no overlap or
replication.

Finally, we conduct abundant experiments to augment various QA datasets, only
using the model trained on CoQA. We also make a manual evaluation and propose
a novel zero-shot method for document-level NLI task (Yin et al., 2021) using
question generation. Successfully, we promote the performance on multiple QA
scenes and prove the expansibility of our model on different NLP tasks.

##  2 Related Work

Question generation is a promising task which has been well studied in many
researches. Initially, rule-based or traditional machine learning methods are
widely used in producing questions. Heilman and Smith (2010) adopt verb
transformations and Berant et al. (2013) use semantic parsing to synthesize
questions. Recently, deep learning techniques have given a further development
of question generation. Du et al. (2017) use an LSTM (Hochreiter and
Schmidhuber, 1997) model, and Sultan et al. (2020) adopt RoBERTa (Liu et al.,
2019) model to generate questions.

At the same time, the strategies like multitask learning and self-training
have been applied to improve the quality of generated questions. Zhou et al.
(2019) and Ma et al. (2020) employ a multitask structure to generate coherent
and fluent questions. Sachan and Xing (2018) and Rennie et al. (2020) adopt a
self-training strategy to jointly learn to ask and answer questions. Alberti
et al. (2019) use roundtrip consistency to filter out inconsistent results.
Shinoda et al. (2021) generate noisy data and Sultan et al. (2020) employ
nucleus sampling (Holtzman et al., 2020) to improve the diversity of
questions. However, they mainly focus on only one quality aspect and most of
them are based on pre-defined answers or original data.

As QG can produce meaningful questions, it has been widely used to promote
other NLP tasks. Liu et al. (2020) use a constrained question rewriting way to
generate new data for QA tasks. Wang et al. (2020) and Nan et al. (2021) check
the faithfulness of summaries through answering generated questions. Pan et
al. (2021) generate question-answer pairs and convert them for fact
verification. Nevertheless, the researches above mainly produce each question
independently and ignore the connections between questions.

As for generating a set of questions over a specific passage, Krishna and
Iyyer (2019) propose a pipelined system to ask different levels of questions
from general to specific. Lee et al. (2020) use conditional variational
autoencoder to generate multiple robust questions for a given paragraph.
Similar to us, Chai and Wan (2020) generate sequential and related questions
under dual-graph interaction, but use ground-truth answers. To the best of our
knowledge, we are the first to consecutively synthesize a series of connected
question-answer pairs to understand an entire passage, with the comprehensive
consideration of accuracy, coverage, and informativeness.

##  3 Multitask Framework

In our CQG strategy, the foundation is five various but unified tasks. The
effects of these tasks are dynamically spread throughout our whole strategy.
In section 4 we use them to compose four related methods to enhance different
stages.

We first symbolically define the four key elements used in our work. $S$
denotes the story from which questions are produced; $Q_{n}$ means the
$n^{th}$ question and $A_{n}$ is the answer; $R_{n}$ is the corresponding
rationale (always one sentence) based on which $Q_{n}$ is generated. Since the
Q-A pairs are generated dependently on previous questions, $C_{n}$ denotes the
context which composes of previous $n-1$ Q-A pairs and the story.333Please be
aware that story is the text content, and context is story plus previous $n-1$
Q-A pairs. Table 2 is an example. Then we define the main task and the four
auxiliary tasks using the $n^{th}$ turn as follows:
Task $main$: $C_{n}+R_{n}\rightarrow Q_{n}+A_{n}$
Task $a$: $C_{n}+Q_{n}\rightarrow A_{n}$
Task $q$: $C_{n}+A_{n}\rightarrow Q_{n}$
Task $r$: $C_{n}+Q_{n}+A_{n}\rightarrow R_{n}$
Task $h$: $\sum_{i=1}^{n}(Q_{i}+A_{i})\rightarrow\bigcup_{i=1}^{n}R_{i}$

In Task $main$, because we think the extractive answer is usually simple and
it is inconsistent to get a Q-A in two steps, different from traditional
methods, we input the context and rationale and output the question and answer
simultaneously.

The design of Task $a$ and Task $q$ aims to guarantee that the generated
question and answer are accurate: given the question we can get the answer and
given the answer we can get the question. Here Task $a$ follows traditional QA
form. We do not input the rationale in Task $q$ because previous Q-A pairs are
included in the context, so if $A_{n}$ is an accurate answer, the model should
recognize the connection between the answer and the previous Q-A pairs, and
restore the question easily.

Moreover, although we input the rationale in Task $main$, it does not
necessarily imply that the question-answer pair is derived from it. So we
design Task $r$ ($C_{n}+Q_{n}+A_{n}\rightarrow R_{n}$) to verify that the
model indeed uses the information in input rationale to get the question and
answer. Task $r$ helps the model to recognize the corresponding rationale, and
then increase the coverage of a Q-A series, which means more events or more
segments are precisely referred to.

Finally, to generate an informative and useful question, which means the
knowledge it asks for does not overlap with previous ones, we consider that
the more unseen information included in the Q-A pair, the better. We introduce
the history of the context as the coverage of all previous rationales, which
represents the total background information till the current Q-A turn.
Therewith, we present Task $h$:
$\sum_{i=1}^{n}(Q_{i}+A_{i})\rightarrow\bigcup_{i=1}^{n}R_{i}$, which uses Q-A
pairs to restore the history. “$\bigcup$” means cover, with no overlap or
replication, and “+” means append or plus.

Both Task $r$ and Task $h$ use Q-A pairs to restore the context, but focus on
coverage and informativeness differently. Specifically, a part of a story is
covered means a question is asked based on it, but a informative question
means it is non-trivial and important and contains no repetitive information.
Also, in Task $r$ we input the context, so the model only needs to locate the
correct rationale, but in Task $h$, it has to generate the history completely
based on Q-A pairs. Therefore in Task $h$, if the $n^{th}$ Q-A pair carries
more unseen information, it will be easier to restore the history compared
with a Q-A pair with repetitive or trivial information.

$BART$training$C_{n}$$Q_{n}$$C_{n}$$A_{n}$$C_{n}$$R_{n}$$Q_{n}$$C_{n}$$A_{n}$$Q_{i}$$A_{i}$$\sum_{i=1}^{n}($$)$$A_{n}$$Q_{n}$$Q_{n}$$A_{n}$$R_{n}$$\bigcup_{i=1}^{n}$$R_{i}$reranking$BART$$C_{n}$$R_{n}$$generate$$Q_{n1}$$A_{n1}$$Q_{n2}$$A_{n2}$$Q_{n3}$$A_{n3}$$Q_{n4}$$A_{n4}$$BART$$C_{n}$$Q_{nj}$$C_{n}$$A_{nj}$$Q_{nj}$$C_{n}$$A_{nj}$$Q_{nj}$$A_{nj}$$Q_{i}$$A_{i}$$\sum_{i=1}^{n-1}($$)$$+$$A_{nj}$$Q_{nj}$$R_{n}$$\bigcup_{i=1}^{n}$$R_{i}$$use\;total\;loss\;to\;rerank$
Figure 1: An overview of our dynamic multitask framework during joint training
and self-reranking. One main task generates Q-A pairs and four auxiliary tasks
generate other four CQG elements. In training, the five tasks are jointly
trained in one model. In inference, the model uses the main task to generate
candidates and then uses the auxiliary tasks to self-rerank them. We use the
$n^{th}$ turn of a series of questions as an example and generate 4 candidates
in inference. $j\in\\{1,2,3,4\\}$.

##  4 Training and Inference

Based on the dynamic multitask framework, we jointly train a BART (Lewis et
al., 2020) model. In inference, we use the main task to generate several
candidates and self-rerank them using the auxiliary tasks. With the reranking
losses, we design a formula to assess the information and automatically sample
the rationales. Globally, we beam-search for the best Q-A series on sentence
level.

###  4.1 Joint Training

We randomly shuffle the five kinds of training instances and use a BART model
to jointly train the five tasks together. We also train the model to generate
a “?” between a Q-A to split it, and adopt five hand-made prompts (Liu et al.,
2021). Table 2 shows an example of our data structure. Given the Seq2Seq model
parameterized by $\theta$, the input sequence $\bm{x}$ with $n$ tokens =
$\\{x_{1},\cdots,x_{n}\\}$ and label $\bm{y}$ with $m$ tokens =
$\\{y_{1},\cdots,y_{m}\\}$, the generation probability and loss are as
follows:

$p(\bm{y}|\bm{x},\theta)=\prod_{z=1}^{m}p(\bm{y}_{z}|\bm{y}_{<z},\bm{x},\theta)$
(1)
$loss(\bm{y}|\bm{x},\theta)=-\frac{1}{m}\sum_{z=1}^{m}log\,p(\bm{y}_{z}|\bm{y}_{<z},\bm{x},\theta)$
(2)

Through joint training we train a model to learn from different views and
allow every task to benefit each other mutually. We also acquire the ability
to do all five tasks in one model.

###  4.2 Self-Reranking

During the inference stage, through the main task we can obtain many candidate
question-answer pairs using a decoding strategy like nucleus sampling. To
select the best result, inspired by Shen et al. (2021), we employ these
candidates to the same model to do Task $a$,$q$,$r$, and $h$, and then rank
the candidates using the inference losses of the four auxiliary tasks. In
another word, we use one model as both the generator and ranker. During
reranking, the corresponding question and answer of the auxiliary tasks are
those generated from Task $main$. Specifically, we multiply the four losses
together as the reranking loss, as Eq.3, where the subscript $i$ refers to
different tasks. We also design other loss aggregation methods to calculate
the reranking losses, as in Appendix B.3, which shows that using $\prod$ or
$\sum$ are the same in nature.

$loss_{rank}(\bm{y}|\bm{x},\theta)=\prod_{i\in\\{a,q,r,h\\}}loss(\bm{y}_{i}|\bm{x}_{i},\theta)$
(3)

We consider the candidate with the lowest reranking loss as the one who excels
in accuracy, coverage, and informativeness generally. This is inspired by the
idea of evaluating generated text as text generation (Yuan et al., 2021).
Through this strategy we also unify the form of training and reranking process
and manage to do them in the same model. Figure 1 shows the structure of our
multitask joint training and self-reranking.

###  4.3 Rationale Sampling

The aforementioned methods are useful to generate one good Q-A pair. Still,
how to effectively generate consecutive questions on a passage remains
unsettled. By default, we select every rationale as the next sentence of
previous one. However, one rationale does not necessarily correspond to only
one question, because a long informative sentence may be suitable for several
Q-A pairs.

$sentence_{t}$$sentence_{t+1}$$sentence_{t-1}$$\cdots$$QA_{n}$$QA_{n-1}$$QA_{n-2}$$QA_{n+1}$$\cdots$$kp$$1-kp$
Figure 2: An example of rationale sampling, in which there is a probability of
$kp$ that $R_{n+1}$ is $sentence_{t}$, and $1-kp$ it is $sentence_{t+1}$.
Specifically in this example, $n^{\prime}$ is $n-2$, $R_{n^{\prime}}$ is
$sentence_{t-1}$, and $m_{n^{\prime}}$ is the length of
$\sum_{i=1}^{t-1}sentence_{i}.$

Hence, we propose the rationale sampling strategy, which introduces a
probability that the next rationale keeps the same sentence as the current
one, as Figure 2 shows. We use $kp$ as the keeping probability. Then
intuitively, we let $kp$ be linearly related to the amount of information left
in the current rationale. Traditionally, the information is hard to be
calculated quantitatively. However, recall that we use the loss of Task $h$ to
measure the information of a Q-A series, so similarly, we design a inference
loss to represent the rest information in current rationale. We want a higher
loss to mean that less information of $R_{n}$ is included in the Q-A series,
and more information is still left in $R_{n}$.

Naturally, we first separate out the Q-A pairs on $R_{n}$. Given current step
$n$, we find $n^{\prime}$, which is the most recent step where
$R_{n^{\prime}}\neq R_{n}$. Then, we use

$\begin{split}&loss(R_{n}|\sum_{i=n^{\prime}+1}^{n}(Q_{i}+A_{i})+\bigcup_{i=1}^{n^{\prime}}R_{i},\theta)\\\
&\approx\frac{m_{n}loss_{h_{n}}-m_{n^{\prime}}loss_{h_{n^{\prime}}}}{m_{n}-m_{n^{\prime}}}\triangleq
a\end{split}$

to represent the rest information in $R_{n}$444$m_{n}$ =
len($\bigcup_{i=1}^{n}R_{i}$). The details are in Appendix B.4, which is the
loss of using previous sentences and the Q-A pairs on $R_{n}$ to restore
$R_{n}$. Given our multitask framework, we use the ready-calculated losses of
Task $h$ to approximate this loss, without introducing more computation and
complexity.

The approximation is $a$. Particularly if $n$ is 1, $a$ is $loss_{h_{1}}$.
Empirically, we set the slope to be 0.2 and set a bound of 0-0.75. Finally, we
get Eq.(4), and the average $kp$ is 0.32 in the experiments, resulting in
about 1.3 questions from one sentence.

$kp=\left\\{\begin{aligned} 0,\quad&\ a\leq 0\\\ 0.2a,\quad&\ 0<a<3.75\\\
0.75,\quad&\ a\geq 3.75\\\ \end{aligned}\right.$ (4)

Besides, we also design other two rationale sampling strategies as in Appendix
B.5, which shows that our strategy which bases on Task $h$ to calculate
information performs better than other hand-made probability formulas.

###  4.4 Sentence-Level Beam-Search

Although rationale sampling helps catch more information and improves
flexibility, it brings about more uncertainty. The mutually dependent
generation may also lead to deviation (Li et al., 2021). Thus, it is crucial
to guide the flow direction in every step and ensure the quality of the whole
series.

Naturally, inspired by traditional beam-search (token-level), we propose the
sentence-level beam-search, as Figure 3 shows. Different from traditional
beam-search, which generates a token in each search step, we generate a QA
pair, and we adopt the reranking loss of each QA pair to take the place of the
generation probability. Thus, in each step, we maintain several candidates
with the lowest product of all previous reranking losses, which is calculated
as Eq.5, where $L$ is the final loss of our sentence-level beam-search method.

$\displaystyle L(Q_{1}A_{1}\cdots
Q_{n}A_{n}|\bm{x},\theta)=\prod_{j=1}^{n}{loss_{rank}}_{{}_{j}}$ (5)

To summarize, 4.2 to 4.4 are for inference. Practically, in each generation
step, we first use previous results to do rationale sampling to locate the
rationale, then generate some candidates and calculate the current reranking
losses, and finally we use the total losses to sentence-level beam-search and
keep several Q-A flows for the next step.

$S$$QA_{00}$$QA_{01}$$QA_{02}$$QA_{03}$$QA_{10}$$QA_{11}$$QA_{12}$$QA_{13}$$QA_{14}$$QA_{15}$$QA_{16}$$QA_{17}$$QA_{20}$$QA_{21}$$QA_{22}$$QA_{23}$$QA_{24}$$QA_{25}$$QA_{26}$$QA_{27}$$QA_{30}$$QA_{31}$$QA_{32}$$QA_{33}$$QA_{34}$$QA_{35}$$QA_{36}$$QA_{37}$
Figure 3: An overview of the sentence-level beam-search strategy. In this
example each step the model generates 4 question-answer candidates and the
sentence-level beam size is 2.

##  5 Experiments

###  5.1 Experimental Setup

We employ CoQA (Reddy et al., 2019) training set as our training data. CoQA is
a large-scale dataset for building Conversational Question Answering systems.
The questions are conversational, and thus, every question after the first is
dependent on the conversation history. The answers are free-form text with
their corresponding rationales in the story. We expand the rationales to whole
sentences and remove the questions with unknown answers. Finally, we get 7199
stories and each story has 15 turns of Q-A pairs on average. The training
details and experiments are in Appendix A, where we also analyze the effect of
joint training.

After training a model $\theta$ on CoQA, we evaluate our model by applying its
question generation ability to two downstream tasks: data augmentation for QA
and document-level NLI. Further, under the synthetic results on CoQA, we
analyze their accuracy, coverage, and informativeness using human evaluations
and a repeat-pose experiment.

###  5.2 Experiments to Augment QA Data

Data augmentation is one common way to employ generated questions and verify
QG models. To augment QA dataset $D$, we (1) use $\theta$ to synthesize Q-A
pairs $D^{\prime}$ on the training set of $D$; (2) train another BART model
$\theta^{\prime}$ on $D^{\prime}$ or $D+D^{\prime}$ to answer
questions555Since our synthetic Q-A pairs are free-form, we still use BART to
generate the answers on both CoQA and SQuAD.; (3) test $\theta^{\prime}$ on
the dev set of $D$.

#### Results on CoQA

First we test our strategy to augment CoQA dataset. The setting $Origin$ means
the model $\theta^{\prime}$ is trained on the original CoQA training set, and
$Synth$ means it is trained with synthetic Q-A pairs. Inspired by Yuan et al.
(2021), we additionally use the inference losses to measure the performance

In $Synth$, we conduct single q, two step, and single m as three baseline
models, where single q means we use a single Task $q$ model to ask questions
based on the origin answers, like the traditional QG methods. Two step means
we first extract an answer666Use a BERT model to locate the start and end
tokens., then generate a question on it using the single Task $q$ model.
Single m is a Task $main$ model, which generates Q-A pairs.

Joint train is a multitask jointly trained model. Based on joint train model,
we further add the self-reranking method, using all four auxiliary tasks. Then
on this joint train + rerank model, we conduct four ablation studies of
auxiliary tasks.

Under joint train + rerank model, we also introduce other two conditions,
independent and relay. By default, we generate the question series in an
automatic way, which means every step the previous Q-A pairs are the Q-A pairs
generated in previous steps. In independent condition, we let previous Q-A
pairs be empty in all steps, which means the model generates every question
like the first question, but when training QA model $\theta^{\prime}$, we
still input the previous QA pairs to align the data format with CoQA. In
relay, the previous Q-A pairs of every synthetic instance are from CoQA
training set, and the rationale is the ground-truth rationale sentence, which
means the model inherits the Q-A flow from authentic CoQA’s context.

Finally, still under joint train + rerank model, we add rationale sampling and
sentence-level beam-search. Additionally, we merge the original training set
with synthetic data to create the merging setting ($D+D^{\prime}$). Note that
RS and SBS are not suitable for independent or relay condition.

CoQA | Bleu | Infer Loss | F1qa
---|---|---|---
$Origin$ |  |  |
Bart | 38.52 | 0.777 | 78.54
$Synth$ |  |  |
Single q | 35.43/37.85 | 5.429/0.869 | 70.82/78.35
Two step | 15.41/39.92 | 5.078/0.817 | 56.00/77.85
Single m | 27.04/41.42 | 5.538/0.776 | 65.66/79.20
Joint train | 26.97/38.92 | 5.613/0.765 | 65.90/80.11
+rerank | 24.88/38.26 | 5.674/0.768 | 65.05/80.52
+RS | 31.73/46.24 | 5.323/0.758 | 72.33/81.83
+SBS | 32.01/47.86 | 5.431/0.766 | 72.49/81.98
Table 3: Results on CoQA dev set. In $Synth$, results without and with merging are separated by “/”. In the middle are four ablation experiments of auxiliary tasks with Bart joint train+rerank. RS: rationale sampling. SBS: sentence-level beam-search. CoQA | Bleu | Infer Loss | F1qa
---|---|---|---
Joint train | 26.97/38.92 | 5.613/0.765 | 65.90/80.11
+rerank a | 25.31/38.03 | 5.612/0.764 | 63.71/80.23
+rerank q | 24.66/37.83 | 5.401/0.773 | 64.44/80.29
+rerank r | 24.03/38.05 | 5.487/0.768 | 63.73/80.18
+rerank h | 23.10/37.32 | 5.499/0.789 | 63.01/80.27
+rerank all | 24.88/38.26 | 5.674/0.768 | 65.05/80.52
Table 4: Results of ablation studies of four auxiliary tasks, on CoQA dev set. CoQA | Bleu | Infer Loss | F1qa
---|---|---|---
Joint train + rerank | 24.88/38.26 | 5.674/0.768 | 65.05/80.52
indep | 20.38/39.03 | 5.490/0.783 | 56.54/78.29
relay | 35.11/45.24 | 5.477/0.781 | 75.90/81.79
+RS+SBS | 32.01/47.86 | 5.431/0.766 | 72.49/81.98
Table 5: Results of different conditions on CoQA dev set. “+RS+SBS” means
Joint train+rerank+RS+SBS.

Table 3 shows the main results. Table 4 and Table 5 are the results of
ablation studies and different conditions. The single q and two step model
make relatively low scores when merged with original data, which means they
generate relatively simple and low-quality questions. Using our one step Q-A
pairs generation, in merging setting the single m model leads to higher scores
even than single q, which based on origin answers. Joint train and reranking
further improve the F1qa scores by 1.32 points. From the four ablation studies
in Table 4, it is not hard to see that every auxiliary task filters the
results effectively, leading to 0.07 to 0.18 higher F1qa scores.

As for our consecutive generation strategy, in Table 5, comparing the
independent condition with our model, we can see that the consecutive
generation largely improves the quality of questions by 2.23 F1qa scores.
Moreover, although the relay model based on the original Q-A flow truly gets
better performance, when we add RS and SBS strategy to get our best model, the
F1qa score is further increased by 1.46 points, and finally it outperforms
relay generation by 0.19 points. It shows that the Q-A series searched by RS
and SBS are more proper even than the ground-truth flow.

#### Results on SQuAD and more data

To check our QG ability on out-of-domain passages, we augment SQuAD (Rajpurkar
et al., 2018) dataset using our best model trained on CoQA. We select the
instances without unknown answers and with a story longer than 128 words.
Since the questions in SQuAD are independent but also well-organized, we
manually add previous Q-A pairs to align with CoQA.

To truly reveal the ability of our model, we employ it to synthesize more
questions on a large number of unlabeled passages. We randomly collect 10000
Wikipedia passages whose lengths are from 100 to 500 words. Then we use our
model trained on CoQA to generate questions on them, resulting in about 0.15
million Q-A pairs, which we use to augment both CoQA and SQuAD.

SQuAD | Bleu | Infer Loss | F1qa
---|---|---|---
$Origin$ |  |  |
Bart | 65.52 | 0.675 | 84.26
+preQA | 68.67 | 0.625 | 85.32
$Synth$ |  |  |
Ours | 41.91/67.43 | 4.639/0.691 | 67.57/85.59
+Wiki | 50.58/65.39 | 4.010/0.630 | 74.90/85.88
CoQA |  |  |
Ours | 32.01/47.86 | 5.431/0.766 | 72.49/81.98
+Wiki | 33.01/47.43 | 5.441/0.758 | 72.58/82.21
Large | 52.36 | 0.521 | 87.90
Table 6: Results of out-of-domain generation on SQuAD dev set, and on
Wikipedia passages. “Ours” means Joint train+rerank+RS+SBS. “Large” means both
the QG model and QA model are Bart Large, and the synthesized data for it is
from CoQA and Wiki under the Joint train+rerank+RS+SBS setting. In $Synth$,
results without and with merging are separated by “/”.

Table 6 shows the results. We can see that the Q-A series indeed enhances
question answering. It also indicates that even if our model is trained on
different dataset, its synthesized questions still help a QA model gain 0.27
more F1qa points on SQuAD. With more Wikipedia questions, in both CoQA and
SQuAD, we manage to further improve F1qa by 0.29 and 0.23 scores. It shows
that our model performs well when transferring to another dataset and can
augment the QA training sets with large-scale unlabeled data. Finally we adopt
$large$ model to get 87.90 F1qa points on CoQA.

###  5.3 Understand a Whole Passage (DocNLI)

To prove that our generated questions can really explore most information in
an entire passage, we adopt our model for document-level NLI (DocNLI) task.
Models are required to predict the relation (entailment or not) between a
document-level premise and a hypothesis.

Traditionally, a model predicts the relation in a sequence classification way.
However, given our ability to synthesize consecutive questions to understand a
passage, we propose a zero-shot method to predict the relation based on
question generating and answering. Since entailment requires the hypothesis to
be derived from the premise, we first generate Q-A pairs given the hypothesis,
and then answer these questions based on the premise. If we can get the same
answers, we predict entailment. In detail, we (1) use $\theta$ to synthesize a
series of Q-A pairs on the hypothesis; (2) use $\theta$ to answer $Q$ on the
premise, obtaining $A^{\prime}$; (3) check the overlap (F1qa) between $A$ and
$A^{\prime}$. If the F1qa exceeds a given threshold, it is entailment.

To make sure that the passages are long enough to generate a series of Q-A
pairs, we select the instances whose premise and hypothesis are 200 to 1000
words from all train, dev, and test set of DocNLI, to be our evaluation set.
It is 1677 instances in all, and we averagely generate 15 turns of Q-A each
instance with rationale sampling. We use 60 points of F1qa as the threshold of
entailment.

DocNLI | Infer Loss | F1qa | F1nli
---|---|---|---
$Finetune$ |  |  |
Bert | - | - | 48.56
$QG$ |  |  |
Two step | 1.142/2.020 | 65.69/51.54 | 47.67
Single m | 3.376/4.273 | 61.00/47.73 | 46.85
Joint train | 3.223/4.119 | 63.32/49.56 | 46.90
+rerank | 3.217/4.149 | 63.04/49.68 | 47.91
indep | 2.811/3.857 | 63.90/49.18 | 47.88
+RS | 2.633/3.601 | 65.98/50.99 | 49.88
+SBS | 2.376/3.353 | 66.19/51.19 | 49.98
Table 7: Results of DocNLI task. Finetune is a BERT-base model fine-tuned on
about 0.8 million other DocNLI instances. When using our zero-shot method, QA
results of entailment and not entailment are separated by “/”. We use
different models for QG, and the QA model is the same as our best model
$\theta$.

Tabel 7 shows the results. F1nli is the harmonic mean of the precision and
recall on the classification task. Impressively, using the zero-shot method,
our best model surpasses the fine-tuned BERT model by 1.42 points of F1nli
score. Among different QG settings, although two step model gets very low
losses, its F1nli score is not very high, indicating that it generates
relatively simple questions which cannot extract much information. Our one
step model gets a lower F1nli score initially but with the joint training and
reranking strategy, it improves the score by 0.98 points. Moreover, we can see
clearly that the RS and SBS strategies improve the result significantly by
2.10 F1nli scores. They also manage to enlarge the discrimination between
entailment and not entailment. It suggests that our consecutive generation
strategy really produces question-answer pairs with most of the information in
a passage, which can help understand the passage effectively.

###  5.4 Analyses

#### Accuracy and Coverage (Task $a$, $q$ and $r$)

Here we conduct two human evaluations, to prove that our strategy improves Q-A
accuracy and story coverage, which are the effects of Task $a$, $q$ and Task
$r$. Since the coverage requires the model to ask for more points of a
passage, we use the question-rationale consistency (accuracy of rationale) to
reflect it. This is because all sentences are asked at least once, and
rationale sampling further guarantees the rationales to be well-distributed,
so if the rationales are all precisely questioned, the coverage should be as
well satisfactory.

We randomly collect 10% stories from CoQA dev set and use different methods to
generate Q-A pairs. We, the authors, then manually measure whether every
question is correctly asked and answered and whether every question-answer
pair is derived from its corresponding rationale.

Acc of | Ours | -SBS | -Rerank | -Joint train
---|---|---|---|---
Q-A pair | 94.85 | 92.71 | 90.32 | 88.33
rationale | 95.65 | 93.89 | 90.97 | 90.26
Table 8: Human evaluations of accuracy of Q-A and rationale. We do not ablate
RS here because it is not relevant here and will make the data unaligned.

Table 8 clearly shows that multitask joint training and reranking and
sentence-level beam-search increase the accuracy of Q-A by 6.52 % and
rationale by 5.39 %. Thus, we can say that our strategy, main due to Task $a$,
$q$ and Task $r$, helps generate questions more correctly and locate the
rationale more precisely, leading to higher Q-A accuracy and coverage in a
series of questions.

#### Informativeness (Task $h$)

To evaluate the ability to utilize information in a rationale, we present the
repeat-pose experiment on CoQA. It is adapted from relay condition, and
requires the model to pose another question based on the same rationale and
same context as the original question. In other words, the model has to
“squeeze” more information from the same rationale, so the key is whether Task
$h$ can rank the informativeness of each candidate precisely.

CoQA | Bleu | Infer Loss | F1qa
---|---|---|---
Joint train relay w/o rerank | 41.01 | 0.737 | 81.21
Joint train repeat w/o rerank | 41.97 | 0.741 | 81.28
Joint train repeat w/ rerank | 43.40 | 0.708 | 81.57
Table 9: Results of the repeat-pose experiment. Synthetic data are merged with
the original training set.

Table 9 shows the results, which demonstrate that repeat-pose with self-
reranking strategy further improves the F1qa scores by 0.36 points, indicating
that Task $h$ indeed helps select the more informative question-answer pairs.

##  6 Conclusion

In this paper, we propose the consecutive question generation task, which
synthesizes mutually connected question-answer pairs to fully explore the
information in a passage. By constructing a novel multitask framework with one
main task and four unified auxiliary tasks, we generate optimum Q-A series
using four sub-methods, which help “generate good questions” as well as “find
worth-asking information”. With extensive experiments, we prove that our model
is able to generate high-quality Q-A pairs to understand a whole passage and
has the power to benefit various NLP tasks.

## Limitations

In this paper, we propose a novel question generation strategy which can
benefit multiple NLP scenes. For this work, we summarize two limitations as
follows. First, CQG has high requirements for the training data. In this work,
we adopt the CoQA corpus which is originally developed for the conversational
QA task. To the best of our knowledge, CoQA is the only existing dataset which
is suitable for our task. Without more datasets for evaluation, we try to
improve the performance on SQuAD and DocNLI to a certain degree by generating
questions zero-shot or generating questions on large-scale Wikipedia passages.
In future, we hope to build a CQG specific corpus and draw more attention to
this novel task.

Second, the time cost of our strategy is higher than others’, because we need
to train five tasks jointly and rerank on four auxiliary tasks during
inference. Specifically, it is about three times more in training and four
times more in inference. Detailed analysis is in Appendix B.2. In our future
work, we will focus on the simplification of our strategy and the distillation
of our model. Also, we will examine if a small model or a base model with
fewer training data can get the same performance as other common models when
using our strategy.

## Acknowledgement

We thank the anonymous reviewers for their helpful comments on this paper.
This work was partially supported by National Key Research and Development
Project (2019YFB1704002) and National Natural Science Foundation of China
(61876009). The corresponding author of this paper is Sujian Li.

## References

  * Alberti et al. (2019) Chris Alberti, Daniel Andor, Emily Pitler, Jacob Devlin, and Michael Collins. 2019\.  Synthetic QA corpora generation with roundtrip consistency.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 6168–6173, Florence, Italy. Association for Computational Linguistics.
  * Bartolo et al. (2021) Max Bartolo, Tristan Thrush, Robin Jia, Sebastian Riedel, Pontus Stenetorp, and Douwe Kiela. 2021.  Improving question answering model robustness with synthetic adversarial data generation.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 8830–8848, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Berant et al. (2013) Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013.  Semantic parsing on Freebase from question-answer pairs.  In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing_ , pages 1533–1544, Seattle, Washington, USA. Association for Computational Linguistics.
  * Bordes and Weston (2017) Antoine Bordes and Jason Weston. 2017.  Learning end-to-end goal-oriented dialog.  _ArXiv_ , abs/1605.07683.
  * Chai and Wan (2020) Zi Chai and Xiaojun Wan. 2020.  Learning to ask more: Semi-autoregressive sequential question generation under dual-graph interaction.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 225–237.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Du et al. (2017) Xinya Du, Junru Shao, and Claire Cardie. 2017.  Learning to ask: Neural question generation for reading comprehension.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1342–1352, Vancouver, Canada. Association for Computational Linguistics.
  * Duan et al. (2017) Nan Duan, Duyu Tang, Peng Chen, and Ming Zhou. 2017.  Question generation for question answering.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 866–874, Copenhagen, Denmark. Association for Computational Linguistics.
  * Durmus et al. (2020) Esin Durmus, He He, and Mona Diab. 2020.  FEQA: A question answering evaluation framework for faithfulness assessment in abstractive summarization.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5055–5070, Online. Association for Computational Linguistics.
  * Heilman and Smith (2010) Michael Heilman and Noah A. Smith. 2010.  Good question! statistical ranking for question generation.  In _Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics_ , pages 609–617, Los Angeles, California. Association for Computational Linguistics.
  * Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber. 1997.  Long short-term memory.  _Neural Computation_ , 9:1735–1780.
  * Holtzman et al. (2020) Ari Holtzman, Jan Buys, Maxwell Forbes, and Yejin Choi. 2020.  The curious case of neural text degeneration.  _ArXiv_ , abs/1904.09751.
  * Jia et al. (2020) Xin Jia, Wenjie Zhou, Xu Sun, and Yunfang Wu. 2020.  How to ask good questions? try to leverage paraphrases.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 6130–6140, Online. Association for Computational Linguistics.
  * Khullar et al. (2018) Payal Khullar, Konigari Rachna, Mukul Hase, and Manish Shrivastava. 2018.  Automatic question generation using relative pronouns and adverbs.  In _Proceedings of ACL 2018, Student Research Workshop_ , pages 153–158, Melbourne, Australia. Association for Computational Linguistics.
  * Krishna and Iyyer (2019) Kalpesh Krishna and Mohit Iyyer. 2019.  Generating question-answer hierarchies.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 2321–2334, Florence, Italy. Association for Computational Linguistics.
  * Lee et al. (2020) Dong Bok Lee, Seanie Lee, Woo Tae Jeong, Donghwan Kim, and Sung Ju Hwang. 2020.  Generating diverse and consistent QA pairs from contexts with information-maximizing hierarchical conditional VAEs.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 208–224, Online. Association for Computational Linguistics.
  * Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020.  BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7871–7880, Online. Association for Computational Linguistics.
  * Lewis et al. (2021) Patrick Lewis, Yuxiang Wu, Linqing Liu, Pasquale Minervini, Heinrich Kuttler, Aleksandra Piktus, Pontus Stenetorp, and Sebastian Riedel. 2021.  Paq: 65 million probably-asked questions and what you can do with them.  _Transactions of the Association for Computational Linguistics_ , 9:1098–1115.
  * Li et al. (2021) Huihan Li, Tianyu Gao, Manan Goenka, and Danqi Chen. 2021.  Ditch the gold standard: Re-evaluating conversational question answering.  _ArXiv_ , abs/2112.08812.
  * Lindberg et al. (2013) David Lindberg, Fred Popowich, John Nesbit, and Phil Winne. 2013.  Generating natural language questions to support learning on-line.  In _Proceedings of the 14th European Workshop on Natural Language Generation_ , pages 105–114, Sofia, Bulgaria. Association for Computational Linguistics.
  * Liu et al. (2020) Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Jiancheng Lv, Nan Duan, and Ming Zhou. 2020.  Tell me how to ask again: Question data augmentation with controllable rewriting in continuous space.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 5798–5810, Online. Association for Computational Linguistics.
  * Liu et al. (2021) Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2021.  Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing.  _ArXiv_ , abs/2107.13586.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _ArXiv_ , abs/1907.11692.
  * Ma et al. (2020) Xiyao Ma, Qile Zhu, Yanlin Zhou, Xiaolin Li, and Dapeng Oliver Wu. 2020.  Improving question generation with sentence-level semantic matching and answer position inferring.  _ArXiv_ , abs/1912.00879.
  * Nan et al. (2021) Feng Nan, Cicero Nogueira dos Santos, Henghui Zhu, Patrick Ng, Kathleen McKeown, Ramesh Nallapati, Dejiao Zhang, Zhiguo Wang, Andrew O. Arnold, and Bing Xiang. 2021.  Improving factual consistency of abstractive summarization via question answering.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 6881–6894, Online. Association for Computational Linguistics.
  * Pan et al. (2021) Liangming Pan, Wenhu Chen, Wenhan Xiong, Min-Yen Kan, and William Yang Wang. 2021\.  Zero-shot fact verification by claim generation.  In _ACL/IJCNLP_.
  * Puri et al. (2020) Raul Puri, Ryan Spring, Mohammad Shoeybi, Mostofa Patwary, and Bryan Catanzaro. 2020\.  Training question answering models from synthetic data.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 5811–5826, Online. Association for Computational Linguistics.
  * Rajpurkar et al. (2018) Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018.  Know what you don’t know: Unanswerable questions for SQuAD.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 784–789, Melbourne, Australia. Association for Computational Linguistics.
  * Reddy et al. (2019) Siva Reddy, Danqi Chen, and Christopher D. Manning. 2019.  CoQA: A conversational question answering challenge.  _Transactions of the Association for Computational Linguistics_ , 7:249–266.
  * Rennie et al. (2020) Steven Rennie, Etienne Marcheret, Neil Mallinar, David Nahamoo, and Vaibhava Goel. 2020.  Unsupervised adaptation of question answering systems via generative self-training.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1148–1157, Online. Association for Computational Linguistics.
  * Sachan and Xing (2018) Mrinmaya Sachan and Eric Xing. 2018.  Self-training for jointly learning to ask and answer questions.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 629–640, New Orleans, Louisiana. Association for Computational Linguistics.
  * Shen et al. (2021) Jianhao Shen, Yichun Yin, Lin Li, Lifeng Shang, Xin Jiang, Ming Zhang, and Qun Liu. 2021.  Generate & rank: A multi-task framework for math word problems.  In _Findings of the Association for Computational Linguistics: EMNLP 2021_ , pages 2269–2279, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Shinoda et al. (2021) Kazutoshi Shinoda, Saku Sugawara, and Akiko Aizawa. 2021.  Improving the robustness of QA models to challenge sets with variational question-answer pair generation.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: Student Research Workshop_ , pages 197–214, Online. Association for Computational Linguistics.
  * Sultan et al. (2020) Md Arafat Sultan, Shubham Chandel, Ramón Fernandez Astudillo, and Vittorio Castelli. 2020.  On the importance of diversity in question generation for QA.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5651–5656, Online. Association for Computational Linguistics.
  * Sun et al. (2019) Kai Sun, Dian Yu, Dong Yu, and Claire Cardie. 2019.  Improving machine reading comprehension with general reading strategies.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 2633–2643, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Sun et al. (2018) Xingwu Sun, Jing Liu, Yajuan Lyu, Wei He, Yanjun Ma, and Shi Wang. 2018.  Answer-focused and position-aware neural question generation.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 3930–3939, Brussels, Belgium. Association for Computational Linguistics.
  * Vaswani et al. (2017) Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _ArXiv_ , abs/1706.03762.
  * Wang et al. (2020) Alex Wang, Kyunghyun Cho, and Mike Lewis. 2020.  Asking and answering questions to evaluate the factual consistency of summaries.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5008–5020, Online. Association for Computational Linguistics.
  * Wang et al. (2019) Siyuan Wang, Zhongyu Wei, Zhihao Fan, Yang Liu, and Xuanjing Huang. 2019.  A multi-agent communication framework for question-worthy phrase extraction and question generation.  In _AAAI_.
  * Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and Jamie Brew. 2020.  Transformers: State-of-the-art natural language processing.  In _EMNLP_.
  * Yin et al. (2021) Wenpeng Yin, Dragomir Radev, and Caiming Xiong. 2021.  DocNLI: A large-scale dataset for document-level natural language inference.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 4913–4922, Online. Association for Computational Linguistics.
  * Yuan et al. (2021) Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021.  Bartscore: Evaluating generated text as text generation.  _ArXiv_ , abs/2106.11520.
  * Zhou et al. (2019) Wenjie Zhou, Minghua Zhang, and Yunfang Wu. 2019.  Multi-task learning with language modeling for question generation.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3394–3399, Hong Kong, China. Association for Computational Linguistics.

##  Appendix A Implementation and Training Details

We use PyTorch to implement our models. We acquire the pre-trained BART
model777https://huggingface.co/facebook/bart-base from the Transformers
library (Wolf et al., 2020).

During training, we set the batch size to 64 and learning rate to 1e-5. The
maximum input length is 1024. In inference, we use beam-search with beam size
4 to generate answers for QA. Following Sultan et al. (2020), we combine top-k
sampling(k=50) with top-p sampling(p=0.95) to generate question-answer pairs.
We averagely return 4 candidates each step and set sentence-level beam size to
4, which means in our best model, every step we select 4 out of 16 candidate
Q-A flows. The models we use are $base$ size.

After training we evaluate the losses of five tasks on CoQA dev set, and the
F1qa scores using Task $a$. Table 10 shows the results with different training
settings. We can see that joint training improves the performance on four out
of five tasks, suggesting that different tasks benefit each other effectively.
Prompts also enhance the Q-A ability and decrease the losses on three out of
five tasks.

CoQA | Ours | w/o Prompts | w/o Joint
---|---|---|---
$Loss$ a | 0.767 | 0.771 | 0.777
$Loss$ q | 1.364 | 1.370 | 1.377
$Loss$ m | 1.372 | 1.378 | 1.388
$Loss$ r | 0.062 | 0.058 | 0.068
$Loss$ h | 2.554 | 2.543 | 2.536
$F1_{qa}$ a | 80.60 | 80.07 | 78.54
Table 10: Inference losses and F1qa scores on CoQA dev set using different
training method.

During reranking, the scales of different losses are also not far from Table
10.

##  Appendix B Supplementary Analyses

###  B.1 Beam-Search or Nucleus Sampling

As argued in (Sultan et al., 2020), nucleus sampling leads to higher diversity
and is better than beam-search in QG. To verify that, we train two sets of
models on different tasks with full strategies. We adopt beam-search with size
4 and nucleus sampling with top-k(k=50) and top-p(p=0.95). Table 11 shows that
nucleus sampling truly gains better results than beam-search.

Tasks | Beam-Search | Nucleus Sampling
---|---|---
CoQA | 0.765/81.60 | 0.766/81.98
SQuAD | 0.679/85.51 | 0.691/85.59
DocNLI | 2.380/49.33 | 2.376/49.98
Table 11: Results using beam-search or nucleus sampling.

###  B.2 Efficiency Analysis

When training the multitask model, we jointly train five tasks in one model,
so the efficiency of our strategy is an inevitable topic. Here in Figure 4, we
demonstrate the training curves of Task $a$ and $main$ using single model and
multitask model.

SingleMultitaskSingleMultitask$step$$loss\ of\
main$$(13000,1.388)$$(55000,1.372)$$step$$loss\ of\
a$$(19000,0.777)$$(57500,0.767)$ Figure 4: The training curves of Task $a$ and
$main$ using single model and multitask model. The optimum points are marked
in the figures. Note that our batch size is 64.

We can clearly see that the convergence speed of multitask model is not five
times slower than the single model. In fact, it only takes about three times
of steps in Task $a$ and four times in Task $main$, for our multitask model to
meet the optimum point compared with the single model. Also, the initial
convergence speed in the first few steps of the single model is only about
twice as fast as the joint model. Thus, in training we can say that the five
tasks mutually benefit each other. In inference our multitask model takes
about five times as long to generate a question.

###  B.3 Different Reranking Losses

Besides the reranking losses defined in 4.2, we also conduct another version
which uses $\sum$ to aggregate single losses. We use Joint train+rerank+RS+SBS
model to augment CoQA dataset and do DocNLI task, using $\prod$ and $\sum$ 036
respectively. Table 12 shows that the methods using $\prod$ gain almost the
same performance as $\sum$.

CoQA | Bleu | Loss | F1qa
---|---|---|---
$\prod$ | 32.01/47.86 | 5.431/0.766 | 72.49/81.98
$\sum$ | 31.96/47.67 | 5.404/0.756 | 72.50/81.91
DocNLI | Loss | F1qa | F1nli
$\prod$ | 2.503/3.457 | 66.04/50.91 | 50.01
$\sum$ | 2.376/3.353 | 66.19/51.19 | 49.98
Table 12: Results of Joint train+rerank+RS+SBS model on augmenting CoQA
dataset and DocNLI task, using different loss aggregation methods.

###  B.4 Mathematically Analysis of Rationale Sampling

Although the intuition of our rationale sampling is to use previous sentences
and the Q-A pairs on $R_{n}$ to restore $R_{n}$,
$\sum_{i=n^{\prime}+1}^{n}(Q_{i}+A_{i})$ is dependent on and logically
connected with $\sum_{i=1}^{n^{\prime}}(Q_{i}+A_{i})$. Also, since the
information of $\sum_{i=1}^{n^{\prime}}(Q_{i}+A_{i})$ is totally contained in
$\bigcup_{i=1}^{n^{\prime}}R_{i}$, we might as well do the following
transformation.

$\begin{split}loss(R_{n}|\sum_{i=n^{\prime}+1}^{n}(Q_{i}+A_{i})+\bigcup_{i=1}^{n^{\prime}}R_{i},\theta)\\\
\approx
loss(R_{n}|\sum_{i=1}^{n}(Q_{i}+A_{i})+\bigcup_{i=1}^{n^{\prime}}R_{i},\theta).\end{split}$

Also, since the information of $\sum_{i=n^{\prime}+1}^{n}(Q_{i}+A_{i})$
contribute not much to generate $\bigcup_{i=1}^{n^{\prime}}R_{i}$, we can say
that

$\begin{split}p(\bigcup_{i=1}^{n^{\prime}}R_{i}|\sum_{i=1}^{n}(Q_{i}+A_{i}),\theta)\\\
\approx
p(\bigcup_{i=1}^{n^{\prime}}R_{i}|\sum_{i=1}^{n^{\prime}}(Q_{i}+A_{i},\theta).\end{split}$

Then,

$\begin{split}&\quad
loss(R_{n}|\sum_{i=n^{\prime}+1}^{n}(Q_{i}+A_{i})+\bigcup_{i=1}^{n^{\prime}}R_{i},\theta)\\\
&\approx
loss(R_{n}|\sum_{i=1}^{n}(Q_{i}+A_{i})+\bigcup_{i=1}^{n^{\prime}}R_{i},\theta)\\\
&=-\frac{log\,p(R_{n}|\sum_{i=1}^{n}(Q_{i}+A_{i})+\bigcup_{i=1}^{n^{\prime}}R_{i},\theta)}{m_{n}-m_{n^{\prime}}}\\\
&=-\frac{1}{m_{n}-m_{n^{\prime}}}[\\\ &\quad
log\,p(R_{n}|\sum_{i=1}^{n}(Q_{i}+A_{i})+\bigcup_{i=1}^{n^{\prime}}R_{i},\theta)\\\
&+log\,p(\bigcup_{i=1}^{n^{\prime}}R_{i}|\sum_{i=1}^{n}(Q_{i}+A_{i}),\theta)\\\
&-log\,p(\bigcup_{i=1}^{n^{\prime}}R_{i}|\sum_{i=1}^{n}(Q_{i}+A_{i}),\theta)]\\\
&=-\frac{1}{m_{n}-m_{n^{\prime}}}[log\,p(\bigcup_{i=1}^{n}R_{i}|\sum_{i=1}^{n}(Q_{i}+A_{i}),\theta)\\\
&-log\,p(\bigcup_{i=1}^{n^{\prime}}R_{i}|\sum_{i=1}^{n}(Q_{i}+A_{i}),\theta)]\
(use\ Eq.\ref{eq2})\\\
&\approx-\frac{1}{m_{n}-m_{n^{\prime}}}[log\,p(\bigcup_{i=1}^{n}R_{i}|\sum_{i=1}^{n}(Q_{i}+A_{i}),\theta)\\\
&-log\,p(\bigcup_{i=1}^{n^{\prime}}R_{i}|\sum_{i=1}^{n^{\prime}}(Q_{i}+A_{i}),\theta)]\\\
&=\frac{1}{m_{n}-m_{n^{\prime}}}(m_{n}loss_{h_{n}}-m_{n^{\prime}}loss_{h_{n^{\prime}}})\triangleq
a.\end{split}$

###  B.5 Other Rationale Sampling Strategies

Besides the rationale sampling strategy in 4.3, we also conduct two other
versions. The first one is a constant function with a value of 0.3, as Eq.6.
In the second version, we use the length of each rationale on behalf of its
amount of information. We let $x$ mean the ratio between the current rationale
length and the story length and make $kp$ linear related to $x$. Empirically,
we set the slope to 3 and an upper bound of 0.75, as Eq.7.

$kp=0.3.$ (6)

$kp=\left\\{\begin{aligned} 3x,\quad&\ 0\leq x\leq 0.25\\\ 0.75,\quad&\
0.25<x\leq 1\\\ \end{aligned}\right.$ (7)

Tasks | Eq.6 | Eq.7 | Ours
---|---|---|---
CoQA | 0.772/81.62 | 0.762/81.88 | 0.766/81.98
SQuAD | 0.660/85.43 | 0.651/85.61 | 0.691/85.59
DocNLI | 2.382/49.12 | 2.375/49.88 | 2.376/49.98
Table 13: Results (F1qa for CoQA and SQuAD, F1nli for DocNLI) using different
rationale sampling strategies.

Using these three rationale sampling methods, we train three sets of models on
different tasks with full strategies. The results are in Table 13. We can see
that the dynamic probability is more suitable than the constant value. Also,
our strategy based on auxiliary Task $h$ performs better than that based on
sentence length. Specifically, it gets 0.1 points higher on CoQA and DocNLI
and gets almost the same score on SQuAD.


# Siren - Advancing Cybersecurity through Deception and Adaptive Analysis

Narayanan Ganesh*, Kulathumani Girish, Ananthanarayanan Samhruth,
School of Computer Science and Engineering
Vellore Institute of Technology
Chennai - 600127
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Siren represents a pioneering research effort aimed at fortifying
cybersecurity through strategic integration of deception, machine learning,
and proactive threat analysis. Drawing inspiration from mythical sirens, this
project employs sophisticated methods to lure potential threats into
controlled environments. The system features a dynamic machine learning model
for real-time analysis and classification, ensuring continuous adaptability to
emerging cyber threats. The architectural framework includes a link monitoring
proxy, a purpose-built machine learning model for dynamic link analysis, and a
honeypot enriched with simulated user interactions to intensify threat
engagement. Data protection within the honeypot is fortified with
probabilistic encryption. Additionally, the incorporation of simulated user
activity extends the system’s capacity to capture and learn from potential
attackers even after user disengagement. Siren introduces a paradigm shift in
cybersecurity, transforming traditional defense mechanisms into proactive
systems that actively engage and learn from potential adversaries. The
research strives to enhance user protection while yielding valuable insights
for ongoing refinement in response to the evolving landscape of cybersecurity
threats.

###### Index Terms:

Cybersecurity, Deception, Machine learning, Honeypot, Threat analysis,
Adaptive security, Threat detection, Dynamic analysis

##  I Introduction

The ongoing transformation of cyber threats underscores the critical need for
adaptive and innovative defense strategies. Confronted with an expanding
attack surface and increasingly sophisticated threat vectors, the traditional
reactive measures often prove inadequate. Recognizing this imperative for
state-of-the-art cybersecurity solutions, this research introduces the Siren
system as an inventive and proactive defense mechanism. By integrating
deception, machine learning, and real-time threat analysis, Siren seeks to
contribute to the ongoing discourse on cybersecurity, offering insights and
solutions tailored to the dynamic challenges posed by the ever-changing threat
landscape.
Reactive strategies, while foundational, inherently operate in response to
known threats, leaving organizations vulnerable to novel and evolving attack
vectors. The escalating sophistication of cyber threats necessitates a
paradigm shift toward proactive defense mechanisms. Proactive approaches
enable anticipation and mitigation of potential threats before they
materialize, offering a more robust and adaptive defense posture. This shift
is imperative to stay ahead of adversaries, addressing vulnerabilities in
advance and minimizing the impact of unforeseen and rapidly evolving cyber
threats. The research presented here underscores the critical need for
proactive cybersecurity measures.
Existing defensive paradigms predominantly hinge on reactive methodologies,
responsive to known threats but lacking preemptive capabilities against
unforeseen risks. The motivation to bridge this gap stems from the imperative
to cultivate a cybersecurity approach that is not merely responsive but
anticipatory, affording organizations the ability to proactively thwart
potential threats. By introducing a proactive system, the Siren project
aspires to revolutionize cybersecurity protocols, minimizing the lag time
between threat emergence and defensive response. This research endeavors to
address the critical need for anticipatory cybersecurity measures, aiming to
contribute to the ongoing discourse in the field and fortify organizations
against the ever-evolving landscape of cyber threats.
The aim is to create an environment where attackers are lured into controlled
spaces by simulating vulnerabilities, facilitating an analysis of their
behaviors. This unique approach seeks to leverage the power of deception to
proactively engage with and understand potential threats, marking a departure
from conventional cybersecurity methodologies. This involves the seamless
integration of deception, adaptive machine learning, and real-time threat
analysis within the Siren system. The research seeks to contribute empirically
validated insights into the efficacy of proactive cybersecurity, examining its
potential to preemptively identify and neutralize emerging threats.
Furthermore, the project aims to enhance the current understanding of
adversary behaviors by meticulously analyzing their responses within
controlled environments. By achieving these objectives, the Siren project
aspires to establish a novel benchmark in cybersecurity practices, redefining
the industry’s approach to threat mitigation and contributing substantively to
the evolution of proactive defense strategies. The foundational strategy of
the Siren system involves a cohesive integration of deception, adaptive
machine learning, and real-time threat analysis. This approach leverages the
dynamic interplay between these core components to create a proactive defense
mechanism. The Siren system is meticulously designed to anticipate and respond
to emerging cyber-threats, offering a comprehensive solution that transcends
the limitations of reactive cybersecurity approaches.
Distinctive attributes define the Siren system, setting it apart in the realm
of cybersecurity. An adaptive machine learning model forms the cognitive
backbone, ensuring continuous evolution in response to emerging threats. The
inclusion of a link monitoring proxy adds an additional layer of scrutiny,
while the honeypot, fortified with probabilistic encryption, creates a secure
and controlled environment. Furthermore, the incorporation of simulated user
activity enhances the system’s capacity to engage and learn from potential
threats, marking a paradigm shift in cybersecurity strategies.
The envisioned outcomes of the Siren research project are twofold. Firstly,
the system aims to elevate user protection by proactively identifying and
mitigating potential threats. Through empirical validation and real-world
application, Siren endeavors to demonstrate the efficacy of its approach in
enhancing overall cybersecurity resilience. Secondly, the research anticipates
contributing valuable insights to the broader cybersecurity domain. By
analyzing adversary behaviors within controlled environments, this system aims
to provide nuanced perspectives that contribute substantively to the ongoing
discourse on proactive defense strategies.This research paper unfolds
systematically, beginning with an introduction that establishes the context of
the current cybersecurity landscape and underscores the importance of
proactive defense mechanisms.
The subsequent section provides a detailed overview of the Siren system’s
approach, elucidating its core components and their integration. A dedicated
segment highlights the unique features of Siren, emphasizing the adaptive
machine learning model, link monitoring proxy, honeypot with probabilistic
encryption, and simulated user activity. The outcomes section provides a
glimpse into the expected impact of the research on user protection and its
contribution to cybersecurity insights. Finally, the paper concludes with a
summary and recommendations for future research, providing a comprehensive
roadmap for readers to grasp the significance and implications of the Siren
project.

##  II Related Works

In this paper, the possibilities of imbuing machine learning and probabilistic
encryption with one another to strengthen security are explored. The current
state of honeypots is that they are currently used as a secondary method to
firewalls as a largely reactionary method according to M.Nawrocki.[1].
Honeypots, even if interacted with, generally just serve as a deterrent for
attackers to waste time but can become a vulnerability if exposed for too long
or largely interacted with. A honeypot design can be loosely based off a
scanning tool, a fingerprinting device, and a few other tools from L.Kuwatly.
[2]. However, far more novel methods such adding an additional layer with user
reconfirmation from Li and Schmitz[11] present far more security even with
relatively prolonged interactions as well with very minimal if any leakage at
all. This not only encourages the attacker to interact with the honeypot
entity as a whole but can in fact keep the attacker from exploiting the
vulnerability detected altogether.
The main feature that makes honeypots a weak point is their easy access and
vulnerability due to features such as encryption. However, the encryption
itself can be strengthened if made probabilistic as put by J.Benaloh [6]. The
probabilistic encryption scheme can be made using the standard encryption
function of input and cipher but with a third variable that is random in
nature.
Harn and Kiesler [5] presented a fairly efficient probabilistic scheme based
on the concept of using a randomized variable to simplify functions such as
Rabin’s 4-1 scheme with randomized variables to ensure that the complexity of
the encryption is retained but there is gain in speed. A probabilistic scheme
does however pose the challenge of choosing the random variable as well and
that in itself would need to be addressed in its entirety.
Lastly, deep learning frameworks to further protect the honeypot without
making changes to the vulnerability itself can also be used. The model,
according to Tang et al. [3], can extract most of the detail using RNN-GNU
models for decently engineered precision and some speed as well. Xuan, Nguyen,
and Nikolaevich [4] also further present that the URL can be sent to the
machine learning model from the website itself to give near instant
recognition to the user and administrator as part of the whole security of the
organization. Matin [7] further enhances this using the honeypot itself to
lure a phisher to come to the vulnerability it creates. Yet none of the papers
have managed to address the consistent vulnerability that the honeypot creates
with the prolonged interaction of the phisher. Detection alone may not be
entirely feasible on its own.

##  III Methodology

###  III-A Overview

The work proposes a framework that uses a deep learning model in order to
learn the nature of the URL by training on a dataset of malicious URLs so that
it can identify and differentiate malicious and non-malicious URLs from one
another. If the malicious URL has been identified, the honeypot to the server
gets activated along with the probabilistic encryption scheme which can
further re-encrypt the already encrypted data adding another layer of
protection to it.
This helps to overcome the major bottleneck of keeping honeypots open for long
periods of time due to security risks as the honeypot here is fairly isolated
while also having files be fairly inaccessible due to the repeated
probabilistic encryption usage.

###  III-B Dataset And Feature Extraction

The dataset consists of two main columns, one for the specific URLs and the
type of URL that they are. There are 4 main types of URLs in the dataset –
benign, defacement, phishing and malware. The objective here is to simply
classify the data based on the given link after extracting all possible
features from it.
Features are extracted based on a few different categories from the URL
itself. Firstly, the protocol, host, path, parameters, query and fragment are
all separately extracted from the URL itself and placed into different columns
of the dataset. This allows us to parse each component down to its most basic
units and decide which features can be retained and which do not need
retention later on. Any URL that does not have such features like query or
fragment will be stored as null values.
There is a clear imbalance in features with a majority of the URLs provided
belonging to the benign category. Even within the malevolent links, the
majority of links belong to the category of defacement with phishing having a
few lesser and barely any belonging to malware.

Figure 1: Feature Distribution Of Dataset

  * •

URL Features: Those columns with predominant null values are subsequently
removed as they do not have much effect on the model itself. Here, the values
removed were parameters, query and fragment. Any columns with remaining null
values were then extracted and the null values were removed. Regular
expressions were then used to extract IP addresses that were valid,
considering IPv4, IPv4 with port, IPv4 in hexadecimal and IPv6 addresses. The
regex used for the extractions are given below:

IPv4:

    * –

(([01]?\\\d\\\d?|2[0-4]\\\d|25[0-5])\\\\.
([01]?\\\d\\\d?|2[0-4]\\\d|25[0-5])\\\\.([
01]?\\\d\\\d?|2[0-4]\\\d|25[0-5])\\\\.

    * –

([01]?\\\d\\\d?|2[0-4]\\\d|25[0-5])\\\/)|

IPv4 With Port:

    * –

(([01]?\\\d\\\d?|2[0-4]\\\d|25[0-5])\\\\.
([01]?\\\d\\\d?|2[0-4]\\\d|25[0-5])\\\\.
([01]?\\\d\\\d?|2[0-4]\\\d|25[0-5])\\\\.|

    * –

([01]?\\\d\\\d?|2[0-4]\\\d|25[0-5])\\\/)|

IPv4 Hexadeceimal:

    * –

((0x[0-9a-fA-F]{1,2})\\\\.(0x[0-9a-fA-
F]{1,2})\\\\.(0x[0-9a-fA-F]{1,2})\\\
.(0x[0-9a-fA-F]{1,2})\\\/)

IPv6:

    * –

(?:[a-fA-F0-9]{1,4}:){7}[a-fA-F0-9]
{1,4}|

    * –

([0-9]+(?:\\.[0-9]+)3:[0-9]+)|

    * –

((?:(?:\d|[01]?\d\d|2[0-4]\d|25[0-5]
)\\.){3}(?:25[0-5]|2[0-4]\d|[01]?\d\d|
\d)(?:\/\d{1,2})?)

URLs with special symbols were then extracted and stored in specific columns
to check if any predominant symbol was used in the making of the URL itself.
The length of each URL was also extracted as a feature to check for
correlations on possible addon domains in malicious websites. Shortening
services such as bit.ly tinyurl and such were also considered for the usage of
URLs to not exclude links that have been shortened and put them in their own
category for classification. URL characteristics such as digit count, email
addresses, letter counts, vowel counts and top-level domain counts were also
considered. Only top-level domain counts were retained as a prominent feature.
Any possibilities of a server client being mentioned for purposes of
classification were also used to make sure that fake malicious copies of real
websites could be identified.

  * •

Path Features: The first feature that extracted on path were special symbols,
the same as in URLs. Similarly extracted length, digit counts, letter counts
and vowel counts were also extracted.

  * •

Host Features: Here too the special symbols, length, digit counts, letter
counts and vowel counts were extracted.

  * •

Final Feature Engineering: Due to the presence of several zeroes, the columns
with very low mean values are removed. Columns such as some special symbols
for path and host and protocol were removed. The remaining columns were merged
together. All the symbols for path, URL and the HTTPS and FTP columns were
merged together. The labels are then extracted from the data and categorically
numbered to make all of them into values between 0 and 3, inclusive using one-
hot encoding.
To ensure that the data can allow for easier gradient descent, the data was
scaled using a standard scale to have all columns with a mean of 0 and a
standard deviation of 1. The labels are then added back to the data for the
purpose of further feature engineering.
The data is first split into training, validation and testing data using
Stratified K-Fold algorithm and shuffling. This ensures a balanced mean value
to be present in each set by allowing an equal percentage of labels in each
set for higher model accuracy. Around 65% of the data in training with 30% for
validation and 5% for testing will be used. The data split ensures that a good
chunk of data can be used for training while maintaining a sizable amount of
data for validation and testing.

###  III-C Model

The model that is used is a basic perceptron for classification using a 68
dimensional input to make sure all the features extracted thus far can be
input into the neural network. The hidden layers have a dimension of 300, 200
and 100 respectively. To ensure the highest possible chance of generalization,
two dropout layers to the model with a probability of 0.2 and a batch
normalization layer have been added before it is fed to the output, assuming a
softmax activation. The hidden layers are all activated using the Scaled
Exponential Linear Unit or SELU activation to ensure that gradients are fed
back properly and account for any chances of the activation function dying or
causing the gradient to vanish or explode.

Figure 2: Model Architecture

###  III-D Honeypot

The honeypot is made using a standard architecture with the intention of
giving the attacker a method of trying to access the system under the guise of
being a dummy that simply serves no purpose in and of itself other than bait.
The honeypot needs to being able to sustain itself to make sure that the
attacker remains within the honeypot itself for as long as the attacker’s
belief on the fact that the system is a real vulnerable system that can be
exploited can be maintained.
The honeypot itself also needs to ensure a sort of detachment from the main
systems to make sure the attacker cannot use the honeypot as a breaching point
very easily. The main methodology suggested here is the use of a one-way
connection to connect the mainframe to the honeypot so that some communication
is possible.
The one-way further allows encryption commands to be sent to the honeypot so
that the files can be encrypted before access just to make sure the attacker
cannot gain access to the files very easily, even if they are dummy files so
as to waste the attacker’s time and effort.
This further enables the study of the attacker’s patterns if monitoring the
honeypot is added so that the attack patterns can be learnt with time as the
attacker is executing them.
The main stalling feature that will be used is a probabilistic key generation
algorithm on the RSA encryption to prevent cryptanalysis of any sorts between
texts. The major architecture of the honeypot is shown in the diagram below.

###  III-E Probabilistic RSA Encryption

Probabilistic RSA encryption is an enhancement of the traditional RSA
encryption algorithm that introduces randomness into the encryption process to
bolster security against certain types of attacks. RSA encryption, named after
its inventors Rivest, Shamir, and Adleman, is an asymmetric encryption scheme
widely used for securing communications and data transmission.
In traditional RSA encryption, plaintext is encrypted using the recipient’s
public key, and the resulting ciphertext can only be decrypted with the
corresponding private key. However, this basic scheme is susceptible to
certain vulnerabilities, particularly when encrypting small or predictable
plaintexts. Probabilistic RSA encryption addresses these vulnerabilities by
incorporating randomness during encryption.
The fundamental principle behind probabilistic RSA encryption is to introduce
randomness into both the plaintext and the encryption process itself. This
randomization helps thwart attacks that exploit regularities or patterns in
the plaintext data. Unlike deterministic encryption schemes, where the same
plaintext always encrypts to the same ciphertext, probabilistic encryption
ensures that each encryption operation produces a different ciphertext, even
for the same plaintext.
The process of probabilistic RSA encryption using weather data without padding
can be summarized as follows:

  * •

Key Generation:

    * –

Take 2 random numbers between 1 and the total number of cities being
considered.

    * –

Find the city associated with the first number.

    * –

Take the maximum and minimum temperature for that day in Kelvin.

    * –

Find the difference of the two temperatures.

    * –

Multiply it by an arbitrary number x to get a value and find the nearest prime
to that value.

    * –

Repeat the above process to get the two primes needed for the encryption.

    * –

Once the primes have been received, product, totient of the product, public
and private key can be generated.

    * –

The public key and product are sent to the sender of the ciphertext.

  * •

Sender:

    * –

The sender receives the public key and encrypts his plaintext using modular
exponentiation by raising the text to the value of the public key modulo
product.

    * –

The ciphertext can be sent to the receiver.

  * •

Receiver:

    * –

The receiver can decrypt the ciphertext by raising to the value of the private
key modulo product.

##  IV Implementation:

The model was implemented on the Tensorflow library on the Python programming
language. All the data preprocessing was done on the same language as well.
All weight and kernel initializations were done using Lecun to make sure that
samples were drawn from a standard Gaussian curve of mean 0 and standard
deviation 1.
The loss function used for this is a Categorical Crossentropy loss function
with a Nesterov-accelarated Adaptive Moment Estimation optimizer (NAdam) to
ensure adequate change in learning rate depending on the time. The metrics
used for the model are Recall and Loss.
All training was done on a quad-core intel CPU and an NVIDIA T4 GPU. The model
will also be early stopped at the best epoch and saved on callback to make
sure that the best point of training and validation can be taken for
evaluating links.
The honeypot was engineered using the Cowrie library on Ubuntu to emulate a
honeypot-like architecture on Ubuntu itself which can be attacked using tools
such as the Nmap scanner on Kali Linux and thus tested for security and to
verify whether or not the data is secure or compromised within the honeypot.
The probabilistic encryption was made using weather data provided by the
OpenWeather API to generate the keys from the weather data of 371 cities and
considering the decimal places of the weather for primes of 4 digits or higher
allows us to get a minimum probability of getting the exact right number as
given by us as:

$\frac{1}{3710000}$

.

Figure 3: Model Loss Figure 4: Model Recall

###  IV-A Results

The model was saved at epoch 28 due to the closeness of the precision and
recall itself with recall differences being limited to around 0.0028 for
training and validation recall and 0.0032 for training and validation loss.
The model can now be used for detection of fraudulent or malevolent links
being clicked within the framework so as to activate the honeypot and check if
any intruder has entered within the given time frame.
Cowrie had further proven to be successful at emulating the honeypot itself
with proper file access and the ability to add and remove files from the
honeypot as we please.
The probabilistic encryption also worked as expected returning different
strings for each encryption take done at any given point in time. Assuming the
test string to be Hello World:
The ASCII representation of Hello World in numbers :
72, 101, 108, 108, 111, 32, 87, 111, 114, 108, 100

  * •

Time 1:
Encrypted: 553, 582, 330, 330, 777, 106, 87, 777, 595, 330, 618
Decrypted: 72, 101, 108, 108, 111, 32, 87, 111, 114, 108, 100

  * •

Time 2:
Encrypted: 901, 640, 823, 823, 777, 483, 725, 777, 102, 823, 676
Decrypted: 72, 101, 108, 108, 111, 32, 87, 111, 114, 108, 100

Figure 5: Accuracy On Data Split Figure 6: Precision On Data Split

###  IV-B Conclusion

This paper presents research on Siren, a proactive cybersecurity protection
mechanism that combines machine learning, deception, and real-time threat
analysis. Siren makes it easier to analyze prospective attacker activity and
gives the system the ability to proactively stop threats by deliberately
luring them into controlled surroundings. The fundamental principle of Siren
is its capacity to detect and react to new threats, presenting a major
departure from reactive cybersecurity techniques.
A link monitoring proxy for increased scrutiny, a honeypot strengthened with
probabilistic encryption to ensure data security, an adaptive machine learning
model that constantly adapts in response to new threats, and the addition of
simulated user activity to increase the system’s ability to learn from
potential threats are some of Siren’s unique features.
By adding randomness to the encryption process, probabilistic RSA encryption
improves the honeypot’s security posture and increases its resistance to
different types of cryptographic attacks.
There’s a good chance that this research will significantly advance
cybersecurity. User protection will be strengthened by the empirical
evaluation of Siren’s effectiveness in proactively recognizing and reducing
potential threats. Moreover, studying how adversaries behave in controlled
settings will provide insightful information that will significantly influence
the creation of proactive defensive plans in the future.
The results of this study highlight how urgent it is to advance beyond
reactive cybersecurity strategies. Organizations may fortify their defenses
and remain ahead of the constantly changing threat landscape by implementing
proactive solutions like Siren. Subsequent research endeavors encompass the
extensive implementation and assessment of Siren in real-world settings, in
conjunction with constant enhancement of the system’s constituent elements to
guarantee its effectiveness against innovative and intricate cyber hazards.
Siren clears the path for a day where cybersecurity is strengthened and
proactive rather than reactive, protecting digital assets and reducing the
dangers associated with persistent cyberattacks.
The results of this study highlight how urgent it is to advance beyond
reactive cybersecurity strategies. Organizations may fortify their defenses
and remain ahead of the constantly changing threat landscape by implementing
proactive solutions like Siren. Subsequent research paths involve the
extensive implementation and assessment of Siren in real-world settings, in
addition to constant improvement of the system’s elements to guarantee its
effectiveness against new and complex dangers from cyberspace. Siren clears
the path for a day where cybersecurity is strengthened and proactive rather
than reactive, protecting digital assets and reducing the dangers associated
with persistent cyberattacks.

## Acknowledgment

The authors would like to express their gratitude to VIT Chennai for providing
the resources necessary to conduct this research. Additionally, they are
deeply thankful to Dr. Ganesan Narayanan for his assistance and mentorship
throughout this research.

## References

  * [1] M. Nawrocki, M. Wahlisch, T. C. Schmidt, C. Keil, and J. Schonfelder, ”A survey on honeypot software and data analysis,” IEEE Transactions on Information Forensics and Security, vol. 14, no. 10, pp. 2567-2582, Oct. 2019. [Online]. Available: arXiv:1608.06249v1 [cs.CR], Aug. 22, 2016.
  * [2] I.Kuwatly, M.Sraj, Z.AI Masri, and H.Artail, ”A dynamic honeypot design for intrusion detection,” in Proceedings of the IEEE/ACS International Conference on Pervasive Services, 2004, American University of Beirut, Department of Electrical and Computer Engineering, Beirut, Lebanon, pp. 3623-3623.
  * [3] L.Tang and Q.H.Mahmoud, ”A deep learning-based framework for phishing website detection,” IEEE Access, vol. 9, pp. 32082-32092, 2021
  * [4] C. D. Xuan, H. Nguyen, and V. N. Tisenko, ”malicious URL detection based on machine learning,” International Journal of Advanced Computer Science and Applications, vol. 11, no. 1, pp. 315-321, 2020. [Online]. Available: DOI: 10.14569/ijacsa.2020.0110119
  * [5] L.Harn and T.Kiesler, ”An efficient probabilistic encryption scheme,” Information Processing Letters, vol. 36, no. 4, pp. 195-200, 1990.
  * [6] J.Benaloh, ”dense probabilistic encryption,” in Proceedings of the Workshop on Selected Areas of Cryptography (SAC), 1994, pp. 1-15
  * [7] I. M. Matin and B. Rahardjo, ”malware detection using honeypot and machine learning,” in Proceedings of the 7th International Conference on Cyber and IT Service Management (CITSM), 2019, pp. 1-6.
  * [8] S.Gajek and A. R.Sadeghi, ”A forensic framework for tracing phishers,” in Proceedings of the IFIP International Summer School on the Future of Identity in the Information Society, 2007, pp. 236-248
  * [9] E. Alata, V. Nicomette, M. Kaâniche, M. Dacier, and M. Herrb, ”Lessons learned from the deployment of a high-interaction honeypot,” in Proceedings of the 6th European Dependable Computing Conference (EDCC), 2006, pp. 59-68.
  * [10] O.Osho, A.Oluyomi, S.Misra, R.Ahuja, R.Damasevicius, and R. Maskeliunas, ”Comparative evaluation of techniques for detection of phishing URLs,” in Proceedings of the Second International Conference on Applied Informatics (ICAI, 2019, pp. 121-132.
  * [11] S.Li and R.Schmitz, ”A novel anti-phishing framework based on honeypots,” in Proceedings of the IEEE Conference on Communications and Network Security, 2009, pp. 1-5.