# A multi-task semi-supervised framework for Text2Graph & Graph2Text

Oriol Domingo1 Marta R. Costa-Jussà2 Carlos Escolano2 1BATOU XYZ
2Universitat Politècnica de Catalunya
<EMAIL_ADDRESS>{marta.ruiz<EMAIL_ADDRESS>
###### Abstract

The Artificial Intelligence industry regularly develops applications that
mostly rely on Knowledge Bases, a data repository about specific, or general,
domains, usually represented in a graph shape. Similar to other databases,
they face two main challenges: information ingestion and information
retrieval. We approach these challenges by jointly learning graph extraction
from text and text generation from graphs. The proposed solution, a T5
architecture, is trained in a multi-task semi-supervised environment, with our
collected non-parallel data, following a cycle training regime. Experiments on
WebNLG dataset show that our approach surpasses unsupervised state-of-the-art
results in text-to-graph and graph-to-text. More relevantly, our framework is
more consistent across seen and unseen domains than supervised models. The
resulting model can be easily trained in any new domain with non-parallel
data, by simply adding text and graphs about it, in our cycle
framework.111Source code is available under request and non-paralled data is
released at: https://github.com/uridr/GTWiki

##  1 Introduction

In recent years, Artificial Intelligence industry has leveraged the power of
computation along deep learning models to build cutting-edge applications.
Some of these applications, such as question answering systems Sharath and
Banafsheh (2021), chat-bot Ait-Mlouk and Jiang (2020), recommender systems Guo
et al. (2020a) or personal assistants Bellegarda (2013), heavily rely on
Knowledge Bases (KB). A KB is a database that stores complex structured and
unstructured facts about specific, or generic, areas, commonly expressed in a
graph shape. The following challenges arise in the aforementioned applications
when dealing with KBs:

Figure 1: Overview of the problem. A Deep Learning model jointly learns Text-
to-Graph (gray) and Graph-to-Text (red) in a cycle training regime.

  1. 1.

Continuous information ingestion to keep the KB updated according to the
(world) situation Ji and Grishman (2011).

  2. 2.

Building a data representation on top of the information retrieval level that
is human comprehensible Ferreira et al. (2020).

To address these challenges, we apply Text-to-Graph (T2G) to uncover new
hidden facts given a textual source (1st challenge). For instance, the
information embedded in the text ”The Palau de la Música Catalana is a concert
hall in Barcelona” can be summarised as “(Palau de la Música Catalana,
instance of, Concert Hall), (Palau de la Música Catalana, location,
Barcelona)”. Besides, we perform Graph-to-Text (G2T) to generate text that
embeds the whole information of the retrieved knowledge graph (2nd challenge).
For example, the graph “(Sagrada Família, location, Barcelona), (Palau de la
Música Catalana, location, Barcelona)” can be expressed as “The Sagrada
Família and Palau de la Música Catalana are well known monuments located in
Barcelona.” By approaching both tasks together, we can consider them as
complementary tasks [Figure 1].

Despite being very well known tasks, there are still very little datasets, and
those are on the magnitude of tens of thousands samples Ferreira et al.
(2020), making it hard for models to reach human-level performance. For this
reason, our work investigates a framework that approaches both tasks in a
multi-task semi-supervised environment, aiming to gain data efficiency, reduce
overfitting through shared representations, and fast learning by leveraging
auxiliary information Crawshaw (2020).

To summarise, our main contributions are two fold:

  * •

Building a model capable of learning G2T and T2G in a multi-task semi-
supervised environment. Experiments on WebNLG dataset show that our framework
surpasses state-of-the-art results in G2T & T2G for unsupervised models. More
relevantly, our framework is on par or even more consistent across seen and
unseen domains than supervised and unsupervised models on both tasks.

  * •

Collecting and releasing non-parallel data, text and graphs, for unsupervised
models of around 250k instances per each data type.

The rest of the paper is organised as follows. In section 2 both challenges
are mathematically formulated, before related work is presented in section 3.
Following, the methodology of this work is explained in section 4, where we
detail each part of our framework. After that, section 5 introduces our
experimental set-up, and then, our results on these experiments are discussed
in section 6. Finally, conclusions are presented in section 7.

##  2 Problem Formulation

We formulate information ingestion and information retrieval as a T2G
(parsing) and G2T (generation) tasks respectively, using a dataset of
supervised examples
$\mathcal{S}:(g,t)_{i=1}^{N}\in\mathcal{G}\times\mathcal{T}$. On one hand, our
graph dataset $\mathcal{G}:=\\{g_{i}\\}_{i=1}^{N}$ consists of $N$ graphs in a
triple format, where triples are composed of a subject ($s_{n}$), predicate
($p_{n}$) and object ($o_{n}$). On the other hand, our text corpus
$\mathcal{T}:=\\{t_{i}\\}_{i=1}^{N}$ consists of $N$ text sequences, made up
by tokens. Presumably, aligned graphs and sequences on $\mathcal{S}$ share the
same latent content, but differ in terms of surface realisation. Thus, we aim
to generate text from graphs using some function $g2t_{\theta}$, and
inversely, we aim to parse graphs from text using some function $t2g_{\phi}$,
where each output embeds the semantic meaning of the aligned input [Eq 1].
Both functions are parameterized by $\theta$ and $\phi$ respectively.

$\begin{cases}g2t_{\theta}(g)=\hat{t}\simeq t\\\ t2g_{\phi}(t)=\hat{g}\simeq
g\end{cases}$ (1)

Ideally, these functions are optimised over $\mathcal{S}$ by means of a
maximum log-likelihood estimation on $\theta$ and $\phi$[Eq 2]. By this
optimisation pass, models aim to learn the corresponding T2G and G2T tasks.

$\mathcal{J}(\theta,\phi)=\mathbb{E}_{(g,t)\sim\mathcal{S}}[\log\ p(t\shortmid
g;\theta)+\log\ p(g\shortmid t;\phi)]$ (2)

##  3 Related Work

Unsupervised Learning. Recently, Schmitt et al. (2020) presented the first
approach to unsupervised G2T and T2G. They proposed two different methods: a
rule-based system; they considered as a baseline; and a neural sequence-to-
sequence system; which considerably improved baseline results. On one hand,
the rule-based system relies on several steps such as preprocessing, removing
stop words, part of speech tagging (similar to semantic parsers), heuristics
and template linearisation. On the other hand, they proposed a BiLSTM
Hochreiter and Schmidhuber (1997) sequence-to-sequence system trained also in
a multi-task environment and fine-tuned with noisy source samples Veit et al.
(2017). Regarding the neural training regime, firstly, they obtained a
language model for both graphs and text, and later on iterative Back
Translation Sennrich et al. (2015) is applied on graphs and text in which
their corresponding alignment has been removed, but come from a supervised
dataset in which the system is evaluated on. Similarly, Guo et al. (2020b)
developed an unsupervised training method that can bootstrap from fully non-
parallel graphs and text data, and iteratively back translate between the two
forms. However, unsupervised samples come from a supervised dataset as well,
so a real data distribution exists among their training examples unlike in our
work. This cycle training framework achieves state-of-the-art results for
unsupervised models in this domain, however, they rely on two different
models. Particularly, T2G task is solved with a system made up by an off-the-
shelf entity extraction model Qi et al. (2020) and a BiLSTM to predict the
relation between any pair of entities. Shortly, G2T is solved with a T5 model
Raffel et al. (2019), as we do (see 4.1). Consequently, not only does this
approach seek to have a common data representation between these two models,
but this also constraints the optimisation procedure as both models cannot be
updated together, rather they are updated one-by-one, making loss function
non-differentiable.

Multi-task Learning. As previously mentioned, Martin Schmitt et al. trained
their neural model in a multi-task set-up sharing encoder and decoder.
However, they tell the decoder which type of output should be produced (text
or graphs) by means of cell state initialisation in the decoder side, with an
embedding corresponding to the desired output type. Alternatively, Oshin
Agarwal et al. Agarwal et al. (2020) overcome the need of task specification
at decoder level thanks to the input format of the chosen model. The model is
also a pre-trained T5 model, in which the task to be solved is simply
identified by adding task tokens to the input. Moreover, they build a
bilingual model for English and Russian languages. Stated by the authors,
these capabilities remarkably improves on unseen relations and Russian. Apart
from that, they also perform data augmentation but not in an unsupervised
manner, they rather pre-trained the original model on a parallel corpus of
news data: WMT-News corpus Tiedemann (2012).

Figure 2: (1) The cycle training approach is based on two mapping functions
$g2t:\mathcal{G}\rightarrow\mathcal{T}$ and
$t2g:\mathcal{T}\rightarrow\mathcal{G}$, however, our approach uses a single
model to learn both functions. This framework introduces two cycle losses that
capture the intuition that if we translate from one side to the other, and
back again we should obtain the original sample: (2) forward cycle loss:
$g\rightarrow g2t(g)\rightarrow t2g(g2t(g))\simeq g$, and (3) backward cycle
loss: $t\rightarrow t2g(t)\rightarrow g2t(t2g(t))\simeq t$.

##  4 Methodology

In this section, we will present our approach towards multi-task learning and
unsupervised learning. Afterwards, the optimisation procedure is detailed.

###  4.1 Multi-task Learning

We propose a multi-task environment, where a single model ($z_{\alpha}$),
parameterized by $\alpha$, can learn both functions ($g2t_{\theta}$ and
$t2g_{\phi}$) simultaneously, leading to [Eq 3]. Thereafter, our main
challenge is to approximate [Eq 3] with unsupervised samples using the model
itself through a cycle training framework (see 4.2).

$\mathcal{J}(\alpha)=\mathbb{E}_{(g,t)\sim\mathcal{S}}[\log\ p(t\shortmid
g;\alpha)\ +\log\ p(g\shortmid t;\alpha)]$ (3)

We are going to use the pre-trained T5-Base model Raffel et al. (2019). This
model is a Transformer architecture, that it is roughly equivalent to the
original one Vaswani et al. (2017), however, it introduces an unified
framework that converts all text-based language problems into text-to-text
format by means of a task token specification. This framework provides a
consistent training objective for both, pre-training and fine-tuning, being
the latter of special interest in our work as it eases the optimisation of our
single model ($z_{\alpha}$) on both downstream tasks at the same time.

###  4.2 Unsupervised Learning

Cycle training was originally suggested as an image-to-image translation,
rather than text-to-text (our current approach), a problem where the goal is
to learn a mapping between an input image and an output image Zhu et al.
(2017). It was presented as a solution to the absence of paired examples. This
solution was based on a cycle consistency loss relying on Generative
Adversarial Networks Goodfellow et al. (2014).

The main constraint for using cycle training is that there must exist two
complementary tasks that guarantees that the input of one task is the output
of the other task, and vice-versa [Figure 2 (1)]. For instance, a graph can be
fed into a model to generate some text. The resulting text can also be fed
into this model to generate a graph, which should resemble the original one
[Figure 2 (2)]. The same procedure is applied in the reverse direction, i.e.
starting from text and generating synthetic graph [Figure 2 (3)]. At this
point, we can cycle-train our model since a reference of our hypothesis exists
on both sides. These steps constitute the iterative loop in which cycle
training is based on.

In our case, previous constraint holds, existence of complementary tasks, so
it is possible to build a bijective mapping function that given a variable $x$
satisfies $x=t2g(g2t(x))$, where $g2t$ is the inverse function of $t2g$.
However, we approach both tasks using a single model ($z_{\alpha}$), following
a multi-task set-up, hence, it must hold that $z:=g2t=t2g^{-1}=t2g$, which is
an involutory function. This constraint needs to be slightly relaxed as our
variable $x$, representing text or graphs, is concatenated with an extra token
($w$ and $k$)222These are task tokens for our T5 model. at the beginning of
the sentence in order to specify the model which output should be generated,
and the output does not contain this extra token to again specify the model
which task is solving. Thus, the input vector space is marginally modified
with respect to the original input, text or graph, before passing into the
model [Eq 4].

$x\simeq\hat{x}=z(z(x))\longrightarrow x\simeq\hat{x}=z(w\ \parallel\ (z(k\
\parallel\ x))$ (4)

This space modification helps the model to avoid learning the identity
function - the most simple involutory function - because the input of the
function is slightly modified with respect to the output of the function due
to task token specification. Furthermore, we use different data on each
iteration of the cycle training (see 5) to prevent our model to memorize all
the training data regardless being synthetic or real, and not generalize well
on test data.

| Overall | Seen Categories | Unseen Entities | Unseen Categories
---|---|---|---|---
| BLEU | TER | chrF++ | BLEU | TER | chrF++ | BLEU | TER | chrF++ | BLEU | TER | chrF++
bt5 (♢△) Agarwal et al. (2020) | 51.7 | 0.43 | 0.67 | 61.1 | 0.39 | 0.72 | 50.8 | 0.41 | 0.68 | 44.0 | 0.47 | 0.63
CycleGT (♡) Guo et al. (2020b) | 44.6 | 0.47 | 0.63 | 47.4 | 0.49 | 0.65 | 46.6 | 0.44 | 0.65 | 40.9 | 0.48 | 0.61
T5-Baseline (♢△) [our implementation] | 44.6 | 0.51 | 0.54 | 51.7 | 0.49 | 0.57 | 45.4 | 0.49 | 0.55 | 38.0 | 0.52 | 0.51
T5-Finetune (♢△) [our implementation] | 42.5 | 0.51 | 0.60 | 46.5 | 0.52 | 0.61 | 44.4 | 0.48 | 0.63 | 38.2 | 0.51 | 0.57
T5-MSSF (♢⨝) [our framework] | 45.3 | 0.48 | 0.62 | 50.9 | 0.49 | 0.64 | 45.8 | 0.45 | 0.64 | 40.0 | 0.49 | 0.59

Table 1: Multi-task (${\diamondsuit}$). Unspervised (${\heartsuit}$). Semi-Supervised ($\Join$). Supervised (${\triangle}$). Summary of related work results in Graph-to-Text (G2T) with multi-task, supervised and/or unsupervised learning. Our multi-task semi-supervised framework is T5-MSSF. |  Test (G2T) |  Test (T2G)
---|---|---
Seen categories | 490 (28%) | 606 (28%)
Unseen entities | 393 (22%) | 457 (21%)
Unseen categories | 896 (50%) | 1,092 (51%)
Total | 1,779 | 2,155
Table 2: Number of the test instances for Graph-to-Text (G2T) and Text-to-
Graph (T2G) with respect to the different data types.

###  4.3 Optimisation Procedure

This mathematical framework allows training without or with few parallel data.
The main idea is that the model can learn from unlabeled data: unlabeled
graphs $\mathcal{U}_{\mathcal{G}}$ and unlabeled text
$\mathcal{U}_{\mathcal{T}}$; using its own predictions [Eq 5 left] as
reference during training [Eq 5 right]. The optimisation pass for both cycle
losses can be backpropagated together on each batch [Eq 6].

$\displaystyle\begin{cases}z_{\alpha}(t)=\hat{g}\longrightarrow\mathcal{L}_{cycleG2T}=\mathbb{E}_{t\in\mathcal{U}_{\mathcal{T}}}[\
-\log\ p(t\shortmid\hat{g};\alpha)\ ]\\\
z_{\alpha}(g)=\hat{t}\longrightarrow\mathcal{L}_{cycleT2G}=\mathbb{E}_{g\in\mathcal{U}_{\mathcal{G}}}[\
-\log\ p(g\shortmid\hat{t};\alpha)\ ]\end{cases}$ (5)

$\displaystyle\mathcal{L}_{cycle}=\mathcal{L}_{cycleG2T}+\mathcal{L}_{cycleT2G}$
(6)

The [Eq 6] resembles the supervised one [Eq 2], however, with predicted inputs
from the unlabeled samples. Notice here that there is no-alignment between
graphs and text from both $\mathcal{U}_{\mathcal{G}}$ and
$\mathcal{U}_{\mathcal{T}}$. On every cycle step, our model uses its
predictions as inputs, which are synthetic and so can change on every
prediction. This variability helps the model to avoid over-fitting training
data.

Before training our model ($z_{\alpha}$) on the unsupervised data, we have to
apply a fine-tuning step to learn a representation of these tasks. To do so,
we optimise our model, over $\alpha$, on a maximum likelihood estimation [Eq
7], in which both tasks are simultaneously considered - following our multi-
task approach.

$\displaystyle\alpha^{*}=argmax_{\alpha}\prod_{(g,t)\sim S}p(t\shortmid
g;\alpha)\cdot p(g\shortmid t;\alpha)$ (7)

Finally, we can train our model on the unsupervised samples through the cycle
training framework. On every cycle training step, the parameters of the model
are optimised using a maximum likelihood estimation [Eq 8], where synthetic
samples are the inputs, but human written text and graphs are the reference
respectively.

$\displaystyle\alpha^{*}=argmax_{\alpha}\prod_{(\hat{g},t)\sim\mathcal{U}_{\mathcal{T}}}p(t\shortmid\hat{g};\alpha)\prod_{(g,\hat{t})\sim\mathcal{U}_{\mathcal{G}}}p(g\shortmid\hat{t};\alpha)$
(8)

##  5 Experimental Framework

Dataset. The WebNLG corpus Gardent et al. (2017) is a common benchmark on
which to evaluate and compare G2T & T2G
systems333https://gitlab.com/shimorina/webnlg-dataset. In 2020, they released
the latest version of the corpus Ferreira et al. (2020), which included
Russian language and data for T2G (parsing). The latest English version of the
corpus is the 3.01 version, in which the training and development sets
comprise 16 different DBpedia categories. Apart from that, the test sets have
three different data types:

  * •

Seen categories: triples containing the entities and categories seen in the
training data.

  * •

Unseen entities: triples containing the categories seen in the training data,
but not entities.

  * •

Unseen categories: triples containing the categories not present in the
training data.

Test split’s statistics for both tasks are shown in [Table 2].

| Overall | Seen Categories | Unseen Entities | Unseen Categories
---|---|---|---|---
| F1 | Precision | Recall | F1 | Precision | Recall | F1 | Precision | Recall | F1 | Precision | Recall
bt5 (♢△) Agarwal et al. (2020) | 0.675 | 0.663 | 0.695 | 0.877 | 0.875 | 0.880 | 0.645 | 0.614 | 0.697 | 0.539 | 0.528 | 0.555
CycleGT (♡) Guo et al. (2020b) | 0.309 | 0.306 | 0.315 | 0.545 | 0.538 | 0.558 | 0.179 | 0.178 | 0.182 | 0.181 | 0.179 | 0.183
T5-Baseline (♢△) [our implementation] | 0.394 | 0.389 | 0.401 | 0.429 | 0.426 | 0.433 | 0.459 | 0.456 | 0.464 | 0.345 | 0.335 | 0.356
T5-Finetune (♢△) [our implementation] | 0.179 | 0.179 | 0.182 | 0.171 | 0.170 | 0.173 | 0.175 | 0.175 | 0.178 | 0.201 | 0.201 | 0.205
T5-MSSF (♢⨝) [our framework] | 0.431 | 0.432 | 0.431 | 0.428 | 0.439 | 0.423 | 0.476 | 0.476 | 0.476 | 0.408 | 0.407 | 0.416

Table 3: Multi-task (${\diamondsuit}$). Unspervised (${\heartsuit}$). Semi-
Supervised ($\Join$). Supervised (${\triangle}$). Summary of related work
results in Text-to-Graph (T2G) (in strict matching schema) with multi-task,
supervised and/or unsupervised learning. Our multi-task semi-supervised
framework is T5-MSSF.

Non-parallel Dataset. Cycle training makes use of large unlabeled data, so we
collected two datasets, one for natural English text and another one for
graphs in English language. These datasets are extracted by means of our
crawling-scraping algorithm which uses Wikipedia444https://www.wikipedia.org
and Wikidata555https://www.wikidata.org for content retrieval as described:

  1. 1.

In the first iteration, we have to define an origin entity, but afterwards,
the crawling algorithm will select the last entity (LIFO policy) following a
Depth First Search strategy.

  2. 2.

Send query, SPARQL-based, to the Wikidata - API service to retrieve all the
graphs in which their subject is the queried entity.

  3. 3.

Scrape at most the 4th first paragraphs of entity’s page in Wikipedia.

  4. 4.

For each object in the retrieved graph, go to the first step, unless it has
reached a depth666This depth represents the number of parents already analysed
w.r.t to the current entity. of 5 or it has already been crawled.

By executing this crawling algorithm 4 times and pre-processing all data, we
extracted a total of 176,000 unique entities with 532,288 triples and 240,024
paragraphs. Finally, we built two datasets, one for the graphs with 271,095
instances (1 to 6 triples per each), and another one for the natural text,
with 240,024 instances (one sentence or more per each) of 459.67characters on
average length.

Models. T5-Baseline is trained with all supervised samples from the WebNLG
dataset. T5-MSSF is finetuned with 15% of the supervised data, but then it is
cycle-trained with 30,000 different synthetic samples on each iteration along
the supervised ones. Similary, T5-Finetune is trained with 15% of the
supervised data, however, the cycle-training part is removed to asses the
advantages of such approach.

Training Details. We trained on mini-batches (8 samples) with accumulation
steps (4 steps), at a learning rate of $2.0e^{-4}$ (fine-tuning) and
$1.0e^{-5}$ (cycle training). The maximum number of epochs are set to 50
(fine-tuning) and 30 (cycle training) with 5 epochs of patience. The maximum
source and target length is limited to 64 tokens with 4-beam search.
Repetition and length penalty are applied, 2.5 and 1.0 respectively, along
early stopping. The number of cycle steps are restricted to 3. For the sake of
simplicity, each iteration of the cycle training is configured with the same
hyper-parameters.

Evaluation Metrics. On one hand, G2T performance is studied with the following
metrics: BLEU Papineni et al. (2002), chrF++ Popović (2015); the greater, the
better performance, and TERSnover et al. (2006); the lower, the better
performance. On the other hand, T2G performance is analysed with the well-
known metrics: F1, Precision and Recall, on strict measurement, i.e. it
evaluates an exact match of the hypothetical triple with the reference triple.

##  6 Results

As shown in Table 1, for G2T generation in overall analysis, even with only
15% of pairing information between text and graphs, our T5-MSSF model (multi-
task semi-supervised) can achieve a 45.3 BLEU score, which improves the
performance of our model trained on the full supervised data, T5-Baseline, and
the CycleGT model (unsupervised). Similarly, T5-MSSF attains the best
performance on TER and chrF++ within our work, but fails to surpass
performance from both CycleGT (unsupervised) and bt5 (multi-task) models on
these metrics. There is an interesting pattern that reveals that unsupervised
or semi-supervised learning perform more similar across the different data
types (seen categories, unseen entities and unseen categories) than supervised
ones. For instance, bt5 and T5-Baseline reach differences of 17.1 and 13.7
BLEU points respectively, between the best (seen categories) and the worst
(unseen categories) performance of themselves. However, CycleGT and T5-MSSF
obtain differences of 6.5 and 10.9 BLEU points respectively between the same
data types. Probably, given the same performance on the overall, then, for
unseen domains these unsupervised learning methods are more likely to
generalize better.

In Table 3, T2G parsing, T5-MSSF reaches best performance among our models and
surpasses again CycleGT in unsupervised learning by a 39.4% F1-Score
improvement in the overall analysis. In fact, T5-MSSF greatly exceeds (140.7%
F1-Score improvement) T5-Finetune performance, however, it is not yet capable
of reaching bt5 performance. As previously observed, the performance of our
multi-task semi-supervised model across the different data types is very
similar, reaching at most a difference of 0.068 F1-Score points between the
best (unseen entities) and the worst (unseen categories). Contrary, bt5 model
presents a difference of 0.338 F1-Score points between the best (seen
categories) and the worst (unseen categories), but even the unsupervised
method, CycleGT, holds a difference of 0.364 F1-Score points between the same
data types. This trend might suggests that synthetic data distribution in our
non-parallel samples helps our framework to better generalize than using
decoupled parallel data in which there exists a real distribution, as CycleGT
does.

##  7 Conclusion

This paper proposed a multi-task semi-supervised framework based on the
T5-Base model for Graph-to-Text & Text-to-Graph. Our cycle framework improves
unsupervised state-of-the-art resuts on both tasks using only 15% of
supervised examples, but in comparison to supervised models there is still
room for improvement. This framework consists in translating text to graphs,
and vice-versa, using the model itself, so it obtains parallel data from fully
non-parallel samples. These samples are added (optionally) to the original
parallel data before training the final model. This approach resulted to be
very convenient to obtain similar performance across the different domains.
Furthermore, the resulting model can be trained in any new domain with non-
parallel data, by simply adding text and graphs in our framework. We release
our collected non-parallel dataset. Thus, future work can research whether
this framework would be useful for lifelong learning.

## References

  * Agarwal et al. [2020] Oshin Agarwal, Mihir Kale, Heming Ge, Siamak Shakeri, and Rami Al-Rfou.  Machine translation aided bilingual data-to-text generation and semantic parsing.  In Proceedings of the 3rd International Workshop on Natural Language Generation from the Semantic Web (WebNLG+), pages 125–130, 2020.
  * Ait-Mlouk and Jiang [2020] Addi Ait-Mlouk and Lili Jiang.  Kbot: A knowledge graph based chatbot for natural language understanding over linked data.  IEEE Access, PP:1–1, 08 2020.
  * Bellegarda [2013] Jerome R Bellegarda.  Large-scale personal assistant technology deployment: the siri experience.  In INTERSPEECH, pages 2029–2033, 2013.
  * Crawshaw [2020] Michael Crawshaw.  Multi-task learning with deep neural networks: A survey.  arXiv preprint arXiv:2009.09796, 2020.
  * Ferreira et al. [2020] Thiago Ferreira, Claire Gardent, Nikolai Ilinykh, Chris van der Lee, Simon Mille, Diego Moussallem, and Anastasia Shimorina.  The 2020 bilingual, bi-directional webnlg+ shared task overview and evaluation results (webnlg+ 2020).  In Proceedings of the 3rd International Workshop on Natural Language Generation from the Semantic Web (WebNLG+), 2020.
  * Gardent et al. [2017] Claire Gardent, Anastasia Shimorina, Shashi Narayan, and Laura Perez-Beltrachini.  Creating training corpora for nlg micro-planning.  In 55th annual meeting of the Association for Computational Linguistics (ACL), 2017.
  * Goodfellow et al. [2014] Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial networks.  arXiv preprint arXiv:1406.2661, 2014.
  * Guo et al. [2020a] Qingyu Guo, Fuzhen Zhuang, Chuan Qin, Hengshu Zhu, Xing Xie, Hui Xiong, and Qing He.  A survey on knowledge graph-based recommender systems.  IEEE Transactions on Knowledge and Data Engineering, pages 1–1, 2020.
  * Guo et al. [2020b] Qipeng Guo, Zhijing Jin, Xipeng Qiu, Weinan Zhang, David Wipf, and Zheng Zhang.  Cyclegt: Unsupervised graph-to-text and text-to-graph generation via cycle training.  arXiv preprint arXiv:2006.04702, 2020.
  * Hochreiter and Schmidhuber [1997] Sepp Hochreiter and Jürgen Schmidhuber.  Long short-term memory.  Neural computation, 9(8):1735–1780, 1997.
  * Ji and Grishman [2011] Heng Ji and Ralph Grishman.  Knowledge base population: Successful approaches and challenges.  In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pages 1148–1158, 2011\.
  * Papineni et al. [2002] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu.  Bleu: a method for automatic evaluation of machine translation.  In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318, 2002.
  * Popović [2015] Maja Popović.  chrf: character n-gram f-score for automatic mt evaluation.  In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, 2015.
  * Qi et al. [2020] Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D Manning.  Stanza: A python natural language processing toolkit for many human languages.  arXiv preprint arXiv:2003.07082, 2020.
  * Raffel et al. [2019] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu.  Exploring the limits of transfer learning with a unified text-to-text transformer.  arXiv preprint arXiv:1910.10683, 2019.
  * Schmitt et al. [2020] Martin Schmitt, Sahand Sharifzadeh, Volker Tresp, and Hinrich Schütze.  An unsupervised joint system for text generation from knowledge graphs and semantic parsing.  In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7117–7130, 2020.
  * Sennrich et al. [2015] Rico Sennrich, Barry Haddow, and Alexandra Birch.  Improving neural machine translation models with monolingual data.  arXiv preprint arXiv:1511.06709, 2015.
  * Sharath and Banafsheh [2021] Japa Sai Sharath and Rekabdar Banafsheh.  Conversational question answering over knowledge base using chat-bot framework.  2021 IEEE 15th International Conference on Semantic Computing (ICSC), pages 84–85, 2021.
  * Snover et al. [2006] Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul.  A study of translation edit rate with targeted human annotation.  In Proceedings of association for machine translation in the Americas, volume 200. Citeseer, 2006.
  * Tiedemann [2012] Jörg Tiedemann.  Parallel data, tools and interfaces in opus.  In Lrec, volume 2012, pages 2214–2218. Citeseer, 2012.
  * Vaswani et al. [2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  arXiv preprint arXiv:1706.03762, 2017.
  * Veit et al. [2017] Andreas Veit, Neil Alldrin, Gal Chechik, Ivan Krasin, Abhinav Gupta, and Serge Belongie.  Learning from noisy large-scale datasets with minimal supervision.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 839–847, 2017.
  * Zhu et al. [2017] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros.  Unpaired image-to-image translation using cycle-consistent adversarial networks.  In Proceedings of the IEEE international conference on computer vision, pages 2223–2232, 2017.


# On a Generalized Framework for Time-Aware Knowledge Graphs

Franz Krause Corresponding Author. E-mail<EMAIL_ADDRESS>Tobias
Weller  Heiko Paulheim  Data and Web Science Group, University of Mannheim,
Germany

###### Abstract

Knowledge graphs have emerged as an effective tool for managing and
standardizing semistructured domain knowledge in a human- and machine-
interpretable way. In terms of graph-based domain applications, such as
embeddings and graph neural networks, current research is increasingly taking
into account the time-related evolution of the information encoded within a
graph. Algorithms and models for stationary and static knowledge graphs are
extended to make them accessible for time-aware domains, where time-awareness
can be interpreted in different ways. In particular, a distinction needs to be
made between the validity period and the traceability of facts as objectives
of time-related knowledge graph extensions. In this context, terms and
definitions such as dynamic and temporal are often used inconsistently or
interchangeably in the literature. Therefore, with this paper we aim to
provide a short but well-defined overview of time-aware knowledge graph
extensions and thus faciliate future research in this field as well.

###### keywords:

Knowledge Graph Dynamic Knowledge Graph Temporal Knowledge Graph Time-Aware
Knowledge Graph Semantic Web

, and

##  1 Introduction

Knowledge graphs (KGs) and their integration into domain-specific use cases
represent a topic that has been gaining popularity in recent research. Their
inherent information is usually encoded in the form of triples $(h,r,t)$
$\widehat{=}$ $(head,relation,tail)$ where a node $h$ has the relation $r$ to
another node or attributive literal $t$. KGs are used to improve the
performance in areas like question answering [1] and recommendation [2]
regarding various domains, e.g., industrial manufacturing [3] and biomedicine
[4]. Furthermore, extension approaches exist which aim at enriching triples
with additional metadata [5, 6, 7], such as annotations or timestamps.
However, discrepancies regarding terminology can be found in the literature.
For example, the term knowledge graph is often used interchangeably, although
enrichment by metadata usually cannot be assumed without loss of generality.
Furthermore, especially in the context of time-aware knowledge graph
extensions, frequently used terms such as dynamic, temporal, and static are
applied inconsistently.

Therefore, in this work, the distinction between standard knowledge graphs by
means of stationary sets of triples [8, 9] and time-aware KG extensions is
elaborated. Reminiscent, mutable and incremental knowledge graphs are
introduced as special cases of dynamic and temporal KGs. These definitions
should ultimately serve to standardize time-aware KG extensions and thus
facilitate future research in this field as well.

##  2 Related Work

State of the art knowledge graphs as sets of triples with entries $(h,r,t)$
already provide a limited possibility of expressing time-awareness by
assigning additional time-related annotations to nodes [9]. For example, the
information that the European Union (EU) was founded in $1951$ can be encoded
by $(EU,founded,1951)$ where $1951$ is an attributive timestamp literal, in
this case the corresponding year. However, time-related node annotations are
not sufficient to encode the information that the United Kingdom (UK) joined
the EU in $1973$ since the year $1973$ refers to the edge $(UK,member,EU)$ and
not to a single node. To provide the encoded triples with further information,
general approaches like RDF* [5] already exist which assign additional
metadata to the edges. In the following, we restrict ourselves to time-related
metadata in order to standardize time-aware KG extensions. In fact, there are
already numerous works dealing with this problem, but most of them consider
successive applications such as KG embeddings [6, 10] or graph completion [11,
12]. However, although knowledge graphs with additional time-related metadata
within the edges are always considered, there is no generalized definition for
this kind of encoding. As most of these approaches refer to such graphs as
temporal KGs, we adopt this notion as well. In particular, temporal extensions
represent a local form of time-awareness as timestamps are added to each edge
individually.

Additionally, several works exist which consider entire knowledge graphs as
being non-stationary, i.e. dynamic, which is to be interpreted as global time-
awareness. Similarly to temporal KGs, these apply methods for standard
knowledge graphs to dynamic KGs to make them accessible for areas such as KG
embeddings [13, 14] and KG completion [15]. However, these approaches do not
attempt to adapt the original methods to enriched knowledge graphs. Rather,
they try to make previous models and results reusable in an efficient way so
that, for example, full retraining of an embedding model is not required after
the information encoded within the knowledge graph is updated.

To the best of our knowledge, no general definitions of temporal or dynamic
KGs exist yet. Usually, respective assumptions are similar but not identical.
Furthermore, there are several works where these terms are used
interchangeably, inversely, or not at all. Finally, there is no well-defined
approach for combining local and global time-awareness.

##  3 Preliminaries

In this work, the term knowledge graph is used as a generalized notion for
approaches that manage semistructured data based on formal conceptualizations,
e.g., ontologies, as well as collections of instantiation rules indicating the
validity of a graph’s topology, i.e., of its inherent edges. However, this
generalization is based on the most common implementation form of a knowledge
graph as a set of triples, which will be referred to as a standard KG.
Furthermore, ontologies which conceptualize a domain by means of triples
$(h,r,t)$ are referred to as static ontologies as they allow no further
extensions of the triple structure such that facts are to be regarded as final
and static. Moreover, a KG may be interpreted in both stationary and dynamic
ways, i.e., we allow the consideration of time-related graph evolutions with
respect to a set of timestamps $\mathcal{T}$. We assume $\mathcal{T}$ to be of
a strict order, so that for $\tau,\tau^{\prime}\in\mathcal{T}$ with
$\tau\neq\tau^{\prime}$ either $\tau<\tau^{\prime}$ or $\tau^{\prime}<\tau$
follows. Therefore, for $\tau<\tau^{\prime}$ the timestamp $\tau$ occurred
before $\tau^{\prime}$. In this context, we also define the closure
$\overline{\mathcal{T}}:=\mathcal{T}\cup\\{-\infty,\infty\\}$ such that
$-\infty<\tau<\infty$ holds for all $\tau\in\mathcal{T}$.

Figure 1: Overview of the time-aware knowledge graph extensions regarding
possible combinations of local extensions (edge timestamps) and global
extensions (consideration of multiple consecutive versions of a graph).

##  4 Time-Aware Knowledge Graph Extensions

In this section, a generalized framework for extending KGs with time-awareness
is introduced, which intends to cover existing and future work in this field.
As indicated in Section 2, we are concerned with local extensions, i.e.,
enriching edges with timestamps, as well as global extensions, i.e.,
considering the evolution of a graph. The overview in Figure 1 shows the
different types of time-aware KGs introduced in this paper. According to the
most common terms in the literature, we refer to locally extended KGs as
temporal and globally extended KGs as dynamic. If the respective extension is
not considered, then the KG is referred to as static and/or stationary. For
example, a standard KG is static because all edges in the graph must be
interpreted as static triples, and stationary because it models domain
knowledge with respect to a fixed point in time.

We investigate different types of KG extensions. A reminiscent KG models the
domain knowledge for a fixed point in time, but is provided with memory in the
form of additional edge timestamps. Regarding a fixed timestamp, mutable KGs
do not contain this memory, but can be observed over the time period
$\mathcal{T}$. As a combination, incremental KGs are equipped with additional
edge metadata and are observable with respect to $\mathcal{T}$.

###  4.1 Stationary Knowledge Graphs

Standard KGs represent special cases of static and stationary KGs as they are
stationary instantiations of static ontologies
$\mathcal{O}=(\mathcal{C},\mathcal{L},\mathcal{R},\rho)$. Such ontologies
include concepts $\mathcal{C}$, i.e. entity types, attributive literals
$\mathcal{L}$, as well as attributive and contextual relations in
$\mathcal{R}$. In addition, $\rho$ denotes the instantiation rules of
$\mathcal{O}$ which assess whether triples are valid or not, based on the
triples themselves and the topology of the graph. For example, a member of the
$EU$ must necessarily be a country. Given such an ontology $\mathcal{O}$, a
triplestore including a set of entites $V$ and a set of edges $E$ with entries
$(h,r,t)$ is called a standard knowledge graph $\mathcal{G}=(V,E)$ if $h\in
V$, $r\in\mathcal{R}$, and $t\in V$ or $t\in\mathcal{L}$ holds such that the
validity of triples can be assessed using the rules in $\rho$. In some works,
blank nodes are considered as head or tail nodes, but we omit this here
without loss of generality, since they can always be added by including the
concept of a blank node in $\mathcal{C}$. To extend the facts in $\mathcal{G}$
with additional timestamps, an adaptation of static ontologies is required.

######  Definition 1 (Temporal Ontology)

Given a time set $\mathcal{T}$ as well as a standard ontology
$\mathcal{O}=(\mathcal{C},\mathcal{L},\mathcal{R},\rho)$, a temporal ontology
is defined as
$\mathcal{O}^{+}=(\mathcal{C},\mathcal{L},\mathcal{R},\mathcal{T},\rho^{+})$
such that triples $(h,r,t)$ are replaced by quintuples
$(h,r,t,\tau_{start},\tau_{end})$ with additional timestamps
$\tau_{start},\tau_{end}\in\overline{\mathcal{T}}$, defining the start and end
of validity of an edge. Further, the instantiation rules $\rho$ are extended
by time-related rules which determine the validity of quintuples.

Regarding the additional time-related rules in $\rho^{+}$, there are some
obvious rules, such as the one that $\tau_{start}\leq\tau_{end}$ should always
hold. To ensure this rule also for quintuples for which no additional time-
related information is available, we assume the previously introduced closure
$\overline{\mathcal{T}}$, so that $\tau_{start}=-\infty$ or
$\tau_{end}=\infty$ may be used if necessary. By means of
$\mathcal{T}=\emptyset$ and $\overline{\mathcal{T}}=\\{-\infty,\infty\\}$, one
also recognizes that static ontologies represent special cases of temporal
ontologies. Therefore, with respect to temporal ontologies, we always assume
$\mathcal{T}\neq\emptyset$. Accordingly, a real-world instantiation of a
temporal ontology for a fixed timestamp is called a reminiscent knowledge
graph which is defined as follows.

######  Definition 2 (Reminiscent Knowledge Graph)

Let
$\mathcal{O}^{+}=(\mathcal{C},\mathcal{L},\mathcal{R},\mathcal{T},\rho^{+})$
be a temporal ontology. Then a quintuple store including entites $V$ and a set
of edges $E^{+}$ with entries $(h,r,t,\tau_{start},\tau_{end})$ is a
reminiscent knowledge graph $\mathcal{G}^{+}=(V,E^{+})$ if $h\in V$,
$r\in\mathcal{R}$, $t\in V$ or $h\in\mathcal{L}$, and
$\tau_{start},\tau_{end}\in\overline{\mathcal{T}}$ holds and there are no
violations of the rules in $\rho^{+}$.

Similar graph implementations as special cases of the above definition are
already used in existing works, for example to optimize KG embeddings with
temporal aspects or to integrate past information in a KG and successive
applications [10, 12]. Since often only single timestamps $\tau_{start}$ are
considered, we introduce the notion of semi-temporality which is present if
$\tau_{end}=\infty$ holds for all quintuples. Accordingly, we introduce semi-
reminiscent KGs as special cases of reminiscent KGs. However, in this case,
the deactivation of an edge inevitably leads to its deletion, i.e., only
active edges are present.

###  4.2 Dynamic Knowledge Graphs

Unlike standard and reminiscent knowledge graphs as stationary domain
representations, many KG applications are meant to go beyond the original
encoding of semistructured data for a fixed point in time. For example, KG
embeddings and graph neural networks are supposed to be adaptive such that
they can be efficiently reused after the topology of the KG is updated [13,
14]. Therefore, it is necessary to consider KG representations that are
dynamic with respect to a set of timestamps $\mathcal{T}$, which justifies the
following definition of a dynamic KG as a mapping from $\mathcal{T}$ to an
appropriate set of KGs.

######  Definition 3 (Dynamic Knowledge Graph)

We assume a set of timestamps $\mathcal{T}$, a set of either static or
temporal ontologies $\\{\mathcal{O}_{\tau}~{}:~{}\tau\in\mathcal{T}\\}$ with
corresponding sets $\mathbb{G}_{\tau}$ of all stationary KGs according to
$\mathcal{O}_{\tau}$ and we define
$\mathbb{G}:=\bigcup_{\tau\in\mathcal{T}}\mathbb{G}_{\tau}$. Then, a dynamic
knowledge graph is defined as a mapping
$\Gamma:\mathcal{T}\rightarrow\mathbb{G}$ such that
$\Gamma(\tau)\in\mathbb{G}_{\tau}$ holds for all $\tau\in\mathcal{T}$.

Apparently, Definition 3 does not specify whether static or temporal
ontologies are considered. Figure 1 suggests that this may result in different
types of time-aware KG extensions, which we discuss in the following. First,
we assume static ontologies.

######  Definition 4 (Mutable Knowledge Graph)

A dynamic knowledge graph $\Gamma$ is called a mutable knowledge graph if the
underlying ontologies $\\{\mathcal{O}_{\tau}:~{}\tau\in\mathcal{T}\\}$ are
static ontologies and $\Gamma(\tau)$ yields a standard knowledge graph for all
$\tau\in\mathcal{T}$.

A mutable KG thus offers the possibility to infer when triples were added to
the graph, i.e., when the inherent information became accessible. However, the
stationary images $\Gamma(\tau)$ for $\tau\in\mathcal{T}$ are static and
therefore do not contain information about the period of validity of an edge.
Thus, regarding time-awareness in general, one has to distinguish between the
validity and the accessibility of facts to the knowledge graph.

######  Definition 5 (Incremental Knowledge Graph)

A dynamic KG $\Gamma$ is called an incremental knowledge graph and is denoted
as $\Gamma^{+}$ if the underlying ontologies
$\\{\mathcal{O}_{\tau}:~{}\tau\in\mathcal{T}\\}$ are temporal ontologies and
$\Gamma^{+}(\tau)$ yields a reminiscent KG for all $\tau\in\mathcal{T}$.

An incremental KG thus offers the possibility to trace the accessibility of
facts as well as their validity. Since deactivated edges are kept in the
graph, for timestamps $\tau,\tau^{\prime}\in\mathcal{T}$ with
$\tau<\tau^{\prime}$, $\Gamma^{+}(\tau^{\prime})$ contains at least as many
facts as $\Gamma(\tau)$. Therefore, according to the notion of semi-
temporality from Section 4.1, we also allow semi-incremental KGs as special
cases of incremental KGs whose stationary images $\Gamma^{+}(\tau)$ are semi-
reminiscent, i.e., they only contain active facts. However, assuming the prior
existence of an edge, the time of its deletion is reconstructable by
inspecting the previous versions of the graph.

###  4.3 Application Example

As indicated in Section 2, standard KGs are not sufficient to encode
temporally extended facts like the membership of the UK in the EU from $1973$
to $2020$. Therefore, reminiscent KGs with additional timestamps
$\tau_{start},\tau_{end}\in\overline{\mathcal{T}}$ are considered so that the
fact is encodable as $(UK,member,EU,1973,2020)$. In addition, if only active,
i.e. currently valid edges are to be contained in the graph, a semi-
reminiscent graph can be implemented.

However, stationary KGs do not not trace the time-related evolution of the
information encoded within a graph, since they only contain the knowledge
available for a fixed timestamp. Therefore, dynamic KGs are introduced in this
paper to account for such evolutions. In Table 1, the different dynamic
extension types are exemplified. Thus, in 2012, the information that the UK is
a member of the EU since $1973$ is added to each KG. However, the year $1973$
is only explicitly encodable in the (semi-)incremental KGs such that this
additional information is still available in $2020$. Due to its staticness,
this is not the case for the mutable KG. After the Brexit in $2020$, the
information about the previous membership is removed from the mutable and the
semi-incremental KG, since they only contain active edges. In the incremental
KG, on the other hand, the fact is updated by means of the new timestamp
$\tau_{end}=2020$, making it available for future timestamps $\tau>2020$ as
well. However, this also leads to a continual expansion of the graph.

###  4.4 Time-Awareness in Existing Knowledge Graphs and Applications

Many existing and established knowledge graphs such as DBpedia, YAGO, and
Wikidata already satisfy certain requirements for time-awareness. Indeed, KGs
are mostly based on dynamic domains such that a versioning or the
implementation and utilization of a SPARQL update endpoint results in a
dynamic KG. However, these dynamics are mostly not explicitly considered, but
rather the stationary images of the graphs. In particular, successive
applications, such as embeddings and graph neural networks, are developed or
trained for stationary images without taking the underlying dynamics into
account.

Table 1: Application of dynamic, i.e., mutable (m), semi-incremental (s-i) and incremental (i) KG extensions. | 2012 | … | 2020 | 2021
---|---|---|---|---
m | $(UK,member,EU)$ |  | $(UK,member,EU)$ | -
s-i | $(UK,member,EU,1973,\infty)$ |  | $(UK,member,EU,1973,\infty)$ | -
i | $(UK,member,EU,1973,\infty)$ |  | $(UK,member,EU,1973,\infty)$ | $(UK,member,EU,1973,2020)$

##  5 Summary

This work contributes to the Semantic Web community by providing a generalized
framework for time-aware knowledge graph extensions. Current research shows
that further progress is needed to establish knowledge graphs in non-
stationary and non-static domains. In this context, some promising approaches
already exist that extend or adapt methods for standard knowledge graphs to
make them usable for dynamic or temporal KGs as well. However, so far, these
approaches do not share a common vocabulary to compare their respective
results. The definitions introduced in this work provide this kind of
vocabulary to facilitate the comparison and integration of existing, but also
future works and thus can serve as an accelerator for the desired progress of
knowledge graph implementations and applications in time-aware, i.e., dynamic
and temporal domains.

Acknowledgements. This work is part of the TEAMING.AI project which receives
funding in the European Commission’s Horizon 2020 Research Programme under
Grant Agreement Number 957402 (www.teamingai-project.eu).

## References

  * [1] Diefenbach D, Giménez-García J, et al.  QAnswer KG: Designing a Portable Question Answering System over RDF Data.  In: The Semantic Web: ESWC 2020; 2020. p. 429-45.
  * [2] Palumbo E, Rizzo G, et al.  Knowledge Graph Embeddings with node2vec for Item Recommendation.  In: The Semantic Web: ESWC 2018 Satellite Events; 2018. p. 117-20.
  * [3] Bader SR, Grangel-Gonzalez I, et al.  A Knowledge Graph for Industry 4.0.  In: The Semantic Web: ESWC 2020; 2020. p. 465-80.
  * [4] Vidal ME, Endris KM, et al.  Semantic Data Integration of Big Biomedical Data for Supporting Personalised Medicine.  In: Current Trends in Semantic Web Technologies: Theory and Practice. Springer International Publishing; 2019. p. 25-56.
  * [5] Hartig O.  Foundations of RDF$\star$ and SPARQL$\star$ (An Alternative Approach to Statement-Level Metadata in RDF).  In: Proceedings of the 11th Alberto Mendelzon International Workshop on Foundations of Data Management and the Web. 2017.
  * [6] Liu Y, Hua W, et al.  Context-Aware Temporal Knowledge Graph Embedding.  In: Web Information Systems Engineering – WISE 2019; 2019. p. 583-98.
  * [7] Pelgrin O, Galárraga L, Hose K.  TrieDF: Efficient In-memory Indexing for Metadata-augmented RDF.  In: Proceedings of the 7th Workshop on Managing the Evolution and Preservation of the Data Web (MEPDaW) co-located with ISWC 2021; 2021. p. 1-10.
  * [8] Ehrlinger L, Wöß W.  Towards a Definition of Knowledge Graphs.  In: Joint Proceedings of the Posters and Demos Track of SEMANTiCS2016 and SuCCESS’16. 2021.
  * [9] Hogan A, Blomqvist E, et al.  Knowledge Graphs.  ACM Comput Surv. 2021;54(4).
  * [10] Xu C, Nayyeri M, et al.  Temporal Knowledge Graph Completion Based on Time Series Gaussian Embedding.  In: The Semantic Web – ISWC 2020; 2020. p. 654-71.
  * [11] García-Durán A, Dumančić S, Niepert M.  Learning Sequence Encoders for Temporal Knowledge Graph Completion.  In: Proceedings of the EMNLP 2018; 2018. p. 4816-21.
  * [12] Xu C, Chen YY, et al.  Temporal Knowledge Graph Completion using a Linear Temporal Regularizer and Multivector Embeddings.  In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies; 2021. p. 2569-78.
  * [13] Tay Y, Luu A, et al.  Non-Parametric Estimation of Multiple Embeddings for Link Prediction on Dynamic Knowledge Graphs.  Proceedings of the AAAI Conference on Artificial Intelligence. 2017;31(1).
  * [14] Wu T, Khan A, et al.  Efficiently Embedding Dynamic Knowledge Graphs.  ArXiv. 2019; abs/1910.06708.
  * [15] Xie W, Wang S, et al.  Dynamic Knowledge Graph Completion with Jointly Structural and Textual Dependency.  In: Algorithms and Architectures for Parallel Processing; 2020. p. 432–448.


# The Extended Thermodynamic Properties of
a topological Taub-NUT/Bolt-AdS spaces

Chong Oh Lee<EMAIL_ADDRESS>Department of Physics, Kunsan National
University, Kunsan 573-701, Korea

###### Abstract

We consider higher dimensional topological Taub-NUT/Bolt-AdS solutions where a
cosmological constant is treated as a pressure. The thermodynamic quantities
of these solutions are explicitly calculated. Furthermore, we find these
thermodynamic quantities satisfy the Clapeyron equation. In particular, a new
thermodynamically stable region for the NUT solution is found by studying the
Gibbs free energy. Intriguingly, we also find that like the AdS black hole
case, the $G-T$ diagram of the Bolt solution has two branches which are joined
at a minimum temperature. The Bolt solution with the large radius, at the
lower branch, becomes stable beyond a certain temperature while the Bolt
solution with the small radius, at the upper branch, is always unstable.

##  I Introduction

It has been found that the area of the event horizon of a black hole is
proportional to its physical entropy in search for similarities between black
hole physics and thermodynamics Bekenstein:1973ur . It has been successively
suggested that by using the thermodynamic relationship between the thermal
energy, temperature, and entropy, the first law of black hole thermodynamics
can be expressed in the similar forms to the first law of standard
thermodynamics Bardeen:1973gs . It has been also found that there is a phase
transition in the Schwarzschild-AdS black hole through investigations of
complete analogy between black hole system and standard thermodynamic system
Hawking:1982dh . Since then, it has been studied for the phase transitions and
critical phenomena in a variety of black hole solutions Lousto:1994jd
-Myung:2012xc  and extensively investigated in various thermodynamic issues of
black hole in higher dimensional AdS space Chamblin:1999tk -Banerjee:2011au .

It has been recently suggested that by considering $(d+1)$-dimensional AdS
black holes, the thermodynamic pressure $p$ is given by

$\displaystyle p=-\frac{1}{8\pi}\Lambda=\frac{u(2u+1)}{8\pi l^{2}},$ (1)

in units where $G=c=\hbar=k_{B}=1$, and $u$ is of the form $d+1=2u+2$ with a
positive integer $u$. Several series of relevant investigations have been
performed Kastor:2009wy -Belhaj:2015hha .

Recently, it has been shown that the thermodynamic volume in the Taub-NUT-AdS
case can be negative. This negative thermodynamic volume may be interpreted in
that the environment (universe) applies work to the system (Taub-NUT-AdS black
hole) in the process of the Taub-NUT-AdS black hole formation, while the
positive thermodynamic volume may be interpreted as applying the work on the
environment (universe) by the system (the whole black hole) considering the
process of forming the black hole Johnson:2014yja . They also have found that
there is the first order phase transition from Taub-NUT-AdS to Taub-Bolt-AdS
with considering the phase structure of these black holes Johnson:2014pwa .
This issue has been studied for the Kerr-Bolt-AdS case MacDonald:2014zaa  and
investigated extensively in higher dimensional NUT/Bolt case. In these higher
dimensional cases, it has been particularly found that the Taub-NUT-AdS
solution has a thermodynamically stable range as a function of the temperature
for any odd $u$ and there is the transition from Taub-NUT-AdS to Taub-Bolt-AdS
for all odd $u$ only Lee:2014tma .

Furthermore, it has been shown that this higher dimensional NUT/Bolt case with
a discrete parameter $k$ can be generalized Mann:2005ra . In the context of
the extended thermodynamics, the thermodynamic properties of the case $k=1$
has been studied Lee:2014tma  only. Thus, it would be interesting to be a
similar discussion of the generalizations in the $k=0,-1$ topological
solutions. More intriguingly, their thermodynamic phase structure would be
investigated through exploring the behaviour of the Gibbs free energy since
understanding its behaviour is essential for uncovering possible thermodynamic
phase transitions. In this paper, we address these questions.

The paper is organized as follows: in the next section we investigate
thermodynamic properties in topological Taub-NUT/Bolt-AdS spaces for any $u$.
We explicitly obtain the general forms of thermodynamic quantities such as the
entropy, the enthalpy, the specific heat, the temperature, the thermodynamic
volume, the Gibbs free energy, and the latent heat. In particular, by
introducing the Gibbs free energy we discuss their phase structure and their
instability. In the last section we give our conclusion.

##  II Topological Taub-NUT/Bolt-AdS Spaces

We consider topological Taub-NUT/Bolt-AdS metric in higher dimensional
spacetime and the general solution in the Euclidean section is given by
Chamblin:1998pz -Astefanesei:2004kn  (for the generalized versions of the
issue, see e.g., Mann:2005ra )

$\displaystyle\textstyle ds^{2}$ $\displaystyle=$ $\displaystyle\textstyle
f(r)\Big{\\{}dt_{E}+4N\sum_{i=1}^{u}f_{k}^{2}(\frac{\theta_{i}}{2})d\phi_{i}\Big{\\}}^{2}+\frac{dr^{2}}{f(r)}$
(2)
$\displaystyle+\textstyle(r^{2}-N^{2})\sum_{i=1}^{u}\Big{\\{}d\theta_{i}^{2}+f_{k}^{2}(\theta_{i})d\phi_{i}^{2}\Big{\\}},$

where $N$ represents a NUT charge for the Euclidean section, and the metric
function $f(r)$ is found to be

$\displaystyle\textstyle f(r)$ $\displaystyle=$
$\displaystyle\textstyle\frac{r}{(r^{2}-N^{2})^{u}}\int^{r}\Big{\\{}\frac{(a^{2}-N^{2})^{u}}{a^{2}}k+\frac{(2u+1)(a^{2}-N^{2})^{u+1}}{l^{2}a^{2}}\Big{\\}}da$
(3) $\displaystyle~{}~{}~{}~{}~{}~{}\textstyle-\frac{2mr}{(r^{2}-N^{2})^{u}}.$

with a cosmological parameter $l$ and a geometric mass $m$. The discrete
parameter $k$ takes the values 1, 0, $-1$ and defines the form of the function
$f_{k}(\theta_{i})$

$\displaystyle
f_{k}(\theta_{i})=\left\\{\begin{array}[]{ll}\sin\theta_{i},&\mathrm{for}\ \
k=1\\\ \theta_{i},&\mathrm{for}\ \ k=0\\\ \sinh\theta_{i},&\mathrm{for}\ \
k=-1,\end{array}\right.$ (7)

and the space ${\cal M}^{2}$ corresponds to a two dimensional sphere for
$k=1$, plane for $k=0$, and pseudohyperboloid for $k=-1$, respectively. For
$k=1$, the NUT solution occurs when solving $f(r)|_{r=N}$=0. The inverse of
the temperature $\beta$ is obtained by requiring regularity in the Euclidean
time $t_{E}$ and radial coordinate $r$ Chamblin:1998pz -Astefanesei:2004kn

$\displaystyle\textstyle\beta=\left.\frac{4\pi}{f^{\prime}(r)}\right|_{r=N}=\frac{4(u+1)\pi}{\sigma}N,$
(8)

where $\sigma$ is a positive integer and $\beta$ is the period of $t_{E}$. The
$\sigma$ appears since the period cannot be bigger than $4(u+1)\pi N$, so that
the Misner-string singularities vanish. However, when $\sigma$ has an integer
value, the period can smaller than $4(u+1)\pi N$. This property is not
guaranteed for $k=0$ and $k=-1$ since there is no Misner-string and no
periodicity of time $t$. However, it was shown that this holds for the
$k=0,-1$ cases through checking the self-consistency of the thermodynamic
relations Astefanesei:2004kn .

Let us first consider the NUT solution ($r=N$). Then after taking $\sigma=1$
for convenience without loss of generality, we obtain the following formula
for the temperature:

$\displaystyle\textstyle\frac{1}{\beta}=T=\frac{k}{4(u+1)\pi N}$ (9)

where $k$ has 1 or 0 since there are no hyperbolic NUT solutions. Thus, for
the NUT solution we consider the cases $k=1,0$ only.

Using counter term subtraction method, we get the regularized action
Chamblin:1998pz -Astefanesei:2004kn

$\displaystyle\textstyle I_{\rm
NUT}=\frac{(4\pi)^{u}N^{2u-1}(2uN^{2}-kl^{2})}{16\pi^{\frac{3}{2}}l^{2}}\Gamma(\frac{1}{2}-u)\Gamma(u+1)\beta,$
(10)

where the gamma function $\Gamma(t)$ is defined as
$\Gamma(t)=\int_{0}^{\infty}x^{t-1}e^{-x}dx$.

Employing the Gibbs-Duhem relation $S=\beta M-I$ and substituting in (1), the
entropy is found to be

$\displaystyle\textstyle S_{\rm NUT}$ $\displaystyle=$
$\displaystyle\textstyle\frac{(4\pi)^{u}N^{2u-1}\Big{\\{}16\pi
N^{2}p-(2u-1)k\Big{\\}}}{16\pi^{\frac{3}{2}}}$ (11)
$\displaystyle\textstyle\times\Gamma(\frac{1}{2}-u)\Gamma(u+1)\beta.$

Here $M$ is the conserved mass $M=u(4\pi)^{u-1}m$, and this will be identified
with enthalpy $H$ ($M\equiv H=U+pV$) Kastor:2009wy , which leads to

$\displaystyle\textstyle H_{\rm NUT}$ $\displaystyle=$
$\displaystyle\textstyle
u(4\pi)^{u-1}\Gamma(\frac{3}{2}-u)\Gamma\left(u+1\right)\Big{\\{}\frac{N^{2u-1}}{\sqrt{\pi}(2u-1)}k$
(12) $\displaystyle-$
$\displaystyle\textstyle\frac{16\sqrt{\pi}(u+1)N^{2u+1}}{u(2u-1)(2u+1)}p\Big{\\}}.$

One the other hand, by thermal relation $C=-\beta\partial_{\beta}S$, the
specific heat is given as

$\displaystyle\textstyle C_{\rm NUT}$ $\displaystyle=$
$\displaystyle\textstyle\frac{2\pi(4u+1)(u+1)^{2}S_{\rm
NUT}T^{2}}{\pi(2u-1)(u+1)^{2}T^{2}-kp},$ (13)

which is negative for $p>\pi(2u-1)(u+1)^{2}T^{2}/k$ while is positive
$p<\pi(2u-1)(u+1)^{2}T^{2}/k$ and diverges at $p=\pi(2u-1)(u+1)^{2}T^{2}/k$.

The thermodynamic volume is also obtained as

$\displaystyle\textstyle V_{\rm NUT}$ $\displaystyle=$
$\displaystyle\textstyle-\frac{u(4\pi)^{u}N^{2u+1}}{2\sqrt{\pi}}\Gamma(-\frac{1}{2}-u)\Gamma(u).$
(14)

Then from the above thermodynamic quantities, the generalized Smarr formula
due to dimensional scaling arguments for any value of $k$ is given as

$\displaystyle\frac{1}{2}H-\frac{u}{2u-1}TS+\frac{1}{2u-1}pV=0,$ (15)

which is precisely matched with that of static $d$-dimensional black holes
with negative cosmological constant Caldarelli:1999xj ; Kastor:2009wy ;
Cvetic:2010jb .

Using an thermal relation $U=H-pV$, the internal energy of Taub-NUT is
obtained as

$\displaystyle\textstyle U_{\rm NUT}$ $\displaystyle=$
$\displaystyle\textstyle\frac{u(2u+1)}{8\pi}\Big{\\{}(2\sqrt{\pi}N)^{2u-1}k-\frac{(2u+1)(2\sqrt{\pi}N)^{2u-1}N^{2}}{l^{2}}\Big{\\}}$
(16)
$\displaystyle~{}~{}~{}~{}~{}~{}\textstyle\times\Gamma(-\frac{1}{2}-u)\Gamma(u+1).$

Figure 1: Plot of the entropy $S_{4}$ (yellow solid curve), specific heat
$C_{4}$ (green solid curve) and pressure $p$ (red solid curve for $p=4\pi
T^{2}$, and blue solid curve for $p=12\pi T^{2}$, respectively) as a function
of the temperature $T$ in four dimensions for $k=1$.

Finally, employing the Legendre transform of enthalpy $G=H-TS$, the Gibbs free
energy is given as

$\displaystyle\textstyle G_{\rm NUT}$ $\displaystyle=$
$\displaystyle\textstyle\frac{k^{2u}}{(2\sqrt{\pi})^{2u+3}}\frac{\pi(2u+1)(u+1)^{2}T^{2}-kp}{\big{\\{}(u+1)T\big{\\}}^{2u+1}}$
(17)
$\displaystyle~{}~{}~{}~{}~{}~{}~{}~{}~{}\textstyle\times\Gamma(-\frac{1}{2}-u)\Gamma(u+1).$

Requiring both the entropy and the specific heat for $k=1$ are positive, we
get the following thermally stable range of $T$ for the NUT solution

$\displaystyle\textstyle\sqrt{\frac{p}{\pi(2u-1)(u+1)^{2}}}<T<\sqrt{\frac{p}{\pi
u(2u-1)(u+1)}}.$ (18)

For example, the NUT solution in four dimensions is thermally stable in the
region inside dashed box in Fig. 1. since both the entropy and the specific
heat are positive when $\sqrt{\frac{p}{4\pi}}<T<\sqrt{\frac{p}{2\pi}}$.
However when the Gibbs free energy is introduced, the NUT solution in some
areas of this region is still unstable since the Gibbs free energy (17) is
positive for $p<4\pi T^{2}$ or $p>12\pi T^{2}$ (as you see in Fig. 1, the NUT
solution in the shaded areas is thermally unstable), that is to say that the
Taub-NUT-AdS system evaporates to a stable cold remnant (the pure AdS
spacetime) whereas the Taub-NUT-AdS system is a more thermally stable
configuration than the pure AdS spacetime since the Gibbs free energy is
negative for $4\pi T^{2}<p<12\pi T^{2}$. For any $u$, the NUT solution is
thermally unstable since the Gibbs free energy (17) is positive for
$p<\pi(2u-1)(u+1)^{2}T^{2}$ or $p>\pi(2u+1)(u+1)^{2}T^{2}$ while the NUT
solution is thermally stable since the Gibbs free energy is negative for
$\pi(2u-1)(u+1)^{2}T^{2}<p<\pi(2u+1)(u+1)^{2}T^{2}$.

Figure 2: Plot of the $(2u+2)$-dimensional Gibbs free energy $G_{\rm NUT}$ as
a function of temperature $T$ for $k=1$ & $p=1$ (red solid curve for $u=1$,
yellow solid curve for $u=2$, brown solid curve for $u=3$, and green solid
curve for $u=4$, respectively).

For $k=1$, as shown in Fig 2., when the temperature $T$ is bigger than a
certain temperature, the Gibbs free energy is positive for odd $u$ while the
Gibbs free energy is negative for even $u$. In this range of the temperature
$T$ the phase transition from the Taub-NUT-AdS spaces to the pure AdS
spacetime occurs for odd $u$ whereas that of the opposite direction occurs for
even $u$. It is quite natural that this thermodynamic behavior of the Taub-
NUT-AdS spaces changes due to odd $u$ or even $u$ since the sign of the gamma
function in the Gibbs free energy (17) alternates due to odd $u$ or even $u$.

Remarkably, this topological Taub-NUT-AdS solution has the Hawking-Page
transition between the Taub-NUT-AdS spaces and the pure AdS spacetime, which
occurs on the line $\tilde{p}_{\rm coex}={(2u+1)(u+1)^{2}\pi T^{2}}/{k}$, or
$S_{\rm
NUT}=\frac{\Gamma(\frac{1}{2}-u)\Gamma(u+2)}{2\sqrt{\pi}}\left\\{\frac{k}{2\sqrt{\pi}(u+1)}\right\\}^{2u}\left(\frac{1}{T}\right)^{2u}$.

As shown in Fig 3., there is the NUT solution phase for $p<\tilde{p}_{\rm
coex}$ at any fixed temperature while there is AdS phase for $p>\tilde{p}_{\rm
coex}$ and the two states exist together for $p=\tilde{p}_{\rm coex}$. The
coexistence lines of the topological Taub-NUT-AdS solution phases and the
divergent lines of the topological Taub-NUT-AdS solution specific heat move
more to the left as the dimension of spacetime increases.

Figure 3: For $k=1$, the coexistence lines of the Hawking-Page phase
transition are solid curves and the specific heat diverges on dashed curves
(red solid/dashed curve for $u=1$, orange solid/dashed curve for $u=2$, brown
solid/dashed curve for $u=3$, and green solid/dashed curve for $u=4$,
respectively). (For interpretation of the references to color in this figure
legend, the reader is referred to the web version of this article.)

The shift of entropy occurs on the coexistence lines of the topological Taub-
NUT-AdS solution phases, and so the displacement of entropy is given by

$\displaystyle\textstyle\Delta
S=\frac{\Gamma(\frac{1}{2}-u)\Gamma(u+2)}{2\sqrt{\pi}}\left\\{\frac{k}{2\sqrt{\pi}(u+1)}\right\\}^{2u}\left(\frac{1}{T}\right)^{2u},$
(19)

which leads to the latent heat

$\displaystyle\textstyle L_{\rm
NUT}=\frac{\Gamma(\frac{1}{2}-u)\Gamma(u+2)}{2\sqrt{\pi}}\left\\{\frac{k}{2\sqrt{\pi}(u+1)}\right\\}^{2u}\left(\frac{1}{T}\right)^{2u-1},$
(20)

by an thermal relation $L=T\Delta S$, and the latent heat in terms of $p$ is
obtained as

$\displaystyle\textstyle L_{\rm
NUT}=\frac{\big{\\{}4(2u+1)\big{\\}}^{u-\frac{1}{2}}}{\pi}\Gamma(\frac{1}{2}-u)\Gamma(u+2)\left(\frac{1}{p}\right)^{u-\frac{1}{2}}.$
(21)

Then at any given temperature $T$ for any $u$, the displacement of entropy and
the latent heat are zero in the case $k=0$ and for odd $u$ are negative in the
case $k=1$. The former is trivial since the temperature $T$ (9) is zero as
well as the Gibbs free energy $G_{\rm NUT}$ (17). However, the latter is
related with energy supplied by the process of forming the Taub-NUT-AdS system
from the pure AdS spacetime. It indicates a net release of latent energy back
into the environment because of evaporating of Taub-NUT-AdS system. The latent
heat (20), like the AdS black hole case Dolan:2014mra , vanishes as the
temperature goes to infinity instead of a certain finite value (critical
temperature) and a second order phase transition cannot take place at a
certain finite temperature. The pressure $p$ vanishes for asymptotically flat
spacetime and the latent heat (21) becomes infinity. This means that it is not
possible to spontaneously form the black hole from the Minkowski spacetime.

For AdS spacetime the Gibbs free energy is $G_{\rm AdS}=0$ and for the Taub-
NUT-AdS solution

$\displaystyle G_{\rm NUT}=U_{\rm NUT}-TS_{\rm NUT}+pV_{\rm NUT}.$ (22)

Their differentials on the coexistence line are $dG_{\rm AdS}=0$ and $dG_{\rm
NUT}=-S_{\rm NUT}\,dT+V_{\rm NUT}\,dp$, which lead to

$\displaystyle 0=dG_{\rm AdS}-dG_{\rm NUT}=S_{\rm NUT}\,dT-V_{\rm NUT}\,dp$
(23)

where the coexistence line with two states may be defined by $G_{\rm
AdS}=G_{\rm NUT}$. Hence, the insertion of (11) and (14) into (23) yields

$\displaystyle\textstyle\frac{dp}{dT}=\frac{S_{\rm NUT}}{V_{\rm
NUT}}=\frac{2\pi(2u+1)(u+1)^{2}}{k}.$ (24)

Furthermore, since the thermodynamic volume and the entropy become zero for
AdS spacetime we can write

$\displaystyle\Delta V=V_{\rm NUT}~{}~{}~{}{\rm and}~{}~{}~{}\Delta S=S_{\rm
NUT},$ (25)

and have the Clapeyron equation ${dp}/{dT}={\Delta S}/{\Delta V}$.

As checking the self-consistency of the thermodynamic relations, we can
reproduce the above result (24) through
$\frac{dp}{dT}=\frac{2\pi(2u+1)(u+1)^{2}}{k}$, which shows that the Clapeyron
equation still holds for the NUT solution.

Let us consider the Bolt solution ($r=r_{\rm B}>N$). Requiring
$f(r)|_{r=r_{\rm B}>N}$ and $f^{\prime}(r)|_{r=r_{\rm M}}=\frac{1}{N(u+1)}$,
the Bolt solution occurs. In Taub-Bolt-AdS metric, the inverse of the
temperature, the action, and the mass are respectively

$\displaystyle\textstyle\beta=\left.\frac{4\pi}{f^{\prime}(r)}\right|_{r=r_{\rm
B}}=\frac{4\pi l^{2}r_{\rm B}}{kl^{2}+(2u+1)(r_{\rm B}-N^{2})},$ (26)
$\displaystyle\textstyle I_{\rm Bolt}$ $\displaystyle=$
$\displaystyle\textstyle\frac{(4\pi)^{u-1}}{4l^{2}}\Big{[}\frac{(2u+1)(-1)^{u}N^{2u+2}}{r_{\rm
B}}$ (30) $\displaystyle\textstyle+\sum_{i=0}^{u}\Big{(}\begin{array}[]{l}u\\\
i\end{array}\Big{)}(-1)^{i}N^{2i}r_{\rm
B}^{2u-2i}\Big{\\{}\frac{l^{2}}{(2u-2i-1)r_{\rm B}}k$
$\displaystyle\textstyle-\frac{(2u+1)(u-2i+1)r_{\rm
B}}{(2u-2i+1)(u-i+1)}\Big{\\}}\Big{]}\beta.$

Here the Bolt radius $r_{\rm B}$ is

$\displaystyle\textstyle r_{\rm
B,\pm}=\frac{l^{2}\pm\sqrt{l^{4}+(2u+1)(2u+2)^{2}N^{2}[(2u+1)N^{2}-kl^{2}]}}{(2u+1)(2u+2)N},$
(31)

where $r_{\rm B,+}$ denote the large radius of the Bolt solution and $r_{\rm
B,-}$ is the small radius of the Bolt solution. The discriminant of the square
root in the Bolt radius (31) is always positive for the $k=0,-1$ cases but
sometimes negative for $k=1$, and so the former has no upper limit on $N$
while the latter has the maximum magnitude of the NUT charge $N_{\rm max}$

$\displaystyle\textstyle
N\leq\frac{l}{\sqrt{2(u+1)(2u+1)[u+1+\sqrt{u(u+2)}]}}=N_{\rm max}.$ (32)

Furthermore, from the Bolt radius (31) one can find in the cases $k=0,-1$ the
large radius $r_{\rm B,+}$ exists only for $N>0$ since the Bolt solution
occurs for $r_{\rm B,\pm}>N$.

Using parallel way as in the case of the NUT solution, the enthalpy $H_{\rm
Bolt}$, the entropy $S_{\rm Bolt}$, and thermodynamics volume $V_{\rm Bolt}$
for the Bolt solution yields respectively

$\displaystyle\textstyle H_{\rm Bolt}$ $\displaystyle=$
$\displaystyle\textstyle\frac{u(4\pi)^{u-1}}{2}\Big{\\{}\sum_{i=0}^{u}\Big{(}\begin{array}[]{l}u\\\
i\end{array}\Big{)}\frac{(-1)^{i}N^{2i}r_{\rm B}^{2u-2i-1}}{(2u-2i-1)}k$ (35)
$\displaystyle+$
$\displaystyle\textstyle\frac{8\pi}{u}\sum_{i=0}^{u+1}\Big{(}\begin{array}[]{l}u+1\\\
~{}~{}~{}i\end{array}\Big{)}\frac{(-1)^{i}N^{2i}r_{\rm
B}^{2u-2i+1}}{(2u-2i+1)}p\Big{\\}},$ (38) $\displaystyle\textstyle S_{\rm
Bolt}$ $\displaystyle=$
$\displaystyle\textstyle\frac{(4\pi)^{u-1}}{4}\Big{[}\sum_{i=0}^{u}\Big{(}\begin{array}[]{l}u\\\
i\end{array}\Big{)}\frac{(2u-1)(-1)^{i}N^{2i}r_{\rm B}^{2u-2i-1}}{2u-2i-1}k$
(45)
$\displaystyle\textstyle+\Big{\\{}\sum_{i=0}^{u}\Big{(}\begin{array}[]{l}u\\\
i\end{array}\Big{)}\frac{8\pi(2u^{2}+3u-2i+1)(-1)^{i}N^{2i}}{u(u-i+1)(2u-2i+1)}$
$\displaystyle~{}~{}~{}~{}~{}\textstyle\times r_{\rm
B}^{2u-2i+1}+\frac{8\pi(2u-1)N^{2u+2}}{ur_{\rm B}}\Big{\\}}p\Big{]}\beta,$
$\displaystyle\textstyle V_{\rm Bolt}$ $\displaystyle=$
$\displaystyle\textstyle\frac{\pi^{u}(2r_{\rm
B})^{u-1}}{2u+1}\textstyle\Big{\\{}-2(N^{2}-r_{\rm
B}^{2})\Big{(}1-\frac{N^{2}}{r_{\rm B}^{2}}\Big{)}^{u}$ (46)
$\displaystyle\textstyle+r_{\rm B}^{2}\Big{(}\frac{N}{r_{\rm
B}}\Big{)}^{2u+1}{\rm B}\Big{(}\frac{N^{2}}{r_{\rm
B}^{2}},\frac{1}{2}-u,u+1\Big{)}\Big{\\}},$

where the incomplete beta function ${\rm B}\left(x,a,b\right)$ is defined as
${\rm B}\left(x,a,b\right)=\int_{0}^{x}t^{a-1}(1-t)^{b-1}dt$. Like the
previous NUT case, these thermodynamic quantities satisfy the generalized
Smarr formula (15).

The specific heat is given as

$\displaystyle\scriptstyle C_{\rm Bolt,\pm}$ $\displaystyle=$
$\displaystyle\scriptstyle\frac{u(2u+1)^{2}p}{2\sqrt{{\cal A}}{\cal
B_{\pm}}^{2}T^{2}}\left(\frac{1}{4\pi}\right)^{u}\scriptstyle\left(\frac{\pi
uT}{kp}+\frac{\sqrt{{\cal A}}}{(u+1)(2u+1)kpT}\right)^{2u}$ (47)
$\displaystyle\scriptstyle\times\Big{[}\frac{T^{2}}{p}\left(1-\frac{k^{4}(2u+1)^{2}p^{2}}{{\cal
B_{\pm}}^{2}}\right)^{u}$ $\displaystyle\times\scriptstyle\Big{\\{}\sqrt{{\cal
A}}\scriptstyle\Big{(}k^{4}p^{2}-\pi
k^{3}u(u+1)^{2}pT^{2}-2\pi^{2}u^{2}(u+1)^{2}T^{4}\Big{)}$
$\displaystyle~{}\scriptstyle+(u+1)(2u+1)\Big{(}-k^{7}p^{3}+2\pi
k^{6}u(u+1)^{2}p^{2}T^{2}$
$\displaystyle~{}~{}~{}\scriptstyle+\pi^{2}k^{3}u^{2}(u+1)^{2}pT^{4}-2\pi^{3}u^{3}(u+1)^{2}T^{6}\Big{)}\Big{\\}}$
$\displaystyle\scriptstyle+\frac{k^{3}}{\pi u(2u-1)}\Big{(}\pi u(u+1){\cal
B_{\pm}}T^{2}+k^{4}(2u+1)p^{2}$
$\displaystyle~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}\scriptstyle-2\pi
k^{3}u(2u+1)(u+1)^{2}pT^{2}\Big{)}$
$\displaystyle~{}~{}~{}\scriptstyle\times\Big{(}-kp+\pi
u(u+1)(2u-1)T^{2}\Big{)}$
$\displaystyle~{}~{}~{}\scriptstyle\times{}_{1}F_{2}\left(\frac{1}{2}-u,-u;\,\frac{3}{2}-u;\,\frac{k^{4}(2u+1)^{2}p^{2}}{{\cal
B_{\pm}}^{2}}\right)\Big{]},$

where $C_{\rm Bolt,+}$ denotes the specific heat with the radius $r_{\rm
B,+}$, and $C_{\rm Bolt,-}$ is the specific heat with the radius $r_{\rm
B,-}$. Here, ${\cal A}$ and ${\cal B_{\pm}}$ are

$\displaystyle{\cal A}=(2u+1)^{2}(k^{4}p^{2}-2\pi
k^{3}u(u+1)^{2}pT^{2}+\pi^{2}u^{2}(u+1)^{2}T^{4},$ $\displaystyle{\cal
B_{\pm}}=\sqrt{{\cal A}}\pm\pi u(u+1)(2u+1)T^{2},$

and the hypergeometric function ${}_{1}F_{2}(a,b;c;,z)$ is defined for $|z|<1$
by the power series

$\displaystyle\textstyle{}_{1}F_{2}(a,b;c;,z)=\sum_{n=0}^{\infty}\frac{(a)_{n}(b)_{n}}{(c)_{n}}\frac{z^{n}}{n!}.$
(48)

For example, the four-dimensional specific heat is obtained as

$\displaystyle\scriptstyle C_{\rm Bolt,\pm}$ $\displaystyle=$
$\displaystyle\scriptstyle-\frac{\pi
T^{2}}{4k^{2}p^{2}}\scriptstyle\pm\frac{(k^{8}p^{4}-6\pi
k^{7}p^{3}T^{2}+2\pi^{2}k^{4}p^{2}T^{4}\scriptstyle-8\pi^{3}k^{3}pT^{6}+8\pi^{4}T^{8})}{16\pi^{2}k^{2}p^{2}T^{4}\sqrt{k^{4}p^{2}-8\pi
k^{3}pT^{2}+4\pi^{2}T^{4}}},$

which is well matched with the result in Astefanesei:2004kn . Here, as far as
we know, even if including the Bolt solution without a cosmological constant
treated as a pressure, the specific heat (47) is firstly shown in the
analytical expressions. In fact, since the specific heat of the Bolt solution
for arbitrary dimensions has a highly complicated high-order polynomial terms,
the analytical expressions have not been obtained even for the case $k=1$.
Furthermore for $k=1$ the four-dimensional specific heat $C_{\rm Bolt,\pm}$
(II) diverges at $T=T_{2}$

$\displaystyle\textstyle T_{2}=\sqrt{\frac{p}{\pi}+\frac{\sqrt{3}p}{2\pi}},$
(50)

and for $k=1$ the higher dimensional specific heat (47) also diverges at $T$

$\displaystyle\textstyle T=\frac{\sqrt{p}}{\sqrt{\pi
u}}\sqrt{1+\frac{\sqrt{u}\sqrt{u+2}}{u+1}}.$ (51)

The internal energy of AdS-Taub-Bolt $U$ is given as

$\displaystyle\textstyle U_{\rm Bolt}$ $\displaystyle=$
$\displaystyle\textstyle\frac{(4\pi)^{u-1}}{4l^{2}}uN^{2u-1}\Big{\\{}(2u+1)N^{2}-kl^{2}\Big{\\}}$
(52) $\displaystyle~{}~{}~{}\textstyle\times{\rm B}\Big{(}\frac{N^{2}}{r_{\rm
B}^{2}},\frac{1}{2}-u,u+1\Big{)},$

and the Gibbs free energy

$\displaystyle\textstyle G_{\rm Bolt}$ $\displaystyle=$
$\displaystyle\textstyle u\Big{(}\frac{N}{r_{\rm B}}\Big{)}^{2u}{\rm
B}\Big{(}\frac{N^{2}}{r_{\rm B}^{2}},\frac{1}{2}-u,u+1\Big{)}k+\frac{16\pi
Nr_{\rm B}}{2u+1}\Big{\\{}$
$\displaystyle\textstyle\Big{(}1-\frac{N^{2}}{r_{\rm
B}^{2}}\Big{)}^{u+1}-\Big{(}\frac{N}{r_{\rm B}}\Big{)}^{2u+1}{\rm
B}\Big{(}\frac{N^{2}}{r_{\rm B}^{2}},\frac{1}{2}-u,u+1\Big{)}\Big{\\}}p.$

Figure 4: Plot of the $(2u+2)$-dimensional Gibbs free energy $G_{\rm Bolt}$ as
a function of temperature $T$ for any $u$ (red solid/dahsed curve for $u=1$,
yellow solid/dashed curve for $u=2$, brown solid/dashed curve for $u=3$, and
green solid/dashed curve for $u=4$, respectively) for pressure $p=3$ and
$k=1$.

As shown in Fig 4., two branches (the red solid curve and the red dashed
curve) are joined at the temperature $T_{2}$. At this temperature two phases
occur, that is to say that the upper branch (red dashed curve) is the phase of
the Taub-Bolt-AdS system with the small radius $r_{\rm B,-}$ and the lower
branch (red solid curve) is the phase of the Taub-Bolt-AdS system with the
large radius $r_{\rm B,+}$. Like the AdS black hole case, at $T_{2}$ (50) the
four-dimensional specific heat of the Bolt solution also diverges and the
higher dimensional cases diverge at $T$ (51). Since the Gibbs free energy is
positive, the Bolt solution for the upper branch is unstable whereas since the
lower branch includes the negative value of the Gibbs free energy, this
solution in such region becomes stable. Thus, when $G_{\rm Bolt}=0$, like the
Taub-NUT-AdS black hole case, there are the Hawking-Page transition between
the Taub-Bolt-AdS black hole and the pure AdS spacetime, which occurs on the
lines for the upper branch $r_{\rm B,+}$

$\displaystyle\textstyle\bar{p}_{\rm coex,+}$ $\displaystyle=$
$\displaystyle\textstyle\Big{[}\frac{\pi^{2}u(2u+1)(u+1)^{2}T^{4}}{k^{5}}$
(54) $\displaystyle\times$ $\displaystyle\textstyle\frac{{\cal
D}^{u+1}B\left({\cal D},\frac{1}{2}-u,u+1\right)}{(1-{\cal
D})^{u+1}/\sqrt{{\cal D}}+u{\cal D}^{u}B\left({\cal
D},\frac{1}{2}-u,u+1\right)}\Big{]}^{1/3},$

with

$\displaystyle\textstyle{\cal D}$ $\displaystyle=$
$\displaystyle\textstyle\frac{k^{4}p^{2}}{\left[\sqrt{k^{4}p^{2}-2\pi
u(u+1)^{2}k^{3}pT^{2}+\pi^{2}u^{2}(1+u)^{2}T^{4}}+\pi
u(u+1)T^{2}\right]^{2}}.$

Figure 5: For the four-dimensional NUT solution and Bolt solution, plot of $p$
as a function of $T$. Here $T_{2}$ and $T_{3}$ are the same values used in Fig
4.

Furthermore as shown in Fig 5., the thermodynamic instability of the Taub-
NUT/Bolt-AdS system is classified by the value of the temperature $T$. When
$T>T_{0}$, the Taub-NUT-AdS system evaporates to a stable cold remnant. When
$T_{0}\leq T\leq T_{1}$, the Taub-NUT-AdS system is a thermally stable
configuration. When $T_{1}<T<T_{2}$, and then the Taub-NUT-AdS system
evaporates again since the four-dimensional entropy $S_{4}$ is negative. When
$T_{2}\leq T<T_{3}$, the Taub-Bolt-AdS system with two phases occurs at
$T_{2}$ but still evaporates since the Gibbs free energy of the Taub-Bolt-AdS
system is positive. Finally, when $T\geq T_{3}$, the Taub-Bolt-AdS system with
the large radius $r_{\rm B,+}$ becomes stable since the Gibbs free energy of
the Taub-Bolt-AdS system is negative. This thermodynamic nature of the Taub-
NUT/Bolt-AdS system may hold for higher dimensional cases since $p-T$ phase
diagrams in higher dimension are similar shapes Lee:2014tma .

From now on, considering the action difference of Taub-NUT-AdS and Taub-Bolt-
AdS, we investigate the instability of the Taub-NUT/Bolt-AdS system. Their
action difference, ${\cal I}_{D}$ is defined as $I_{\rm Bolt}-I_{\rm NUT}$

$\displaystyle\textstyle{\cal I}_{D}$ $\displaystyle=$
$\displaystyle\textstyle\frac{(4\pi)^{u}}{8r_{\rm
B}l^{2}}\Big{[}2N(N^{2}-r_{\rm B}^{2})(u+1)\left(1-\frac{N^{2}}{r_{\rm
B}^{2}}\right)^{u}$ (55)
$\displaystyle\textstyle+(2uN^{2}-kl^{2})\Big{\\{}(u+1)N^{2u}{\rm
B}\Big{(}\frac{N^{2}}{r_{\rm B}^{2}},\frac{1}{2}-u,u+1\Big{)}$
$\displaystyle~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}\textstyle-\frac{2N^{2u}}{\sqrt{\pi}}\Gamma(\frac{1}{2}-u)\Gamma(2+u)\Big{\\}}\Big{]}.$

Figure 6: Plot of the four-dimensional action difference ${\cal I}_{4}$ as a
function of N for cosmological parameter $l$ (from left to right) 1 to 5 for
pressure $p=3$ and $k=0$. Solid lines are curves of ${\cal I}_{4}$ with large
radius $r_{\rm B,+}$.

Considering the four-dimensional case ($u=1$), for $k=1$ taking the
cosmological parameter $l$ and the pressure $p$ as fixed parameters, the
action difference ${\cal I}_{D}$ becomes negative as $N$ increases. This means
that the Taub-Bolt-AdS system with $r_{\rm B,+}$ is a more thermally stable
configuration than the Taub-NUT-AdS system for ${\cal I}_{D}<0$, and so there
is the first order phase transition from Taub-NUT-AdS system to Taub-Bolt-AdS
system with $r_{\rm B,+}$ at a critical NUT chargeJohnson:2014pwa ;
Lee:2014tma . As shown in Fig 6., for the case $k=0$ Taub-Bolt-AdS system with
$r_{\rm B,+}$ is stable only since the lower branch is always negative whereas
the upper branch is always positive. It is shown that the curves of ${\cal
I}_{4}$ move more to the right as the pressure $p$ decreases (the cosmological
parameter $l$ grows up) since $p$ is inversely proportional to $l$. For $u=2$
a similar result is obtained as with the $u=1$ case. Furthermore, ${\cal I}-N$
diagrams in higher dimension are similar shapes Lee:2014tma . Thus, this
thermodynamic instability of the Taub-NUT/Bolt-AdS system may hold for higher
dimensional cases, and is well matched with the result in Astefanesei:2004kn .

##  III Conclusion

We considered higher dimensional topological Taub-NUT/Bolt-AdS solutions and
in the context of the extended thermodynamics, explicitly calculated their
thermal quantities. In particular, we gave the analytical expressions of the
specific heat for the $k=1,0,-1$ topological Bolt solutions.

By using these thermal quantities we also investigated their phase structure.
By introducing Gibbs free energy, it was particularly found that there is a
new thermodynamically stable region of the NUT solution.

Furthermore, it was found that like the AdS black hole case, for $k=1$ Taub-
Bolt-AdS system with two phases occurs at a minimum temperature and at this
temperature the specific heat diverges.

Recently using AdS/CFT correspondence, thermodynamics and thermodynamic
geometry have been covered for Schwarzschild-AdS black hole in the extended
phase space Zhang:2014uoa . It has been explored through the extensive
application of a RN-AdS black hole. They have shown that its thermodynamic
behavior is qualitatively the same as that in the Schwarzschild-AdS case
Zhang:2015ova . It would be of interest to study the Taub-NUT/Bolt-AdS case in
this context.

## References

  * (1) J. D. Bekenstein, Phys. Rev. D 7, 2333 (1973).
  * (2) J. M. Bardeen, B. Carter and S. W. Hawking, Commun. Math. Phys. 31, 161 (1973).
  * (3) S. W. Hawking and D. N. Page, Commun. Math. Phys. 87, 577 (1983).
  * (4) C. O. Lousto, Phys. Rev. D 51, 1733 (1995) [gr-qc/9405048].
  * (5) C. S. Peca and J.P.S. Lemos, Phys. Rev. D 59, 124007 (1999) [gr-qc/9805004].
  * (6) R. Banerjee, S. K. Modak and S. Samanta, Phys. Rev. D 84, 064024 (2011) [arXiv:1005.4832 [hep-th]].
  * (7) Y. S. Myung, Eur. Phys. J. C 72, 2116 (2012) [arXiv:1203.1367 [hep-th]].
  * (8) A. Chamblin, R. Emparan, C. V. Johnson and R. C. Myers, Phys. Rev. D 60, 064018 (1999) [hep-th/9902170].
  * (9) M. Cvetic and S. S. Gubser, JHEP 9904, 024 (1999) [hep-th/9902195].
  * (10) M. M. Caldarelli, G. Cognola and D. Klemm, Class. Quant. Grav. 17, 399 (2000) [hep-th/9908022].
  * (11) S. S. Gubser and I. Mitra, JHEP 0108, 018 (2001) [hep-th/0011127].
  * (12) R. -G. Cai, Phys. Rev. D 65, 084014 (2002) [hep-th/0109133].
  * (13) T. K. .Dey, Phys. Lett. B 595, 484 (2004) [hep-th/0406169].
  * (14) T. K. Dey, S. Mukherji, S. Mukhopadhyay and S. Sarkar, JHEP 0704, 014 (2007) [hep-th/0609038].
  * (15) N. Banerjee and S. Dutta, JHEP 0707, 047 (2007) [arXiv:0705.2682 [hep-th]].
  * (16) T. K. Dey, S. Mukherji, S. Mukhopadhyay and S. Sarkar, JHEP 0709, 026 (2007) [arXiv:0706.3996 [hep-th]].
  * (17) R. A. Konoplya and A. Zhidenko, Phys. Rev. D 78, 104017 (2008) [arXiv:0809.2048 [hep-th]].
  * (18) R. Banerjee and D. Roychowdhury, JHEP 1111, 004 (2011) [arXiv:1109.2433 [gr-qc]].
  * (19) D. Kastor, S. Ray and J. Traschen, Class. Quant. Grav. 26 (2009) 195011 [arXiv:0904.2765 [hep-th]].
  * (20) M. Cvetic, G. W. Gibbons, D. Kubiznak and C. N. Pope, Phys. Rev. D 84, 024037 (2011) [arXiv:1012.2888 [hep-th]].
  * (21) B. P. Dolan, Class. Quant. Grav. 28, 235017 (2011) [arXiv:1106.6260 [gr-qc]].
  * (22) D. Kubiznak and R. B. Mann, JHEP 1207, 033 (2012) [arXiv:1205.0559 [hep-th]].
  * (23) S. Gunasekaran, R. B. Mann and D. Kubiznak, JHEP 1211, 110 (2012) [arXiv:1208.6251 [hep-th]].
  * (24) S. H. Hendi and M. H. Vahidinia, Phys. Rev. D 88, no. 8, 084045 (2013) [arXiv:1212.6128 [hep-th]].
  * (25) R. -G. Cai, L. -M. Cao, L. Li and R. -Q. Yang, JHEP 1309, 005 (2013) [arXiv:1306.6233 [gr-qc]].
  * (26) W. Xu, H. Xu and L. Zhao, arXiv:1311.3053 [gr-qc].
  * (27) D. C. Zou, S. J. Zhang and B. Wang, Phys. Rev. D 89, 044002 (2014) [arXiv:1311.7299 [hep-th]].
  * (28) D. Kubiznak and R. B. Mann, arXiv:1404.2126 [gr-qc].
  * (29) D. C. Zou, Y. Liu and B. Wang, arXiv:1404.5194 [hep-th].
  * (30) C. V. Johnson, arXiv:1404.5982 [hep-th].
  * (31) Y. Liu, D. C. Zou and B. Wang, arXiv:1405.2644 [hep-th].
  * (32) H. Xu, W. Xu and L. Zhao, arXiv:1405.4143 [gr-qc].
  * (33) M. -S. Ma and Y. -Q. Ma, arXiv:1405.7609 [hep-th].
  * (34) W. Xu and L. Zhao, arXiv:1405.7665 [gr-qc].
  * (35) S. MacDonald, arXiv:1406.1257 [hep-th].
  * (36) C. V. Johnson, arXiv:1406.4533 [hep-th].
  * (37) A. M. Frassino, D. Kubiznak, R. B. Mann and F. Simovic, arXiv:1406.7015 [hep-th].
  * (38) B. P. Dolan, Phys. Rev. D 90, no. 8, 084002 (2014) [arXiv:1407.4037 [gr-qc]].
  * (39) B. P. Dolan, A. Kostouki, D. Kubiznak and R. B. Mann, arXiv:1407.4783 [hep-th].
  * (40) B. P. Dolan, Mod. Phys. Lett. A 30 (2015) 03n04, 1540002 [arXiv:1408.4023 [gr-qc]].
  * (41) E. Caceres, P. H. Nguyen and J. F. Pedraza, arXiv:1507.06069 [hep-th].
  * (42) A. Belhaj, M. Chabab, H. El Moumni, K. Masmar, M. B. Sedra and A. Segui, JHEP 1505, 149 (2015) [arXiv:1503.07308 [hep-th]].
  * (43) C. O. Lee, Phys. Lett. B 738, 294 (2014) [arXiv:1408.2073 [hep-th]].
  * (44) R. B. Mann and C. Stelea, Phys. Lett. B 634, 448 (2006) [arXiv:hep-th/0508203].
  * (45) A. Chamblin, R. Emparan, C. V. Johnson and R. C. Myers, Phys. Rev. D 59, 064010 (1999) [arXiv:hep-th/9808177].
  * (46) S. W. Hawking, C. J. Hunter and D. N. Page, Phys. Rev. D 59, 044033 (1999) [arXiv:hep-th/9809035].
  * (47) R. B. Mann, Phys. Rev. D 61, 084013 (2000) [arXiv:hep-th/9904148].
  * (48) A. Awad and A. Chamblin, Class. Quant. Grav. 19, 2051 (2002) [arXiv:hep-th/0012240].
  * (49) R. Clarkson, L. Fatibene and R. B. Mann, Nucl. Phys. B 652, 348 (2003) [arXiv:hep-th/0210280].
  * (50) D. Astefanesei, R. B. Mann and E. Radu, Phys. Lett. B 620, 1 (2005) [arXiv:hep-th/0406050].
  * (51) D. Astefanesei, R. B. Mann and E. Radu, JHEP 0501, 049 (2005) [arXiv:hep-th/0407110].
  * (52) J. L. Zhang, R. G. Cai and H. Yu, JHEP 1502, 143 (2015) [arXiv:1409.5305 [hep-th]].
  * (53) J. L. Zhang, R. G. Cai and H. Yu, Phys. Rev. D 91, no. 4, 044028 (2015) [arXiv:1502.01428 [hep-th]].