#  Predicting Discourse Trees from Transformer-based Neural Summarizers

Wen Xiao, Patrick Huber, Giuseppe Carenini
Department of Computer Science
University of British Columbia
Vancouver, BC, Canada, V6T 1Z4
{xiaowen3, huberpat<EMAIL_ADDRESS>
###### Abstract

Previous work indicates that discourse information benefits summarization. In
this paper, we explore whether this synergy between discourse and
summarization is bidirectional, by inferring document-level discourse trees
from pre-trained neural summarizers. In particular, we generate unlabeled RST-
style discourse trees from the self-attention matrices of the transformer
model. Experiments across models and datasets reveal that the summarizer
learns both, dependency- and constituency-style discourse information, which
is typically encoded in a single head, covering long- and short-distance
discourse dependencies. Overall, the experimental results suggest that the
learned discourse information is general and transferable inter-domain111The
code can be found in https://github.com/Wendy-Xiao/summ_guided_disco_parser.

##  1 Introduction

Extractive summarization is a common and important task within the area of
Natural Language Processing (NLP) , which can be useful in a multitude of
diverse real-life scenarios. Current extractive summarizers typically use
exclusively neural approaches, in which the importance of extracted units
(i.e., sentences or clauses) and relationship between them are learned by the
model from a large amount of data (e.g., Liu and Lapata (2019b)).

Inspired by previous work in pre-neural times, indicating that discourse
information, especially discourse trees according to the Rhetorical Structure
Theory (RST) Mann and Thompson (1988), can benefit the summarization task
Marcu (1999), several very recent neural summarizers have tried to explicitly
encode discourse information to support summarization. Overall, it seems that
adding these encodings, consistent with pre-neural results, is beneficial. In
particular, injecting discourse has been shown to either improve performance
on the extractive summarization task itself Xu et al. (2020), or allow for a
substantial reduction in the number of the summarizer’s parameters, while
keeping competitive performance Xiao et al. (2020).

The central hypothesis we are exploring in this paper is whether the synergy
between discourse parsing and summarization is bidirectional. In other words,
we examine if summarization is a useful auxiliary task to infer discourse
structures. Liu et al. (2019b) performed a preliminary investigation of this
conjecture, showing that structural information can be inferred from attention
mechanisms while training a neural model on auxiliary tasks. However, they did
not perform any comparison against ground-truth discourse trees. Further,
recent work showed that discourse trees implicitly induced during training are
oftentimes trivial and shallow, not representing valid discourse structures
Ferracane et al. (2019).

In this paper, we address these limitations by explicitly exploring the
relationship between summarization and discourse parsing through the inference
of document-level discourse trees from pre-trained summarization models,
comparing the results against ground-truth RST discourse trees. Besides Liu et
al. (2019b), our idea and approach are inspired by recent works on extracting
syntactic trees from pre-trained language models Wu et al. (2020) or machine
translation approaches Raganato and Tiedemann (2018), as well as previous work
on knowledge graph construction from pre-trained language models Wang et al.
(2020). Specifically, we generate full RST-style discourse trees from self-
attention matrices of a pre-trained transformer-based summarization model. We
use three different tree-aggregation approaches (CKY Jurafsky and Martin
(2014), Eisner Eisner (1996) and CLE Chu and Liu (1965); Edmonds (1967)),
generating a set of constituency and dependency trees representing diverse
discourse-related attributes.

Our proposal is thereby addressing one of the key limitations in discourse
parsing, namely the lack of large training corpora. We aim to overcome this
limitation by generating a large number of reasonable quality discourse trees
from a pre-trained summarization model, similar in spirit to what Huber and
Carenini (2020) did with sentiment. Admittedly, the discourse information
captured with our approach is summarization task-specific, however, our
generated discourse treebank can be combined with further task-dependent
treebanks (e.g. from sentiment) to train more powerful discourse parsers in a
multitask framework.

Generally speaking, the ability to infer discourse trees as a “by-product" of
the summarization task can also be seen as a form of unsupervised discourse
parsing, where instead of leveraging pre-trained language models like in
Kobayashi et al. (2019), we exploit a pre-trained neural summarizer.

We empirically evaluate our method on three datasets with human RST-style
annotations, covering different text genres. Multiple experiments show that
the summarization model learns discourse information implicitly, and that more
dependency information are captured, compared to structural (i.e.,
constituency) signals. Interestingly, an additional exploration of the
attention matrices of individual heads suggests that, for all models, most of
the discourse information is concentrated in a single head, and the best
performing head is consistent across all datasets. We further find that the
dependency information learned in the attention matrix covers long distance
discourse dependencies. Overall, the results are consistent across datasets
and models, indicating that the discourse information learned by the
summarizer is general and transferable inter-domain.

##  2 Related Work

Rhetorical Structure Theory (RST) Mann and Thompson (1988) is one of the most
popular theories of discourse, postulating that a document can be represented
as a constituency tree, where leaves are clause-like Elementary Discourse
Units (EDUs), and internal nodes combine their respective children by
aggregating them into a single, joint constituent. Each internal node also has
a nuclearity attribute222In this paper we do not consider rhetorical
relations., representing the local importance of their direct child-nodes in
the parent context from the set of {Nucleus-Nucleus, Nucleus-Satellite,
Satellite-Nucleus}. “Nucleus" child-nodes thereby generally play a more
important role when compared to a “Satellite" child-node. Although standard
RST discourse trees are encoded as constituency trees, they can be converted
into dependency trees with near isomorphic transformations. In this work, we
infer both, constituency and dependency trees.

Over the past decades, RST discourse parsing has been mainly focusing on
supervised models, typically trained and tested within the same domain using
human annotated discourse treebanks, such as RST-DT Carlson et al. (2002),
Instruction-DT Subba and Di Eugenio (2009) or GUM Zeldes (2017). The intra-
domain performance of these supervised models has consistently improved, with
a mix of traditional models by Joty et al. (2015) and Wang et al. (2017), and
neural models Yu et al. (2018) reaching state-of-the-art (SOTA) results. Yet,
these approaches do not generalize well inter-domain Huber and Carenini
(2020), likely due to the limited amount of available training data.

Huber and Carenini (2019) recently tackled this data-sparsity issue through
automatically generated discourse structures from distant supervision, showing
that sentiment information can be used to infer discourse trees. Improving on
their initial results, Huber and Carenini (2020) published a large-scale,
distantly supervised discourse corpus (MEGA-DT), showing that a parser trained
on such treebank delivers SOTA performance on the more general inter-domain
discourse parsing task. In this paper, we also tackle the data sparsity
problem in discourse parsing, however, using a significantly different
approach. First, instead of relying on sentiment, we leverage the task of
extractive summarization. Second, instead of a method for distant supervision,
we propose an unsupervised approach.

The area of unsupervised RST-style discourse parsing has been mostly
underlooked in the past, with recent neural approaches either taking advantage
of pre-trained language models to predict discourse Kobayashi et al. (2019) or
using pre-trained syntactic parsers and linguistic knowledge Nishida and
Nakayama (2020) to infer discourse trees in an unsupervsied manner. Similarly.
our proposal only relies on a pre-trained neural summarization model to
generate discourse trees.

Recent neural summarization models are typically based on transformers Liu and
Lapata (2019a); Zhang et al. (2019). One advantage of these models is that
they learn the relationship between input units explicitly using the dot-
product self-attention, which allows for some degree of exploration of the
inner working of these complex and distributed models. Here, we investigate if
the attention matrices of a transformer-based summarizer effectively capture
discourse information (i.e., how strongly EDUs are related) and therefore can
be used to derive discourse trees for arbitrary documents.

Marcu (1999) pioneered the idea to directly apply RST-style discourse parsing
to extractive summarization, and empirically showed that RST discourse
information can benefit the summarization task, by simply extracting EDUs
along the nucleus path. This initial success was followed by further work on
leveraging discourse parsing in summarization, including McDonald (2007),
Hirao et al. (2013), and Kikuchi et al. (2014). More recently, the benefits of
discourse for summarization have also been confirmed for neural summarizers,
e.g. in Xiao and Carenini (2019) and Cohan et al. (2018), using the structure
of scientific papers (i.e. sections), and in Xu et al. (2020), successfully
incorporating RST-style discourse and co-reference information in the BERTSUM
summarizer Liu and Lapata (2019b).

In contrast to previous approaches demonstrating how discourse can enhance
summarization performance, we have recently shown that discourse enables the
specification of simpler neural summarizers, without affecting their
performance Xiao et al. (2020). In particular, by using a fixed discourse-
based attention they achieve competitive results compared to learnable dot-
product self-attention mechanisms, as used in the original transformer model.
Inspired by these findings, suggesting that transformer-based summarization
models learn effective discourse representations, we explore if useful
discourse structures can be inferred from learnt transformer self-attention
weights.

Admittedly, Liu and Lapata (2018) and Liu et al. (2019b) presented preliminary
work on inferring discourse structures from attention mechanisms, while
training a neural model on auxiliary tasks, like text classification and
summarization. However, they did not perform any comparison against ground-
truth discourse trees as we do here. More importantly, we employ a more
explicit approach to infer discourse structures, not as part of the learning
process, but extracting the structures after the summarization model is
completely trained and applied to new documents.

While our focus is on discourse, extracting syntactic constituency and
dependency trees from transformer-based models has been recently attempted in
both, machine translation and language modelling. In machine translation,
Mareček and Rosa (2019) and Raganato and Tiedemann (2018) show that trained
translation models can capture syntactic information within their attention
heads, using the CKY and CLE algorithms, respectively. In pre-trained language
models, Wu et al. (2020) propose a parameter-free probing method to construct
syntactic dependency trees based on a pre-trained BERT model, only briefly
elaborating on possible applications to discourse. In contrast to our work,
they do not directly use attention heads, but instead build an impact matrix
based on the distance between token representations. Furthermore, while their
BERT-based model cannot deal with long sequences, our two-level encoder can
effectively deal with sequences of any length, which is critical in discourse.

Figure 1: The pipeline of our whole method.

##  3 Our Model

###  3.1 Framework Overview

Our main goal is to show the ability of a previously trained summarization
model to be directly applied to the task of RST-style discourse parsing. Along
this line, we explore the relationship between information learned by the
transformer-based sumarizer and the task of discourse parsing. We leverage the
synergies between units learned in the transformer model by following Xiao et
al. (2020), previously proposing the use of a transformer document-encoder on
top of a pretrained BERT EDU encoder. This standard summarization model is
presented in Figure 1 (left). In the transformer-based document encoder, each
head internally contains a self-attention matrix, learned during the training
of the summarization model, representing the relationship between EDUs (Figure
1 (center)). In this paper, we analyze these learned self-attention matrices,
not only to confirm our intuition that they contain relevant discourse
information, but also to computationally exploit such information for
discourse parsing. We therefore generate a set of different
(constituency/dependency) discourse trees from the self-attention matrices,
focusing on different attributes of discourse, as shown in Figure 1 (right).
Our generated constituency trees only reveal the discourse tree structure
without additional nuclearity and relation attributes. More interestingly, we
complement the constituency interpretation of the self-attention by
additionally inferring a dependency tree, also partially guided by discourse
structures, but mostly driven by the RST nuclearity attribute, which is shown
to be more related to the summarization task where the importance of the
different text spans is critical Hirao et al. (2013). We present and discuss
the different parsing algorithms to extract discourse information from the
self-attention matrix next.

###  3.2 Parsing Algorithms

Formally, for an input document $D=\\{u_{1},..,u_{n}\\}$ with $n$ EDUs, each
attention head returns an attention matrix $A\in\mathbf{R}^{n\times n}$ where
entry $A_{ij}$ contains a score measuring how much the $i$-th EDU relies on
the $j$-th EDU. Given those bidirectional scores defining the relationship
between every two EDUs in a document, we build a tree such that EDU pairs with
higher reciprocal attention scores are more closely associated in the
resulting tree. In the constituency case, this means that EDUs with higher
mutual attention should belong to sub-trees on lower levels of the tree, while
in the dependency case this implies that the path between such EDUs should
contain less intermediate nodes. In essence, these requirements can be
formalized as searching for the tree within the set of possible trees, which
maximizes a combined score.

####  3.2.1 Constituency Tree (C-Tree) Parsing

To generate a constituency tree from the attention matrix, we follow a large
body of previous work in discourse parsing (e.g., Joty et al. (2015)), where
constituency discourse trees are generated using the CKY algorithm Jurafsky
and Martin (2014). Specifically, we fill a $n\times n$ matrix
$P\in\mathbf{R}^{n\times n}$ generating the optimal tree in bottom-up fashion
using the dynamic programming approach according to:

$P_{ij}=\begin{cases}0,&i>j\\\ \sum_{k=1}^{n}(A_{ki}),&i=j\\\
\max_{k=i}^{j-1}(P_{ik}+P_{(k+1)j}\\\
\quad+\operatorname{avg}(A_{i:k,(k+1):j})\\\
\quad+\operatorname{avg}(A_{(k+1):j,i:k}))/2,&i<j\end{cases}$

where $P_{ij}$ with $i=j$ contains the overall importance of EDU $i$, computed
as the attention paid by others to unit $i$. $P_{ij}$ with $i<j$ represents
the score of the optimal sub-tree spanning from EDU $i$ to EDU $j$. We select
the best combination of sub-trees $k$, such that the sum of the left sub-tree
spanning $[i:k]$ and the right one spanning $[(k+1):j]$, along with the
average score of connections between the two sub-trees is maximized.

Figure 2: Example of CKY constituency parsing. Figure 3: Chu-Liu-Edmonds
Algorithm with sentence constraints

For example, to pick the structure of the sub-tree spanning EDUs $[3:5]$ (see
Fig. 2), we need to decide between the potential sub-tree aggregation of
$((34)5)$ and $(3(45))$. The respective scores are computed based on the
scores in green and blue blocks in both the CKY and the Attention Matrices.
Following this algorithm, two sub-trees with a high attention score between
them tend to be combined on lower levels of the tree, indicating they are more
related in the discourse tree.

Besides the standard CKY algorithm described above, we also explore a
hierarchical CKY approach with sentence and paragraph constraints.
Specifically, we do not aggregate $P_{ij}$ if the span $[i:j]$ crosses a
sentences boundary where either sentence is incomplete. In the previous
example, if $EDU_{3}$ and $EDU_{4}$ were in the same sentence, even if the
score of the blue aggregation candidate was higher, we would choose the green
sub-tree aggregation. Plausibly, this hierarchical approach will perform
better, since the ground-truth treebanks mostly contain sentences and
paragraphs that are covered by a complete discourse sub-trees.

####  3.2.2 Dependency Tree (D-Tree) Parsing

For the dependency tree generation, we use the Eisner Eisner (1996) and Chu-
Liu-Edmonds algorithm Chu and Liu (1965); Edmonds (1967) to generate
projective and non-projective dependency trees, respectively333“Mixed”
approaches, dealing with mildly non-projective trees Kuhlmann and Nivre
(2006), are left for future work.. First, we convert the attention matrix $A$
into a fully connected graph $G=(N,E)$, where $N$ contains all the EDUs, and
$e_{ij}$, indicating how much the $i$-th EDU influences the $j$-th EDU,
corresponds to $A_{ji}$, which is the attention that the $j$-th EDU pays to
the $i$-th EDU. Based on this graph, we apply the following algorithms:

##### Eisner Algorithm:

We apply this dynamic programming algorithm to generate projective dependency
trees. Thereby, we build a matrix $P\in\mathbf{R}^{n\times n\times 2\times
2}$, in which the first and second dimensions contain the start and end
indexes of sub-trees, similar to the CKY algorithm; while the third and fourth
dimensions indicate whether the head is the start or the end unit, and whether
the sub-tree is completed. As done for constituency parsing, we also use a
hierarchical version of Eisner’s algorithm, in which we restrict inter-
sentence connections for incomplete sentence trees. Since the Eisner algorithm
can only generate projective dependency trees, it will be inaccurate for
documents with a non-projective discourse structure.

##### Chu-Liu-Edmonds (CLE) Algorithm:

Originally proposed as a recursive approach to find the maximum spanning tree
of a graph given its root, CLE can generate non-projective trees. In the
unconstrained case, we simply follow the standard CLE algorithm, selecting the
EDU with the highest importance score, computed similar to Sec. 3.2.1, i.e.
$root=\operatorname*{argmax}_{i}\sum_{k=1}^{n}(A_{ki})$, as the root. From
there, the algorithm selects the “optimal edges", i.e. the maximum in-edges
for each node except the root, breaking the cycles recursively.

Again, as we did for CKY and Eisner, we also apply the additional sentence
constraint. Unlike for the dynamic programming approaches, which build the
trees in a bottom-up fashion and can directly be constrained to avoid cross-
sentence aggregations of incomplete sentences, we need to substantially modify
CLE to allow for sentence constraints.

In particular, we first build a sentence graph $G^{s}=\\{N^{s},E^{s}\\}$ from
the EDU graph (Figure 3 (b)), in which
$e^{s}_{SD}=\operatorname{avg}\limits_{s\in S,d\in D}e_{sd}$, and record the
maximum edge corresponding to the edge between sentences, i.e.
$\operatorname*{argmax}_{s\in S,d\in D}e_{sd}$. After that, we use the CLE
algorithm within the sentence containing the root EDU as the root sentence to
find the maximum spanning tree in $G^{s}$ (Figure 3 (c)). We then add the
corresponding EDU edges to the final tree (Figure 3 (d)). For example, the
edge $(s_{0},s_{1})$ in $G^{s}$ corresponds to the EDU edge $(e_{0},e_{2})$ in
$G$. Next, we treat nodes with incoming edges from other sentences as the root
of the sentence itself and run the CLE algorithm within each sentence (Figure
3 (e)). The final tree (Figure 3 (f)) is eventually formed as the combination
of inter-sentence edges derived in sentence graph $G_{s}$ and intra-sentence
edges found within each sentence.

##  4 Experiments and Analysis

###  4.1 The Summarization Task

In order to show the generality of the discourse structures learned in the
summarization model, we train our summarizer across a variety of datasets and
hyper-parameter settings. More specifically, we train on two separate, widely-
used news corpora – CNN Daily Mail (CNNDM) Nallapati et al. (2016) and NYT
Sandhaus (2008) –, as well as under three hyper-parameter settings with
different numbers of layers and attention heads: (a) A simple model with 2
layers and a single head. (b) 6 layers with 8 heads each, proposed in the
original transformer modelVaswani et al. (2017). (c) 2 layers with 8 heads
each, constituting a middle ground between the previous two settings. By
considering two corpora (CNNDM and NYT) and the three settings, we train six
models, which we call: CNNDM-2-1, CNNDM-6-8, CNNDM-2-8, NYT-2-1, NYT-6-8,
NYT-2-8444Complete evaluation results for all six models are presented in
Appendix A..

###  4.2 Discourse Datasets

The quality of the attention-generated trees is assessed on three discourse
datasets (see Table 1).

RST-DT is the largest and most frequently used RST-style discourse treebank
Carlson et al. (2002), containing news articles from the Wall Street Journal.
Since this is the genre of both our summarization training corpora, the
experiments testing on this dataset are intra-domain.

Instruction-DT contains documents in the home-repair instructions domain Subba
and Di Eugenio (2009). We categorize the experiments on this dataset as cross-
domain.

GUM contains documents from eight domains including news, interviews, academic
papers and more Zeldes (2017). Since the GUM corpus is multi-domain, the
performance on this dataset will reveal the generalizability of generated
trees in a broader sense.

Dataset | # Docs | #EDU/doc | #Sent/doc | #words/doc
---|---|---|---|---
RST-DT | 385 | 56.6 | 22.5 | 549
Instruction | 176 | 32.7 | 19.5 | 318
GUM | 127 | 107 | 45 | 874

Table 1: Key RST-style discourse dataset dimensions.

Model | No Cons. | Sent Cons.
---|---|---
Attn#0 | Attn#1 | Attn#0 | Attn#1
RST-DT
CNNDM-2-1 | 61.2 | 59.7 | 76.2 | 74.6
CNNDM-6-8 | 60.3 | 60.8 | 75.4 | 75.0
NYT-6-8 | 62.4 | 62.2 | 76.7 | 75.6
Random | 58.6$\pm$0.1 | 74.1$\pm$0.1
Instruction
CNNDM-2-1 | 61.1 | 59.8 | 71.4 |  $\downarrow$ 70.3
CNNDM-6-8 | 60.3 | 61.2 | 71.2 | 70.9
NYT-6-8 | 61.3 | 61.3 | 71.3 |  $\downarrow$ 70.0
Random | 59.5$\pm$0.3 | 70.5$\pm$0.1
GUM
CNNDM-2-1 | 58.7 | 57.7 | 72.7 | 71.9
CNNDM-6-8 | 58.9 | 59.3 | 72.4 | 72.7
NYT-6-8 | 59.6 | 59.3 | 72.2 | 71.6
Random | 57.5$\pm$0.1 | 71.5$\pm$0.2

Table 2: RST Parseval Scores of generated constituency trees on the three
datasets, expressed as ’Avg. $\pm$ Std’. Green means the result is better than
Random $+$ Std, and Red along with $\downarrow$ means worse. Results for
Random are obtained by applying the parser to random matrices for 10 times.
Attn#0/1 are the first two layers.

All three discourse datasets contain ground-truth RST-style consituency trees.
While all corpora contain potential non-binary sub-trees, Instruction-DT also
includes multi-root documents. To account for these cases, we apply the right-
branching binarization following Huber and Carenini (2019). Furthermore, we
convert constituency trees with nuclearity into ground truth dependency trees
using the algorithm proposed in Li et al. (2014) .

###  4.3 Evaluation Metric

To evaluate how well the generated trees align with ground-truth trees, we use
RST Parseval Scores for constituency trees and Unlabeled Attachment Score for
dependency trees, measuring the ratio of matched spans and the ratio of
matched dependency relations, respectively.

###  4.4 Overall Results

For each model configuration, we run a set of experiments using the average
attention matrix across all heads in a layer, i.e. $A_{avg}=\sum_{h}A^{h}/H$,
with $H$ as the number of heads. This initial setup is intended to provide
insights into the discourse information learned in each layer.

Model | No Cons. | Sent Cons.
---|---|---
Attn#0 | Attn#1 | Attn#0 | Attn#1
RST-DT
CNNDM-2-1 | 23.7 |  $\downarrow$ 4.8 | 28.2 |  $\downarrow$ 18.2
CNNDM-6-8 |  $\downarrow$ 7.9 | 20.5 |  $\downarrow$ 13.8 | 27.8
NYT-6-8 | 15.7 | 12.5 | 24.3 |  $\downarrow$ 18.9
Random | 11.2$\pm$0.2 | 20.3$\pm$0.2
Instruction
CNNDM-2-1 | 31.1 |  $\downarrow$ 4.4 | 29.3 |  $\downarrow$ 13.5
CNNDM-6-8 |  $\downarrow$ 8.5 | 19.5 |  $\downarrow$ 9.9 | 22.0
NYT-6-8 | 16.2 |  $\downarrow$ 12.1 | 22.8 |  $\downarrow$ 16.4
Random | 13.1$\pm$0.3 | 19.3$\pm$0.4
GUM
CNNDM-2-1 | 21.3 |  $\downarrow$ 2.24 | 27.3 |  $\downarrow$ 16.1
CNNDM-6-8 |  $\downarrow$ 4.7 | 15.8 |  $\downarrow$ 11.5 | 24.80
NYT-6-8 | 12.6 |  $\downarrow$ 9.6 | 23.4 |  $\downarrow$ 17.1
Random | 10.4$\pm$0.2 | 19.2$\pm$0.3

Table 3:  Unlabeled Attachment Scores of dependency trees generated by the
Eisner algorithm.

Model | No Cons. | Sent Cons.
---|---|---
Attn#0 | Attn#1 | Attn#0 | Attn#1
RST-DT
CNNDM-2-1 | 21.6 |  $\downarrow$ 1.5 | 29.3 | 19.6
CNNDM-6-8 | 7.3 | 17.3 |  $\downarrow$ 16.1 | 28.5
NYT-6-8 | 13.7 | 10.6 | 25.0 | 21.1
Random | 1.7$\pm$0.1 | 18.7$\pm$0.1
Instruction
CNNDM-2-1 | 28.1 |  $\downarrow$ 2.1 | 37.4 | 18.1
CNNDM-6-8 | 6.9 | 15.9 |  $\downarrow$ 14.9 | 25.8
NYT-6-8 | 14.8 | 9.8 | 25.4 | 21.1
Random | 2.9$\pm$0.2 | 17.9$\pm$0.4
GUM
CNNDM-2-1 | 19.5 |  $\downarrow$ 0.7 | 28.8 | 17.9
CNNDM-6-8 | 4.0 | 13.1 |  $\downarrow$ 14.9 | 25.4
NYT-6-8 | 10.7 | 8.2 | 23.0 | 19.5
Random | 0.9$\pm$0.05 | 17.0$\pm$0.2

Table 4: Unlabeled Attachment Scores of dependency trees generated by the CLE
algorithm

The results of the three tree-generation algorithms are shown in Table 2, 3
and 4 along with the performance of a random baseline obtained by running the
algorithms on 10 random matrices. Here, we present the results of three
selected models, limited to the performance of the first two layers for the
6-layer models, to allow for a direct comparison to the 2-layer
models555Results for all six models can be found in Appendix B.. Across
evaluations, the layer-wise performance within the same models are rather
distinct, indicating that different properties are learned in the layers. This
finding is in line with previous work Liu et al. (2019a), especially given
that the performance of each layer is consistent across constituency and
dependency parsing outputs for all datasets. Furthermore, the more layers the
summarization model contains, the smaller the performance gap between layers
becomes. We believe that this could be caused by the discourse information
being further spread across different layers. Generally, we observe that
models trained on the CNNDM dataset perform better than models trained on the
NYT corpus, despite the larger size of the NYT dataset. Plausibly, the
superior performance of our models trained on CNNDM potentially reflects a
higher diversity within documents in the CNNDM dataset.

Comparing the constituency tree performance in Table 2 against the dependency
tree results in Tables 3 and 4, we can clearly see that the improvement of the
constituency parsing approach over the random baseline is much smaller than
the improvements for the generated dependency parse-trees. Presumably, this
larger improvement for the dependency trees is due to the fact that dependency
relationships (strongly encoding the nuclearity attribute) are more directly
related to the summarization task than the plain structure information. This
is in line with previous work on applying dependency trees to the
summarization task Hirao et al. (2013); Xu et al. (2020) and indicates that
the learned attention matrices contain valid discourse information.

Figure 4: The Unlabeled Attachment Score of trees generated by the attention
matrix per head on three datasets under two conditions with the CNNDM-6-8
model.

As for the two approaches to dependency parsing, although Eisner generally
outperforms CLE, the improvement over random trees is larger for CLE. We
believe that this effect is due to the reduced constraints imposed on the CLE
algorithm, which is not limited to generate projective trees.

Considering all three methods, the results of the CLE-generated dependency
tree seem most promising. A possible explanation is that both CKY and Eisner
build the discourse tree in a bottom-up fashion with dynamic programming. This
way, only local information is used on lower levels of the tree. On the other
hand, the CLE algorithm uses global information, potentially more aligned with
the summarization task, where all EDUs are considered to predict importance
scores.

###  4.5 Performance of Heads

While all previous results rely on the average attention matrices, we now
analyze whether discourse information is evenly distributed across attention
heads, or if a subset of the heads contains the majority of discourse related
information.

We describe this analysis only for CLE for two reasons: (a) the summarization
model seemingly captures more dependency-related discourse information than
structure information; (b) compared with Eisner, the CLE approach is more
flexible, by also covering non-projective dependency trees.

Since the results across all summarization models are consistent, we only show
the accuracy heatmap for the CNNDM-6-8 model on the three RST-style discourse
datasets in Figure 4. Remarkably, for all three datasets, there is one head in
the model capturing the vast majority of discourse information, especially in
the unconstrained case. Furthermore, the performance of the best single
attention head is much better than the one of the average attention matrix
shown in section 4.4 (e.g. $34.53$ compared to $19.51$ on the GUM dataset
without sentence constraints). These intriguing findings will be further
explored in future work.

###  4.6 Analysis of Generated Trees

##### Localness of Trees:

To further verify that the generated trees are non-trivial, for instance
simply connecting adjacent EDUs, we analyze the quality of the trees produced
with the second attention head on the second layer, which is the top performer
among all the heads shown in Figure 4. First, we separate all dependency
relationships into two classes: local, holding between two adjacent EDUs, and
distant, including all other relations between non-adjacent EDUs.

Measurement(%) | No Cons. | Sent Cons.
---|---|---
RST-DT
Local Ratio Corr. | 77.78 | 79.17
Local Ratio GT | 53.22
Local Ratio Ours | 46.52 | 58.35
Instruction
Local Ratio Corr. | 81.15 | 84.90
Local Ratio GT | 59.82
Local Ratio Ours | 47.90 | 60.54
GUM
Local Ratio Corr. | 77.99 | 80.20
Local Ratio GT | 53.28
Local Ratio Ours | 39.97 | 53.76

Table 5: Measurements on the locality of the generated dependency trees, all
numbers are in $\%$. Corr. represents all the correct predictions, GT the
ground-truth trees, and Ours the generated tree respectively.

Then we compute the ratio of the correctly predicted dependencies which are
local (Local Ratio Corr.), as well as the ratio of local dependencies in the
generated trees (Local Ratio Ours), and in the ground-truth trees (Local Ratio
GT). The results of this analysis are shown in Table 5. For all datasets, the
ratio of correctly predicted local dependencies (Local Ratio Corr.) (being
$>50$) is larger than the ratio for distant relations, which appears
reasonable, since local dependency predictions are easier to predict than
distant ones. Further, comparing (Local Ratio GT) and (Local Ratio Ours)
without the sentence constraint (first column) shows that the number of local
dependency relations in the ground-truth discourse trees is consistently
larger than the predicted number. This indicates that the discourse
information learned in the attention matrices goes beyond the oftentimes
predominant local positional information. However, even without the sentence
constraint (first column), when the CLE algorithm can predict trees of any
form, more than $40\%$ of the relations are predicted as local, suggesting
that the standard CLE approach can already capture local information well.

Adding the sentence constraint (second column), we find that the local
dependency ratio of the generated trees (Local Ratio Ours) further increases
by more than $10\%$ across all three datasets. This makes intuitive sense,
since the sentence constraint forces the generated trees to purely focus on
local aspects within each sentence. To sum up, we find that the learned
attention matrices contains both local and distant dependency information,
although local dependency predictions perform better.

##### Properties of Trees:

Following Ferracane et al. (2019), we structurally inspect the generated
dependency trees, and compare them with the gold trees on all three datasets.
This comparison is presented in Table 6, showing the average branch width,
average height, average leaf ratio (micro) and average normalized arc length
of the trees as well as the percentage of vacuous trees in each dataset666A
vacuous tree is a special tree in which the root is one of the first two EDUs,
with all nodes are children of the root..

Looking at Table 6, it appears that our tree structure properties are similar
to the ground-truth properties in regards to all measures except the height of
the tree, which indicates that our trees tend to be generally deeper than gold
standard trees, despite having a similar branch width and leaf ratio.
Furthermore, our trees are even deeper when using the sentence constraint.
Plausibly, by forcing each sentence to have its own sub-tree can make
shallower inter-sentential structures less likely. Exploring potential causes
for the difference in tree-height, possibly due to the summarization task
itself, are left as future work.

| Branch | Height | Leaf | Arc | vac. (%)
---|---|---|---|---|---
RST-DT
Ours(Sent Cons) | 1.50 | 27.06 | 0.37 | 0.10 | 3%
Ours(No Cons) | 1.74 | 25.76 | 0.49 | 0.12 | 3%
GT Tree | 2.10 | 8.19 | 0.51 | 0.13 | 2%
Instruction
Ours(Sent Cons) | 1.56 | 15.74 | 0.39 | 0.13 | 3%
Ours(No Cons) | 1.80 | 14.35 | 0.50 | 0.14 | 3%
GT Tree | 1.59 | 8.49 | 0.41 | 0.15 | 1%
GUM
Ours(Sent Cons) | 1.61 | 44.94 | 0.40 | 0.05 | 0%
Ours(No Cons) | 2.14 | 43.08 | 0.54 | 0.08 | 0%
GT Tree | 2.02 | 12.17 | 0.51 | 0.04 | 0%

Table 6: Statistics of our generated trees and the gold standard trees in
terms of the average branch width, average height, average leaf ratio (micro),
average normalized arc length of the trees and percentage of the Vacuous
trees.

###  4.7 Additional Results on Model Sensitivity to Initialization and
Summarizer Quality

To investigate whether the performance is consistent cross different random
initializations, and to explore the influence of the results with respect to
the quality of the summarizer, we perform additional experiments with the
’CNNDM-6-8’ model777More details can be found in Appendix C.. Overall, we find
that the performance is rather similar across random initializations.
Interestingly, a single head consistently shows better performance than all
other heads across different initialization as well as datasets; however,
while the position of the top-performing head is not always the same, it is
often located in the second layer of the model. Regarding the second
experiment exploring sensitivity to the summarizer quality, we create
summarizers of increasing quality by providing more and more training. As
expected, we find that as the summarization model is trained for additional
steps, more accurate discourse information is learnt, concentrated in a single
head.

##  5 Conclusions and Future Work

We present a novel framework to infer discourse trees from the attention
matrices learned in a transformer-based summarization model. Experiment across
models and datsets indicates that both dependency and structural discourse
information are learned, that such information is typically concentrated in a
single head, and that the attention matrix also covers long distance discourse
dependencies. Overall, consistent results across datasets and models suggest
that the learned discourse information is general and transferable inter-
domain.

In the future, we want to explore if simpler summarizers like BERTSUM Liu and
Lapata (2019b) can also capture discourse info; specifically studying if the
importance of the heads corresponds to the captured discourse info, which may
help pruning summarization model by incorporating discourse info, in spirit of
Xiao et al. (2020).

With respect to dependency tree generation possible improvements could come by
looking for additional strategies balancing between guidance and flexibility,
as Kuhlmann and Nivre (2006) explore for syntactic dependency parsing.

To address the problem of data sparsity in discourse parsing, we want to
synergistically leverage other discourse-related tasks, in addition to
sentiment and summarization, like topic modeling.

## Acknowledgments

We thank the anonymous reviewers and the UBC-NLP group for their insightful
comments. This research was supported by the Language & Speech Innovation Lab
of Cloud BU, Huawei Technologies Co., Ltd.
We further acknowledge the support of the Natural Sciences and Engineering
Research Council of Canada (NSERC).
Nous remercions le Conseil de recherches en sciences naturelles et en génie du
Canada (CRSNG) de son soutien.

## References

  * Carlson et al. (2002) Lynn Carlson, Mary Ellen Okurowski, and Daniel Marcu. 2002.  _RST discourse treebank_.  Linguistic Data Consortium, University of Pennsylvania.
  * Chu and Liu (1965) Y. Chu and T. Liu. 1965.  On the shortest arborescence of a directed graph.
  * Cohan et al. (2018) Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, and Nazli Goharian. 2018.  A discourse-aware attention model for abstractive summarization of long documents.  _NAACL HLT 2018 - 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \- Proceedings of the Conference_ , 2:615–621.
  * Edmonds (1967) Jack Edmonds. 1967.  _Optimum Branchings_.
  * Eisner (1996) Jason M. Eisner. 1996.  Three new probabilistic models for dependency parsing: An exploration.  In _COLING 1996 Volume 1: The 16th International Conference on Computational Linguistics_.
  * Ferracane et al. (2019) Elisa Ferracane, Greg Durrett, Junyi Jessy Li, and Katrin Erk. 2019.  Evaluating discourse in structured text representations.  _arXiv preprint arXiv:1906.01472_.
  * Hirao et al. (2013) Tsutomu Hirao, Yasuhisa Yoshida, Masaaki Nishino, Norihito Yasuda, and Masaaki Nagata. 2013.  Single-Document Summarization as a Tree Knapsack Problem.  Technical report.
  * Huber and Carenini (2019) Patrick Huber and Giuseppe Carenini. 2019.  Predicting discourse structure using distant supervision from sentiment.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 2306–2316.
  * Huber and Carenini (2020) Patrick Huber and Giuseppe Carenini. 2020.  MEGA RST discourse treebanks with structure and nuclearity from scalable distant sentiment supervision.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 7442–7457, Online. Association for Computational Linguistics.
  * Joty et al. (2015) Shafiq Joty, Giuseppe Carenini, and Raymond T Ng. 2015.  Codra: A novel discriminative framework for rhetorical analysis.  _Computational Linguistics_ , 41(3):385–435.
  * Jurafsky and Martin (2014) Dan Jurafsky and James H Martin. 2014.  _Speech and language processing_ , volume 3.  Pearson London.
  * Kikuchi et al. (2014) Yuta Kikuchi, Tsutomu Hirao, Hiroya Takamura, Manabu Okumura, and Masaaki Nagata. 2014.  Single document summarization based on nested tree structure.  _52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014 - Proceedings of the Conference_ , 2:315–320.
  * Kobayashi et al. (2019) Naoki Kobayashi, Tsutomu Hirao, Kengo Nakamura, Hidetaka Kamigaito, Manabu Okumura, and Masaaki Nagata. 2019.  Split or merge: Which is better for unsupervised rst parsing?  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 5801–5806.
  * Kuhlmann and Nivre (2006) Marco Kuhlmann and Joakim Nivre. 2006.  Mildly non-projective dependency structures.  In _Proceedings of the COLING/ACL 2006 Main Conference Poster Sessions_ , pages 507–514, Sydney, Australia. Association for Computational Linguistics.
  * Li et al. (2014) Sujian Li, Liang Wang, Ziqiang Cao, and Wenjie Li. 2014.  Text-level discourse dependency parsing.  In _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 25–35, Baltimore, Maryland. Association for Computational Linguistics.
  * Liu et al. (2019a) Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, and Noah A. Smith. 2019a.  Linguistic knowledge and transferability of contextual representations.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 1073–1094, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Liu and Lapata (2018) Yang Liu and Mirella Lapata. 2018.  Learning structured text representations.  _Transactions of the Association for Computational Linguistics_ , 6:63–75.
  * Liu and Lapata (2019a) Yang Liu and Mirella Lapata. 2019a.  Hierarchical transformers for multi-document summarization.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 5070–5081, Florence, Italy. Association for Computational Linguistics.
  * Liu and Lapata (2019b) Yang Liu and Mirella Lapata. 2019b.  Text Summarization with Pretrained Encoders.  _EMNLP-IJCNLP 2019 - 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, Proceedings of the Conference_ , pages 3730–3740.
  * Liu et al. (2019b) Yang Liu, Ivan Titov, and Mirella Lapata. 2019b.  Single document summarization as tree induction.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 1745–1755, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Mann and Thompson (1988) William C. Mann and Sandra A. Thompson. 1988.  Rhetorical Structure Theory: Toward a functional theory of text organization.
  * Marcu (1999) Daniel Marcu. 1999.  Discourse Trees are Good Indicators of Importance in Text.  _Advances in Automatic Text Summarization_ , pages 123–136.
  * Mareček and Rosa (2019) David Mareček and Rudolf Rosa. 2019.  From balustrades to pierre vinken: Looking for syntax in transformer self-attentions.  In _Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP_ , pages 263–275, Florence, Italy. Association for Computational Linguistics.
  * McDonald (2007) Ryan McDonald. 2007.  A study of global inference algorithms in multi-document summarization.  In _Proceedings of the 29th European Conference on IR Research_ , ECIR’07, page 557–564, Berlin, Heidelberg. Springer-Verlag.
  * Nallapati et al. (2016) Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Çağlar GuÌ‡lçehre, and Bing Xiang. 2016.  Abstractive text summarization using sequence-to-sequence RNNs and beyond.  In _Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning_ , pages 280–290, Berlin, Germany. Association for Computational Linguistics.
  * Nishida and Nakayama (2020) Noriki Nishida and Hideki Nakayama. 2020.  Unsupervised discourse constituency parsing using viterbi em.  _Transactions of the Association for Computational Linguistics_ , 8:215–230.
  * Raganato and Tiedemann (2018) Alessandro Raganato and Jörg Tiedemann. 2018.  An analysis of encoder representations in transformer-based machine translation.  In _Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP_ , pages 287–297, Brussels, Belgium. Association for Computational Linguistics.
  * Sandhaus (2008) Evan Sandhaus. 2008.  The new york times annotated corpus.
  * Subba and Di Eugenio (2009) Rajen Subba and Barbara Di Eugenio. 2009.  An effective discourse parser that uses rich linguistic information.  In _Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics_ , pages 566–574. Association for Computational Linguistics.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _Advances in Neural Information Processing Systems_ , 2017-Decem(Nips):5999–6009.
  * Wang et al. (2020) Chenguang Wang, Xiao Liu, and Dawn Song. 2020.  Language models are open knowledge graphs.
  * Wang et al. (2017) Yizhong Wang, Sujian Li, and Houfeng Wang. 2017.  A two-stage parsing method for text-level discourse analysis.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 184–188.
  * Wu et al. (2020) Zhiyong Wu, Yun Chen, Ben Kao, and Qun Liu. 2020.  Perturbed masking: Parameter-free probing for analyzing and interpreting BERT.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 4166–4176, Online. Association for Computational Linguistics.
  * Xiao and Carenini (2019) Wen Xiao and Giuseppe Carenini. 2019.  Extractive summarization of long documents by combining global and local context.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3011–3021, Hong Kong, China. Association for Computational Linguistics.
  * Xiao et al. (2020) Wen Xiao, Patrick Huber, and Giuseppe Carenini. 2020.  Do we really need that many parameters in transformer for extractive summarization? discourse can help ! In _Proceedings of the First Workshop on Computational Approaches to Discourse_ , pages 124–134, Online. Association for Computational Linguistics.
  * Xu et al. (2020) Jiacheng Xu, Zhe Gan, Yu Cheng, and Jingjing Liu. 2020.  Discourse-aware neural extractive text summarization.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5021–5031, Online. Association for Computational Linguistics.
  * Yu et al. (2018) Nan Yu, Meishan Zhang, and Guohong Fu. 2018.  Transition-based neural rst parsing with implicit syntax features.  In _Proceedings of the 27th International Conference on Computational Linguistics_ , pages 559–570.
  * Zeldes (2017) Amir Zeldes. 2017.  The GUM corpus: Creating multilayer resources in the classroom.  _Language Resources and Evaluation_ , 51(3):581–612.
  * Zhang et al. (2019) Xingxing Zhang, Furu Wei, and Ming Zhou. 2019.  HIBERT: Document level pre-training of hierarchical bidirectional transformers for document summarization.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 5059–5069, Florence, Italy. Association for Computational Linguistics.

##  Appendix A Performance of the Summarization Task

Table.7 shows the performance of different summarization models. In general,
adding additional layers and heads does not consistently increase the
performance on the summarization task itself.

Dataset | #Layer | #Head | R-1 | R-2 | R-L
---|---|---|---|---|---
CNNDM | 2 | 1 | 40.92 | 18.69 | 37.85
CNNDM | 2 | 8 | 41.02 | 18.78 | 37.96
CNNDM | 6 | 8 | 41.03 | 18.69 | 37.86
NYT | 2 | 1 | 43.64 | 25.58 | 36.87
NYT | 2 | 8 | 44.11 | 26.08 | 37.34
NYT | 6 | 8 | 43.93 | 25.99 | 37.15

Table 7: The in-domain performance of the summarizers.

Model | Const | Eisner | CLE
---|---|---|---
No Cons. | Sent Cons. | No Cons. | Sent Cons. | No Cons. | Sent Cons.
| RSTDT
CNNDM-2-1 | 61.16 / 59.67 | 76.23 / 74.63 |  23.65 / 4.80 |  28.24 / 18.23 |  21.56 / 1.45 |  29.29 / 19.56
CNNDM-2-8 | 62.65 / 59.75 | 76.42 / 74.28 | 22.09 / 8.40 | 26.23 / 21.29 | 20.31 / 6.13 | 26.57 / 22.67
CNNDM-6-8 | 60.33 / 60.79 | 75.44 / 75.04 | 7.89 / 20.48 | 13.83 / 27.78 | 7.28 / 17.30 | 16.10 / 28.50
NYT-2-1 | 60.27 / 60.23 | 75.57 / 75.29 | 9.76 / 14.84 | 23.18 / 20.61 | 6.18 / 12.68 | 21.06 / 21.73
NYT-2-8 |  63.20 / 59.65 | 76.63 / 75.23 | 7.35 / 9.74 | 16.04 / 21.27 | 6.44 / 7.09 | 16.72 / 22.90
NYT-6-8 | 62.42 / 62.17 |  76.65 / 75.58 | 15.74 / 12.51 | 24.30 / 18.90 | 13.71 / 10.59 | 25.04 / 21.14
Random | 58.60 (0.1) | 74.10 (0.1) | 11.16 (0.2) | 20.28 (0.2) | 1.67 (0.08) | 18.72 (0.11)
| Instruction
CNNDM-2-1 | 61.06 / 59.84 | 71.39 / 70.29 |  31.07 / 4.39 |  29.33 / 13.45 |  28.06 / 2.08 |  37.38 / 18.12
CNNDM-2-8 | 61.44 / 60.55 | 71.13 / 71.09 | 26.98 / 8.89 | 24.72 / 14.75 | 24.56 / 5.23 | 28.70 / 20.58
CNNDM-6-8 | 60.32 / 61.22 | 71.24 / 70.88 | 8.53 / 19.51 | 9.93 / 21.96 | 6.92 / 15.85 | 14.93 / 25.78
NYT-2-1 | 60.31 / 61.30 | 71.40 / 71.43 | 10.67 / 21.15 | 18.99 / 21.19 | 7.82 / 17.59 | 21.30 / 24.54
NYT-2-8 | 61.27 / 60.51 | 70.80 / 70.90 | 6.26 / 12.59 | 13.64 / 19.34 | 5.25 / 7.96 | 13.39 / 21.92
NYT-6-8 |  61.32 / 61.27 | 71.30 / 70.03 | 16.22 / 12.14 | 22.79 / 16.37 | 14.81 / 9.81 | 25.44 / 21.10
Random | 59.49 (0.3) | 70.53 (0.1) | 13.14 (0.33) | 19.31 (0.44) | 2.94 (0.24) | 17.88 (0.42)
| GUM
CNNDM-2-1 | 58.74 / 57.69 |  72.73 / 71.92 |  21.28 / 2.24 |  27.26 / 16.12 |  19.50 / 0.70 |  28.77 / 17.92
CNNDM-2-8 | 59.98 / 58.43 | 72.69 / 71.95 | 19.45 / 4.98 | 25.00 / 19.25 | 18.03 / 2.92 | 25.07 / 20.40
CNNDM-6-8 | 58.92 / 59.30 | 72.40 / 72.69 | 4.74 / 15.80 | 11.53 / 24.79 | 4.01 / 13.14 | 14.85 / 25.37
NYT-2-1 | 57.81 / 58.84 | 71.95 / 72.23 | 5.64 / 12.84 | 19.94 / 20.19 | 2.92 / 9.79 | 18.23 / 19.68
NYT-2-8 |  60.17 / 58.22 | 71.98 / 71.82 | 5.66 / 7.22 | 15.21 / 18.81 | 4.54 / 3.96 | 15.25 / 19.31
NYT-6-8 | 59.62 / 59.25 | 72.19 / 71.56 | 12.58 / 9.61 | 23.35 / 17.14 | 10.67 / 8.23 | 22.99 / 19.53
Random | 57.47 (0.1) | 71.50 (0.2) | 10.37 (0.23) | 19.15 (0.26) | 0.92 (0.05) | 17.01 (0.2)

Table 8: The RST Parseval Scores of generated constituency trees, Unlabeled
Attachment Score of generated dependency trees by Eisner algorithm and CLE
algorithm on the three datasets. The numbers in each cell are represented as
the performance of (Layer 0 / Layer 1) the results of Random are obtained by
applying the parser on random generated matrices for 10 times, and are
represented as ’Average (Std)’.

##  Appendix B Full Results on Overall Tree Parsing

We show the overall results of all the six summarization models on
constituency/dependency parsing in Table.8, the results of three of them are
shown in Table.2, Table.3 and Table.4 in the main paper.

Model | Const | Eisner | CLE
---|---|---|---
No Cons. | Sent Cons. | No Cons. | Sent Cons. | No Cons. | Sent Cons.
| RSTDT
CNNDM-6-8 | 61.13 / 61.63 | 75.81 / 75.41 | 10.32 / 20.99 | 16.42 / 27.08 | 9.40 / 18.16 | 18.89 / 28.33
(1.11) / (1.35) | (0.26) / (0.34) | (4.03) / (2.80) | (3.62) / (1.37) | (3.92) / (3.25) | (4.19) / (1.59)
Random | 58.6 (0.1) | 74.10 (0.1) | 11.16 (0.2) | 20.28 (0.2) | 1.67 (0.08) | 18.72 (0.11)
| Instruction
CNNDM-6-8 | 61.87 / 61.06 | 70.84 / 70.94 | 11.50 / 19.78 | 12.91 / 22.45 | 9.79 / 16.53 | 17.81 / 26.30
(1.17) / (0.98) | (0.51) / (0.34) | (5.71) / (2.17) | (4.39) / (1.48) | (5.31) / (2.73) | (4.70) / (1.64)
Random | 59.49 (0.3) | 70.53 (0.1) | 13.14 (0.33) | 19.31 (0.44) | 2.94 (0.24) | 17.88 (0.42)
| GUM
CNNDM-6-8 | 58.19 / 58.71 | 72.28 / 72.48 | 7.62 / 15.69 | 14.62 / 24.02 | 6.77 / 13.23 | 17.32 / 25.13
(0.82) / (0.97) | (0.27) / (0.16) | (3.97) / (2.87) | (3.77) / (1.36) | (3.96) / (3.29) | (3.85) / (1.22)
Random | 57.47 (0.1) | 71.50 (0.2) | 10.37 (0.23) | 19.15 (0.26) | 0.92 (0.05) | 17.01 (0.2)

Table 9: The average RST Parseval Scores of generated constituency trees, the
average Unlabeled Attachment Scores of generated dependency trees by the
Eisner and CLE algorithms, on the three datasets with 5 random initialization,
the numbers in parenthesis are the standard deviation across different run
Figure 5: The heatmap of average UAS across three discourse datasets for all
attention heads in the models with different initialization by the CLE
algorithm.

##  Appendix C Results on Sensitivity to Initialization and Summarizer Quality

To explore if the models with different random initialization have consistent
performances, we train 5 models with 6 layers and 8 heads on the CNNDM dataset
with different initialization, and the results of each layer for
constituency/dependency parsing are shown in Table.9. We can find that the
results are relatively consistent across different initialization, and
additional exploration on the performance of all the heads (Fig.5) show that,
with different initialization of the model, there is consistently one head
containing most of the discourse information, but the position of that head is
not fixed.

We further do the experiments on dependency parsing during training the
summarizer, to see how the performance changes as the summarizer become
better, and show the max and mean UAS over three datasets for all attention
heads in the ’cnndm-6-8’ model by the CLE algorithms after training for (0,
1k, 5k, 10k, 20k) steps in Fig.6. We also show the heatmaps of the average UAS
across three datasets for all the heads in Fig.7. We can find that as the
summarizer is trained for more steps, more discourse information is learned,
and it’s more concentrated in one head. Interestingly, the mean UAS of
dependency trees generated by CLE algorithm with sentence constraints show a
different trend, which may due to the concentration of the discourse
information on single head as the model trained for more steps, as it shows in
Figure.7.

Figure 6: Max and Mean UAS of dependency trees generated by CLE algorithm on
all attention heads (48) of the model ’cnndm-6-8’, after training for
(0,1,5,10,20,23) K steps on RST-DT(top), Instructional(middle) and GUM(bottom)
datasets. The corresponding ROUGE scores are increasing. Figure 7: The
heatmaps of the average UAS across the three discourse datasets for all the
heads during training the summarization model.


# Finite volume effects of the Nambu-Jona-Lasinio model with the running
coupling constant

Shou-Zheng Su1, Ye-Yin Zhao2 and Xin-Jian Wen3 1Basic Experimental Teaching
Center, Anhui Sanlian University, Hefei 230601, China  2Sichuan University of
Science and Engineering(SUSE), Zigong 643000, China<EMAIL_ADDRESS>3Institute of Theoretical Physics, Shanxi University, Taiyuan 030006, China
<EMAIL_ADDRESS>
###### Abstract

With the Schwinger’s proper-time formalism of the Nambu-Jona-Lasinio model, we
investigate the finite volume effects in the presence of magnetic fields.
Since the coupling constant $G$ can be influenced by strong magnetic fields,
the model is solved with a running coupling constant $G(B)$ which is fitted by
the lattice average $(\Sigma_{u}+\Sigma_{d})/2$ and difference
$\Sigma_{u}-\Sigma_{d}$. The investigation mainly focuses on the constituent
quark mass and the thermal susceptibility depending on the magnetic fields,
the temperatures and the finite sizes. For the model in finite or infinite
volume, the magnetic fields can increase the constituent quark mass while the
temperatures can decrease it inversely. There is a narrow range of the box
length that makes the effects of finite volume perform prominently. The model
will behave close to infinite volume limit for larger box length. It is shown
that the influence of finite volume can be changed by magnetic fields and
temperatures. Finally, we discuss the thermal susceptibility depending on the
temperature in finite volume in the presence of magnetic fields.

  *

##  1 Introduction

The investigations of finite volume effects have great importance for the
strongly interacting matter and attract many authors to enthusiastically do
theoretical and experimental work [1]. The strongly interacting matter is
essentially described by the Quantum ChromoDynamics(QCD) which correctly give
most features of interactions between quarks and gluons. Experimentally the
study of strongly interacting matter is mainly performed by the heavy-ion
collisions at present. The strongly interacting matter produced in a heavy-ion
collision always has finite volume which depends on the size of the colliding
nuclei,the collision center of mass energy and the centrality of the collision
[2]. The hadronic fireballs in relativistic nuclear collision reaction have
volumes corresponding to a size of $2fm$ in radius or more [3]. The quark-
gluon plasma(QGP) produced in high-energy heavy-ion collisions, which is also
thought to have permeated the first microseconds of the Universe and to have
cooled sufficiently to transform to hadronic matter soon, has sizes estimated
between $2fm$ and $10fm$ [4, 5]. The fireballs of QGP formed in ultra-
relativistic heavy-ion collisions undergo phase transition in a special range
of temperature, volume and chemical potential [6, 7]. Therefore the volume
plays an important role in the properties of strongly interacting matter
produced in heavy-ion collisions.

The effects of finite volume have been thought over for decades in QCD and,
especially, their analyses are encouraged by the simulations of QCD on finite,
discrete Euclidean space-time lattices [1, 8]. The finite volume effects
appear sensibly until the box sizes up to $L\simeq 5fm$ in the lattice
simulations of light quark mass and strange quark mass [9]. Theoretically the
investigation of finite volume effects has been completed by many effective
approaches such as the Dyson-Schwinger equations of QCD [10, 11, 12], the
quark-meson model [13, 14], the non-interacting bag model [15], the Nambu-
Jona-Lasinio(NJL) model [16, 17, 18, 19], the linear sigma model [20, 21] and
others [22, 23, 24]. The boundary conditions should be imposed on the
effective models when the strongly interacting matter constrained in finite
volume. There are many results worked on antiperiodic boundary condition(APBC)
as well as periodic boundary condition(PBC) [25, 26, 27, 28], sice no
restrictions impose on the spatial direction for finite size systems [19]. In
Ref. [19], the authors also put forward the application of stationary wave
condition(SWC) considering that quark’s wave function equals zero on the
boundary. The effects of finite volume may be induced by the spherical MIT
boundary condition when the strongly interacting matter is considered to be
constrained in a sphere [29]. By means of the Multiple Reflection
Expansion(MRE) formalism, the finite volume effects can be taken into account
in the Polyakov loop Nambu-Jona-Lasinio model and the deconfinement phase
transition could be influenced by a finite radius [30]. In this work we adopt
the well-known antiperiodic boundary condition(APBC).

The magnetic field has been shown by plenty of investigations that it has
great influence on the thermodynamics and the phase transition of strongly
interacting matter [31, 32]. In the presence of magnetic fields, the effects
of finite volume occur consequently and the magnetic catalysis effect remains
in all considered ranges of finite sizes [33]. Furthermore the magnetic field
is also confirmed that it has influence on the coupling constant [34]. Many
authors have made efforts in constructing a magnetic-field-dependent running
coupling constant [35, 36, 37, 38]. As the running coupling constant depends
on magnetic fields, it will have important influence on the phase transition
as well as the stability of quark matter [39, 40]. It is reasonable that we
investigate finite volume effects in this work with a running coupling
constant in the presence of magnetic fields.

In this paper, we will investigate finite volume effects of strongly
interacting matter with the framework of the two-flavor NJL model. In the
following content, we first show a general Schwinger’s proper-time formalism
of the NJL model in the presence of magnetic fields in Sec. 2. The finite
temperatures are taken into account by applying the Matsubara formalism. In
Sec. 3, the model is generalized to finite volume with the antiperiodic
boundary condition. The running coupling constant depending on magnetic field
is determined through fitting dimensionless quantities to the lattice results.
By solving the gap equation in finite volume, we show the numerical results in
Sec. 4 and the results are compared with the cases of infinite volume.
Finally, a short summary is given in Sec. 5.

##  2 Schwinger’s Proper Time Formalism of the NJL model

In the presence of an constant magnetic field, the two flavor Nambu-Jona-
Lasinio model, which is an effective low-energy model for QCD and is superior
to investigate quark matter at finite density or temperature, can be described
by the Lagrangian

$\displaystyle\mathcal{L}_{NJL}=\bar{\psi}(i\not{D}-\hat{m}_{c})\psi+G[(\bar{\psi}\psi)^{2}-(\bar{\psi}\gamma_{5}\vec{\tau}\psi)^{2}].$
(1)

The two flavor quark field $\psi=(\psi_{u},\psi_{d})^{T}$, the current quark
mass matrix is $\hat{m}_{c}=diag(m_{u},m_{d})$ and $G$ is the conpling
constant. For simplicity we adopt $m_{u}=m_{d}=m_{c}$ with the isospin-
symmetric limit. The covariant derivative
$D^{\mu}=\partial^{\mu}+i\hat{Q}A^{\mu}$ with the electric charge matrix
$\hat{Q}=diag(2e/3,-e/3)$ in flavor space and $A^{\mu}$ is the electromagnetic
gauge field. To investigate finite size effects within a constant magnetic
background field $B$, we can choose the Landau gauge $A^{\mu}=(0,-By,0,0)$
which means the magnetic field along $z$ direction. In the mean-field
approximation, the interaction terms are assumed to be deviated small from
their thermal average and then the Lagrangian can be simplified as

$\displaystyle\mathcal{L}_{MF}=\bar{\psi}(i\not{D}-M)\psi+G\langle\bar{\psi}\psi\rangle^{2},$
(2)

where the constituent quark mass $M$ is self-consistently determined by the
gap equation

$\displaystyle M=m_{c}-2G\langle\bar{\psi}\psi\rangle.$ (3)

The thermal average fields $\langle\bar{\psi}\psi\rangle$ in this formula is
called quark condensate. It can be defined by the trace of the dressed quark
propagator

$\displaystyle\langle\bar{\psi}\psi\rangle=-\int\frac{d^{4}p}{(2\pi)^{4}}\Tr[iS(p)].$
(4)

The original purpose of the Schwinger’s proper-time method is to maintain
invariance properties in field calculations [41]. Then it is widely used to
calculate higher loop and investigate the hadron and chiral phase transition
as well as finite volume effects [42]. The Schwinger’s proper-time method
starts with the Green’s function for the particle field

$\displaystyle(i\not{D}-M)S(x,y)=\delta(x,y).$ (5)

In the coordinate space $S(x,y)$ and $\delta(x,y)$ can be regarded as the
matrix elements of operators $\hat{S}$ and $\hat{1}$ respectively.
Consequently the Green’s function can be expressed as

$\displaystyle\hat{S}=\frac{1}{i\not{D}-M}=\frac{-\not{D}+M}{-(\not{D})^{2}+M^{2}}=(-\not{D}+M)i\int_{0}^{\infty}dse^{-is(H-i\epsilon)},$
(6) $\displaystyle S(x,y)=\langle
x|\hat{S}|y\rangle=(-\not{D}+M)i\int_{0}^{\infty}ds\langle
x|e^{-is(H-i\epsilon)}|y\rangle,$ (7)

where we have defined $H=-(\not{D})^{2}+M^{2}$. The meaningful idea of
Schwinger’s proper-time method is to consider $H$ as an Hamiltonian that
describes evolution of some systems in proper time $s$. The state is defined
as $|x(s)\rangle=e^{iHs}|x\rangle$. Then the matrix element of $e^{-iHs}$ can
be viewed as transformation function from a state $|y(s=0)\rangle$ to another
state $|x(s)\rangle$, i.e.

$\displaystyle\langle x|e^{-iHs}|y\rangle=\langle x(s)|y(0)\rangle.$ (8)

The operators also depend upon the proper time parameter and evolute as the
equation of motion in the Heisenberg Picture:

$\displaystyle i\frac{dx_{\mu}}{ds}=[x_{\mu},H]=-2i\Pi_{\mu},$ (9)
$\displaystyle
i\frac{d\Pi_{\mu}}{ds}=[\Pi_{\mu},H]=-2iq_{f}F_{\mu\nu}\Pi^{\mu}.$ (10)

The transformation function can be solved from the differential equations

$\displaystyle i\frac{\partial\langle x(s)|y(0)\rangle}{ds}=\langle
x(s)|H|y(0)\rangle,$ (11) $\displaystyle[i\frac{\partial}{\partial
x^{\mu}}-q_{f}A_{\mu}(x)]\langle x(s)|y(0)\rangle=\langle
x(s)|\Pi_{\mu}(s)|y(0)\rangle,$ (12) $\displaystyle[i\frac{\partial}{\partial
y^{\mu}}-q_{f}A_{\mu}(y)]\langle x(s)|y(0)\rangle=\langle
x(s)|\Pi_{\mu}(0)|y(0)\rangle,,$ (13)

with the boundary condition

$\displaystyle\langle x(s)|y(0)\rangle|_{s\rightarrow 0}=\delta(x-y).$ (14)

Following the details in Ref. [41, 43], the transition function is finally
expressed as

$\displaystyle\langle x(s)|y(0)\rangle=\frac{-i}{(4\pi
s)^{2}}e^{-\frac{i}{4}(x-y)q_{f}F\coth(eFs)(x-y)-\frac{1}{2}Tr\ln\frac{\sinh(q_{f}Fs)}{q_{f}Fs}-is(\frac{q_{f}}{2}\sigma
F+m^{2})},$ (15)

where the integration involved the Wilson line is neglected since it has no
effect on the gap equations. Taking the Fourier transformation of $S(x,y)$ and
carrying out the integration with respect to the coordinate variables, the
quark propagator in momentum space is calculated as

$\displaystyle S(p)=$
$\displaystyle\int_{0}^{\infty}dse^{-is\\{M^{2}-[(p^{0})^{2}-(p^{3})^{2}]+\frac{(p^{1})^{2}+(p^{2})^{2}}{q_{f}Bs\cot(q_{B}s)}\\}}$
(16)
$\displaystyle\times[M-\gamma^{\mu}p_{\mu}-(\gamma^{1}p_{2}-\gamma^{2}p_{1})\tan(q_{f}Bs)][1-\tan(q_{f}Bs)\gamma^{1}\gamma^{2}].$

With the quark propagator Eq. (16), the Schwinger’s proper-time formalism of
the NJL model can be constructed eventually. By taking the trace in the Dirac
space, the flavor space and the color space, the quark condensate can be
calculated as

$\displaystyle\langle\bar{\psi}\psi\rangle=-4MN_{c}\sum_{f=u}^{d}\int\frac{d^{4}p}{(2\pi)^{4}}\int_{0}^{\infty}dse^{-is\\{M^{2}-[(p^{0})^{2}-(p^{3})^{2}]+\frac{(p^{1})^{2}+(p^{2})^{2}}{q_{f}Bs\cot(q_{f}Bs)}\\}}$
(17)

The integration of the quark condensate with respect to momentum carries out
without difficulty using Gaussian integral. After transferring this expression
into the Euclidean space by taking $s\rightarrow-i\tau$, the quark condensate
is finally calculated as

$\displaystyle\langle\bar{\psi}\psi\rangle=-\frac{MN_{c}}{4\pi^{2}}\sum_{f=u}^{d}\int_{0}^{\infty}\frac{d\tau}{\tau}\frac{|q_{f}B|}{\tanh(|q_{f}B|\tau)}e^{-\tau
M^{2}}$ (18)

In order to take account of finite temperature, the integral over the four-
dimensional momentum in Eq. (17) should be replaced by the Matsubara formalism
[44], namely,

$\displaystyle\int\frac{d^{4}p}{(2\pi)^{4}}f(p)\rightarrow
iT\sum_{n=-\infty}^{+\infty}\int\frac{d^{3}p}{(2\pi)^{3}}f(i\omega_{n},\vec{p}).$
(19)

The zeroth component of the momentum is discretized by the fermion Matsubara
frequencies $p_{0}=i\omega_{n}=i(2n+1)\pi T$. As a result, the quark
condensate in finite temperature is expressed as

$\displaystyle\langle\bar{\psi}\psi\rangle=-\frac{MN_{c}}{4\pi^{2}}\sum_{f=u}^{d}|q_{f}B|\int_{0}^{\infty}\frac{d\tau}{\tau}\frac{e^{-\tau
M^{2}}}{\tanh(|q_{f}B|\tau)}\\{1+2\sum_{n=1}^{+\infty}(-1)^{n}e^{-\frac{n^{2}}{4T^{2}\tau}}\\},$
(20)

where we have used the properties of the Jacobi’s theta function

$\displaystyle\vartheta_{3}(z|x)=\sum_{n=-\infty}^{+\infty}e^{i\pi
xn^{2}}e^{2niz}=1+2\sum_{n=1}^{+\infty}e^{i\pi xn^{2}}\cos(2nz).$ (21)
$\displaystyle(-ix)^{\frac{1}{2}}\vartheta_{3}(z|x)=e^{-\frac{iz^{2}}{\pi
x}}\vartheta_{3}(-\frac{z}{x}|-\frac{1}{x}).$ (22)

Since the NJL model is nonrenormalizable for its quadratic fermionic
interaction, it is necessary to employ a regularization scheme to handle
divergent integrals skillfully in the model. In this proper-time formalism we
choose an ultraviolet cutoff $\Lambda$ to replace the lower limit of
integrations in above equations, namely,

$\displaystyle\int_{0}^{+\infty}f(\tau)d\tau\rightarrow\int_{1/\Lambda^{2}}^{+\infty}f(\tau)d\tau.$
(23)

Therefore the cutoff $\Lambda$, together with the current quark mass $m_{c}$
and the coupling strength $G$ are three parameters of the model which shoud be
determined by the pion decay constant $f_{\pi}$, the pion mass $m_{\pi}$, and
the quark condensate in vacuum. In this work we adopt the parameters to be
$m_{c}=4.516MeV$, $\Lambda=1164.1MeV$, $G\Lambda=3.608$ in Ref. [45] where
$f_{\pi}=92.4MeV$, $m_{\pi}=138MeV$ and
$-\langle\bar{\psi}\psi\rangle_{0}^{1/3}=260MeV$.

##  3 Finite volume effects in the presence of magnetic field

The Schwinger’s proper-time formalism of the NJL model in Sec. 2 describes
systems in infinite volume. It is meaningful to generalize the previous
results to the systems of finite volume since quark matter is always produced
in restricted space regions by high-energy collision experiments [5]. Assuming
that the system under consideration is restricted in a box with equal side
lengths $L$, the quantum fields will satisfy boundary conditions leading to
the rules

$\displaystyle\int\frac{d^{3}p}{(2\pi)^{3}}f(p_{1},p_{2},p_{3})\rightarrow\frac{1}{L^{3}}\sum_{n_{1}=-\infty}^{+\infty}\sum_{n_{2}=-\infty}^{+\infty}\sum_{n_{3}=-\infty}^{+\infty}f(\omega_{n_{1}},\omega_{n_{2}},\omega_{n_{3}}),$
(24)

which the momenta are discretized as

$\displaystyle
p_{i}\rightarrow\omega_{n_{i}}=\frac{2\pi}{L}(n_{i}+\alpha),\quad n_{i}=0,\pm
1,\pm 2,....$ (25)

The boundary condition is usually called antiperiodic boundary condition if
$\alpha=1$ and periodic one if $\alpha=0$. In this work we adopt the
antiperiodic boundary condition. Consequently, by replacing the momentum
integration in Eq. (17) with Eq. (19), Eq. (24) and Eq. (25), the quark
condensate at finite temperature and volume can be expressed as

$\displaystyle\langle\bar{\psi}\psi\rangle=$
$\displaystyle-\frac{2MN_{c}}{L^{3}}\sum_{f=u}^{d}\int_{1/\Lambda^{2}}^{\infty}\frac{d\tau}{\sqrt{\pi\tau}}e^{-\tau
M^{2}}\\{1+2\sum_{n=1}^{+\infty}(-1)^{n}e^{-\frac{n^{2}}{4T^{2}\tau}}\\}$ (26)
$\displaystyle\times\sum_{n_{1}=-\infty}^{+\infty}e^{-\frac{\tanh(q_{f}B\tau)}{q_{f}B}\omega_{n_{1}}^{2}}\sum_{n_{2}=-\infty}^{+\infty}e^{-\frac{\tanh(q_{f}B\tau)}{q_{f}B}\omega_{n_{2}}^{2}}\sum_{n_{1}=-\infty}^{+\infty}e^{-\tau\omega_{n_{3}}^{2}},$

where the proper time has been transferred into the Euclidean space and the
ultraviolet cutoff is taken on.

As the quark condensate is given by Eq. (26) in the presence of constant
magnetic field, the gap equation Eq. (3) can be solved with the fixed three
parameters $\Lambda$, $m_{c}$ and $G$. The coupling constant $G$ controls the
strength of strongly interaction in QCD and, however, can be influenced by
sufficiently strong magnetic fields [33, 46]. As a consequence, a appropriate
form of running coupling constant depending on magnetic fields has been
attempted by many authors to fit the lattice results [36, 37, 38, 47]. A
magnetic-field-dependent running coupling constant has influence on the
constituent quark mass as well as the phase transition of the model [39, 47].
In this work we adopt the magnetic-field-dependent running coupling constant
[36]

$\displaystyle
G(B)=\frac{G}{1+\alpha\ln(1+\beta\frac{eB}{\Lambda_{QCD}^{2}})},$ (27)

where $\Lambda_{QCD}^{2}=200MeV$. The free parameters $\alpha$ and $\beta$ are
fixed to get reasonable results of the lattice average
$(\Sigma_{u}+\Sigma_{d})/2$ for $T=0MeV$. The lattice average
$(\Sigma_{u}+\Sigma_{d})/2$ relates to quark condensate in NJL model by
defining the dimensionless quantity [48]

$\displaystyle\Sigma_{f}(B,T)=\frac{2m_{c}}{m_{\pi}^{2}f_{\pi}^{2}}[\langle\bar{\psi}_{f}\psi_{f}\rangle(B,T)-\langle\bar{\psi}_{f}\psi_{f}\rangle(0,0)]+1$
(28)

In figure 1, by solving the model in infinite volume with the running coupling
constant Eq. (27), the average $(\Sigma_{u}+\Sigma_{d})/2$ as well as the
difference $\Sigma_{u}-\Sigma_{d}$ at $T=0MeV$ is fitted to the lattice
results of Ref. [48]. The reasonable results are obtained with $\alpha=2.39$
and $\beta=0.002515$.

Figure 1: The average $(\Sigma_{u}+\Sigma_{d})/2$ and the difference
$\Sigma_{u}-\Sigma_{d}$ at $T=0MeV$ are fitted to the lattice results of Ref.
[48]. The reasonable results are obtained with $\alpha=2.39$ and
$\beta=0.002515$ in the running coupling constant Eq. (27).

##  4 Numerical results

With the Schwinger’s proper-time formalism, the model in the background of
magnetic fields is extended to finite volume with antiperiodic boundary
condition at finite temperatures. As the parameters $\alpha$ and $\beta$ in
the running coupling constant Eq. (27) are fixed with the lattice data, the
model can be presented by solving the gap equation Eq. (3). In this section we
will concentrate on the finite volume effects with the running coupling
constant Eq. (27) in the presence of magnetic fields.

Figure 2: The constituent quark mass $M$ at vanishing temperature depends on
the inverse length $1/L$ in the presence of magnetic fields. The values at
$1/L=0MeV$ correspond to ?those in infinite volume.

When the quark condensate is calculated in finite volume, the constituent
quark mass $M$ solved from the gap equation Eq. (3) will be influenced
apparently by finite volume. In figure 2, the constituent quark mass at
vanishing temperature is presented as a function of the inverse length $1/L$
in the presence of magnetic fields $eB=0.0GeV^{2}$, $eB=0.3GeV^{2}$,
$eB=0.5GeV^{2}$, $eB=0.7GeV^{2}$, $eB=1.0GeV^{2}$. The results at $1/L=0MeV$
correspond to the cases in infinite volume. As the magnetic field becomes
stronger, the constituent quark mass is obviously increased especially when
the box length is close to infinite volume. While the increase falls off as
the box length becomes smaller. When the box length is quite small, for
example $1/L=250MeV$ which corresponds to $L\simeq 0.79fm$, the constituent
quark mass is small enough to close to chiral limit. As the box length $L$
increases away from quite small values, the constituent mass $M$ will sharply
increases for all cases of $eB$ until it is close to the infinite volume
limit. $M$ is close to the infinite volume limit at about $L=15fm$ for the
case of $eB=0.0GeV^{2}$ and about $L=8fm$ for $eB=1.0GeV^{2}$. The box length
that is close to the infinite volume limit is reduced by stronger magnetic
fields.

Figure 3: The partial derivative respecting to length $\partial M/\partial L$
depends on the box length $L$ in the presence of magnetic fields
$eB=0.1GeV^{2}$,$0.3GeV^{2}$ and $0.5GeV^{2}$. The temperatures are
appropriately selected as $T=0MeV$,$150MeV$ and $200MeV$ for investigation.

To find out the range of the box length which sharply affects $M$, the partial
derivative respecting to length $\partial M/\partial L$ is presented in figure
3. Overall, the constituent quark mass $M$ varies sharply with the box length
when $\partial M/\partial L$ is massively greater than $0$. According to the
cases of $T=0MeV$ at $eB=0.1GeV^{2}$, $0.3GeV^{2}$ and $0.5GeV^{2}$, the
constituent quark mass $M$ will varies more sharply at stronger magnetic
field. While it will varies less sharply at higher temperature according to
the cases of $eB=0.5GeV^{2}$ at $T=0MeV$, $150MeV$ and $200MeV$. In addition,
the box length that is close to the infinite volume limit can also be reduced
by higher temperatures. The location of protrusions in figure 3 means that the
constituent quark mass $M$ varies sharply for most cases when the box length
is restricted in a narrow range between about $0.5fm$ and $4fm$. The narrow
range can be reduced by stronger magnetic fields and by higher temperatures.
It means that the finite volume effects could be weakened when the system is
in stronger magnetic fields and higher temperatures.

Figure 4: The constituent quark mass $M$ depends on the temperature $T$ in the
presence of magnetic field $eB=0.7GeV^{2}$ at the length $L=1.7fm$, $1.4fm$,
$1.2fm$, $1.1fm$ and $1.0fm$. The solid line stands for the results in
infinite volume.

Therefore, the box length in the following numerical results is mainly set in
the narrow range where the effects of finite volume appear obviously. In
figure 4, the constituent quark mass $M$ is presented as a function of
magnetic field $eB$ at $L=1.7fm$, $1.4fm$, $1.2fm$, $1.1fm$ and $1.0fm$. For
larger values of length, the behavior of $M$ is close to the line of infinite
volume. Evidently $M$ decreases as the temperature increases as usual for all
cases. While it decreases as the length decreases especially when the
temperature is not high enough. When the temperature becomes high enough, for
example $T=250MeV$, $M$ will decreases slightly by the decrease of $L$.

Figure 5: The constituent quark mass $M$ depends on the magnetic field $eB$ at
the box length $L=2fm$, $1.7fm$, $1.5fm$, $1.2fm$ and $1.0fm$ when the
temperature $T=100MeV$. The solid line stands for the results in infinite
volume.

The dependence of constituent quark mass $M$ on the magnetic field $eB$ is
presented in figure 5 at $L=2fm$, $1.7fm$, $1.5fm$, $1.2fm$ and $1.0fm$ when
the temperature is $T=100MeV$. For larger values of length, the behavior of
$M$ is also close to the line of infinite volume. For the system in infinite
volume, $M$ will decreases as $eB$ increases beginning with $0GeV^{2}$. Soon
afterwards $M$ will apparently increases by the increase of the magnetic
field. Contrasting the lines of finite volume with the infinite volume, the
constituent quark mass $M$ can be significantly decreased by the box length.

Figure 6: The thermal susceptibility $\chi_{T}$ depends on the temperature $T$
in the presence of magnetic field $eB=0.1GeV^{2}$ at the length $L=2fm$,
$1.5fm$, $1.3fm$ and $1.0fm$. The solid line stands for the results in
infinite volume.

The thermal susceptibility is defined as

$\displaystyle\chi_{T}=-m_{\pi}\frac{\partial\sigma}{\partial T},$ (29)

where $\sigma$ is given by

$\displaystyle\sigma=\frac{\langle\bar{\psi}_{u}\psi_{u}\rangle(B,T)+\langle\bar{\psi}_{d}\psi_{d}\rangle(B,T)}{\langle\bar{\psi}_{u}\psi_{u}\rangle(B,0)+\langle\bar{\psi}_{d}\psi_{d}\rangle(B,0)}.$
(30)

In figure 6, the thermal susceptibility $\chi_{T}$ is presented as a function
of temperature $T$ in the presence of magnetic field $eB=0.1GeV^{2}$ at the
box length $L=2fm$, $1.5fm$, $1.3fm$ and $1.0fm$. Similarly the lines of
$\chi_{T}$ is also close to the infinite volume for larger values of the
length. As the box length decreases to small values showed in figure 6,
$\chi_{T}$ will move to the right, which leads to the peaks of the lines lying
at higher temperatures. The peak of the thermal susceptibility $\chi_{T}$
defines a pseudocritical temperature.

##  5 Summary

In this work we investigate the effects of finite volume at finite
temperatures in the presence of magnetic fields with the Schwinger’s proper-
time formalism of the NJL model. The system in finite volume is considered by
the antiperiodic boundary condition. Since the coupling constant $G$ can be
influenced by sufficiently strong magnetic fields, the investigations of this
paper work with the running coupling constant Eq. (27) depending on the
magnetic field. The running coupling constant Eq. (27) is properly determined
by fitting the free parameters with the lattice average
$(\Sigma_{u}+\Sigma_{d})/2$ and difference $\Sigma_{u}-\Sigma_{d}$ at
$T=0MeV$. The numerical results of the model in finite volume are presented by
solving the gap equation.

The magnetic field has the effect that increases the constituent quark mass of
the model in both infinite and finite volume according to the dependence of
$M$ on the inverse length. However, the increase falls off as the box length
becomes smaller. When the length is quite small, the constituent quark mass is
small enough to close to chiral limit. When the value of box length is
appropriately larger, the constituent quark mass depending on the length will
behave close to the infinite volume limit. Especially the box length that is
close to the infinite volume limit can be reduced by stronger magnetic fields
and by higher temperatures. For the system in finite temperatures and volume,
there is a narrow range of the box length that makes the effects of finite
volume perform prominently. This narrow range can also be reduced by stronger
magnetic fields and by higher temperatures.

At finite temperatures, the constituent quark mass decreases by higher
temperatures for both infinite and finite volume in the presence of magnetic
fields. When the temperature is not very high, the constituent quark mass can
decreases obviously as the box length decreases in the narrow range which
effects of finite volume perform prominently. While the constituent quark mass
will decrease slightly when the temperature is high enough. Considering the
dependence of $M$ on the magnetic field at finite temperature, the constituent
quark mass can also decrease obviously as the length decreases in the narrow
range which effects of finite volume perform prominently. For the constituent
quark mass depending on the temperature and the magnetic field, $M$ will
behave close to the infinite volume limit when the box length is appropriate
large. While $M$ has small values when the box length is quite small.

The thermal susceptibility $\chi_{T}$ changes obviously by the effects of
finite volume. In the presence of magnetic fields, $\chi_{T}$ also behave
close to the infinite volume limit when the length has appropriately large
values. The diagram of $\chi_{T}$ will move to the right in the $T$-axis when
the box length decreases in the narrow range which effects of finite volume
perform prominently. Consequently the peaks of the $\chi_{T}$, which define
pseudocritical temperature, move to higher temperatures.

The authors would like to thank support from the National Natural Science
Foundation of China (under the Grant Nos. 11875181, 11705163, 12275102 and
11475110), the National Key Research and Development Program of China (under
the Grant No. 2022YFA1604900) and the Natural Science Foundation of Anhui
Sanlian University (under the Grant Nos. KJZD2021003, KJZD2022010,
KJZD2023007).

## References

## References

  * [1] Klein B, Phys. Rep. 1, 707-708 (2017).
  * [2] Bhattacharyya A, Deb P, Ghosh S K, Ray R, Sur S, Phys. Rev. D 87, 054009 (2013).
  * [3] Koch P, Müller B, Rafelski J, Phys. Rep. 142, 167-262 (1986).
  * [4] Braun-Munzinger P, Stachel J, Nature 448, 302-309 (2007).
  * [5] Palhares L F, Fraga E S, Kodama T, J. Phys. G 38, 085101 (2011).
  * [6] Ramanathan R, Mathur Y K, Gupta K K, Jha K A, Phys. Rev. C 70, 027903 (2004).
  * [7] Wu S, Shen C, Song H, Chin. Phys. Lett 38, 081201 (2021).
  * [8] Hörz B, Hanlon A, Phys. Rev. Lett 123, 142002 (2019).
  * [9] Dürr S, Fodor Z, Hoelbling C, Katz S D, Krieg S, Kurth T, Lellouch L, Lippert T, Szabó K K, Vulvert G, J. High Energy Phys. 08 (2011) 148.
  * [10] Luecher J, Fischer C S, Williams R, Phys. Rev. D 81, 094005 (2010).
  * [11] Li B L, Cui Z F, Zhou B W, An S, Zhang L P, Zong H S, Nucl. Phys. B 938, 298 (2019).
  * [12] Shi C, Jia W, Sun A, Zhang L, Zong H, Chin. Phys. C 42, 023101 (2018).
  * [13] Braun J, Klein B, Pirner H J, Phys. Rev. D 71, 014032 (2005).
  * [14] Braun J, Klein B, Pirner H J, Rezaeian A H, Phys. Rev. D 73, 074010 (2006).
  * [15] Elze H T, Greiner W, Phys. Lett. B 179, 385 (1986).
  * [16] Abreu L M, Malbouisson A P C, Malbouisson J M C, Phys. Rev. D 83, 025001 (2011).
  * [17] Bhattacharyya A, Ray R, Samanta S, Sur S, Phys. Rev. C 91, 041901 (2015).
  * [18] Kohyama H, Kimura D, Inagaki T, Nucl. Phys. B 906, 524 (2016).
  * [19] Wang Q, Xiq Y, Zong H, Mod. Phys. Lett. A 33, 1850232 (2018).
  * [20] Braun J, Klein B, Piasecki P, Eur. Phys. J. C 71, 1576 (2011).
  * [21] Braun J, Klein B, Schefer B J, Phys. Lett. B 713, 216 (2012).
  * [22] Gasser J, Leutwyler, Phys. Lett. B 188, 477 (1987).
  * [23] Samanta S, Ghosh S, Mohanty B, J. Phys. G 45, 075101 (2018).
  * [24] Abreu L M, Nery E S, Phys. Rev. C 96, 055204 (2017).
  * [25] Davoudi Z, Harrison J, Jüttner A, Portelli A, Savage M J, Phys. Rev. D 99, 034510 (2019).
  * [26] Gasser J, Leutwyler H, Nucl. Phys. B 307, 763 (1988).
  * [27] Braun J, Klein B, Pirner H J, Phys. Rev. D 72, 034017 (2005).
  * [28] Carrizal N B M, Ordón̈ez E V, Aguirre A J G, Sotomayor F J B, Ibarra J R M, Universe 8, 264 (2022).
  * [29] Zhang Z, Shi C, Zong H S, Phys. Rev. D 101, 043006 (2020).
  * [30] Grunfeld A G, Lugones G, Eur. Phys. J. C 78, 640 (2018).
  * [31] Andersen J O, Naylor W R, Tranberg A, Rev. Mod. Phys. 88, 025001 (2016).
  * [32] Cao G, Eur. Phys. J. A 57, 264 (2021).
  * [33] Abreu L M, Corrêa E B S, Linhares C A, Malbouisson A P C, Phys. Rev. D 99, 076001 (2019).
  * [34] Miransky V A and Shovkovy I A, Phys. Rev. D 66, 045006 (2002).
  * [35] Moreira J, Costa P, Restrepo, Phys. Rev. D 102, 014032 (2020).
  * [36] Farias R L S, Gomes K P, Krein G, Pinto M B, Phys. Rev. C 90, 025203 (2014).
  * [37] Ferreira M, Costa P, Lourenco O, Frederico T, and Providência C, Phys. Rev. D 89, 116011 (2014).
  * [38] Farias R L S, Timóteo V S, Avancini S S, Pinto M B, and Krein G, Eur. Phys. J. A 53, 101 (2017).
  * [39] Su S Z, Wen X J, J. Phys. G 48, 075004 (2021).
  * [40] Costa P, Ferreira M, Menezes D P, Moreira J and Providência C, Phys. Rev. D 92, 036012 (2015).
  * [41] Schwinger J, Phys. Rev. 82, 664 (1951).
  * [42] Cao G, He L, Zhuang P, Phys. Rev. D 90, 056005 (2014).
  * [43] Cao G, Huang X G, Phys. Rev. D 93, 016007 (2016).
  * [44] Kapusta J I, Gale C, Finite-Temperature Field Theory, Cambridge University Press, Cambridge, 2006.
  * [45] Avancini S S, Farias R L S, Scoccola N N and Tavares W R, Phys. Rev. D 99, 116002 (2019).
  * [46] Prosperi G M, Raciti M, Simolo C, Progr. Part. Nucl. Phys. 58, 387 (2007).
  * [47] Andersen J O, Naylor W R and Tranberg A, J. High Energy Phys. 02 (2015) 042.
  * [48] Bali G S, Bruckmann F, Endrödi G, Fodor Z, Katz S D, Schäfer A, Phys. Rev. D 86, 071502(R) (2012).