# Aspect and Opinion Term Extraction Using Graph Attention Network

Abir Chakraborty
Microsoft
Abir.Chakraborty@microsoft

###### Abstract

In this work we investigate the capability of Graph Attention Network for
extracting aspect and opinion terms. Aspect and opinion term extraction is
posed as a token-level classification task akin to named entity recognition.
We use the dependency tree of the input query as additional feature in a Graph
Attention Network along with the token and part-of-speech features. We show
that the dependency structure is a powerful feature that in the presence of a
CRF layer substantially improves the performance and generates the best result
on the commonly used datasets from SemEval 2014, 2015 and 2016. We experiment
with additional layers like BiLSTM and Transformer in addition to the CRF
layer. We also show that our approach works well in the presence of multiple
aspects or sentiments in the same query and it is not necessary to modify the
dependency tree based on a single aspect as was the original application for
sentiment classification.

_K_ eywords aspect $\cdot$ opinion $\cdot$ graph attention $\cdot$ CRF

##  1 Introduction

Extracting information from customer feedback is a key capability required for
identifying current drawbacks and scope for further improvement. Online
shoppers routinely provide feedback on their experience with the purchased
product that are not just important for the other potential customers but also
a critical feedback to the product manufacturers for the next cycle of
iteration. Similar feedbacks are available in various other domains ranging
from Manufacturing to Healthcare where granular opinions (sentiments) about
various dimensions (aspects) of the used product (or service) are available in
textual form but need to be understood. Due to the presence of multiple
aspects (and the corresponding sentiments) extraction of these aspect-
sentiment pairs is a challenging task and since its introduction in 2014
(SemEval-2014 Task-4, Pontiki et al.) Aspect Based Sentiment Analysis (ABSA)
attracted various different approaches and is still under active
consideration.

ABSA demands semantic understanding of the sentence where it is necessary to
identify the aspect terms (defining "what") and the opinion terms (defining
"why") and the connection between each related pairs (resulting in a positive,
neutral or negative sentiment, i.e., "how"). A relatively simple example would
be “The price is reasonable although the service is poor” where "price" and
"service" are the aspects with the corresponding opinion terms "reasonable"
and "poor", respectively. Here, the relative locations of the aspect and
opinion word can help each other to identify these terms. However, other
examples like “prices are in line” (neutral sentiment) and “For the price, you
can not eat this well in Manhattan” (positive sentiment) it is not obvious
which opinion terms are driving the sentiments and their linkage with the
aspect terms. As a result, while it is possible to capture the syntactical
structure and dependency between words it is necessary to capture a deeper
meaning of each token (word) that should not be relied upon the limited number
of examples (that are hallmark of all ABSA datasets) but rich representations
provided by large language models like BERT and RoBERTA.

Dataset |  | Train | Dev | Test | Total
---|---|---|---|---|---
LAPTOP | #sent | 2741 | 304 | 800 | 4245
| #aspect | 2041 | 256 | 634 | 2931
REST | #sent | 3490 | 387 | 2158 | 6035
| #aspect | 3893 | 413 | 2287 | 6593
Table 1: Statistics of the dataset in Li et al. (2019c)

In addition to large model based initial representations, additional
information from parts-of-speech and dependency structure play a critical role
as can be seen in some of the previous ABSA work where the objective is to
predict only the sentiment associated with a particular aspect. While it is
generally understood that a confluence of deep encoders and graph-based
representation of the input sentence will drive better performance it is not
clear what is the optimal graph representation of a sentence, especially for
ABSA kind of task. In case of polarity detection, it can be argued that only
the part of the dependency tree associated with a particular aspect is
significant and the rest of the tree can be ignored. However, when we convert
the task into aspect and opinion term extraction it is not clear whether any
part of the sentence has higher significance compared to the rest.

In this work we address the problem of aspect and opinion term extraction from
input sentences. As an example, "the weather was gloomy, but the food was
tasty" has two sets of aspect-opinion-sentiment tuples, i.e., (weather,
gloomy, negative) and (food, tasty, positive). We use ABSA datasets with token
level labels - one set for the aspects and one set for the opinions. Thus, the
aspect and opinion term extraction becomes a NER task with different types of
entity classes. The tags for the aspects include the sentiment, (for opinion
it is only the BIEOS tag), however, what is missing is the connection between
the aspects and opinion terms. Thus, our aspect models predict (weather,
negative) and (food, positive), opinion models predict (gloomy) and (tasty),
without making the subsequent contractions. We encode the graph associated
with the dependency parsing of the input sentence and create token (node)
level representations that take care of both the neighborhood (connected
edges) and dependency type (edge type) of each token.

The organization of the paper is as follows. In the next section we provide a
detailed literature survey on the techniques employed for aspect and opinion
extraction task of ABSA. Next, we present the details of the proposed model.
Subsequently, the model predictions and comparisons with other baseline
methods are discussed. Finally, conclusions are drawn and scope for future
works is outlined.

##  2 Related Work

Most of the ABSA work concern with the sentiment polarity detection associated
with a particular aspect and the approaches varied from initial SVM classifier
with handcrafted features to deep learning classifiers based on RNN (Wang et
al., 2016, 2018; He et al., 2018; Ma et al., 2017), Transformer (Hoang et al.,
2019; Zeng et al., 2019; Xu et al., 2019) and memory network (Tang et al.,
2016; Chen et al., 2017; Tay et al., 2017; Lin et al., 2019). There are few
graph based approaches as well, e.g., graph convolution network (GCN) based
model of Zhang et al. (2019) and Sun et al. (2019), graph attention network
(GAT) based model of Huang and Carley (2019) and Wang et al. (2020) where the
latter modified the original dependency tree to create an aspect-oriented
dependency tree that was used further in a relational GAT (R-GAT) where
different relations contributed differently in the computation of nodal
representations. However, the approach assumes the presence of only one aspect
at a time in the given input.

On the other hand, an end-to-end ABSA tries to extract all the aspect terms in
a given query simultaneously along with the corresponding sentiments. The
first approach towards E2E-ABSA was provided by Li et al. (2019c) where a
unified tagging scheme (combining the position and sentiment) was used, i.e.,
the token labels were one of B-{POS,NEG,NEU}, I-{POS,NEG,NEU},
E-{POS,NEG,NEU}, S-{POS,NEG,NEU} or O, denoting the beginning, inside or end
of an aspect, single-word aspect, with positive, negative or neutral sentiment
respectively, and finally the outside of that aspect. In this work, BERT
(Devlin et al., 2019) was used to embed the tokens and other layers like, GRU,
Transformer (Vaswani et al., 2017) or CRF was used on the BERT output.

Li et al. (2019c) ignored the opinion term extraction and that was addressed
by Peng et al. (2020a) who proposed a two-stage pipeline framework. In the
first stage, aspect-sentiment pairs were extracted using the above mentioned
unified tagging scheme. In addition, opinion spans were extracted using BIEOS
tagging scheme. The aspect-sentiment and opinion pairs thus extracted were
matched against each other in the second stage where an MLP-based classifier
was used to find the compatibility of these pairs. Zhang et al. (2020)
proposed a multi-task framework to jointly detect aspects, opinions, and
sentiment dependencies. However, instead of using the unified tagging scheme
they used two sets of BIOS tags to identify the aspects and sentiments before
connecting them with the corresponding sentiment. While all these approaches
extract aspects and opinion pairs in isolation a recent approach based on
pointer network is proposed by Mukherjee et al. (2021) where an encoder-
decoder architecture is used to generate all the aspect-opinion-sentiment
tuples.

Dataset | Train | Dev | Test | Total
---|---|---|---|---
| #s | #p | #s | #p | #s | #p | #s
LAPTOP’14 | 920 | 1265 | 228 | 337 | 339 | 490 | 1487
REST’14 | 1300 | 2145 | 323 | 524 | 496 | 862 | 2119
REST’15 | 593 | 923 | 148 | 238 | 318 | 455 | 1059
REST’16 | 842 | 1289 | 210 | 316 | 320 | 465 | 1372
Table 2: Statistics of the dataset ASTE-V1 as given in Peng et al. (2020b) (#s and #p denote the number of sentences and aspect-opinion pairs, respectively. Model | Laptop | Restaurant
---|---|---
|  | P | R | F1 | P | R | F1
| Li et al. (2019a) | 61.27 | 54.89 | 57.90 | 68.64 | 71.01 | 69.80
Existing Models | Luo et al. (2019) | - | - | 60.35 | - | - | 72.78
| He et al. (2019) | - | - | 58.37 | - | - | -
| Lample et al. (2016) | 58.61 | 50.47 | 54.24 | 66.10 | 66.30 | 66.20
LSTM-CRF | Ma and Hovy (2016) | 58.66 | 51.26 | 54.71 | 61.56 | 67.26 | 64.29
| Liu et al. (2018) | 53.31 | 59.40 | 56.19 | 68.46 | 64.43 | 66.38
| BERT+Linear | 62.16 | 58.90 | 60.43 | 71.42 | 75.25 | 73.22
| BERT+GRU | 61.88 | 60.47 | 61.12 | 70.61 | 76.20 | 73.24
BERT Models | BERT+SAN | 62.42 | 58.71 | 60.49 | 72.92 | 76.72 | 74.72
Li et al. (2019c) | BERT+TFM | 63.23 | 58.64 | 60.80 | 72.39 | 76.64 | 74.41
| BERT+CRF | 62.22 | 59.49 | 60.78 | 71.88 | 76.48 | 74.06
Our Model | RGAT-BERT | 62.17 | 60.08 | 61.11 | 69.70 | 73.29 | 71.45
| RGAT-BERT-CRF | 64.72 | 59.09 | 61.78 | 73.15 | 69.82 | 71.45
| RGAT-BERT-BiLSTM-CRF | 65.34 | 61.46 | 63.34 | 75.19 | 71.89 | 73.48
| RGAT-BERT-TRFMR-CRF | 65.03 | 60.28 | 62.56 | 80.16 | 74.01 | 76.96
Table 3: Comparison of predictions on the aspect extraction dataset of Li et
al. (2019c). The best F1-scores are shown in bold and the second best ones are
underlined.

##  3 Methodology

We have modified the R-GAT approach of Wang et al. (2020) which was originally
targeted for polarity prediction. The approach utilizes the dependency
structure of the input sentence which captures the grammatical structure by
connecting the words with the corresponding dependency type. The limitations
of common approaches that do not pay attention to the parts-of-speech and the
dependency structure are discussed by Wang et al. (2020) and the importance of
the syntactic relations are emphasized, especially in the context of the
aspect word.

###  3.1 Aspect Oriented Dependency Tree

To bring out the relations of different words with the aspect word(s) an
Aspect Oriented Dependency Tree (AODT) was proposed where the root of the
original dependency tree was shifted to the target aspect word followed by
pruning some of the unnecessary relations. It is to be noted that this
approach assumes that there is no more than one aspect word (with the
possibility of multiple sub-words) in the sentence. As a result, this approach
cannot be applied directly for sentences with multiple aspect words and
sentences with no identified aspect word (say, for opinion term extraction).

###  3.2 Relational Graph Attention Network

AODT can be represented by a graph structure where each node is a word and the
edges between them are represented by the dependency relation, e.g., nominal
subject, adverbial modifier, etc. Given a neighborhood of a node
$\mathcal{N}_{i}$, the node embeddings can be iteratively updated using multi-
head attention as

$h^{l+1}_{{att}_{i}}=concat_{k=1}^{K}\sum_{j\in\mathcal{N}_{i}}\alpha_{ij}^{lk}W_{k}^{l}h_{j}^{l},$
(1) $\alpha_{ij}^{lk}=attention(i,j),$ (2)

where $h^{l+1}_{{att}_{i}}$ is the attention head of node-$i$ at layer $l+1$
and $\alpha_{ij}^{lk}$ is the normalized attention coefficient computed by the
$k$-th attention at layer $l$ and $W^{l}_{k}$ is an input transformation
matrix.

In addition to the attention head of word-$i$ a relational head is also
computed for this node as

$h^{l+1}_{{rel}_{i}}=concat_{m=1}^{M}\sum_{j\in\mathcal{N}_{i}}\beta_{ij}^{lm}W_{m}^{l}h_{j}^{l},$
(3) $g_{ij}^{lm}=\sigma(relu\left(r_{ij}W_{m1}+b_{m1}\right)W_{m2}+b_{m2})$
(4)
$\beta_{ij}^{lm}=exp(g_{ij}^{lm})/\sum_{j\in\mathcal{N}_{i}}exp(g_{ij}^{lm})$
(5)

where $r_{ij}$ denotes the relation embedding between node-$i$ and $j$. The
final representation of each word (node) is a concatenation of the attention
and relational embeddings:

$x_{i}^{l+1}=concat(h^{l+1}_{{att}_{i}},h^{l+1}_{{rel}_{i}})$ (6)
$h^{l+1}_{i}=relu\left(W_{l+1}x_{i}^{l+1}+b_{l+1}\right)$ (7)

###  3.3 Named Entity Recognition

While the R-GAT model utilizes only the root representation to predict the
sentiment polarity, here, we use the node representation $h^{l+1}_{i}$ for
node-$i$ to predict the corresponding NER tag.

####  3.3.1 Linear Layer

In this case, $h^{l+1}_{i}$ is directly fed into a fully connected layer with
softmax activation function that generates probabilities over all the NER
classes, $\mathbf{C}$

$p(y_{i}=\mathbf{C})=softmax(W_{p}h^{l+1}_{i}+b_{p}).$ (8)

where $y_{i}$ is NER target label for the word-$i$ and $W_{p},b_{p}$ are
learnable parameters.

####  3.3.2 RNN Layer

While Li et al. (2019c) applied GRU we have used a (Bi-directional Long Short
Term Memory) BiLSTM layer on $h^{l+1}_{i}$ to capture the contributions of the
neighbors of each token. An LSTM layer updates the input $h^{l+1}$ as

$\displaystyle i_{t}=\sigma\left(W_{i}[x_{t};h_{t-1}]+b_{i}\right),$
$\displaystyle f_{t}=\sigma\left(W_{f}[x_{t};h_{t-1}]+b_{f}\right),$
$\displaystyle o_{t}=\sigma\left(W_{o}[x_{t};h_{t-1}]+b_{o}\right),$
$\displaystyle\tilde{c}_{t}=\tanh\left(W_{c}[x_{t};h_{t-1}]+b_{c}\right),$
$\displaystyle c_{t}=f_{t}\odot c_{t-1}+i_{t}\odot\tilde{c}_{t},$
$\displaystyle h_{t}=o_{t}\odot\tanh(c_{t}),$

where $\odot$ denotes the Hadamard product,
$W_{i},W_{f},W_{o},W_{c},b_{i},b_{f},b_{o}$ and $b_{c}$ are learnable
parameters. The input to the LSTM layer $x_{t}=h^{l+1}_{t}$ and the output is
$h^{l+2}_{t}$. A Bi-LSTM layer processes both the original sequence
($\overrightarrow{x}$) and the sequence in reverse order ($\overleftarrow{x}$)
together so that contexts from both left and right are captured for each
token. The final representation of each token is obtained by concatenating the
outputs, i.e., $h^{l+2}=[\overrightarrow{h};\overleftarrow{h}]$ where
$\overrightarrow{h}=LSTM(\overrightarrow{h}^{l+1})$ and
$\overleftarrow{h}=LSTM(\overleftarrow{h}^{l+1})$.

####  3.3.3 Transformer Layer

While an LSTM layer processes the sequence uni-directionally and the effects
of neighboring tokens are accumulated one at a time, a Transformer (Vaswani et
al., 2017) computes attention between all tokens irrespective of the
direction. We apply a Transformer layer that takes $h^{l+1}$ as input and
generates another sequence of token representations, $h^{l+2}$, which is of
the same length.

####  3.3.4 CRF Layer

For all the three previous cases, we have a final layer that generates a score
matrix $\mathbf{P}$ over all the tag classes, where $P_{ij}$ is the score of
the $j-$th tag for the $i-$th input token. The CRF layer learns a transition
probability matrix $A\in\mathbb{R}^{K+2\times K+2}$ where $K$ is the number of
tag classes (and +2 indicates one tag each for the start and the end marker).
For an input sequence $\mathbf{x}$ of length $T$ and the corresponding tags
$\mathbf{y}=\\{y_{1},y_{2},\ldots,y_{T}\\}$ (with the start and end tag
denoted by $y_{0}$ and $y_{T+1}$, the probability score is modified as

$S(\mathbf{x},\mathbf{y})=\sum_{i=0}^{T}A_{y_{i}y_{i+1}}+\sum_{i=1}^{T}P_{i,y_{i}}$
(9)

which is used to calculate the probability of the tag sequence as

$p(\mathbf{y}|\mathbf{x})=e^{S(\mathbf{x},\mathbf{y})}/\sum_{y^{\prime}\in\mathbf{y}}e^{S(\mathbf{x},\mathbf{y^{\prime}})}$
(10)

which replaces Eq. 8. The loss for each word is calculated by the categorical
cross-entropy loss

$\mathcal{L}(\theta)=-\sum_{S\in\mathcal{D}}\sum_{w_{i}\in S}\log(p_{ij})$
(11)

where $p_{ij}$ is the probability of the word-$i$ with label-$j$ where the
word-$i$ appeared in sentence-$S$ and $\mathcal{D}$ represents the entire
corpus. For CRF based models, the loss is calculated using the forward-
backward algorithm.

Model | Restaurant’14 | Laptop’14
---|---|---
| P | R | F1 | P | R | F1
RINANTE | 48.97 | 47.36 | 48.15 | 41.20 | 33.20 | 36.70
CMLA | 67.80 | 73.69 | 70.62 | 54.70 | 59.20 | 56.90
Li-unified | 74.43 | 69.26 | 71.75 | 68.01 | 56.72 | 61.86
Li-unified-R | 73.15 | 74.44 | 73.79 | 66.28 | 60.71 | 63.38
(Li et al., 2019c)–BLSTM | 70.00 | 74.20 | 72.04 | 65.99 | 54.62 | 59.77
(Li et al., 2019c)–TG | 74.41 | 73.97 | 74.19 | 64.35 | 60.29 | 62.26
(Li et al., 2019c)–T | 69.42 | 72.2 | 70.79 | 64.14 | 60.63 | 62.34
(Li et al., 2019c) | 76.60 | 67.84 | 71.95 | 63.15 | 61.55 | 62.34
RGAT-BERT | 71.70 | 78.80 | 75.08 | 60.69 | 62.74 | 61.69
RGAT-BERT-CRF | 82.69 | 78.21 | 80.39 | 70.38 | 62.53 | 66.22
RGAT-BERT-BiLSTM-CRF | 81.20 | 79.39 | 80.28 | 70.43 | 68.21 | 69.30
RGAT-BERT-TRFMR-CRF | 86.04 | 78.44 | 82.07 | 68.94 | 68.21 | 68.57
Table 4: Comparison of predictions on the aspect extraction dataset of Peng et al. (2020b). The best F1-scores are shown in bold and the second best ones are underlined. Model | Restaurant’15 | Restaurant’16
---|---|---
| P | R | F1 | P | R | F1
RINANTE | 46.20 | 37.40 | 41.30 | 49.40 | 36.70 | 42.10
CMLA | 49.90 | 58.00 | 53.60 | 58.90 | 63.60 | 61.20
Li-unified | 61.39 | 67.99 | 64.52 | 66.88 | 71.40 | 69.06
Li-unified-R | 64.95 | 64.95 | 64.95 | 66.33 | 74.55 | 70.20
(Li et al., 2019c)–BLSTM | 63.41 | 65.19 | 64.29 | 69.74 | 71.62 | 70.67
(Li et al., 2019c)–TG | 59.28 | 61.92 | 60.57 | 64.57 | 66.89 | 65.71
(Li et al., 2019c)–T | 62.28 | 66.35 | 64.25 | 62.65 | 71.4 | 66.74
(Li et al., 2019c) | 67.65 | 64.02 | 65.79 | 71.18 | 72.30 | 71.73
RGAT-BERT | 62.37 | 71.59 | 66.67 | 66.35 | 78.15 | 71.77
RGAT-BERT-CRF | 73.10 | 70.19 | 71.62 | 84.63 | 80.63 | 82.58
RGAT-BERT-BiLSTM-CRF | 76.03 | 73.71 | 74.85 | 84.61 | 81.76 | 83.16
RGAT-BERT-TRFMR-CRF | 75.13 | 68.78 | 71.81 | 84.36 | 80.18 | 82.22
Table 5: Comparison of predictions on the aspect extraction dataset of Peng et
al. (2020b). The best F1-scores are shown in bold and the second best ones are
underlined.

##  4 Experiments

###  4.1 Datasets

We used two datasets in our experiments. The first one is created by Li et al.
(2019c), which is originating from SemEval’14 (Pontiki et al., 2014) but
modified by Li et al. (2019b). The statistics of the dataset are summarized in
Table 1 where the number of sentences (queries) and aspects are shown in two
domains, namely, Laptop and Restaurant, across train, validation and test set.
The second dataset is called Aspect Sentiment Triplet Extraction (ASTE)
dataset (version-1) as created by Peng et al. (2020a) where each sentence has
a unified aspect/target tags and opinion tags. The details of the dataset are
shown in Table 2.

Model | Restaurant’14 | Laptop’14
---|---|---
| P | R | F1 | P | R | F1
Distance rule | 58.39 | 43.59 | 49.92 | 50.13 | 33.86 | 40.42
Dependency rule | 64.57 | 52.72 | 58.04 | 45.09 | 31.57 | 37.14
RINANTE | 81.06 | 72.05 | 76.29 | 78.20 | 62.70 | 69.60
CMLA | 69.47 | 74.53 | 71.91 | 51.80 | 65.30 | 57.70
IOG | 82.85 | 77.38 | 80.02 | 73.24 | 69.63 | 71.35
Li-unified-R | 81.20 | 83.18 | 82.13 | 76.62 | 74.90 | 75.70
(Li et al., 2019c)–BLSTM | 80.41 | 86.19 | 83.15 | 78.06 | 68.98 | 73.19
(Li et al., 2019c)–TG | 81.77 | 84.80 | 83.21 | 76.87 | 75.31 | 76.03
(Li et al., 2019c)–T | 80.61 | 85.38 | 82.88 | 76.69 | 73.88 | 75.21
(Li et al., 2019c) | 84.72 | 80.39 | 82.45 | 78.22 | 71.84 | 74.84
RGAT-BERT | 82.05 | 86.43 | 84.18 | 74.71 | 80.20 | 77.36
RGAT-BERT-CRF | 95.43 | 89.56 | 92.40 | 93.97 | 79.59 | 86.19
RGAT-BERT-BiLSTM-CRF | 95.64 | 91.53 | 93.54 | 93.42 | 84.08 | 88.51
RGAT-BERT-TRFMR-CRF | 94.58 | 89.09 | 91.76 | 93.55 | 82.86 | 87.88
Table 6: Comparison of predictions on the opinion extraction dataset of (Peng
et al., 2020b) based on the SemEval’14 (Pontiki et al., 2014) task.

###  4.2 Implementation Details

We use the bi-affine parser (Dozat and Manning, 2016) from AllenNLP for
dependency parsing. While Wang et al. (2020) used the aspect words to orient
the dependency tree, in our case, we cannot use that information. Instead, we
randomly choose a noun word (adjective for opinion tagging) from the input
sentence, if available, otherwise, we select the middle token about which the
dependency tree is re-oriented. For all experiments, the embedding dimension
for the dependency relation is set to 200 and the dropout is fixed at 0.3. The
last hidden state of the pre-trained
BERT111https://github.com/huggingface/transformers is used for the initial
token representations which is subsequently fine-tuned. All models are trained
using Adam optimizer (Kingma and Ba, 2014) with the default parameters. For
all experiments we have used RGAT based feature extraction and BERT based
token encoding. There are four variants of our model, namely, (1) RGAT-BERT,
that does not use any other layer, (2) RGAT-BERT-CRF, that additionally uses a
CRF final layer, (3) RGAT-BERT-BiLSTM-CRF, that uses a Bi-LSTM layer on the
output of BERT before passing the output to a CRF layer and (4) RGAT-BERT-
TRFMR-CRF that uses a Transformer layer instead of a Bi-LSTM.

###  4.3 Results and Discussions

For all experiments we report only the F1-score (and omit precison and recall
due to space limitation) across all the tags. Table 3 shows the performance of
different RGAT models on the dataset of Li et al. (2019c). As can be seen,
RGAT models provide the best F1 score for the Laptop (BiLSTM) and Restaurant
(Transformer) domain surpassing the previous best F1 scores by more than 3%
point. For the laptop domain, the second best result is also given by another
RGAT model (Transformer based), whereas, for the Restaurant domain one of the
prior models stand out (BERT+self-attention network from Li et al. (2019c)).

For the second dataset, we extract aspects and opinions from four domains,
Laptop, Restaurant-2014, Restaurant-2015 and Restaurant-2016. Table 4 and 5
together show the results of aspect extraction for all the four domains. For
aspect extraction, it can be seen that the best F1-score is obtained by the
Transformer and BiLSTM models for the Restaurant-2014 and Laptop domain,
respectively. The second best results are also obtained by our models, by CRF
and Transformer based models. We can see substantial improvement (6-8% point)
over the previously reported F1 scores for both the cases. Even larger
improvement can be seen for the other two Restaurant domain data (2015 and
2016) where the BiLSTM-CRF model generates the best F1 score.

Table 6 shows the performance on the opinion extraction task where the best F1
scores are provided by the BiLSTM-CRF model for both the Restaurant-2014 and
Laptop domain. Here we see 10-12 % point improvement over the best previous F1
score. Following the same trend of aspect extraction, for Restaurant-2015 and
2016 data (shown in Table 7), we see the best performance coming from the
BiLSTM-CRF model exceeding the previous best scores by more than 10% point.
Overall, we can see that all the RGAT-BERT based models perform much better
than all the other baseline models on all the tasks.

Model | Restaurant’15 | Restaurant’16
---|---|---
| P | R | F1 | P | R | F1
Distance rule | 54.12 | 39.96 | 45.97 | 61.90 | 44.57 | 51.83
Dependency rule | 65.49 | 48.88 | 55.98 | 76.03 | 56.19 | 64.62
RINANTE | 77.40 | 57.00 | 65.70 | 75.00 | 42.40 | 54.10
CMLA | 60.80 | 65.30 | 62.90 | 74.50 | 69.00 | 71.70
IOG | 76.06 | 70.71 | 73.25 | 85.25 | 78.51 | 81.69
Li-unified-R | 79.18 | 75.88 | 77.44 | 79.84 | 86.88 | 83.16
(Li et al., 2019c)–BLSTM | 74.29 | 80.48 | 77.21 | 82.12 | 84.95 | 83.46
(Li et al., 2019c)–TG | 75.98 | 76.32 | 76.10 | 82.33 | 85.16 | 83.67
(Li et al., 2019c)–T | 78.13 | 75.22 | 76.60 | 77.14 | 87.10 | 81.77
(Li et al., 2019c) | 78.07 | 78.07 | 78.02 | 81.09 | 86.67 | 83.73
RGAT-BERT | 75.20 | 81.98 | 78.44 | 77.98 | 89.89 | 83.52
RGAT-BERT-CRF | 92.27 | 83.96 | 87.92 | 96.07 | 84.09 | 89.68
RGAT-BERT-BiLSTM-CRF | 94.52 | 90.99 | 92.72 | 95.82 | 93.76 | 94.78
RGAT-BERT-TRFMR-CRF | 93.62 | 80.66 | 86.66 | 94.37 | 83.01 | 88.33
Table 7: Comparison of predictions on the opinion extraction dataset of (Peng
et al., 2020b) for the Restaurant domain based on Pontiki et al., 2015 and
Pontiki et al., 2016.

##  5 Conclusion

In this work we have applied Relational Graph Attention network which was
previously used for classifying sentiment polarity associated with a specific
aspect. However, here we show that the dependency structure of the input query
when encoded by a Graph attention network is powerful enough to improve the
performance and it is not necessary to tie to a particular aspect. We have
used a surrogate aspect/opinion term by selecting a noun/adjective token if
available. The strength of the approach is evident in the superior results
that we have obtained for both aspect and opinion term extraction on four
commonly used datasets. In addition, we have compared BiLSTM and Transformer
as additional layers along with a CRF layer and found that BiLSTM layer
consistently performs better than a Transformer layer.

## Limitations

There are several limitations of the present methodology as shown below:

  1. 1.

The success of the current approach heavily hinges on knowing the dependency
structure of the queries. Thus, we cannot extend this approach easily to other
languages other than English.

  2. 2.

While we have extracted all the aspects and opinions from a given query we
have not matched them in this work. A more complete solution is proposed by
Mukherjee et al. (2021) using pointer network based decoder, which can be the
next step of this work.

## References

  * Chen et al. (2017) Peng Chen, Zhongqian Sun, Lidong Bing, and Wei Yang. 2017.  Recurrent attention network on memory for aspect sentiment analysis.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 452–461, Copenhagen, Denmark. Association for Computational Linguistics.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Dozat and Manning (2016) Timothy Dozat and Christopher D Manning. 2016.  Deep biaffine attention for neural dependency parsing.
  * He et al. (2018) Ruidan He, Wee Sun Lee, Hwee Tou Ng, and Daniel Dahlmeier. 2018.  Effective attention modeling for aspect-level sentiment classification.  In _Proceedings of the 27th International Conference on Computational Linguistics_ , pages 1121–1131, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  * He et al. (2019) Ruidan He, Wee Sun Lee, Hwee Tou Ng, and Daniel Dahlmeier. 2019.  An interactive multi-task learning network for end-to-end aspect-based sentiment analysis.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 504–515, Florence, Italy. Association for Computational Linguistics.
  * Hoang et al. (2019) Mickel Hoang, Oskar Alija Bihorac, and Jacobo Rouces. 2019.  Aspect-based sentiment analysis using BERT.  In _Proceedings of the 22nd Nordic Conference on Computational Linguistics_ , pages 187–196, Turku, Finland. Linköping University Electronic Press.
  * Huang and Carley (2019) Binxuan Huang and Kathleen Carley. 2019.  Syntax-aware aspect level sentiment classification with graph attention networks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 5469–5477, Hong Kong, China. Association for Computational Linguistics.
  * Kingma and Ba (2014) Diederik P. Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization.
  * Lample et al. (2016) Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016.  Neural architectures for named entity recognition.  In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 260–270, San Diego, California. Association for Computational Linguistics.
  * Li et al. (2019a) Xin Li, Lidong Bing, Piji Li, and Wai Lam. 2019a.  A unified model for opinion target extraction and target sentiment prediction.  In _AAAI_ , pages 6714––6721.
  * Li et al. (2019b) Xin Li, Lidong Bing, Piji Li, and Wai Lam. 2019b.  A unified model for opinion target extraction and target sentiment prediction.  In _AAAI_ , pages 6714–6721.
  * Li et al. (2019c) Xin Li, Lidong Bing, Wenxuan Zhang, and Wai Lam. 2019c.  Exploiting BERT for end-to-end aspect-based sentiment analysis.  In _Proceedings of the 5th Workshop on Noisy User-generated Text (W-NUT 2019)_ , pages 34–41, Hong Kong, China. Association for Computational Linguistics.
  * Lin et al. (2019) Peiqin Lin, Meng Yang, and Jianhuang Lai. 2019.  Deep mask memory network with semantic dependency and context moment for aspect level sentiment classification.  In _Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19_ , pages 5088–5094. International Joint Conferences on Artificial Intelligence Organization.
  * Liu et al. (2018) Liyuan Liu, Jingbo Shang, Xiang Ren, Frank F Xu, Huan Gui, Jian Peng, and Jiawei Han. 2018.  Empower sequence labeling with task-aware neural language model.  In _AAAI_ , pages 5253––5260.
  * Luo et al. (2019) Huaishao Luo, Tianrui Li, Bing Liu, and Junbo Zhang. 2019.  DOER: Dual cross-shared RNN for aspect term-polarity co-extraction.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 591–601, Florence, Italy. Association for Computational Linguistics.
  * Ma et al. (2017) Dehong Ma, Sujian Li, Xiaodong Zhang, and Houfeng Wang. 2017.  Interactive attention networks for aspect-level sentiment classification.  In _Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17_ , pages 4068–4074.
  * Ma and Hovy (2016) Xuezhe Ma and Eduard Hovy. 2016.  End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1064–1074, Berlin, Germany. Association for Computational Linguistics.
  * Mukherjee et al. (2021) Rajdeep Mukherjee, Tapas Nayak, Yash Butala, Sourangshu Bhattacharya, and Pawan Goyal. 2021.  PASTE: A tagging-free decoding framework using pointer networks for aspect sentiment triplet extraction.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 9279–9291, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Peng et al. (2020a) Haiyun Peng, Lu Xu, Lidong Bing, Fei Huang, Wei Lu, and Luo Si. 2020a.  Knowing what, how and why: A near complete solution for aspect-based sentiment analysis.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 34(05):8600–8607.
  * Peng et al. (2020b) Haiyun Peng, Lu Xu, Lidong Bing, Fei Huang, Wei Lu, and Luo Si. 2020b.  Knowing what, how and why: A near complete solution for aspect-based sentiment analysis.  In _AAAI_ , volume 34 (5).
  * Pontiki et al. (2016) Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Ion Androutsopoulos, Suresh Manandhar, Mohammad AL-Smadi, Mahmoud Al-Ayyoub, Yanyan Zhao, Bing Qin, Orphée De Clercq, Véronique Hoste, Marianna Apidianaki, Xavier Tannier, Natalia Loukachevitch, Evgeniy Kotelnikov, Nuria Bel, Salud María Jiménez-Zafra, and Gülşen Eryiğit. 2016.  SemEval-2016 task 5: Aspect based sentiment analysis.  In _Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016)_ , pages 19–30, San Diego, California. Association for Computational Linguistics.
  * Pontiki et al. (2015) Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Suresh Manandhar, and Ion Androutsopoulos. 2015.  SemEval-2015 task 12: Aspect based sentiment analysis.  In _Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015)_ , pages 486–495, Denver, Colorado. Association for Computational Linguistics.
  * Pontiki et al. (2014) Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014.  SemEval-2014 task 4: Aspect based sentiment analysis.  In _Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014)_ , pages 27–35, Dublin, Ireland. Association for Computational Linguistics.
  * Sun et al. (2019) Kai Sun, Richong Zhang, Samuel Mensah, Yongyi Mao, and Xudong Liu. 2019.  Aspect-level sentiment analysis via convolution over dependency tree.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 5679–5688, Hong Kong, China. Association for Computational Linguistics.
  * Tang et al. (2016) Duyu Tang, Bing Qin, and Ting Liu. 2016.  Aspect level sentiment classification with deep memory network.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_ , pages 214–224, Austin, Texas. Association for Computational Linguistics.
  * Tay et al. (2017) Yi Tay, Luu Anh Tuan, and Siu Cheung Hui. 2017.  Dyadic memory networks for aspect-based sentiment analysis.  In _Proceedings of the 2017 ACM on Conference on Information and Knowledge Management_ , CIKM ’17, page 107–116, New York, NY, USA. Association for Computing Machinery.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.
  * Wang et al. (2018) Jingjing Wang, Jie Li, Shoushan Li, Yangyang Kang, Min Zhang, Luo Si, and Guodong Zhou. 2018.  Aspect sentiment classification with both word-level and clause-level attention networks.  In _Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18_ , pages 4439–4445. International Joint Conferences on Artificial Intelligence Organization.
  * Wang et al. (2020) Kai Wang, Weizhou Shen, Yunyi Yang, Xiaojun Quan, and Rui Wang. 2020.  Relational graph attention network for aspect-based sentiment analysis.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 3229–3238, Online. Association for Computational Linguistics.
  * Wang et al. (2016) Yequan Wang, Minlie Huang, Xiaoyan Zhu, and Li Zhao. 2016.  Attention-based LSTM for aspect-level sentiment classification.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_ , pages 606–615, Austin, Texas. Association for Computational Linguistics.
  * Xu et al. (2019) Hu Xu, Bing Liu, Lei Shu, and Philip S. Yu. 2019.  Bert post-training for review reading comprehension and aspect-based sentiment analysis.
  * Zeng et al. (2019) Biqing Zeng, Heng Yang, Ruyang Xu, Wu Zhou, and Xuli Han. 2019.  Lcf: A local context focus mechanism for aspect-based sentiment classification.  _Applied Sciences_ , 9(16).
  * Zhang et al. (2019) Chen Zhang, Qiuchi Li, and Dawei Song. 2019.  Aspect-based sentiment classification with aspect-specific graph convolutional networks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 4568–4578, Hong Kong, China. Association for Computational Linguistics.
  * Zhang et al. (2020) Chen Zhang, Qiuchi Li, Dawei Song, and Benyou Wang. 2020.  A multi-task learning framework for opinion triplet extraction.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 819–828, Online. Association for Computational Linguistics.


# On the proof of Taylor’s conjecture
in multiply connected domains

Daniel Faraco  Departamento de Matemáticas, Univeridad Autónoma de Madrid,
E-28049 Madrid, Spain; ICMAT CSIC-UAM-UC3M-UCM, E-28049 Madrid, Spain  Sauli
Lindberg  Department of Mathematics and Statistics, University of Helsinki,
P.O. Box 68,
00014 Helsingin yliopisto, Finland  David MacTaggart  School of Mathematics
and Statistics, University of Glasgow,
Glasgow, G12 8QQ, United Kingdom  Alberto Valli  Department of Mathematics,
University of Trento, 38123 Povo (Trento), Italy

###### Abstract

In this Letter we extend the proof, by Faraco and Lindberg [1], of Taylor’s
conjecture in multiply connected domains to cover arbitrary vector potentials
and remove the need to impose conditions on the magnetic field due to gauge
invariance. This extension allows us to treat general magnetic fields in
closed domains that are important in laboratory plasmas and brings closure to
a problem that has been open for almost 50 years.

###### keywords:

Magnetohydrodynamics, Magnetic Helicity, Magnetic Relaxation, Turbulence

††journal: Applied Mathematics Letters

##  1 Introduction

The Woltjer–Taylor theory of magnetic relaxation is remarkable in the sense
that it is one of the few examples of turbulence theory for which the end
result can be written down explicitly. In short, the theory predicts that for
a magnetically-closed plasma, a magnetic field will evolve through turbulent
relaxation to the equilibrium of a linear force-free field. This relaxation is
based on two key results: Woltjer’s theorem [2] and Taylor’s conjecture [3,
4]. Woltjer’s theorem states that in a fixed and magnetically-closed domain,
the magnetic field that minimizes the magnetic energy whilst keeping the
global magnetic helicity fixed is a linear force-free field. Taylor combined
this theorem with the conjecture that in a non-ideal plasma (one with small
but non-zero resistivity) the global helicity of the entire domain is
approximately conserved. Related to this, Taylor also assumed that the
magnetic helicities of subdomains (sub-helicities) are not important in
determining the final relaxed state since the magnetic field undergoes
significant field line reconnection during turbulent relaxation, thus
destroying the sub-helicities.

Despite some good experimental evidence for Woltjer–Taylor relaxation, see [5,
4, 6], there also exists research which suggests that the assumption that only
the global magnetic helicity is conserved is too restrictive in general.
Simulations, such as [7], show that magnetic fields do not always relax to
linear force-free fields. This deviation from the classic Woltjer–Taylor
theory has been attributed to the influence of other topological invariants
besides the global magnetic helicity [8]. It has also been suggested that
field line helicities (sub-helicities of individual field lines) can play an
important role in magnetic relaxation, and that they are redistributed by
reconnection rather than destroyed. See [9] for a summary of these issues.

Although modifications to the classic Woltjer–Taylor theory seem to be needed
for a more comprehensive theory of magnetic relaxation, one aspect of the
theory that has proved to be robust is Taylor’s conjecture itself. Berger [10]
showed that strict limits on magnetic helicity dissipation can be found, which
depend on the plasma energy, the energy dissipation rate and the magnetic
diffusion. In both astrophysical and laboratory plasmas, the conditions are
often such that magnetic helicity dissipation, following Berger’s bounds, is
very small and so magnetic helicity can be treated as an invariant in non-
ideal (resistive) plasmas as well as ideal plasmas. For a list of other
relevant theoretical works on helicity conservation, we direct the reader to
[1, Sect. 1].

Recently, Faraco and Lindberg [1] provided a rigorous proof of Taylor’s
conjecture in a magnetically-closed turbulent plasma. They described the
turbulent plasma using Onsager’s theory of turbulence [11, 12] and proved the
conjecture by considering limits of Leray–Hopf solutions of non-ideal
magnetohydrodynamics (MHD). The proof considers both simply connected and
multiply connected domains. In the former, the choice of vector potential in
the magnetic helicity is arbitrary, but in the latter this is not the case.
The fact that only certain vector potentials are suitable in multiply
connected domains stems from [1] considering the classical form of magnetic
helicity,

$H=\int_{\Omega}\mathbf{A}\cdot\mathbf{B},$ (1)

where $\mathbf{B}$ is the magnetic field, $\mathbf{A}$ is the vector potential
with $\textup{{\rm curl}}\,\mathbf{A}=\mathbf{B}$ and $\Omega$ is the
magnetically-closed domain. Equation (1) represents the gauge invariant form
of helicity in simply connected domains. MacTaggart and Valli [13] showed that
the definition of magnetic helicity $H$ needs to be extended for multiply
connected domains in order to accommodate arbitrary vector potentials that do
not impose any restrictions on the magnetic field. For example, it can be
shown that a consequence of imposing gauge invariance with equation (1) in
multiply connected domains is that the magnetic flux through internal cuts of
the domain must be zero, e.g., the toroidal magnetic flux in a tokamak would
be zero.

In this Letter, we bring together the results of Faraco and Lindberg [1] and
MacTaggart and Valli [13] to provide a complete proof of Taylor’s conjecture
for arbitray vector potentials of closed magnetic fields with no restrictions
on the magnetic field imposed by gauge invariance. We proceed by only
describing extensions and modifications to specific parts of the proof in [1]
that are relevant for expanding its scope in the manner described above. For
full details of the many technical results involved in the original proof, the
reader is directed to the work by Faraco and Lindberg [1]. Before presenting
the main result, we now introduce some preliminary material that will be key
to understanding the extension to their proof.

##  2 Preliminaries

###  2.1 Geometric set-up

We consider a multiply connected domain $\Omega\subset\mathbb{R}^{3}$ that is
a bounded open connected set with Lipschitz continuous boundary
$\partial\Omega$ and unit outer normal $\mathbf{n}$. Let $g>0$ be the first
Betti number, or genus, of $\Omega$. Then the first Betti number of
$\partial\Omega$ is 2$g$. We can consider 2$g$ non-bounding cycles on
$\partial\Omega$,
$\\{\gamma_{i}\\}_{i=1}^{g}\cup\\{\gamma^{\prime}_{i}\\}_{i=1}^{g}$, that
represent the generators of the first homology group of $\partial\Omega$. The
$\\{\gamma_{i}\\}_{i=1}^{g}$ represent the generators of the first homology
group of $\overline{\Omega}$. The tangent vector on $\gamma_{i}$ is denoted
$\mathbf{t}_{i}$. Similarly, the $\\{\gamma^{\prime}_{i}\\}_{i=1}^{g}$
represent the generators of the first homology group of
$\overline{\Omega^{\prime}}$, where
$\Omega^{\prime}=B\setminus\overline{\Omega}$ and $B$ is an open ball
containing $\overline{\Omega}$. The tangent vector on $\gamma^{\prime}_{i}$ is
denoted $\mathbf{t}^{\prime}_{i}$.

In $\Omega$ there exist $g$ ‘cutting surfaces’ $\\{\Sigma_{i}\\}_{i=1}^{g}$
that are connected orientable Lipschitz surfaces satisfying
$\Sigma_{i}\subset\Omega$. Each cutting surface in $\Omega$ satisfies
$\partial\Sigma_{i}=\gamma^{\prime}_{i}$ and $\overline{\Sigma}_{i}$ ‘cuts’
the corresponding cycle $\gamma_{i}$ in exactly one point. Each cutting
surface $\Sigma_{i}$ has a unit normal vector $\mathbf{n}_{\Sigma_{i}}$
oriented as $\gamma_{i}$. An equivalent definition applies for the cutting
surfaces $\\{\Sigma^{\prime}_{i}\\}_{i=1}^{g}$ of $\Omega^{\prime}$. In
particular, $\partial\Sigma^{\prime}_{i}=\gamma_{i}$.

Two examples of the geometric set-up are displayed in Figure 1 for a 2-fold
torus and in Figure 2 for a toroidal shell.

$\gamma_{1}$$\Sigma^{\prime}_{1}$$\gamma_{2}$$\Sigma^{\prime}_{2}$$\Sigma_{1}$$\gamma^{\prime}_{1}$$\Sigma_{2}$$\gamma^{\prime}_{2}$
Figure 1: An illustration of cutting surfaces (shaded areas) and cycles
(coloured paths) for a 2-fold torus. The notation is as described in the main
text.
$\Sigma^{\prime}_{2}$$\Sigma^{\prime}_{1}$$\gamma_{1}$$\gamma_{2}$$\Sigma_{2}$$\Sigma_{1}$$\gamma^{\prime}_{1}$$\gamma^{\prime}_{2}$(a)(b)
Figure 2: An illustration of cutting surfaces (shaded areas) and cycles
(coloured paths) for a toroidal shell (cut in half). (a) shows the cutting
surfaces of $\Omega^{\prime}$ and their associated cycles. (b) shows the same
but for $\Omega$.

###  2.2 Required function spaces

Using the basic Hilbert spaces listed in Alonso Rodríguez et al. [14, Sect.
2], the space of harmonic Neumann fields is defined by

$\mathcal{K}_{T}\mathrel{:\mkern-0.25mu=}H(\textup{curl}^{0};\Omega)\cap
H_{0}(\textup{div}^{0};\Omega).$

Remember that if $\mathbf{v}\in H_{0}(\textup{div}^{0};\Omega)$ satisfies
$\mathbf{v}\perp\mathcal{K}_{T}$ (where orthogonality has to be intended in
$(L^{2}(\Omega))^{3}$) then it follows that
$\int_{\Sigma_{i}}\mathbf{v}\cdot\mathbf{n}_{\Sigma_{i}}\,\textup{d}S=0$ for
$i=1,\ldots,g$, and viceversa. Thus, if we denote

$\mathcal{Q}\mathrel{:\mkern-0.25mu=}\nabla
H^{1}(\Omega)\stackrel{{\scriptstyle\perp}}{{\oplus}}\mathcal{K}_{T},$

we have

$\displaystyle\mathcal{Q}=H(\textup{curl}^{0};\Omega),$
$\displaystyle\mathcal{Q}^{\perp}=\left\\{\mathbf{v}\in
H_{0}(\textup{div}^{0};\Omega)\colon\int_{\Sigma_{i}}\mathbf{v}\cdot\mathbf{n}_{\Sigma_{i}}=0\textup{
for }i=1,\ldots,g\right\\},$ $\displaystyle
H_{0}(\textup{div}^{0};\Omega)=\mathcal{Q}^{\perp}\stackrel{{\scriptstyle\perp}}{{\oplus}}\mathcal{K}_{T}.$

Analogous spaces can be defined for $\Omega^{\prime}$, e.g.,
$\mathcal{K}_{T}^{\prime}\mathrel{:\mkern-0.25mu=}H(\textup{curl}^{0};\Omega^{\prime})\cap
H_{0}(\textup{div}^{0};\Omega^{\prime})$. For the ease of the reader, let us
also recall that the notation in Faraco and Lindberg [1] is different:
precisely,
$L^{2}_{\sigma}(\Omega;\mathbb{R}^{3})=H_{0}(\textup{div}^{0};\Omega)$,
$L^{2}_{H}(\Omega;\mathbb{R}^{3})=\mathcal{K}_{T}$,
$L^{2}_{\Sigma}(\Omega;\mathbb{R}^{3})=\mathcal{Q}^{\perp}$.

###  2.3 Integral identities

Let the basis functions of $\mathcal{K}_{T}$ and $\mathcal{K}_{T}^{\prime}$ be
denoted by $\boldsymbol{\rho}_{i}$ and $\boldsymbol{\rho}^{\prime}_{i}$,
$i=1\dots g$, respectively. These functions are defined by imposing that
$\oint_{\gamma_{k}}\boldsymbol{\rho}_{i}\cdot\mathbf{t}_{k}=\delta_{ki}$ and
$\oint_{\gamma^{\prime}_{k}}\boldsymbol{\rho}^{\prime}_{i}\cdot\mathbf{t}^{\prime}_{k}=\delta_{ki}$.

Let us recall that, given $\mathbf{v}\in H_{0}(\textup{div}^{0};\Omega)$, it
holds that

$\int_{\Omega}\mathbf{v}\cdot\boldsymbol{\rho}_{i}=\int_{\Sigma_{i}}\mathbf{v}\cdot\mathbf{n}_{\Sigma_{i}}.$
(2)

Given
$\mathbf{w}\in\boldsymbol{\mathcal{X}}\mathrel{:\mkern-0.25mu=}\\{\mathbf{w}\in
H(\textup{curl};\Omega):{\rm
curl}\,\mathbf{w}\cdot\mathbf{n}=0\,\textup{on}\,\partial\Omega\\}$, as in
Alonso Rodríguez et al. [14] we define

$\oint_{\gamma_{i}}\mathbf{w}\cdot\mathbf{t}_{i}\mathrel{:\mkern-0.25mu=}-\int_{\partial\Omega}\mathbf{n}\times\mathbf{w}\cdot\boldsymbol{\rho}^{\prime}_{i},\quad\oint_{\gamma^{\prime}_{i}}\mathbf{w}\cdot\mathbf{t}^{\prime}_{i}\mathrel{:\mkern-0.25mu=}\int_{\partial\Omega}\mathbf{n}\times\mathbf{w}\cdot\boldsymbol{\rho}_{i}$

(as in [14] the generators of the first homology group of $\overline{\Omega}$
are denoted $\gamma^{\prime}_{i}$ and those of the first homology group of
$\overline{\Omega^{\prime}}$ are denoted $\gamma_{i}$). Note that, with these
definitions and taking into account (2), Stokes’ theorem holds on
$\Sigma_{i}$, i.e.,
$\oint_{\gamma^{\prime}_{i}}\mathbf{w}\cdot\mathbf{t}^{\prime}_{i}=\int_{\Sigma_{i}}{\rm
curl}\,\mathbf{w}\cdot\mathbf{n}_{\Sigma_{i}}$.

As in Lemma 7 of [14], for $\mathbf{w}$,
$\mathbf{v}\in\boldsymbol{\mathcal{X}}$ we find

$\begin{array}[]{ll}\int_{\partial\Omega}(\mathbf{w}\times\mathbf{n})\cdot\mathbf{v}\\!\\!\\!&=\sum_{i=1}^{g}\left(\oint_{\gamma_{i}}\mathbf{w}\cdot\mathbf{t}_{i}\right)\left(\oint_{\gamma^{\prime}_{i}}\mathbf{v}\cdot\mathbf{t}^{\prime}_{i}\right)\\\
&\qquad-\sum_{i=1}^{g}\left(\oint_{\gamma^{\prime}_{i}}\mathbf{w}\cdot\mathbf{t}^{\prime}_{i}\right)\left(\oint_{\gamma_{i}}\mathbf{v}\cdot\mathbf{t}_{i}\right).\end{array}$
(3)

For more details on the derivation of this result, the reader is directed to
[14].

###  2.4 Leray–Hopf solutions of viscous, resistive MHD

In [1], the domains studied have a $C^{1,1}$ regular boundary. This smoothness
condition implies, in particular, that
$X_{T}(\Omega)\mathrel{:\mkern-0.25mu=}H(\textup{curl};\Omega)\cap
H_{0}(\textup{div};\Omega)\hookrightarrow H^{1}(\Omega)\hookrightarrow
L^{6}(\Omega)$, and this chain of embeddings allows one to prove the existence
of Leray–Hopf solutions for the velocity $\mathbf{u}\in
L^{\infty}(0,T;H_{0}(\textup{div}^{0}(\Omega)))\cap
L^{2}(0,T;H_{0}^{1}(\Omega))$ and magnetic field $\mathbf{B}\in
L^{\infty}(0,T;H_{0}(\textup{div}^{0}(\Omega)))\cap L^{2}(0,T;H^{1}(\Omega))$
by standard arguments (see [1, Appendix A]).

In fact, Lipschitz regularity of $\partial\Omega$ is also a strong enough
assumption for the existence of Leray–Hopf solutions, as long as the
definition of Leray–Hopf solutions is modified in the manner described below.
First recall that in bounded Lipschitz domains, the weaker embedding result
$X_{T}(\Omega)\hookrightarrow H^{1/2}(\Omega)\hookrightarrow L^{3}(\Omega)$
holds. We relax the smoothness requirements on the magnetic field to
$\mathbf{B}\in L^{\infty}(0,T;H_{0}(\textup{div}^{0}(\Omega)))\cap
L^{2}(0,T;X_{T}(\Omega))$. Whereas in $C^{1,1}$ regular domains the weak
formulation of the Cauchy momentum equation says that, for viscosity $\nu$,
$\langle\partial_{t}\mathbf{u},\varphi\rangle+\int_{\Omega}(\mathbf{u}\cdot\nabla\mathbf{u}-\mathbf{B}\cdot\nabla\mathbf{B})\cdot\varphi+\nu\int_{\Omega}\nabla\mathbf{u}:\nabla\varphi=0$
holds at almost each $t\in[0,T)$ for every $\varphi\in
H(\textup{div}^{0}(\Omega))\cap H^{1}_{0}(\Omega)$, in bounded Lipschitz
domains the formula
$\langle\partial_{t}\mathbf{u},\varphi\rangle+\int_{\Omega}(\mathbf{u}\cdot\nabla\mathbf{u}-\textup{curl}\,\mathbf{B}\times\mathbf{B})\cdot\varphi+\nu\int_{\Omega}\nabla\mathbf{u}:\nabla\varphi=0$
needs to be used instead; note that
$\|\textup{curl}\,\mathbf{B}\times\mathbf{B}\cdot\varphi\|_{L^{1}}\leq\|\textup{curl}\,\mathbf{B}\|_{L^{2}}\|\mathbf{B}\|_{L^{3}}\|\varphi\|_{L^{6}}$
at almost each $t\in[0,T)$ by Hölder’s inequality. Under these modifications,
the main result [1, Theorem 1.5] holds and the proof only requires minor
adjustments.

##  3 Main result

Suppose that $\mathbf{B},\mathbf{u}\in
L^{\infty}(0,T;H_{0}(\textup{div}^{0};\Omega))$ form a weak solution of ideal
MHD with initial data $\mathbf{B}_{0},\mathbf{u}_{0}\in
H_{0}(\textup{div}^{0};\Omega)$, where $\mathbf{u}$ is the velocity. Assume,
furthermore, that $(\mathbf{B},\mathbf{u})$ arises as an ideal (inviscid, non-
resistive) limit of Leray–Hopf solutions. Using the notation from [1] we
denote

$\mathbf{B}=\mathbf{B}^{\Sigma}+\mathbf{B}^{H},\qquad\mathbf{B}^{\Sigma}\in\mathcal{Q}^{\perp}\textup{
and }\mathbf{B}^{H}\in\mathcal{K}_{T}.$ (4)

Let us recall that at almost each time $\mathbf{B}^{H}$ is the unique element
of $\mathcal{K}_{T}$ such that
$\int_{\Sigma_{i}}\mathbf{B}^{H}\cdot\mathbf{n}_{\Sigma_{i}}=\int_{\Sigma_{i}}\mathbf{B}\cdot\mathbf{n}_{\Sigma_{i}}$
for each $i=1,\ldots,g$. Further, we decompose the vector potential as

$\mathbf{A}=\mathbf{A}^{\Sigma}+\mathbf{A}^{H},$ (5)

where at almost each time $\mathbf{A}^{H}$ is the unique element of
$\mathcal{Q}^{\perp}$ such that $\textup{curl}\,\mathbf{A}^{H}=\mathbf{B}^{H}$
(for these existence results see, e.g., [15, Sect. 2.2, and in particular
Remark 1]). Similar notation is used for $\mathbf{B}_{0}$ and
$\mathbf{A}_{0}$.

The generalized expression for magnetic helicity in multiply connected domains
is defined as

$\Upsilon(\mathbf{B})\mathrel{:\mkern-0.25mu=}\int_{\Omega}\mathbf{A}\cdot\mathbf{B}-\sum_{i=1}^{g}\oint_{\gamma_{i}}\mathbf{A}\cdot\mathbf{t}_{i}\int_{\Sigma_{i}}\mathbf{B}\cdot\mathbf{n}_{\Sigma_{i}}$
(6)

(see MacTaggart and Valli [13]). It is important to note that this definition
is gauge invariant, i.e., it is independent of the vector potential
$\mathbf{A}\in\mathcal{X}$.

Using this gauge invariant expression for magnetic helicity we are now in a
position to write an adapted version of Theorem 1.5 of Faraco and Lindberg
[1], which describes the convergence of magnetic helicity; clearly, no
dependence on the vector potential appears.

######  Theorem 1.

In a domain $\Omega\subset\mathbb{R}^{3}$ with the properties described
previously, suppose that $(\mathbf{u},\mathbf{B})$ is a weak ideal limit of
Leray–Hopf solutions $(\mathbf{u}_{k},\mathbf{B}_{k})$, $k\in\mathbb{N}$. The
initial data of $\mathbf{B}$ and $\mathbf{B}_{k}$ are denoted $\mathbf{B}_{0}$
and $\mathbf{B}_{k,0}$, respectively. Then it holds that

$\Upsilon(\mathbf{B}_{k},t)-\Upsilon(\mathbf{B}_{k,0})=-2\eta_{k}\int_{0}^{t}\int_{\Omega}\mathbf{B}_{k}\cdot{\rm
curl}\,\mathbf{B}_{k},$ (7)

where $\eta_{k}>0$ is the resistivity, for all $k\in\mathbb{N}$ and
$t\in[0,T)$. Furthermore

$\Upsilon(\mathbf{B},t)-\Upsilon(\mathbf{B}_{0})=0,$ (8)

for almost each $t\in[0,T)$.

###### Proof.

Let us define

$Z(\mathbf{B}_{k},\mathbf{A}_{k};t)\mathrel{:\mkern-0.25mu=}\int_{\Omega}\mathbf{A}_{k}(t)\cdot\mathbf{B}_{k}(t)-\int_{\partial\Omega}\mathbf{A}_{k}^{\Sigma}(t)\times\mathbf{n}\cdot\mathbf{A}_{k}^{H}(t)$

and similarly for $\mathbf{B}_{k,0}$ and $\mathbf{A}_{k,0}$, $\mathbf{B}$ and
$\mathbf{A}$, $\mathbf{B}_{0}$ and $\mathbf{A}_{0}$. In [1, Lemma 4.3] it is
shown that for Leray–Hopf solutions (and their weak limits) the harmonic part
of the magnetic field is stationary, i.e.,

$\mathbf{B}_{k}^{H}(t)=\mathbf{B}^{H}_{k,0},\quad\mathbf{B}^{H}(t)=\mathbf{B}^{H}_{0},$

and consequently $\mathbf{A}^{H}_{k}(t)=\mathbf{A}^{H}_{k,0}$,
$\mathbf{A}^{H}(t)=\mathbf{A}^{H}_{0}$. Therefore, Theorem 1.5 in [1] (having
corrected for a minus sign that is a misprint) says that

$\displaystyle
Z(\mathbf{B}_{k},\mathbf{A}_{k};t)-Z(\mathbf{B}_{k,0},\mathbf{A}_{k,0})=-2\eta_{k}\int_{0}^{t}\int_{\Omega}\mathbf{B}_{k}\cdot{\rm
curl}\,\mathbf{B}_{k},$ (9a) $\displaystyle
Z(\mathbf{B},\mathbf{A};t)-Z(\mathbf{B}_{0},\mathbf{A}_{0})=0.$ (9b)

Let us compute
$\int_{\partial\Omega}\mathbf{A}_{k}^{\Sigma}(t)\times\mathbf{n}\cdot\mathbf{A}_{k}^{H}(t)$.
By (3) we have

$\displaystyle\int_{\partial\Omega}\mathbf{A}_{k}^{\Sigma}(t)\times\mathbf{n}\cdot\mathbf{A}_{k}^{H}(t)=$
$\displaystyle\sum_{i=1}^{g}\left(\oint_{\gamma_{i}}\mathbf{A}^{\Sigma}_{k}(t)\cdot\mathbf{t}_{i}\right)\left(\oint_{\gamma^{\prime}_{i}}\mathbf{A}_{k,0}^{H}\cdot\mathbf{t}^{\prime}_{i}\right)$
$\displaystyle-\sum_{i=1}^{g}\left(\oint_{\gamma^{\prime}_{i}}\mathbf{A}^{\Sigma}_{k}(t)\cdot\mathbf{t}^{\prime}_{i}\right)\left(\oint_{\gamma_{i}}\mathbf{A}_{k,0}^{H}\cdot\mathbf{t}_{i}\right).$

Using the decomposition in (4), we know that

$\int_{\Sigma_{i}}\mathbf{B}^{\Sigma}_{k}(t)\cdot\mathbf{n}_{\Sigma_{i}}=\int_{\Sigma_{i}}\mathbf{B}_{k,0}^{\Sigma}\cdot\mathbf{n}_{\Sigma_{i}}=0.$
(10)

Since ${\rm curl}\,\mathbf{A}_{k}^{\Sigma}(t)=\mathbf{B}_{k}^{\Sigma}(t)$,
${\rm curl}\,\mathbf{A}_{k,0}^{H}=\mathbf{B}_{k,0}^{H}$ and
$\gamma^{\prime}_{i}=\partial\Sigma_{i}$, applying Stokes’ theorem gives

$\begin{array}[]{ll}&\oint_{\gamma^{\prime}_{i}}\mathbf{A}_{k}^{\Sigma}(t)\cdot\mathbf{t}^{\prime}_{i}=\int_{\Sigma_{i}}\mathbf{B}_{k}^{\Sigma}(t)\cdot\mathbf{n}_{\Sigma_{i}}=0,\\\\[5.0pt]
&\oint_{\gamma^{\prime}_{i}}\mathbf{A}_{k,0}^{H}\cdot\mathbf{t}^{\prime}_{i}=\int_{\Sigma_{i}}\mathbf{B}_{k,0}^{H}\cdot\mathbf{n}_{\Sigma_{i}}=\int_{\Sigma_{i}}\mathbf{B}_{k,0}\cdot\mathbf{n}_{\Sigma_{i}},\end{array}$

and also

$\begin{array}[]{ll}&\int_{\Sigma_{i}}\mathbf{B}_{k}(t)\cdot\mathbf{n}_{\Sigma_{i}}=\int_{\Sigma_{i}}\mathbf{B}_{k}^{H}(t)\cdot\mathbf{n}_{\Sigma_{i}}\\\\[5.0pt]
&\qquad=\oint_{\gamma^{\prime}_{i}}\mathbf{A}_{k}^{H}(t)\cdot\mathbf{t}^{\prime}_{i}=\oint_{\gamma^{\prime}_{i}}\mathbf{A}_{k,0}^{H}\cdot\mathbf{t}^{\prime}_{i}=\int_{\Sigma_{i}}\mathbf{B}_{k,0}\cdot\mathbf{n}_{\Sigma_{i}},\end{array}$
(11)

for $t\in[0,T)$, $i=1,\ldots,g$, $k\in\mathbb{N}$. In conclusion, repeating
the same computation for
$\int_{\partial\Omega}\mathbf{A}_{k,0}^{\Sigma}\times\mathbf{n}\cdot\mathbf{A}_{k,0}^{H}$,
we find

$\begin{array}[]{ll}&Z(\mathbf{B}_{k},\mathbf{A}_{k};t)-Z(\mathbf{B}_{k,0},\mathbf{A}_{k,0})\\\\[5.0pt]
&\qquad=\int_{\Omega}\mathbf{A}_{k}(t)\cdot\mathbf{B}_{k}(t)-\int_{\Omega}\mathbf{A}_{k,0}\cdot\mathbf{B}_{k,0}\\\\[5.0pt]
&\qquad\qquad-\sum_{i=1}^{g}\left(\oint_{\gamma_{i}}[\mathbf{A}^{\Sigma}_{k}(t)-\mathbf{A}^{\Sigma}_{k,0}]\cdot\mathbf{t}_{i}\right)\left(\int_{\Sigma_{i}}\mathbf{B}_{k,0}\cdot\mathbf{n}_{\Sigma_{i}}\right)\\\\[5.0pt]
&\qquad=\int_{\Omega}\mathbf{A}_{k}(t)\cdot\mathbf{B}_{k}(t)-\int_{\Omega}\mathbf{A}_{k,0}\cdot\mathbf{B}_{k,0}\\\\[5.0pt]
&\qquad\qquad-\sum_{i=1}^{g}\left(\oint_{\gamma_{i}}[\mathbf{A}_{k}(t)-\mathbf{A}_{k,0}]\cdot\mathbf{t}_{i}\right)\left(\int_{\Sigma_{i}}\mathbf{B}_{k,0}\cdot\mathbf{n}_{\Sigma_{i}}\right)\\\\[5.0pt]
&\qquad=\Upsilon(\mathbf{B}_{k},t)-\Upsilon(\mathbf{B}_{k,0}),\end{array}$

having used again that $\mathbf{A}^{H}_{k}(t)=\mathbf{A}^{H}_{k,0}$ and
moreover (11). By combining the computation above with (9a) we get (7).

By analogous reasoning,
$Z(\mathbf{B},\mathbf{A};t)-Z(\mathbf{B}_{0},\mathbf{A}_{0})=\Upsilon(\mathbf{B},t)-\Upsilon(\mathbf{B}_{0})$.
Now (9b) leads to (8), and Theorem 1 is proved. ∎

######  Remark 1.

The right-hand side of (7) is bounded by
$3\sqrt{t\eta_{k}}(\|\mathbf{u}_{k,0}\|_{L^{2}}^{2}+\|\mathbf{B}_{k,0}\|_{L^{2}}^{2})$
for all $k\in\mathbb{N}$ and $t\in[0,T)$. This is seen by using Young’s
inequality on $\mathbf{B}_{k}/\sqrt{t}$ and
$\sqrt{t}\,\textup{curl}\,\mathbf{B}_{k}$ and then appealing to the energy
inequality; see [1, Lemma 4.6] where Young’s inequality is used on
$\mathbf{B}_{k}$ and $\textup{curl}\,\mathbf{B}_{k}$ instead.

##  4 Conclusion

This extension to the proof in Faraco and Lindberg [1] of Taylor’s conjecture
allows for the extra consideration of defining magnetic helicity in multiply
connected domains with arbitrary vector potentials, rather than a specific
subset which may impose unwanted conditions on the magnetic field, e.g.,
forcing zero flux through cutting surfaces. The result gives a complete answer
to a problem that has been posed almost 50 years ago.

Acknowledgments. DF acknowledges financial support from the Spanish Ministry
of Science and Innovation through the Severo Ochoa Programme for Centres of
Excellence in R&D(CEX2019-000904-S and by the MTM2017-85934-C3-2-P2. He is
also partially supported by CAM through the Line of excellence for University
Teaching Staff between CM and UAM. DF and SL are partially supported by the
ERC Advanced Grant 834728. DM acknowledges support from AFOSR: grant number
FA8655-20-1-7032. AV acknowledges the support of INdAM-GNCS.

##  5 References

## References

  * Faraco and Lindberg [2020] D. Faraco, S. Lindberg,  Proof of Taylor’s conjecture on magnetic helicity conservation,  Comm. Math. Phys. 373 (2020) 707–738.
  * Woltjer [1958] L. Woltjer,  A theorem on force-free magnetic fields,  Proc. Natl Acad Sci. USA 44 (1958) 489–491.
  * Taylor [1974] J. B. Taylor,  Relaxation of toroidal plasma and generation of reverse magnetic fields,  Phys. Rev. Lett. 33 (1974) 1139–1141.
  * Taylor [1986] J. B. Taylor,  Relaxation and magnetic reconnection in plasmas,  Rev. Mod. Phys. 56 (1986) 741–763.
  * Butt et al. [1976] E. P. Butt, A. A. Newton, A. J. L. Verhage,  Experimental observations of the self reversal of toroidal magnetic field in pinches,  in: D. E. Evans (Ed.), Pulsed High Beta Plasmas, Oxford: Pergamon, 1976, pp. 419–424.
  * Ji et al. [1995] H. Ji, S. C. Prager, J. S. Sarff,  Conservation of magnetic helicity during plasma relaxation,  Phys. Rev. Lett. 74 (1995) 2945–2948.
  * Yeates et al. [2015] A. R. Yeates, A. J. B. Russell, G. Hornig,  Physical role of topological constraints in localized magnetic relaxation,  Proc. Roy. Soc. A 471 (2015) 20150012.
  * Yeates and Hornig [2011] A. R. Yeates, G. Hornig,  Dynamical constraints from field line topology in magnetic flux tubes,  J. Phys. A Math. Theor. 44 (2011) 265501.
  * Yeates [2020] A. R. Yeates,  Magnetohydrodynamics relaxation theory,  in: D. MacTaggart, A. Hillier (Eds.), Topics in Magnetohydrodynamic Topology, Reconnection and Stability Theory, Springer, 2020, pp. 117–143.
  * Berger [1984] M. A. Berger,  Rigorous new limits on magnetic helicity dissipation in the solar corona,  Geophys. Astrophys. Fluid Dyn. 30 (1984) 79–104.
  * Onsager [1949] L. Onsager,  Statistical hydrodynamics,  Nuovo Cimento 6 (1949) 279–287.
  * Eyink and Sreenivasan [2006] G. L. Eyink, K. R. Sreenivasan,  Onsager and the theory of hydrodynamic turbulence,  Rev. Mod. Phys. 787–135 (2006) 279–287.
  * MacTaggart and Valli [2019] D. MacTaggart, A. Valli,  Magnetic helicity in multiply connected domains,  J. Plasma Phys. 85 (2019) 775850501.
  * Alonso Rodríguez et al. [2018] A. Alonso Rodríguez, J. Camaño, R. Rodríguez, A. Valli, P. Venegas,  Finite element approximation of the spectrum of the curl operator in a multiply connected domain,  Found. Comput. Math. 18 (2018) 1493–1533.
  * Alonso Rodríguez et al. [2019] A. Alonso Rodríguez, E. Bertolazzi, A. Valli,  The curl–div system: theory and finite element approximation,  in: U. Langer, D. Pauly, S. Repin (Eds.), Maxwell’s Equations, De Gruyter, Berlin/Boston, 2019, pp. 1–43.