# Comparison of pipeline, sequence-to-sequence, and GPT models for end-to-end
relation extraction: experiments with the rare disease use-case

Shashank Gupta  Computer Science Department, University of Kentucky, USA
Xuguang Ai  Computer Science Department, University of Kentucky, USA
Ramakanth Kavuluru  Division of Biomedical Informatics, Dept. of Internal
Medicine, University of Kentucky, USA  Computer Science Department, University
of Kentucky, USA

###### Abstract

Objective:  End-to-end relation extraction (E2ERE) is an important and
realistic application of natural language processing (NLP) in biomedicine. In
this paper, we aim to compare three prevailing paradigms for E2ERE using a
complex dataset focused on rare diseases involving discontinuous and nested
entities.

Methods: We use the RareDis information extraction dataset to evaluate three
competing approaches (for E2ERE): NER $\rightarrow$ RE pipelines, joint
sequence to sequence models, and generative pre-trained transformer (GPT)
models. We use comparable state-of-the-art models and best practices for each
of these approaches and conduct error analyses to assess their failure modes.

Results:  Our findings reveal that pipeline models are still the best, while
sequence-to-sequence models are not far behind; GPT models with eight times as
many parameters are worse than even sequence-to-sequence models and lose to
pipeline models by over 10 F1 points. Partial matches and discontinuous
entities caused many NER errors contributing to lower overall E2E
performances. We also verify these findings on a second E2ERE dataset for
chemical-protein interactions. Although generative LM-based methods are more
suitable for zero-shot settings, when training data is available, our results
show that it is better to work with more conventional models trained and
tailored for E2ERE.

Conclusion:  More innovative methods are needed to marry the best of the both
worlds from smaller encoder-decoder pipeline models and the larger GPT models
to improve E2ERE. As of now, we see that well designed pipeline models offer
substantial performance gains at a lower cost and carbon footprint for E2ERE.
Our contribution is also the first to conduct E2ERE for the RareDis dataset.
The dataset and code for all our experiments are publicly available:
https://github.com/shashank140195/Raredis

##  1 INTRODUCTION

Named entities and relations among them are basic units of information in many
disciplines including biomedicine. A relation is typically expressed as a
triple that has a subject entity and an object entity connected via a
predicate (or relation type) as in the example (subject: atorvastatin,
predicate: treats, object: hyperlipidemia). Disease and treatment mechanisms
are often driven at the biological level by protein-protein and chemical-
protein interactions while clinical relations such as drug-disease treatment
relations and disease-symptom causative relations are helpful in providing
care. Most new relational information is first discussed in textual narratives
(e.g., scientific literature, clinical notes, or social media posts), and
extracting and storing it as triples enable effective search systems [1],
high-level reasoning, hypothesis generation, and knowledge discovery
applications [2]. As such, named entity recognition (NER) and relation
extraction (RE) have become standard tasks in biomedical natural language
processing (BioNLP) [3].

Many RE efforts in the past assume that the entity spans are already provided
as part of the input and hence addressed an easier problem of relation
classification (RC) [4, 5, 6]. However, a more realistic setting is the
ability to extract both entity spans and associated relations from the raw
text where entities are not provided. RE in this setting is generally called
end-to-end relation extraction (E2ERE). With the recent deluge of deep neural
networks (or deep learning methods), the NLP community has been focusing more
on E2ERE efforts [7, 8, 9, 10]. Efforts have also been expanded from single
sentence E2ERE to a more complex setting of extractions at the document level,
involving cross-sentence relations, where entities expressed in different
sentences are to be linked [11, 12]. Additional intricacies arise when named
entities are discontinuous or when their spans overlap [13]. For example,
consider the string “accumulation of fats (lipids) called GM 2 gangliosides,”
where entity span “accumulation of GM 2 gangliosides” is discontinuous with a
gap involving outside words. In the example phrase “central pain syndrome,”
both the full three-word string and the middle word “pain” can constitute two
different entities, where the latter entity is fully nested in the longer
3-word entity. Thus far, we have not seen efforts handling these complex
document-level E2ERE settings involving discontinuous and overlapping/nested
entities. In this paper, we address this using the recently introduced RE
dataset called RareDis [14], which focuses on information extraction for rare
diseases and has the complex traits indicated earlier. Although there is
another dataset that focuses on rare diseases at the sentence level [15], we
use RareDis since it operates at the document level.

Over the past decade, neural methods especially those involving contextual
dense word embeddings have supplanted conventional NLP methods that relied on
n-gram statistics. For E2ERE, joint learning neural methods that
simultaneously optimized for NER and RE objectives [16, 17] have gained
popularity over pipeline-based methods that build two separate models for NER
and RE, where the NER model’s output is fed to the RE model. However, the
recent Princeton University Relation Extraction (PURE) framework [18] proposed
an intuitive pipeline method that takes advantage of the so-called typed
“entity markers” to encapsulate entity spans provided as input to
contextualized language models (LMs). The PURE method reignited the relevance
of cleverly designed pipeline methods when compared with joint learning
methods. Simultaneously, sequence-to-sequence models that became popular for
machine translation have been repurposed [19] effectively for E2ERE where the
encoder-decoder architecture is used to transform raw text to directly output
relations encoded through so-called “linearization schemas” and “copy
mechanism” [20]. The state-of-the-art (SoTA) for this paradigm of models is
the Seq2Rel architecture [21] that inherently allows for E2ERE. Finally,
generative pre-trained transformers (GPTs) have gained traction and publicity
(thanks to ChatGPT), especially for zero-shot and few-shot settings [22, 23].
In biomedicine, BioGPT [24] and BioMedLM [25] have been shown to work well for
relation extraction and question answering, among generative decoder-only
language models (LMs), producing SoTA scores on a few datasets.

Thus we identify PURE, Seq2Rel, and BioMedLM***Although we experimented with
BioGPT models, they are smaller than BioMedLM and were quite inferior (more
later) and as such our focus in this manuscript is more on BioMedLM, the
latest and largest GPT model exclusively trained on biomedical literature as
representative models for the pipeline, sequence-to-sequence, and generative
LMs, respectively. Now the central question is, which of these approaches
works well for the complex document level E2ERE task involving discontinuous
and overlapping entities manifesting in the RareDis dataset? Toward answering
this, we make the following contributions in this paper.

  * •

We explore and provide descriptive statistics of the RareDis dataset and fix
certain formatting/annotation errors in the original dataset (acknowledged by
its creators) to ensure availability for the community for further
benchmarking.

  * •

We adapt the PURE pipeline approach to the RareDis dataset since the original
method does not handle discontinuous and nested entities.

  * •

We design linearization schemas for the Seq2Rel method and appropriate
supervised prompting strategies for BioMedLM in the context of E2ERE for the
RareDis dataset.

  * •

We provide quantitative evaluations of the three models (and associated
variants) and conduct qualitative evaluations through manual error analyses.
We make publicly available the modified RareDis dataset and code for all our
experiments: https://github.com/shashank140195/Raredis

To our knowledge, our effort is the first to handle E2ERE with the RareDis
dataset and also to compare SoTA approaches arising from three different
competing paradigms in the neural RE landscape.

Statement of Significance Problem: It is not clear what NLP methods work best
in practice for end-to-end relation extraction What is already known: Although
pipeline methods used to be the norm, recent literature shows a rise in
sequence-to-sequence and decoder-only GPT models for information extraction.
There is also general tendency to prefer the fancier latter models considering
the excitement in the field for them. What this paper adds: With the use-case
of a rare disease information extraction task involving discontinuous and
overlapping entities, we compare three different competing paradigms
(pipeline, seq2seq, and GPT) for end-to-end relation extraction. Our findings
show that a well-designed pipeline model is computationally inexpensive and
more effective than other methods.

##  2 METHODS

###  2.1 The RareDis dataset

The National Institutes of Health (NIH) estimates that around 7,000 rare
diseases impact between 25 and 30 million Americans, which translates to
approximately 1 out of every 10 Americans [26]. Around 95% of the known rare
diseases currently lack any treatment options [26]. Because these diseases are
so rare, they can be challenging to diagnose and treat — nearly 95% of rare
diseases have no known cure, and the number of drugs available for treating
these conditions is limited to 100 [27]. The average diagnostic delay is
around seven years [28]. Many rare diseases are genetic in nature and are
caused by mutations in a single gene. However, because there are thousands of
rare diseases, each with unique symptoms and genetic causes, developing
effective treatments can be a significant challenge. Developing a structured
compendium of information about rare diseases has the potential to help
expedite search, discovery, and hypothesis generation for these conditions.
This necessitates developing NLP models for RE in this setting and toward this
goal, Maritinez-deMiguel et al. [14] created an annotated corpus for rare
disease-related information extraction. This resource is based on the database
of articles about rare diseases maintained by the National Organization for
Rare Disorders (https://rarediseases.org/rare-diseases/). The dataset contains
six entity types and six relation types and the annotation process is
described in detail by the authors [14].

Figure 1:  Examples of is_a and anaphora relations in the RareDis dataset.

####  Entity and relations types

The six entity types in RareDis are: disease, rare disease, symptom, sign,
anaphor, and rare skin disease with frequencies shown in the first six rows of
Table 1. There are six relation types (with counts shown in the last six rows
of Table 1): produces (relation between any disease entity and a sign/symptom
produced by that entity), increase_risk_of (relation between a disease entity
and another disease entity where the subject disease increases the likelihood
of suffering from the object disease), is_a (relation between a given disease
and its classification as a more general disease), is_acron (relation between
an acronym and its full or expanded form), is_synon (relation between two
different names designating the same disease) and anaphora (relation of an
anaphor entity with its antecedent entity). Here an anaphor entity refers to
pronouns or pronominal constructs (e.g., ‘it” or “this disease”) that point to
a named entity that is already mentioned in the preceding context (the
“antecedent” of the anaphora relation). An example is shown in Figure 1.

Type | Training | Dev | Test
---|---|---|---
sign | 2945 | 798 | 528
rare disease | 2533 | 624 | 480
disease | 1369 | 278 | 230
anaphor | 913 | 195 | 151
skin rare disease | 393 | 58 | 45
symptom | 275 | 44 | 24
produces | 3256 | 850 | 556
anaphora | 918 | 195 | 151
is_a | 544 | 149 | 88
increase_risk_of | 161 | 8 | 22
is_acron | 142 | 44 | 34
is_synon | 66 | 14 | 16
Table 1:  Statistics of entity types (first six rows) and relation types (last
six rows) in the RareDis corpus.

The dataset contains discontinuous and overlapping/nested entities as
discussed with examples in Section 1; Table 2 throws light on the relative
frequency of these situations where “flat” corresponds to continuous entities.
While in both tables in this section we show training, development, and test
set counts, the original dataset consisted of only training and development
datasets where the authors claim to withhold the test set for a future shared
task, which has not happened yet. We split up their training dataset into
training and development for our experiments and their development split
became our test split.

Dataset | Training | Dev | Test
---|---|---|---
Flat | 7103 | 1666 | 1212
Discontinuous | 528 | 136 | 103
Overlapped | 720 | 166 | 112
Nested | 77 | 29 | 31
Total | 8428 | 1997 | 1458
Table 2:  Counts of entity types in the corpus.

####  Modifications to the original dataset

While exploring the dataset, we observed some annotation issues that we
confirmed with the creators of the RareDis dataset through email
communication. Next, we describe what they are and how we fixed them at a high
level in this section. We created a custom train, validate, test split of the
full dataset after fixing the following errors and made it available as a
Google Drive link on our GitHub page for this project.

##### Relation argument error

Figure 2 shows an example of how the annotations are provided for each
instance. For this example, we see the entities (T1, …, T9) listed first along
with types, character-based offsets, and lexical spans. Next, relations
between entities are listed (R1, …, R5) along with the relation type and the
arguments (subject and object). Although there are only nine entities, we see
for anaphora relation R5, the second argument is T90 with a trailing 0 after
9. This happened several times — arguments in relations referring to entity
IDs that are not present in the preceding entity list. This almost always
happened with a trailing extra zero. We safely removed that zero and it fixed
all these errors, which accounted for 9% of the total number of relations. In
the example in Figure 2, the anaphora relation R5 was referring to the bigram
“This disorder”.

Figure 2:  An example of the argument error due to an extra trailing zero in
entity IDs. Here, T90 ought to be just T9.

##### Span mismatch Error

There were a few occasions (less than 1% of the full dataset) where the
character offsets for entities captured an extra character than needed or
missed the last character of a word. We used simple rules to remove the extra
character or add the missing character. For example, in the sentence
“Balantidiasis is a rare infectious disease caused by the single-celled
(protozoan) parasite Balantidium coli,” the bold phrase was annotated as [T24,
DISEASE,1272 1289, infectious diseas] with a missing trailing character ‘e’.

##### Offset order error

For some discontinuous entities where more than one span is part of the full
entity, the order used for the spans was not left to right and we simply
reordered them as such.

As outlined earlier (in Section 1), we experiment with three different SoTA
approaches each representing a competing paradigm for E2ERE. Each of these
approaches is highly involved and hence we focus on high-level explanations of
how they work.

###  2.2 The three E2ERE methods

####  Pipeline: The PURE Approach

PURE by Zhong and Chen [18] is a span-based model that has two different
models for NER and RE parts of the E2ERE system. It improved upon prior joint
modeling approaches even though it separately trains NER and RE models. The
main argument by Zhong and Chen, the authors of PURE, is that NER and RE need
different representations of tokens because they need different types of
signals to make the predictions; and combining the signals can hurt the
performance of both.

Figure 3:  Pipeline approach using SODNER and PURE models for end-to-end
relation extraction.

One weakness of PURE is that it does not handle discontinuous entities in its
NER component while it easily handles flat and nested entities. So we needed
to adapt the PURE approach to the RareDis setting. Since PURE is pipeline-
based, we could simply use a different NER model for identifying discontinuous
entities and retain the PURE model to spot flat and nested entities. Hence, we
use a specialized model that was exclusively developed for handling
discontinuous entities called SODNER [13], which is also a span-based NER
model that models discontinuous NER task as a classification problem to
predict whether entity fragments with gaps ought to be linked to form a new
entity. To do this, SODNER uses dependency parses of the input document to
guide a graph convolutional neural (GCN) network that obtains enhanced
contextual embeddings to link disparate fragments and form discontinuous
entities. Figure 3 shows the schematic of the pipeline we use. It starts on
the left with the SODNER model identifying discontinuous entities.

Even if SODNER successfully identifies discontinuous entities, PURE’s relation
extraction model cannot handle them. The PURE relation model puts exactly one
start and one end entity marker token around each candidate subject (or
object) entity span. This modified input is passed through the contextual
language model (such as PubMedBERT) and the marker token embeddings are used
to predict the relation type. This is reflected by the purple [S:Disease] and
[$\backslash$S:Disease] tokens on the right side of Figure 3. But SODNER
outputs multiple fragments for discontinuous entities. Rather than change the
PURE relation model architecture, we use the discontinuous entity fragments
and straightforward rules to convert the input sentence to a modified one
where the discontinuous entities are rendered in a continuous format. For
instance, consider the input, “weakness in the muscles of the arms and legs,”
which contains two entities: one flat entity, “weakness in the muscles of the
arms and legs” and one discontinuous entity, “weakness in the muscles of the
arms and legs.” Both entities have the gold entity type Sign. Our modified new
input will read as: “weakness in the muscles of the arms and weakness in the
muscles of the legs”. This transformed sentence is now input through the PURE
NER model and then through the PURE relation model.

Neither the PURE NER model nor SODNER can handle cases where the same span has
more than one entity type (e.g., a span being both a disease and a sign). This
is a special case of overlapped entities where the overlap is exact, leading
to the same span having two types. Since most relations involving such spans
only use one of the entity types, this has not caused major issues in RE
evaluation.

####  Sequence-to-Sequence: The Seq2Rel Model

The Seq2Rel model [21] model uses an encoder-decoder framework to process the
input document and output relations akin to machine translation where the
source language sentence is ingested into the encoder and the target language
sentence is output by the decoder one token at a time. Here the target
sequence is essentially a list of relations. Unlike the machine translation
setting where the target is a natural language sequence where an order is
inherent, relations do not have any order amongst them. Hence, during training
an order is imposed on the relations in a document. Special tokens are also
used to represent entity types. For example, the relation R2 in Figure 2
indicates: (Rare disease “Vitamin D Deficiency Rickets”, produces, sign “bone
disease”), where the entity types are in bold. This will be linearized in
Seq2Rel as: Vitamin D Deficiency Rickets @RareDisease@ bone disease @Sign@
@PRODUCES@, where @ENTITY-TYPE@ and @RELATION-TYPE@ are special tokens
indicating entity and relation types, respectively. The @ENTITY-TYPE@ tokens
are preceded by the actual entity spans in the input. If an input does not
contain any relations, a special @NOREL@ is coded as the output. The order
imposed during training is simply the order in which the entities occur in the
document. This is reflected in Figure 2 where relations involving entities
that occur earlier in the document are annotated before relations that involve
entities that occur later. This left-to-right order is followed until all
relations are output followed by a special end of sequence token @END@
signaling that all relations have been output. Besides this linearization
schema, a “copy mechanism” [20] is applied to the decoder, restricting it to
generate tokens only from the observed input sequence, unlike the full
vocabulary of the target language in machine translation. This mechanism
enables the decoder to output spans of the input text that correspond to
entities, as well as special tokens representing relation labels that connect
these entities. The Seq2Rel model [21] uses a PubMedBERT model as the encoder
and a long short-term memory (LSTM) network as the decoder.

####  Generative Pre-trained Transformers: BioMedLM

Generative pre-trained transformers (GPTs) have captured the fascination of
the general public and researchers alike, especially since the introduction of
ChatGPT in December 2022. However, the in-context learning and few-shot
capabilities have already surfaced in June 2020, when Open AI released GPT-3
[23]. Building on the decoder component of the transformer architecture with
the main objective of autoregressive left to right next token prediction task,
they have excelled at text generation tasks (e.g., summarization). However,
there is a growing interest in assessing their capabilities for language
understanding tasks including relation extraction. BioGPT [24] and BioMedLM
[25] have been pre-trained from scratch on biomedical abstracts from PubMed
and full text articles from PubMed Central (from the corresponding subset from
Pile [29]) based on the GPT-2 model [22]. In this effort, we focus on
BioMedLM, a 2.7B parameter model, comprised of 32 layers, a hidden size of
2560, and 20 attention heads. BioMedLM is an order of magnitude larger than
BioGPT and nearly twice as large as BioGPTlarge. It has been shown to be be
superior to BioGPT models (including in our experiments for this paper where
BioGPT underperforms by 10-15% in F-score) and to our knowledge is the largest
public GPT-style model for biomedicine. Hence, we only show BioMedLM results
in this manuscript for the sake of clarify and simplicity. Unlike Seq2Rel
whose sequence generation capabilities are highly constrained to terms
observed in the input, BioMedLM and BioGPT are purely generative, and
supervised fine-tuning involves using appropriate prompts and output
templates. Technically, we could simply use the linearization schemas
introduced for Seq2Rel. However, these generative models generate natural
language statements and not unnatural-looking templates. So our initial
experiments using a Seq2Rel style output schemas have failed. So, we
considered two types of schemas here:

  * •

rel-is template: This output template is the same as that used by the original
BioGPT paper for E2ERE: “The relation between subject-span and object-span is
relationType.noun,” where relationType.noun is the noun form of the predicate.
With this template, as an example, the output for the gold relation (Wilm’s
tumor, is_a, kidney cancer) is: “The relationship between Wilm’s tumor and
kidney cancer is hyponym”. We can see here that we converted “is a” predicate
to a noun representation “hyponym” in the template and a similar strategy was
followed for all predicates.

  * •

natural-lang: We came up with different natural language templates tailored to
each relation type in RareDis. They are fully specified in Table 3, each with
a representative example.

Relation type | Natural language output template
---|---
(An example for the template)
produces |  $ent_{1}Span$ is a $ent_{1}Type$ that produces $ent_{2}Span$, as a $ent_{2}Type$
(Asherman’s syndrome is a rare disease that produces abdominal pain, as a
symptom)
anaphora | The term $ent_{2}Span$ is an anaphor that refers back to the entity of the $ent_{1}Type$ $ent_{1}Span$
(The term “it” is an anaphor that refers back to the entity of the disease
encephalitis)
is_synon | The $ent_{1}Type$ $ent_{1}Span$ and the $ent_{2}Type$ $ent_{2}Span$ are synonyms
(The disease diastrophic dysplasia and the rare disease disastrophic dwarfism
are synonyms)
is_acron | The acronym $ent_{1}Span$ stands for $ent_{2}Span$, a $ent_{2}Type$
(The acronym LQTS stands for long QT syndrome, a rare disease)
increases_risk_of | The presence of the $ent_{1}Type$ $ent_{1}Span$ increases the risk of developing the $ent_{2}Type$ of $ent_{2}Span$
(The presence of the disease neutropenia increases the risk of developing the
disease infections)
is_a | The $ent_{1}Type$ $ent_{1}Span$ is a type of $ent_{2}Span$, a $ent_{2}Type$
(The rare skin disease Bowen disease is a type of skin disorder, a disease)

Table 3:  Natural language templates used to encode RareDis relations as
BioMedLM outputs.

###  2.3 Training objectives and evaluation metrics

For the SODNER+PURE pipeline model, the training objective is the well-known
cross entropy function for both NER and RE components. Seq2Rel and BioMedLM,
however, produce sequences (based on the schemas and templates selected) that
need to be interpreted back into the triple format (which we accomplish using
regular expressions). Since their outputs are sequences, the training
objective is the well-known auto-regressive language model objective based on
predicting the next token given previously predicted tokens. The loss function
is the average cross-entropy per target word (more details in Chapter 9.7 of
Jurafsky and Martin [30]).

For evaluation, we note that RareDis annotations are at the span level and
hence the same exact relation connecting the same entities can occur multiple
times if it is discussed several times in the document. However, Seq2Rel and
BioMedLM do not keep track of the number of times a relation occurs as they
are generative and do not operate on spans; but the pipeline models output all
connections as they operate at the span level. To ensure fair evaluation, if
the same relation occurs multiple times within an instance, it is collapsed
into a single occurrence. This is natural and harmless because there is no
loss of information if duplicate relations are ignored. Since Seq2Rel and
BioMedLM produce sequences, we use regular expressions on top of the output
templates and schemas to produce the triples we need. The evaluation metrics
are precision, recall, and F1-score, which are standard in RE. For a relation
to be counted as correctly predicted, the subject and object entity types,
their spans, and the relation type all need to exactly match the ground truth
relation.

##  3 RESULTS AND DISCUSSION

Experiments for the pipeline approach were performed on our in-house cluster
of 32GB GPU. All experiments for Seq2Rel were performed on Google Colab Pro+
using an Nvidia a100-sxm4-40gb GPU with access to high RAM. In Seq2Rel, we use
AllenNLP, an open-source NLP library developed by the Allen Institute for
Artificial Intelligence (AI2). Fairseq, a sequence modeling toolkit, is used
for training custom models for text generation tasks for BioGPT on Google
Colab Pro. We used Lambda Labs to fine-tune BioMedLM on a single H100 80GB
GPU.

Next, we describe model configurations and hyperparameters. Our settings for
learning rate, number of epochs, and other hyperparameters are determined
based on experiments on the validation dataset.

  * •

Pipeline (SODNER+PURE): We used a batch size of 8, a learning rate of 1e-3,
and 100 epochs to train the SODNER model for discontinuous entities with a
PubMedBERTbase encoder. For the PURE NER model, we used PubMedBERTbase and
trained for 100 epochs, with a learning rate of 1e-4 and a batch size of 8. We
also experimented with PubMedBERTlarge with the same settings. For the PURE
relation model, we used both PubMedBERTbase and PubMedBERTlarge as encoders
with a learning rate of 1e-5 and trained for 25 epochs with the training batch
size of 8.

  * •

Seq2Rel: Training was conducted for 150 epochs, with a learning rate of 2e-5
for the encoder (PubMedBERTbase or PubMedBERTlarge) and 1.21e-4 for the
decoder (LSTM) with a batch size of 2 and a beam size of 3 (for the decoder).

  * •

BioMedLM: Despite supervised fine-tuning, it is not uncommon for GPT models to
output strings that were not part of the input. We observed that nearly 3%-7%
of entities output by BioMedLM did not exactly match ground truth spans. Since
we require an exact match for a prediction to be correct, we appended explicit
natural language instructions to the input, directing the model to generate
tokens from the input text: “From the given abstract, find all the entities
and relations among them. Do not generate any token outside the abstract.” We
used a batch size of 1 with gradient_accumulation_steps of 16, a learning rate
of 1e-5, and 30 epochs for BioMedLM.

We also needed some post-processing tricks to handle the idiosyncrasies of the
three different models. As we discussed earlier in Section 2.2.1, for the
pipeline models, since discontinuous entities are not handled natively by the
PURE relation model, we had to transform the inputs to render the
discontinuous entities in a flat fashion before passing them on to the PURE
model. For the Seq2Rel model, due to the WordPiece tokenization in BERT
models, the output sometimes contains extra spaces around hyphens and
brackets. To align such output strings with the input text, as a post-
processing step, we removed these additional spaces, specifically around
hyphens, curved brackets, and forward slashes.

The main results of the comparison using different models are presented in
Table 4. We observe that the SODNER+PURE pipeline (with PubMedBERTbase
encoder) produces the best F1-score of 52.2, which is 5 points more than the
best-performing Seq2Rel model with the PubMedBERTlarge encoder (47.15 F1), and
13 points more than best performing BioMedLM model (38.9 F1). The pipeline’s
performance does not increase when using the PubMedBERTlarge model. For
Seq2Rel, using PubMedBERTlarge outperforms a model with PubMedBERTbase (44.53
F1) by 2.5 points, with an increase in both precision and recall. Potentially,
the increased model capacity of PubMedBERTlarge enables it to capture more
complex and subtle relationships between medical terms and concepts. However,
it is not clear why similar gains were not observed with PubMedBERTlarge in
the pipeline.

Method | Configuration | copyInstruct | Score
---|---|---|---
P | R | F
SODNER + PURE | PubMedBERTbase | NA | 55.99 | 48.89 | 52.20
SODNER + PURE | PubMedBERTlarge | NA | 56.20 | 48.52 | 52.08
Seq2Rel | PubMedBERTbase | NA | 47.60 | 40.90 | 44.53
PubMedBERTlarge | NA | 51.46 | 43.51 | 47.15
BioMedLM | rel-is | yes | 40.19 | 29.68 | 34.14
rel-is | no | 42.14 | 36.1 | 38.89
natural-lang | yes | 38.64 | 32.81 | 35.49
natural-lang | no | 44.22 | 33.76 | 38.29
Table 4: Performances of different models under different settings on the
RareDis dataset.

For BioMedLM, the ‘copyInstruct’ column in Table 4 indicates the additional
input prompt discussed earlier in this section where decoder-only auto-
regressive models are directed to generate tokens observed in the input. The
best performance for BioMedLM is an F1 score of 38.89 using the rel-is
template for prompting the model when copy instructions were not provided.
When copy instructions are not provided, rel-is does slightly better (<1% F1)
and when copy instructions are not provided, natural-lang does better job
(1.35 of points gain) So looks like there is no advantage to using copy
instructions. (However, when using the smaller BioGPT models, the natural
language prompting seemed to perform slightly better than the rel-is
template.) Note that, BioMedLM’s best performance is still $\approx 6$ points
lower than then Seq2Rel’s best score and 11 points lower than the pipeline
score.

Note that BioMedLM is over eight times larger than our best-performing
pipeline model (considering it has three encoders based on the encoder
PubMedBERTbase, which has 110M parameters). However, its low performance
compared to the pipeline is not surprising because GPT models are
autoregressive and do not benefit from language understanding arising from the
bidirectional masked language modeling objective used in BERT models. Although
the original BioMedLM [25] effort did not perform RE, it reports SOTA scores
on biomedical Q&A tasks. The smaller BioGPT models were shown to do better
than BERT models for E2ERE too. Hence we repurposed them for this RE task and
as the largest publicly available GPT-based model, BioMedLM outperformed
BioGPT models [24] by 10–15% in F1 score and we do not see these as worthy of
reporting in this manuscript. We believe much larger models (GPT-3, GPT-3.5,
GPT-4) ought to be used to fully leverage the power of generative LMs.
Furthermore, some recent results also show that using GPT-style models to
generate additional training examples to augment the training data may be a
more effective way of using them, rather than fine-tuning them for RE tasks.

Relation type | SODNER+PURE | Seq2Rel | BioMedLM
---|---|---|---
P | R | F | P | R | F | P | R | F
anaphora | 70.40 | 69.84 | 70.11 | 64.60 | 58.00 | 61.08 | 61.26 | 53.96 | 57.38
is_a | 62.67 | 55.29 | 58.75 | 58.67 | 51.76 | 55.00 | 52.77 | 44.70 | 48.40
is_acron | 70.37 | 57.58 | 63.33 | 50.00 | 42.00 | 45.65 | 55.17 | 48.48 | 51.61
produces | 50.21 | 45.09 | 47.51 | 47.48 | 41.13 | 44.00 | 37.20 | 32.82 | 34.87
is_synon | 75.00 | 18.75 | 30.00 | 100.00 | 12.50 | 22.23 | 0.00 | 0.00 | 0.00
increases_risk_of | 50.00 | 4.55 | 8.33 | 11.80 | 9.52 | 10.52 | 0.00 | 0.00 | 0.00
Table 5: Scores for each relation type of best-performing models in the group.

We also wanted to examine scores per relation type in our models to see if
there are any predicates for which we are underperforming more than expected.
From Table 5, we notice that recall is less than 5% for increases_risk_of
relation type. This is quite awful but not surprising given the prevalence of
such relations is very small in the dataset (from Table 1). But what is very
unusual is the F1 of the ‘produces’ relation being less than 50, when it
constitutes over 60% of all relations in the dataset (from Table 1). Upon
deeper investigation, we found that generally longer object entities lead to
NER errors. We checked this more concretely by examining the errors (for
‘produces’) and found out that we missed 43% of the object spans for the best-
performing pipeline method. Thus, a large portion of performance loss is
simply due to the model not being able to predict the object entity span
correctly; especially for long object entities, even missing a single token
can lead to RE errors.

Thus, the overall performance pattern observed for the RareDis dataset is
Pipeline $>$ Seq2Rel $>$ BioMedLM. We wanted to verify this with at least one
other dataset. Considering our prior experiences with the chemical-protein
interaction extraction task [31], we repeated our E2ERE experiments using the
BioCreative Shared Task VI dataset and the results showed the same performance
pattern with pipeline leading to a 69 F1 score, followed by Seq2Rel with 49,
and BioMedLM with 37 points.

##  4 Error Analysis

Before we proceed, we note that many RE errors appear to arise from NER
errors. This can lead to a snowball effect of errors in the RE phase. Consider
a single entity participating in $n$ gold relations. If it is predicted
incorrectly as a partial match, it may potentially lead to $2n$ relation
errors because it can give rise to $n$ false positives (FPs) (because the
relation is predicted with the wrong span) and $n$ false negatives (FNs)
(because the gold relation with the right span is missed). Thus, even a small
proportion of NER errors can lead to a high loss in RE performance. In this
section, we discuss a few error categories that we observed commonly across
models.

  * •

Partial matches: When multi-word entities are involved, the relation error is
often due to the model predicting a partial match (a substring or superstring
of a gold span) and this was frequent in our effort. Consider the snippet
“Kienbock disease changes may produce pain…The range of motion may become
restricted”. Here Kienbock disease is the subject of a produces relation with
the gold object span: “the range of motion may become restricted”. However,
the Seq2Rel model predicted “range of motion restricted” as the object span,
leading to both an FP and FN. But common sense tells us that the model
prediction is also correct (and potentially even better) because it removed
the unnecessary “may become” substring. In a different example, when the
relation involved the gold span “neurological disorder,” the model predicted a
superstring “progressive neurological disorder” from the full context:
“Subacute sclerosing panencephalitis (SSPE) is a progressive neurological
disorder.”

  * •

Entity type mismatch:  Because our evaluation is strict, predicting the entity
spans and relation type correctly, but missing a single entity type can
invalidate the whole relation leading to both an FP and an FN. The models are
often confused between closely related entity types. Rare disease and skin
rare disease were often confused along with the pair sign and symptom.

  * •

Issues with discontinuous entities: Discontinuous entities are particularly
tricky and have led to several errors, even if the prediction is not
incorrect, because the model was unable to split an entity conjunction into
constituent entities. Consider the snippet: “affected infants may exhibit
abnormally long, thin fingers and toes and/or deformed (dysplastic) or absent
nails at birth.” Instead of generating relations with the two gold entities
“abnormally long, thin fingers” and “abnormally long, thin toes”, the model
simply created one relation with “long, thin fingers and toes.”

  * •

BioMedLM generations not in the input: In several cases we noticed spans that
were not in the input but were nevertheless closely linked with the gold
entity span’s meaning. For example, for the gold span “muscle twitching”,
BioMedLM predicted “muscle weakness”. It also tried to form meaningful noun
phrases that capture the meaning of longer gold spans. For instance, for the
gold span “ability to speak impaired”, it predicted “difficulty in speaking”.
For the gold span, “progressive weakness of the muscles of the legs” it
outputs “paralysis of the legs”. All these lead to both FPs and FNs,
unfortunately.

  * •

Errors due to potential annotation issues: In document-level RE settings, it
is not uncommon for annotators to miss certain relations. But when these are
predicted by a model, they would be considered FPs. Consider the context: “The
symptoms of infectious arthritis depend upon which agent has caused the
infection but symptoms often include fever, chills, general weakness, and
headaches.” Our model predicted that “infectious arthritis” produces “fever”.
However, the gold predictions for this did not have this and instead had the
relation “the infection” (anaphor) produces “fever”. While the gold relation
is correct, we believe what our model extracted is more meaningful. However,
since we missed the anaphor-involved relation, it led to an FN and an FP.

##  5 Conclusion

In this paper, we explored three state of the art representative models for
E2ERE from three competing paradigms: pipelines (SODNER + PURE), sequence-to-
sequence models (Seq2Rel), and generative LMs (BioMedLM). Our evaluations used
a complex dataset (RareDis) involving discontinuous, nested, and overlapping
entities. Even with the advances in Seq2Seq models and generative
transformers, a custom-built pipeline still seems to be the best option based
on our experiments in this paper. The performance gap between Seq2Rel and the
pipeline is not as high as that between BioMedLM and pipeline. As such there
could be other datasets where Seq2Rel matches the pipeline methods especially
for simpler NER scenarios without discontinuous entities. We still would not
want readers to conclude that more advanced models are not suitable for this
task and not to take away from the few-shot abilities of GPT models. Also, the
generative aspects of GPT models may not be suitable for the type of strict
evaluation imposed here where an exact match with gold spans is required. In
the future, this may be mitigated by using vector similarity or edit-distance
metrics to map such phrases to the closest matches of the input. Using
inference-only proprietary large models such as GPT-4 [32] to generate
paraphrases for training instances to create larger augmented training
datasets could also be helpful. However, in the end, a small $\approx$ 200M
parameter pipeline model that can run on consumer desktops may be preferable
for several use-cases even in the current era of excitement over generative
transformers.

## Acknowledgment

This work is supported by the NIH National Library of Medicine through grant
R01LM013240. The content is solely the responsibility of the authors and does
not necessarily represent the official views of the NIH.

## References

  * [1] Dietze H, Schroeder M.  GoWeb: a semantic search engine for the life science web.  BMC bioinformatics. 2009;10:1-13.
  * [2] Henry S, McInnes BT.  Literature based discovery: models, methods, and trends.  Journal of biomedical informatics. 2017;74:20-32.
  * [3] Kilicoglu H, Rosemblat G, Fiszman M, Shin D.  Broad-coverage biomedical relation extraction with SemRep.  BMC bioinformatics. 2020;21:1-28.
  * [4] Zeng D, Liu K, Lai S, Zhou G, Zhao J.  Relation classification via convolutional deep neural network.  In: Proceedings of COLING 2014, the 25th international conference on computational linguistics: technical papers; 2014. p. 2335-44.
  * [5] Zhou P, Shi W, Tian J, Qi Z, Li B, Hao H, et al.  Attention-based bidirectional long short-term memory networks for relation classification.  In: Proceedings of the 54th annual meeting of the association for computational linguistics (volume 2: Short papers); 2016. p. 207-12.
  * [6] Kavuluru R, Rios A, Tran T.  Extracting drug-drug interactions with word and character-level recurrent neural networks.  In: 2017 IEEE International Conference on Healthcare Informatics (ICHI). IEEE; 2017. p. 5-12.
  * [7] Miwa M, Bansal M.  End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures.  In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). vol. 1; 2016. p. 1105-16.
  * [8] Zhang M, Zhang Y, Fu G.  End-to-End Neural Relation Extraction with Global Optimization.  In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing; 2017. p. 1730-40.
  * [9] Pawar S, Bhattacharyya P, Palshikar G.  End-to-end relation extraction using neural networks and Markov logic networks.  In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. vol. 1; 2017\. p. 818-27.
  * [10] Tran T, Kavuluru R.  An end-to-end deep learning architecture for extracting protein–protein interactions affected by genetic mutations.  Database. 2018:1-13.
  * [11] Peng N, Poon H, Quirk C, Toutanova K, Yih Wt.  Cross-sentence n-ary relation extraction with graph lstms.  Transactions of the Association for Computational Linguistics. 2017;5:101-15.
  * [12] Yao Y, Ye D, Li P, Han X, Lin Y, Liu Z, et al.  DocRED: A Large-Scale Document-Level Relation Extraction Dataset.  In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics; 2019. p. 764-77.
  * [13] Li F, Lin Z, Zhang M, Ji D.  A Span-Based Model for Joint Overlapped and Discontinuous Named Entity Recognition.  In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers); 2021. p. 4814-28.
  * [14] Martínez-deMiguel C, Segura-Bedmar I, Chacón-Solano E, Guerrero-Aspizua S.  The RareDis corpus: a corpus annotated with rare diseases, their signs and symptoms.  Journal of Biomedical Informatics. 2022;125:103961.
  * [15] Fabregat H, Araujo L, Martinez-Romo J.  Deep neural models for extracting entities and relationships in the new RDD corpus relating disabilities and rare diseases.  Computer methods and programs in biomedicine. 2018;164:121-9.
  * [16] Eberts M, Ulges A.  Span-Based Joint Entity and Relation Extraction with Transformer Pre-Training.  In: ECAI 2020. IOS Press; 2020. p. 2006-13.
  * [17] Tran T, Kavuluru R.  Neural metric learning for fast end-to-end relation extraction.  arXiv preprint arXiv:190507458. 2019.
  * [18] Zhong Z, Chen D.  A Frustratingly Easy Approach for Entity and Relation Extraction.  In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies; 2021. p. 50-61.
  * [19] Nayak T, Ng HT.  Effective modeling of encoder-decoder architecture for joint entity and relation extraction.  In: Proceedings of the AAAI conference on artificial intelligence. vol. 34; 2020. p. 8528-35.
  * [20] Zeng X, Zeng D, He S, Liu K, Zhao J.  Extracting relational facts by an end-to-end neural model with copy mechanism.  In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers); 2018. p. 506-14.
  * [21] Giorgi J, Bader G, Wang B.  A sequence-to-sequence approach for document-level relation extraction.  In: Proceedings of the 21st Workshop on Biomedical Language Processing. Dublin, Ireland: Association for Computational Linguistics; 2022. p. 10-25.  Available from: https://aclanthology.org/2022.bionlp-1.2.
  * [22] Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I. Language models are unsupervised multitask learners; 2019.  Available from: https://insightcivic.s3.us-east-1.amazonaws.com/language-models.pdf.
  * [23] Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, et al.  Language models are few-shot learners.  Advances in neural information processing systems. 2020;33:1877-901.
  * [24] Luo R, Sun L, Xia Y, Qin T, Zhang S, Poon H, et al.  BioGPT: generative pre-trained transformer for biomedical text generation and mining.  Briefings in Bioinformatics. 2022;23(6).
  * [25] Bolton E, Hall D, Yasunaga M, Lee T, Manning C, Liang P. BioMedLM; 2022.  Available from: https://crfm.stanford.edu/2022/12/15/biomedlm.html.
  * [26] National Organization for Rare Disorders (NORD). Rare Disease Database Frequently Asked Questions; 2019.  Accessed: Month Day, Year.  https://rarediseases.org/wp-content/uploads/2019/01/RDD-FAQ-2019.pdf.
  * [27] Klimova B, Storek M, Valis M, Kuca K.  Global view on rare diseases: a mini review.  Current medicinal chemistry. 2017;24(29):3153-8.
  * [28] Global Genes. Facts;.  Accessed May 1, 2023.  https://globalgenes.org/learn/rare-disease-facts/.
  * [29] Gao L, Biderman S, Black S, Golding L, Hoppe T, Foster C, et al.  The pile: An 800gb dataset of diverse text for language modeling.  arXiv preprint arXiv:210100027. 2020.
  * [30] Jurafsky D, Martin JH.  Speech and Language Processing (3rd Edition); 2023.  https://web.stanford.edu/~jurafsky/slp3/.
  * [31] Ai X, Kavuluru R.  End-to-End Models for Chemical-Protein Interaction Extraction: Better Tokenization and Span-Based Pipeline Strategies.  arXiv preprint arXiv:230401344. 2023.
  * [32] Bubeck S, Chandrasekaran V, Eldan R, Gehrke J, Horvitz E, Kamar E, et al.  Sparks of artificial general intelligence: Early experiments with GPT-4.  arXiv preprint arXiv:230312712. 2023.


####  4.1.1 Universal enveloping algebras of Lie algebras

The universal enveloping algebra ${\mathcal{U}}(\mathfrak{g})$ of a Lie
algebra $\mathfrak{g}$ is the quotient of the tensor algebra
$\otimes(\mathfrak{g})$ by the two-sided ideal generated by the identification
of the commutator of generators with their Lie bracket (i.e. the associative
ideal spanned by elements proportional to
$\hat{X}\otimes\hat{Y}-\hat{Y}\otimes\hat{X}-[\hat{X},\hat{Y}]$ for some
$\hat{X},\hat{Y}\in\mathfrak{g}$).

The Poincaré-Birkhoff-Witt theorem asserts that the universal enveloping
algebra ${\mathcal{U}}(\mathfrak{g})$ of the Lie algebra $\mathfrak{g}$ is an
almost-commutative algebra whose associated graded algebra is isomorphic to
the symmetric algebra of $\mathfrak{g}$,

$\text{gr}\,{\mathcal{U}}(\mathfrak{g})\,\cong\,\odot(\mathfrak{g})\,.$ (93)

Note that the universal enveloping algebra ${\mathcal{U}}(\mathfrak{g})$ of a
Lie algebra $\mathfrak{g}$ is never simple, since the subalgebra
${\mathcal{U}}_{>0}(\mathfrak{g})\subset{\mathcal{U}}(\mathfrak{g})$ of
strictly positive degree is always an associative ideal.

Example (Abelian Lie algebra): Any vector space $V$ can be endowed with a
structure of Lie algebra with the trivial Lie bracket. The universal
enveloping algebra of the Abelian Lie algebra $V$ is isomorphic to the
symmetric algebra of $V$: ${\mathcal{U}}(V)\,\cong\,\odot(V)$ .

Remark: When dealing with Lie-Rinehart algebras $\mathfrak{L}$ in the next
subsection, it will be important to distinguish explicitly the universal
enveloping algebra ${\mathcal{U}}_{\mathbb{K}}(\mathfrak{L})$ of the Lie
algebra $\mathfrak{L}$ over the field $\mathbb{K}$ from the universal
enveloping algebra ${\mathcal{U}}_{\mathcal{A}}(\mathfrak{L})$ of the Lie-
Rinehart algebra $\mathfrak{L}$ over the commutative algebra $\mathcal{A}$. In
the former case, the Lie-Rinehart algebra is simply seen as a Lie algebra
(over the field $\mathbb{K}$) while, in the latter case, the richer structure
of Lie-Rinehart algebra (over the commutative algebra $\mathcal{A}$) is taken
into account (see Subsection 4.1.3).

####  4.1.2 Lie-Rinehart algebras and their associated bimodules

To any Lie-Rinehart algebra $\mathfrak{L}$ over a commutative algebra
$\mathcal{A}$, with anchor $\rho$, is associated another Lie-Rinehart algebra
over $\mathcal{A}$: the semidirect sum
${\mathfrak{B}}=\mathfrak{L}\inplus_{\rho}\mathfrak{A}$ of the Lie-Rinehart
algebra $\mathfrak{L}$ and the commutator algebra $\mathfrak{A}$.

Proof: Indeed, the anchor $\rho:\mathfrak{L}\to\mathfrak{der}({\mathcal{A}})$
of any Lie-Rinehart algebra $\mathfrak{L}$ over a commutative algebra
$\mathcal{A}$ is a representation of the Lie algebra $\mathfrak{L}$ on the
associative algebra $\mathcal{A}$. The corresponding Abelian Lie algebra
$\mathfrak{A}$ can be endowed trivially with a structure of $\mathcal{A}$-Lie
algebra (with trivial anchor and bracket). Therefore, the anchor of
$\mathfrak{L}$ provides a representation
$\rho:\mathfrak{L}\to\mathfrak{der}(\mathfrak{A})$ of the Lie-Rinehart algebra
$\mathfrak{L}$ on the $\mathcal{A}$-Lie algebra $\mathfrak{A}$. This allows to
define the semidirect sum
${\mathfrak{B}}=\mathfrak{L}\inplus_{\rho}\mathfrak{A}$. By construction, the
adjoint representation of the subalgebra $\mathfrak{L}\subset\mathfrak{B}$ on
the Abelian ideal ${\mathfrak{A}}$ is defined by the anchor $\rho$ of
$\mathfrak{L}$, i.e. $[\hat{X},\hat{f}]_{{}_{\mathfrak{B}}}:=\hat{X}[f]$ for
any $\hat{X}\in\mathfrak{L}$ and $f\in\mathcal{A}$. The left
$\mathcal{A}$-module structure of ${\mathfrak{B}}$ is defined in the obvious
way $g\cdot(f\oplus\hat{X}):=(g\cdot f)\oplus(g\cdot\hat{X})$. ∎

What is remarkable is that this Lie-Rinehart algebra
${\mathfrak{B}}={\mathfrak{A}}\niplus\mathfrak{L}$ has a richer structure than
the original Lie-Rinehart algebra $\mathfrak{L}$ in the sense that it is not
only a left $\mathcal{A}$-module but also an $\mathcal{A}$-bimodule, where the
right $\mathcal{A}$-module structure is defined via the action

$(f\oplus\hat{X})\bullet g\,:=\,\big{(}\,f\cdot
g+\hat{X}[g]\,\big{)}\oplus\big{(}\,g\cdot\hat{X}\,\big{)}\,.$ (94)

From this perspective, the anchor of $\mathfrak{L}$ can be seen as the
structure that relates the left and right $\mathcal{A}$-module structures of
${\mathfrak{A}}\niplus\mathfrak{L}$. If the left and right actions are written
by the same symbol $\circ$ in an operatorial form, then the relation (94) can
be written in the more balanced form
$(\hat{f}\oplus\hat{X})\circ\hat{g}\,:=\,\big{(}\,\hat{f}\circ\hat{g}+\hat{X}[g]\,\big{)}\oplus\big{(}\,\hat{g}\circ\hat{X}\,\big{)}$,
which already suggests its later interpretation in the universal enveloping
algebra as arising from a commutator of an associative product.

Example (Lie-Rinehart algebra of smooth vector fields) : In the case when
$\mathcal{A}$ and $\mathfrak{L}$ are respectively the structure algebra
${\mathcal{C}}^{\infty}(M)$ and the Lie algebra ${\mathfrak{X}}(M)$ of vector
fields on a manifold $M$, the associated Lie-Rinehart algebra
${\mathfrak{B}}={\mathfrak{A}}\niplus\mathfrak{L}$ is isomorphic to the Lie-
Rinehart algebra
${\mathfrak{D}}^{1}(M)\cong{\mathcal{C}}^{\infty}(M)\niplus{\mathfrak{X}}(M)$
of first-order differential operators on $M$.

####  4.1.3 Universal enveloping algebras of Lie-Rinehart algebras

The universal enveloping algebra ${\mathcal{U}}_{\mathcal{A}}(\mathfrak{L})$
of a Lie-Rinehart algebra $\mathfrak{L}$ over the commutative algebra
$\mathcal{A}$ can be defined (see e.g. [11] and refs therein) as the quotient
of the universal enveloping algebra
${\mathcal{U}}_{\mathbb{K}}({\mathfrak{B}})$ of the Lie algebra
${\mathfrak{B}}=\mathfrak{A}\niplus\mathfrak{L}$ by the two-sided ideal
generated by the left action of $\mathcal{A}$ on ${\mathfrak{B}}$, i.e. by the
associative ideal spanned by elements proportional to
$g\cdot(f\oplus\hat{X})-(g\cdot f)\oplus(g\cdot\hat{X})$ for some
$f,g\in\mathcal{A}$ and $\hat{X}\in\mathfrak{L}$.

Example (Smooth differential operators) : The almost-commutatative algebra
${\mathcal{D}}(M)$ of differential operators on $M$ is the universal
enveloping algebra of the Lie-Rinehart algebra ${\mathfrak{X}}(M)$ of vector
fields on $M$, i.e.

${\mathcal{U}}_{C^{\infty}(M)}\big{(}{\mathfrak{X}}(M)\big{)}\cong{\mathcal{D}}(M)\,.$
(95)

Example (Free modules generated by Lie algebras) : Consider the simplest
examples of $\mathcal{A}$-Lie algebras: free ${\mathcal{A}}$-modules generated
by a Lie algebra $\mathfrak{g}$. The universal enveloping algebra of such an
$\mathcal{A}$-Lie algebra $\mathfrak{L}={\mathcal{A}}\otimes\mathfrak{g}$ is
the free ${\mathcal{A}}$-modules generated by the universal enveloping algebra
of the Lie algebra $\mathfrak{g}$,

${\mathcal{U}}_{\mathcal{A}}({\mathcal{A}}\otimes\mathfrak{g})\cong{\mathcal{A}}\otimes{\mathcal{U}}_{\mathbb{K}}(\mathfrak{g})\,.$
(96)

Example (Invariant differential operators) : Consider a Klein geometry, i.e.,
a pair made of a Lie group $G$ and a closed Lie subgroup $H\subset G$ such
that the coset space $G/H$ is connected. It defines a principal $H$-bundle $G$
over $G/H$ whose Atiyah algebroid is the vector bundle $\frac{TG}{H}$ over
$G/H$. Its global sections are $H$-invariant vector fields on $G$. They span
the Atiyah algebra $\mathfrak{X}(G)^{H}=\Gamma(TG/H)$. The universal
enveloping algebra of the Atiyah algebra of such a Klein geometry $H\subset G$
is spanned by $H$-invariant differential operators on $G$ [9, Example 4.26]

${\cal U}_{C^{\infty}(G/H)}\big{(}\mathfrak{X}(G)^{H}\big{)}\simeq{\cal
D}(G)^{H}\simeq{\cal U}_{C^{\infty}(G)}\big{(}\mathfrak{X}(G)\big{)}^{H}.$
(97)

Counter-example (Covariant differential operators on a module) : Consider an
$\mathcal{A}$-module V. The tensor algebra
$\otimes_{\mathcal{A}}(\textsc{V})=\oplus_{r\in\mathbb{N}}\otimes^{r}_{\mathcal{A}}\textsc{V}$
is an $\mathcal{A}$-algebra, $\mathbb{N}$-graded by the rank of tensors. A
covariant derivative on an $\mathcal{A}$-module V can be defined equivalently
as a derivation of the tensor algebra $\otimes_{\mathcal{A}}\textsc{V}$
preserving the rank of tensors. Consider the algebra
${\mathcal{D}}_{\mathcal{A}}(\otimes_{\mathcal{A}}\textsc{V})=\oplus_{q\in\mathbb{Z}}{\mathcal{D}}_{q}(\otimes_{\mathcal{A}}\textsc{V})$
of differential operators on the tensor algebra
$\otimes_{\mathcal{A}}(\textsc{V})$. It is $\mathbb{Z}$-graded: an element of
${\mathcal{D}}_{q}(\otimes_{\mathcal{A}}\textsc{V})$ increases the tensor rank
by $q$. A differential operator on the tensor algebra
$\otimes_{\mathcal{A}}(\textsc{V})$ preserving the rank of tensors will be
called a covariant differential operator on the $\mathcal{A}$-module V since
they are somehow the higher-order generalisation of covariant derivatives. The
almost-commutative subalgebra
${\mathcal{D}}_{0}(\otimes_{\mathcal{A}}\textsc{V})$ spanned by all covariant
differential operators on the $\mathcal{A}$-module V will be denoted
${\mathcal{C}D}_{\mathcal{A}}(\textsc{V})$. In general, it is not isomorphic
to the universal enveloping algebra
${\mathcal{U}}_{\mathcal{A}}(\,\mathfrak{cder}_{\mathcal{A}}(\textsc{V})\,)$
of the Atiyah algebra $\mathfrak{cder}_{\mathcal{A}}(\textsc{V})$ of covariant
derivatives [9, Example 4.28]. However, if the first-order covariant
differential operators generate the whole algebra of covariant differential
operators, then it is isomorphic to a quotient of the universal enveloping
algebra.

Remember that the universal enveloping algebra ${\mathcal{U}}(\mathfrak{g})$
of a Lie algebra $\mathfrak{g}$ is never simple. This remains true for the
universal enveloping algebra ${\mathcal{U}}_{\mathcal{A}}(\mathfrak{L})$ of an
$\mathcal{A}$-Lie algebra $\mathfrak{L}$. However, for a Lie-Rinehart algebra
$\mathfrak{L}$ with non-trivial anchor the universal enveloping algebra
${\mathcal{U}}_{\mathcal{A}}(\mathfrak{L})$ may be simple.

Example (Lie-Rinehart algebra of polynomial differential operators) : The Weyl
algebra ${\mathcal{D}}(A)$ of polynomial differential operators on the affine
space $A$, modeled on the vector space $V$, is simple although it is
isomorphic to the universal enveloping algebra of the Lie-Rinehart algebra
$\mathfrak{der}(\odot V^{*})$ of polynomial vector fields on $A$,

${\mathcal{U}}_{\odot V^{*}}\big{(}\mathfrak{der}(\odot
V^{*})\,\big{)}\cong{\mathcal{D}}(A)\,.$ (98)

####  4.1.4 Poincaré-Birkhoff-Witt theorem

In order to identify concretely the universal enveloping algebra of a Lie-
Rinehart algebra, the most important result to know is the generalisation [20]
by Rinehart of the Poincaré-Birkhoff-Witt theorem: if $\mathfrak{L}$ is a
projective left $\mathcal{A}$-module, then the universal enveloping algebra
${\mathcal{U}}_{\mathcal{A}}(\mathfrak{L})$ is an almost-commutative algebra
whose graded algebra is isomorphic to the symmetric algebra of $\mathfrak{L}$
over $\mathcal{A}$,

$\text{gr}\,{\mathcal{U}}_{\mathcal{A}}(\mathfrak{L})\,\cong\,\odot_{\mathcal{A}}(\mathfrak{L})\,.$
(99)

It is important to emphasise that the symmetric algebra
$\odot_{\mathcal{A}}(\mathfrak{L})$ over ${\mathcal{A}}$ is much smaller than
the usual symmetric algebra $\odot_{\mathbb{K}}(\mathfrak{L})$ over
${\mathbb{K}}$, because the former takes into account the full
${\mathcal{A}}$-linearity of the corresponding tensor product (while the
latter only takes into account its ${\mathbb{K}}$-linearity).

Example (Smooth differential operators) : The almost-commutatative algebra
${\mathcal{D}}(M)$ of differential operators on $M$ is the universal
enveloping algebra of the Lie-Rinehart algebra ${\mathfrak{X}}(M)$ of vector
fields on $M$. From the generalised Poincaré-Birkhoff-Witt theorem, one
recovers that the classical limit of the almost-commutative algebra
${\mathcal{D}}(M)$ of differential operators on $M$ is isomorphic to the
Schouten algebra ${\mathcal{S}}(M)\cong\Gamma(\odot TM)$ of principal symbols
on $M$, i.e.

$\text{gr}\,{\mathcal{D}}(M)\cong\odot_{{}_{C^{\infty}(M)}}\big{(}{\mathfrak{X}}(M)\big{)}\cong\mathcal{S}(M)\,.$
(100)

Example (Polynomial vs formal differential operators) : The Grothendieck
algebra of differential operators acting on the commutative algebra
$\mathcal{A}=\odot(V^{*})$ of polynomials on the affine space $A$
(respectively, the commutative algebra
$\mathcal{A}=\overline{\odot}(V^{*}):=\odot(V)^{*}$ of formal power series at
the origin of the vector space $V$) is the universal enveloping algebra of the
Lie-Rinehart algebra $\mathfrak{der}({\mathcal{A}})$ of polynomial
(respectively, formal) vector fields $\hat{X}=X^{a}(y)\,\partial_{a}$:

${\mathcal{U}}_{\mathcal{A}}\big{(}\mathfrak{der}({\mathcal{A}})\,\big{)}={\mathcal{D}}({\mathcal{A}})\qquad\text{for}\quad{\mathcal{A}}\,\,=\,\,\text{either}\,\,\odot(V^{*})\,\,\text{or}\,\,\overline{\odot}(V^{*})\,.$
(101)

The Poincaré-Birkhoff-Witt theorem leads to the isomorphism

$\text{gr}\,{\mathcal{D}}({\mathcal{A}})\,\cong\,{\mathcal{A}}\,\otimes\,\odot(V)\qquad\text{for}\quad{\mathcal{A}}\,\,=\,\,\text{either}\,\,\odot(V^{*})\,\,\text{or}\,\,\overline{\odot}(V^{*})\,,$
(102)

since $\mathfrak{der}({\mathcal{A}})\cong{\mathcal{A}}\otimes V$ as vector
spaces. The isomorphism (102) is the property that the commutative algebra
$\text{gr}\,{\mathcal{D}}({\mathcal{A}})$ of symbols

$X=\sum\limits_{r=0}^{k}X^{a_{1}\cdots a_{r}}(y)\,p_{a_{1}}\cdots p_{a_{r}}$
(103)

of differential operators

$\hat{X}=\sum\limits_{r=0}^{k}X^{a_{1}\cdots
a_{r}}(y)\,\partial_{a_{1}}\cdots\partial_{a_{r}}$ (104)

is a free $\mathcal{A}$-module with all monomials $p_{a_{1}}\cdots p_{a_{r}}$
as holonomic basis, i.e. the corresponding components $X^{a_{1}\cdots
a_{r}}(y)$ are polynomials (vs formal power series). In particular, the
symbols of polynomial differential operators are polynomials functions on the
cotangent bundle $T^{*}V=V\oplus V^{*}$, in agreement with the isomorphism

$\odot(V\oplus V^{*})\,\cong\,\odot(V)\,\otimes\,\odot(V^{*})$ (105)

####  4.1.5 Universality property

An equivalent (but more abstract) definition of the universal enveloping
algebra ${\mathcal{U}}_{\mathcal{A}}(\mathfrak{L})$ of the Lie-Rinehart
algebra $\mathfrak{L}$ over the commutative algebra $\mathcal{A}$ is by the
following universality property (see e.g. [11, 21]).343434Apart from the well-
known constructions of Rinehart [20] and Huebschmann [21], the universal
enveloping algebra of a Lie-Rinehart algebra admits other (equivalent)
realisations (see e.g. [40, 41] and refs therein).

Let $\mathfrak{L}$ be a Lie-Rinehart algebra over $\mathcal{A}$ with $\cdot$
denoting the left action of $\mathcal{A}$ on $\mathfrak{L}$. Let $\cal U$ be
an associative algebra with product denoted by $\circ$ and with commutator
algebra denoted by $\mathfrak{U}$. If

$a\,:\,{\mathcal{A}}\to{\mathcal{U}}\,:\,f\mapsto a(f)$ (106)

is a morphism of associative algebras and

$\ell\,:\,\mathfrak{L}\to\mathfrak{U}\,:\,\hat{X}\mapsto\ell({X})$ (107)

is a morphism of Lie algebras, such that they satisfy the compatibility
conditions ($\forall f\in\mathcal{A}$, $\forall{X}\in\mathfrak{L}$) :

$\ell(f\cdot{X})\,=\,a(f)\circ\ell({X})\,,$ (108)

and

$\ell({X})\,\circ\,a(f)\,-\,a(f)\,\circ\,\ell(X)\,=\,a\big{(}\,\hat{X}[f]\,\big{)}\,,$
(109)

then there exists a unique extension

$u\,:\,{\mathcal{U}}_{\mathcal{A}}(\mathfrak{L})\to{\mathcal{U}}\,,\qquad
u|_{\mathcal{A}}=a\,,\quad u|_{\mathfrak{L}}=\ell\,,$ (110)

which is a morphism of associative algebras.

An important corollary of this universality property is that there is a one-
to-one correspondence between modules of a Lie-Rinehart algebras
$\mathfrak{L}$ over $\mathcal{A}$ and modules of its universal enveloping
algebra ${\mathcal{U}}_{\mathcal{A}}(\mathfrak{L})$. In fact, if a vector
space V carries a representation of a Lie-Rinehart algebra $\mathfrak{L}$ over
$\mathcal{A}$ then it is both a left $\mathcal{A}$-module V (i.e. there is a
morphism $a:{\mathcal{A}}\to\text{End}_{\mathcal{A}}(\textsc{V})$ of
associative algebras) and a left $\mathfrak{L}$-module (i.e. there is a
morphism $\nabla:\mathfrak{L}\to\mathfrak{cder}_{\mathcal{A}}(\textsc{V})$ of
Lie-Rinehart algebras). Setting $\ell=\nabla$ and
${\mathcal{U}}={\mathcal{D}}_{\mathcal{A}}(\textsc{V})$, one finds that there
exists a unique extension
${\mathcal{U}}(\nabla):{\mathcal{U}}_{\mathcal{A}}(\mathfrak{L})\to{\mathcal{D}}_{\mathcal{A}}(\textsc{V})$
as a morphism of associative algebras. This makes V a left
${\mathcal{U}}_{\mathcal{A}}(\mathfrak{L})$-module.

Note that the extension of a faithful representation
$\nabla:\mathfrak{L}\hookrightarrow\mathfrak{cder}_{\mathcal{A}}(\textsc{V})$
of a Lie-Rinehart algebra to a representation
${\mathcal{U}}(\nabla):{\mathcal{U}}_{\mathcal{A}}(\mathfrak{L})\to{\mathcal{D}}_{\mathcal{A}}(\textsc{V})$
of the universal enveloping algebra may not be faithful. However, the
representation of the almost-commutative algebra
${\mathcal{U}}_{\mathcal{A}}(\mathfrak{L})\,/\,\text{Ker}\,{\mathcal{U}}(\nabla)$
defined as the quotient of the universal enveloping algebra
${\mathcal{U}}_{\mathcal{A}}(\mathfrak{L})$ by the kernel of
${\mathcal{U}}(\nabla)$ will be faithful by construction. This quotient will
be called the enveloping algebra of the Lie-Rinehart algebra $\mathfrak{L}$
associated to the faithful representation $\nabla$. In this sense, any
faithful representation of a Lie-Rinehart algebra (such as a flat connection)
can be lifted to a faithful representation of the corresponding enveloping
algebra. And, conversely, any (faithful) representation of an almost-
commutative algebra restricts to a (faithful) representation of a Lie-Rinehart
algebra.

####  4.1.6 Almost-commutative algebras and associative algebroids

Consider an almost-commutative algebra $\cal U$ and let us denote by
$\mathfrak{U}$ its commutator algebra. Recall that the component of degree
zero is a commutative algebra ${\mathcal{U}}_{0}$ and that the component of
degree one is a Lie-Rinehart algebra $\mathfrak{U}_{1}$ over
${\mathcal{U}}_{0}$. Furthermore, the quotient
$\mathfrak{U}_{1}/\mathfrak{U}_{0}\subset\text{gr}\,\mathfrak{U}$ is a Lie-
Rinehart subalgebra of the grade one component of the classical limit
$\text{gr}\,{\mathcal{U}}$. The Lie-Rinehart algebra $\mathfrak{U}_{1}$ over
${\mathcal{U}}_{0}$ is the extension (41) of the Lie-Rinehart subalgebra
$\mathfrak{U}_{1}/\mathfrak{U}_{0}\subset\text{gr}\,\mathfrak{U}$ by the
Abelian Lie-Rinehart subalgebra $\mathfrak{U}_{0}\subset\mathfrak{U}$.

Another corollary of the universality property is that, for any almost-
commutative algebra $\cal U$, there exists a morphism of associative algebras
from the universal enveloping algebra
$\mathcal{U}_{{}_{{\mathcal{U}}_{0}}}(\mathfrak{U}_{1}/\mathfrak{U}_{0})$ of
the Lie-Rinehart algebra $\mathfrak{U}_{1}/\mathfrak{U}_{0}$ over
${\mathcal{U}}_{0}$ to the almost-commutative algebra $\cal U$ (see e.g. [42,
Section 2.1] for more details). In this sense, “almost-commutative” algebras
could be called “associative-Rinehart” algebras, since almost-commutative
algebras are to Lie-Rinehart algebras what associative algebras are to Lie
algebras.353535This statement can even be made precise in functorial language.
The “commutator” functor associating a Lie-Rinehart algebra to any almost-
commutative algebra is right-adjoint to the “universal enveloping” functor
associating an almost-commutative algebra to any Lie-Rinehart algebra [42,
Proposition 2.9]. Accordingly, an almost-commutative algebra $\mathcal{U}$
whose degree zero component $\mathcal{U}_{0}$ is the structure algebra
$C^{\infty}(M)$ of a manifold $M$ and such that each component
$\mathcal{U}_{k}$ is locally-free of finite-rank, could be called an
associative algebroid over $M$. In particular, an associative algebroid is the
space of sections of a filtered vector bundle over $M$ with two important
vector sub-bundles: the unit bundle $M\times\mathbb{R}$ at degree zero, and a
Lie algebroid $\mathbb{A}$ at degree one. As argued in the introduction, it is
tempting to speculate that associative algebroids should be the proper arena
for discussing geometrically higher-spin gauge symmetries and connections.

An almost-commutative algebra $\cal U$ generated by its component
${\mathcal{U}}_{1}$ of degree one will be called an enveloping algebra of the
Lie-Rinehart algebra $\mathfrak{U}_{1}/\mathfrak{U}_{0}$ over the commutative
algebra ${\mathcal{U}}_{0}$. Another corollary of the universality property is
that any enveloping algebra $\cal U$ of a Lie-Rinehart algebra $\mathfrak{L}$
over the commutative algebra ${\mathcal{A}}$ is isomorphic to a quotient of
the universal enveloping algebra ${\mathcal{U}}_{\mathcal{A}}(\mathfrak{L})$
of the Lie-Rinehart algebra $\mathfrak{L}$ over $\mathcal{A}$. In fact, the
morphism ${\mathcal{U}}_{\mathcal{A}}(\mathfrak{L})\to\cal U$ of associative
algebras is surjective since $\cal U$ is generated by its component of order
one ${\mathcal{U}}_{1}$, therefore one has a short exact sequence of
associative algebra morphisms

$0\to{\mathcal{I}}\stackrel{{\scriptstyle
i}}{{\hookrightarrow}}{\mathcal{U}}_{\mathcal{A}}(\mathfrak{L})\stackrel{{\scriptstyle\pi}}{{\twoheadrightarrow}}{\mathcal{U}}\to
0$ (111)

where the associative ideal ${\mathcal{I}}$ of
${\mathcal{U}}_{\mathcal{A}}(\mathfrak{L})$ is the kernel of $\pi$.

###  4.2 Weyl algebra as enveloping algebra of Heisenberg algebra

Let $V$ be a finite-dimensional vector space. Its cotangent bundle
$T^{*}V\cong V\oplus V^{*}$ is endowed with a canonical symplectic two-form
$\Omega$ defined by $\Omega(v\oplus\alpha,w\oplus\beta)=\alpha(w)-\beta(v)$
for all $v,w\in V$ and $\alpha,\beta\in V^{*}$. Conversely, any finite-
dimensional symplectic vector space $W$ admits a choice of polarisation
$W=V\oplus V^{*}$.

####  4.2.1 Heisenberg group and algebra

Obviously, the vector space $V\oplus V^{*}$ can be seen as an additive Abelian
Lie group. The Heisenberg group $H(V)$ is a nontrivial central extension

$0\to{\mathbb{K}}\,\hookrightarrow\,H(V)\,\twoheadrightarrow\,V\oplus V^{*}\to
0\,.$ (112)

of the Abelian Lie group $V\oplus V^{*}$ by the Abelian Lie group
${\mathbb{K}}$. It is the vector space $V\oplus V^{*}\oplus{\mathbb{K}}$
endowed with the product

$\displaystyle(v,\alpha,t)\cdot(w,\beta,u)=\Big{(}\,v+w\,,\alpha+\beta\,,\,t+u+\alpha(w)-\beta(v)\,\Big{)}\,,$
(113) $\displaystyle\qquad\forall\,v,w\in V\,,\quad\forall\,\alpha,\beta\in
V^{*},\quad\forall\,t,u\in{\mathbb{K}}\,.$

The Heisenberg group $H(V)$ is a non-Abelian Lie group whose center is
${\mathbb{K}}$.

The Heisenberg algebra $\mathfrak{h}(V)$ is the Lie algebra of the Heisenberg
group $H(V)$. It is the vector space $V\oplus V^{*}\oplus{\mathbb{K}}$ endowed
with the Lie bracket

$\displaystyle\big{[}\,(v,\alpha,t)\,,\,(w,\beta,u)\,\big{]}\,=\,\big{(}\,0,0,\,\alpha(w)-\beta(v)\,\big{)}\,,$
(114) $\displaystyle\qquad\forall\,v,w\in V\,,\quad\forall\,\alpha,\beta\in
V^{*},\quad\forall\,t,u\in{\mathbb{K}}\,.$

Given a basis $\\{e_{a}\\}$ of the vector space $V$, the latter becomes
isomorphic to ${\mathbb{K}}^{n}$ in which case the Heisenberg group
(respectively, algebra) is often denoted by physicists $H_{2n}$ (respectively,
$\mathfrak{h}_{2n}$). Let c denote the central element of $\mathfrak{h}(V)$
corresponding to the unit element $1\in\mathbb{K}$. In the basis
$\\{e_{a},e^{*b},\texttt{c}\\}$ of $\mathfrak{h}_{2n}$, the only nontrivial
Lie brackets are given by $[e^{*b},e_{a}]=\delta_{a}^{b}\texttt{c}$.

####  4.2.2 Unitary irreducible representations

The theorem of Stone and von Neumann asserts (respectively, implies) that all
unitary irreducible representations of the real Heisenberg group $H(V)$
(respectively, of the real Heisenberg algebra $\mathfrak{h}(V)$ ) which are
not trivial on the center ${\mathbb{R}}$ are unitarily equivalent (up to a
scale, i.e. up to a rescaling of the eigenvalue of the central element).

By Schur’s lemma, all unitary irreducible modules of the Heisenberg algebra
$\mathfrak{h}(V)$ are eigenspaces of the central element c. All unitary
irreducible modules of the Heisenberg algebra $\mathfrak{h}(V)$ for non-
vanishing real eigenvalue look exactly the same, so one may take $1$ as
eigenvalue. This faithful representation of the Lie algebra $\mathfrak{h}(V)$
extends to a representation of its universal enveloping algebra
${\mathcal{U}}\big{(}\,\mathfrak{h}(V)\,\big{)}$ which is not faithful. The
associative ideal
$(\texttt{c}-1)\,{\mathcal{U}}\big{(}\,\mathfrak{h}(V)\,\big{)}$ is the
annihilator of the corresponding unitary irreducible $\mathfrak{h}(V)$-module.
The Weyl algebra ${\mathcal{D}}(A)$ is isomorphic to the quotient

${\mathcal{U}}\big{(}\mathfrak{h}(V)\big{)}\,/\,(\texttt{c}-1){\mathcal{U}}\big{(}\mathfrak{h}(V)\big{)}\cong{\mathcal{D}}(A)$
(115)

of the universal enveloping algebra
${\mathcal{U}}\big{(}\mathfrak{h}(V)\big{)}$ of the Heisenberg algebra
$\mathfrak{h}(V)$ by the primitive ideal
$(\texttt{c}-1){\mathcal{U}}\big{(}\mathfrak{h}(V)\big{)}$. In other words,
the Weyl algebra is isomorphic to the enveloping algebra of the Heisenberg
algebra associated to one of its representation on a unitary irreducible
module, non-trivial on the centre.

The classical limit $\text{gr}\,{\mathcal{D}}(A)$ of the Weyl algebra (seen as
an almost-commutative algebra) is isomorphic to the Schouten algebra (105) of
polynomial functions on the cotangent space $T^{*}V\cong V\oplus V^{*}$. This
algebra (105) of polynomial symbols is isomorphic to the quotient of the
universal enveloping algebra ${\mathcal{U}}\big{(}\mathfrak{h}(V)\big{)}$ of
the Heisenberg algebra by the associative ideal
$\texttt{c}\,{\mathcal{U}}\big{(}\mathfrak{h}(V)\big{)}$,

$\odot(V\oplus
V^{*})\cong{\mathcal{U}}\big{(}\mathfrak{h}(V)\big{)}\,/\,\texttt{c}\,{\mathcal{U}}\big{(}\mathfrak{h}(V)\big{)}\,.$
(116)

In fact, this quotient amounts to take the classical limit where position and
momenta commute with each other.

The many faces of Weyl algebras

Consider an affine space $A$ modeled on a vector space $V$. The Weyl algebra
${\mathcal{D}}(A)$ can be defined in various equivalent ways as:

$\bullet$ the Grothendieck algebra ${\mathcal{D}}(\odot V^{*})$ of the
commutative algebra of polynomial functions on $A$,

$\bullet$ the universal enveloping algebra ${\mathcal{U}}_{\odot
V^{*}}\big{(}\mathfrak{der}(\odot V^{*})\,\big{)}$ of the Lie-Rinehart algebra
of polynomial vector fields on $A$,

$\bullet$ the enveloping algebra
$\frac{{\mathcal{U}}\big{(}\mathfrak{h}(V)\big{)}}{(\texttt{c}-1){\mathcal{U}}\big{(}\mathfrak{h}(V)\big{)}}$
of the Heisenberg algebra associated to one of its representation on a unitary
irreducible module non-trivial on the centre,

###  4.3 Universal enveloping algebras of semidirect sums

For the sake of simplicity of the discussion, let us focus first on the
example of Lie algebras over a field $\mathbb{K}$.

The universal enveloping algebra ${\mathcal{U}}(\mathfrak{g})$ of a semidirect
sum

$\mathfrak{g}=\mathfrak{i}\niplus\mathfrak{h}$ (117)

of the Lie ideal $\mathfrak{i}\subset\mathfrak{g}$ and the Lie subalgebra
$\mathfrak{h}\subset\mathfrak{g}$ is isomorphic to the smash product of the
respective universal enveloping algebras ${\mathcal{U}}(\mathfrak{i})$ and
${\mathcal{U}}(\mathfrak{h})$ [43, Subsection 1.7.11],

${\mathcal{U}}(\mathfrak{i}\niplus\mathfrak{h})\,\cong\,{\mathcal{U}}(\mathfrak{i})\rtimes{\mathcal{U}}(\mathfrak{h})\,,$
(118)

where the action of $\mathfrak{h}$ on ${\mathcal{U}}(\mathfrak{i})$ arises via
the Leibnitz rule from the representation of $\mathfrak{h}$ on $\mathfrak{i}$.
This result admits a generalisation [44, 45] to the case of a linearly-split
extension363636In other words, the arrows in the splitting
$0\leftarrow\mathfrak{i}\twoheadleftarrow\mathfrak{g}\hookleftarrow\mathfrak{h}\leftarrow
0$ are morphisms of vector spaces only. $\mathfrak{g}$ of the Lie algebra
$\mathfrak{h}$ by the ideal $\mathfrak{i}$, in which case the symbol $\niplus$
stands for the “curved” semidirect sum [9, Definition 1.7] while the symbol
$\rtimes$ in (118) stands for the “cross” product [44, Definition 4.1]. The
abstract definitions of the smash and cross products $\rtimes$ will not be
reviewed here (because it involves some concepts in bialgebra theory that are
beyond the scope of the present text).373737For those interested, see e.g.
[46] for a thorough introduction to bialgebras, Hopf algebras, etc. Anyway, in
order to understand the meaning of (118), it is enough to appreciate that the
generalised Poincaré-Birkhoff-Witt theorem implies that

$\text{gr}\,{\mathcal{U}}(\mathfrak{i}\niplus\mathfrak{h})\,\,\cong\,\,\odot(\mathfrak{i}\oplus\mathfrak{h})\,\,\cong\,\,\odot(\mathfrak{i})\,\otimes\,\odot(\mathfrak{h})\,,$
(119)

where the associated graded algebra is with respect to both filtrations, i.e.
of ${\mathcal{U}}(\mathfrak{i})$ and of ${\mathcal{U}}(\mathfrak{h})$. More
concretely, there is a natural choice of ordering for
${\mathcal{U}}(\mathfrak{i}\niplus\mathfrak{h})$: the “normal” ordering where
the dependence in $\mathfrak{i}$ is factored on the left while the dependence
on $\mathfrak{h}$ is factored on the right. The product of two normal-ordered
elements of ${\mathcal{U}}(\mathfrak{i}\niplus\mathfrak{h})$ is not any more
normal-ordered in general. The normal ordering requires to recursively compute
commutators of the form
$[{\mathcal{U}}(\mathfrak{h}),{\mathcal{U}}(\mathfrak{i})]$. In some sense,
the abstract notion of smash product is simply a way to formalise the
systematic calculus (use of Leibnitz rule, etc) involved with the normal
ordering of such expressions.

Example (Direct sum) : The universal enveloping algebra
${\mathcal{U}}(\mathfrak{g})$ of a direct sum

$\mathfrak{g}=\mathfrak{h_{1}}\oplus\mathfrak{h_{2}}$ (120)

of two Lie algebra $\mathfrak{h}_{1}$ and $\mathfrak{h}_{2}$ is isomorphic to
the tensor product of the respective universal enveloping algebras
${\mathcal{U}}(\mathfrak{h}_{1})$ and ${\mathcal{U}}(\mathfrak{h}_{2})$,

${\mathcal{U}}(\mathfrak{h}_{1}\oplus\mathfrak{h}_{2})\,\cong\,{\mathcal{U}}(\mathfrak{h}_{1})\otimes{\mathcal{U}}(\mathfrak{h}_{2})\,.$
(121)

This obvious result corresponds to the isomorphism (118) for the case of a
trivial representation.

Interestingly, the factorisation (118) admits a generalisation for Lie-
Rinehart algebras [9]: for any given curved (respectively, flat) connection,
that is, a linear (respectively, Lie-Rinehart) splitting of a Lie-Rinehart
algebra extension (i.e. a generalised connection), a crossed (resp. smash)
product decomposition of the associated universal enveloping algebra is
provided, and vice versa. As a geometric example for Lie algebroids, the
associative algebra generated by the invariant vector fields on the total
space of a principal bundle is described as a crossed product of the algebra
generated by the vertical ones and the algebra of differential operators on
the base. Such a factorisation can be thought as an alternative
characterisation of an infinitesimal connection on a principal bundle. Its
interest for higher-spin geometry is that such an algebraic characterisation
might admit natural generalisations adapted to the characterisation of higher-
spin connections, e.g. by relaxing in [9, Theorem 3.10] the condition that the
coproduct (hence the filtration) of the universal enveloping algebra is
preserved.

## Acknowledgments

I would like to thank Damien Calaque for pointing to me (a long time ago) the
relevance of Lie-Rinehart algebras for defining properly the universal
enveloping algebra of the vector field Lie algebra. I am also very grateful to
Niels Kowalzig and Paolo Saracco for our collaboration on the universal
enveloping algebra of Lie-Rinehart algebra, from which I learned so much.
Finally, I acknowledge Thomas Basile for his patient reading and useful
comments on some early version of these notes.

The author acknowledge support of the Institut Henri Poincaré (UAR 839 CNRS-
Sorbonne Université) and LabEx CARMIN (ANR-10-LABX-59-01) during his
participation to the trimester “Higher Structures in Geometry and Mathematical
Physics” held at the Institut Henri Poincaré (April-July 2023).

## References

  * [1] K. Mackenzie, Lie groupoids and Lie algebroids in differential geometry (Cambridge University Press, 1987).
  * [2] K. Mackenzie, General theory of Lie groupoids and Lie algebroids (Cambridge University Press, 2005).
  * [3] I. Moerdijk and J. Mrcun, Introduction to Foliations and Lie Groupoids (Cambridge University Press, 2003)
  * [4] M. Crampin and D. Saunders, Cartan geometries and their symmetries: a Lie algebroid approach (Atlantis Press, 2016).
  * [5] M. Crainic, R. L. Fernandes, “Lectures on Integrability of Lie Brackets,” Geometry & Topology Monographs 17 (2011) 1 [arXiv:math/0611259].
  * [6] M. A. Vasiliev, “Higher spin gauge theories in four-dimensions, three-dimensions, and two-dimensions,” Int. J. Mod. Phys. D 5 (1996) 763 [hep-th/9611024]; “Higher spin gauge theories in various dimensions,” Fortsch. Phys. 52 (2004) 702 [hep-th/0401177]; “Higher spin gauge theories in any dimension,” Comptes Rendus Physique 5 (2004) 1101 [hep-th/0409260];
X. Bekaert, S. Cnockaert, C. Iazeolla and M. A. Vasiliev, “Nonlinear higher
spin theories in various dimensions,” hep-th/0503128;
V. E. Didenko and E. D. Skvortsov, “Elements of Vasiliev theory,”
arXiv:1401.2975 [hep-th];
M. A. Vasiliev, “Higher-spin theory and space-time metamorphoses,” Lect. Notes
Phys. 892 (2015) 227 [arXiv:1404.1948 [hep-th]];
D. Ponomarev, “Basic introduction to higher-spin theories,” Int. J. Theor.
Phys. 62 (2023) 146 [arXiv:2206.15385 [hep-th]].

  * [7] D. Sorokin, “Introduction to the classical theory of higher spins,” AIP Conf. Proc. 767 (2005) 172 [hep-th/0405069];
X. Bekaert, N. Boulanger and P. Sundell, “How higher-spin gravity surpasses
the spin two barrier: no-go theorems versus yes-go examples,” Rev. Mod. Phys.
84 (2012) 987 [arXiv:1007.0435 [hep-th]];
R. Rahman, “Higher Spin Theory - Part I,” PoS ModaveVIII (2012) 004
[arXiv:1307.3199 [hep-th]]. R. Rahman and M. Taronna, “From Higher Spins to
Strings: A Primer,” arXiv:1512.07932 [hep-th];
P. Kessel, “The Very Basics of Higher-Spin Theory,” PoS Modave2016 (2017) 001
[arXiv:1702.03694 [hep-th]];
A. Bengtsson, Higher Spin Field Theory (Concepts, Methods and History) Volume
1: Free Theory (De Gruyter, 2020);
X. Bekaert, N. Boulanger, A. Campoleoni, M. Chiodaroli, D. Francia, M.
Grigoriev, E. Sezgin and E. Skvortsov, “Snowmass White Paper: Higher Spin
Gravity and Higher Spin Symmetry,” in J. N. Butler, R. S. Chivukula, M. E.
Peskin (eds) Proceedings: 2021 US Community Study on the Future of Particle
Physics: Snowmass 2021 (SLAC, 2023) [arXiv:2205.01567 [hep-th]].

  * [8] R. Argurio, G. Barnich, G. Bonelli and M. Grigoriev (eds), Higher Spin Gauge Theories (International Solvay Institutes, 2004);
L. Brink, M. Henneaux and M. A. Vasiliev (eds), Higher Spin Gauge Theories
(World Scientific, 2017).

  * [9] X. Bekaert, N. Kowalzig and P. Saracco, “Universal enveloping algebras of Lie-Rinehart algebras: crossed products, connections, and curvature,” [arXiv:2208.00266 [math.RA]].
  * [10] M. Crampin, “Cartan connections and Lie algebroids,” SIGMA 5 (2009) 061 [arXiv:0906.2554 [math.DG]];
J. Attard, J. François, S. Lazzarini and T. Masson, “Cartan Connections and
Atiyah Lie Algebroids,” J. Geom. Phys. 148 (2020) 103541 [arXiv:1904.04915
[math-ph]].

  * [11] I. Moerdijk and J. Mrcun, “On the universal enveloping algebra of a Lie algebroid,” Proc. Amer. Math. Soc. 138 (2010) 3135 [arXiv:0801.3929 [math.QA]].
  * [12] X. Bekaert, “Geometric tool kit for higher-spin gravity (Part I): An introduction to the geometry of differential operators,” Int. J. Mod. Phys. A 38 (2023) 2330003 [arXiv:2301.08069 [hep-th]].
  * [13] J. Nestruev, Smooth Manifolds and Observables (Springer, 2020) [2nd edition, augmented].
  * [14] J. J. Rotman, Advanced Modern Algebra: Part 1, Graduate Studies in Mathematics 165 (American Mathematical Society, 2020).
  * [15] Michel Raynaud and Laurent Gruson, “Critères de platitude et de projectivité, Techniques de ‘platification’ d’un module,” Invent. Math. bf 13 (1971) 1.
  * [16] R. G. Swan, “Vector Bundles and Projective Modules,” Trans. Am. Math. Soc. 105 (1962) 264.
  * [17] J.-P. Serre, “Faisceaux Algébriques Cohérents,” Ann. Math. bf 61 (1955) 197.
  * [18] A. S. Morye, “Note on the Serre‐Swan theorem,” Math. Nachr. 286 (2013) 272 [arXiv:0905.0319 [math.AG]].
  * [19] J. Herz, “Pseudo-algèbres de Lie,” C. R. Acad. Sci. Paris 236 (1953) 1935.
  * [20] G. Rinehart, “Differential forms for general commutative algebras”, Trans. Amer. Math. Soc. 108 (1963) 195.
  * [21] J. Huebschmann, “Poisson cohomology and quantization,” J. für die Reine und Angew. Math. 408 (1990) 57 [arXiv:1303.3903 [math.DG]].
  * [22] J. Huebschmann, “On the history of Lie brackets, crossed modules, and Lie-Rinehart algebras,” J. Geom. Mech. 13 (2021) 385 [arXiv:2208.02539 [math.HO]].
  * [23] C. Laurent-Gengoux, M. Stiénon and X. Ping, “Poincaré–Birkhoff–Witt isomorphisms and Kapranov dg-manifolds,” Adv. Math. 387 (2021) 107792 [arXiv:1408.2903 [math.DG]].
  * [24] J. Pradines, “Théorie de Lie pour les groupoïdes différentiables. Calcul différentiel dans la catégorie des groupoïdes infinitésimaux,” C. R. Acad. Sci. Paris (Série A) 264 (1967) 245.
  * [25] A. Douady and M. Lazard, “Espaces fibrés en algèbres de Lie et en groupes,” Invent. Math. 1 (1966) 133.
  * [26] Y. Kosmann-Schwarzbach and K. C. H. Mackenzie, “Differential operators and actions of Lie algebroids,” in T. Voronov (Ed.), Quantization, Poisson Brackets and Beyond, Contemporary Mathematics 315 (American Mathematical Society, 2002) 213 [arXiv:math/0209337 [math.DG]].
  * [27] P. Dazord, “Groupoïde d’holonomie et géométrie globale,” C.R. Acad. Sci. Paris 324 (1997) 77.
  * [28] M. Crainic and R. L. Fernandes, “Lectures on integrability of Lie brackets,” in T. Ratiu, A. Weinstein and N. T. Zung (Eds.), Lectures on Poisson Geometry, Geometry & Topology Monographs 17 (Mathematical Sciences Publishers, 2011) 1.
  * [29] M. F. Atiyah, “Complex Analytic Connections in Fibre Bundles,” Trans. Amer. Math. Soc. 85 (1957) 181.
  * [30] R. Almeida and P. Molino, “Suites d’Atiyah et feuilletages transversalement complets,” C. R. Acad. Sc. Paris (Série I) 300 (1985) 13.
  * [31] A. Grothendieck, “Éléments de géométrie algébrique : IV. Étude locale des schémas et des morphismes de schémas, Quatrième partie,” Publ. Math. IHÉS 32 (1967) 5.
  * [32] J. Huebschmann, “Extensions of Lie-Rinehart algebras and the Chern-Weil construction,” in J. McCleary (Ed.), Higher Homotopy Structures in Topology and Mathematical Physics (Festschrift in honour of Jim Stasheff’s 60’th anniversary) Contemporary Mathematics 227 (American Mathematical Society, 1999) 145 [arXiv:dg-ga/9706002].
  * [33] J. A. Schouten, Der Ricci-Kalkül – Eine Einführung in die neueren Methoden und Probleme der mehrdimensionalen Differentialgeometrie (Springer, 1924).
  * [34] N. Jacobson, “On pseudo-linear transformations,” Proc. Nat. Acad. Sci. 21 (1935) 667.
  * [35] R. S. Palais, Foundations of global non-linear analysis (W. A. Benjamin, 1968).
  * [36] Y. Kosmann, “On Lie transformation groups and the covariance of differential operators” in M. Cahen and M. Flato (Eds.), Differential geometry and relativity (in honour of André Lichnerowicz on his 60th birthday) Mathematical Physic and Applied Mathematics 3 (Springer, 1976) 75.
  * [37] N. Bourbaki, Éléments de mathématique. Algèbre: Chapitre 3 (Masson, 1970) [ Modern edition: N. Bourbaki, Éléments de mathématique. Algèbre: Chapitres 1-3 (Springer, 2007); English translation: Algebra I: Chapters 1-3 (Springer, 2008) ].
  * [38] R. L. Fernandes, “Lie Algebroids, Holonomy and Characteristic Classes,” Adv. Math. 170 (2002) 119 [arXiv:math/0007132 [math.DG]].
  * [39] F. Kamber and P. Tondeur, Foliated bundles and characteristic classes, Lecture Notes in Mathematics 493 (Springer-Verlag, 1975).
  * [40] L. El Kaoutit and P. Saracco, “Topological tensor product of bimodules, complete Hopf algebroids and convolution algebras,” Commun. Contemp. Math. 21 (2019) 1.
  * [41] P. Saracco, “Universal enveloping algebras of Lie-Rinehart algebras as a left adjoint functor,” Mediterr. J. Math. 19 (2022) 92.
  * [42] X. G. Martínez, Lie-Rinehart Algebras (Master thesis of Universidade de Santiago de Compostela, June 2013).
  * [43] J. C. McConnell and J. C. Robson, Noncommutative Noetherian Rings, Graduate Studies in Mathematics 30 (American Mathematical Society, 1987).
  * [44] R. J. Blattner, M. Cohen, and S. Montgomery, _Crossed products and inner actions of Hopf algebras_ , Trans. Amer. Math. Soc. 298 (1986) 671.
  * [45] S. Montgomery, Hopf algebras and their actions on rings, CBMS Regional Conference Series in Mathematics 82 (American Mathematical Society, 1993).
  * [46] M. Hazewinkel, N. Gubareni and V. V. Kirichenko, Algebras, Rings and Modules: Lie Algebras and Hopf Algebras, Mathematical Surveys and Monographs 168 (American Mathematical Society, 2010).