# Active Learning with Contrastive Pre-training for Facial Expression
Recognition

Shuvendu Roy, Ali Etemad  Dept. ECE and Ingenuity Labs Research Institute
Queen’s University, Kingston, Canada
{shuvendu.roy<EMAIL_ADDRESS>
###### Abstract

Deep learning has played a significant role in the success of facial
expression recognition (FER), thanks to large models and vast amounts of
labelled data. However, obtaining labelled data requires a tremendous amount
of human effort, time, and financial resources. Even though some prior works
have focused on reducing the need for large amounts of labelled data using
different unsupervised methods, another promising approach called active
learning is barely explored in the context of FER. This approach involves
selecting and labelling the most representative samples from an unlabelled set
to make the best use of a limited ‘labelling budget’. In this paper, we
implement and study 8 recent active learning methods on three public FER
datasets, FER13, RAF-DB, and KDEF. Our findings show that existing active
learning methods do not perform well in the context of FER, likely suffering
from a phenomenon called ‘Cold Start’, which occurs when the initial set of
labelled samples is not well representative of the entire dataset. To address
this issue, we propose contrastive self-supervised pre-training, which first
learns the underlying representations based on the entire unlabelled dataset.
We then follow this with the active learning methods and observe that our
2-step approach shows up to 9.2% improvement over random sampling and up to
6.7% improvement over the best existing active learning baseline without the
pre-training. We will make the code for this study public upon publication at:
github.com/ShuvenduRoy/ActiveFER.

###### Index Terms:

Facial Expression Recognition, Semi-supervised Learning, Contrastive Learning

##  I Introduction

Facial expression recognition (FER) has seen growing interest in the deep
learning community [1, 2, 3, 4, 5] mainly due to its practical applications
ranging from smart devices and medical care assistants to smart vehicles.
However, the size of the labelled FER datasets is generally one of the
concerns prohibiting further progress in the area. Many of the recently
developed deep learning models, such as Transformer [6], inherently require
very large amounts of data. As a result, many recent works on FER have focused
on developing methods that learn better representations from small amounts of
labelled data [7, 8, 9].

Although labelled datasets are hard to collect and annotate, unlabelled images
are widely available on the Internet. Given a pre-defined labelling budget,
the annotation process involves resolving the choice of which samples of the
unlabelled dataset to annotate [10]. Recently, active learning has emerged as
a viable solution for identifying key samples from the unlabelled set [11].
The basic idea of active learning is to start the training process with a few
randomly selected samples and their corresponding labels. As the training
progresses, a selection criterion is used to find more samples from the
unlabelled set that are the best candidates for annotation. The model is then
trained with this new labelled set along with the previously available
labelled set. This cycle continues as long as the labelling budget is not
exhausted.

A variety of new active learning methods have been proposed in recent years in
different areas [11]. Although a few works have explored active learning
specifically for FER [12, 13], more recent approaches in active learning [14,
15] have not yet been studied in this context. Moreover, to our knowledge, a
comprehensive study to benchmark the performance of different active learning
methods for FER under the same training protocol has not been conducted.

Another well-known fact about active learning training with a small labelling
budget is the ‘cold start’ problem. The cold start problem occurs when the
initial labelled set is either too small or not a good representative for the
entire dataset. In such scenarios, the model fails to learn effective
representations from the initial labelled set, and thus informative samples
are not selected in later cycles of the active learning process. This may
result in a final accuracy that is even worse than not using active sampling
altogether.

Figure 1: Overview of the 2-step training protocol. First, we pre-train the
model with a contrastive self-supervised learning objective to learn the
underlying representation of the data (left). Then we train it further with
active learning (right).

In this work, we address the two problems mentioned above. First, we present a
comprehensive study of different active learning methods for FER. To this end,
we compare 8 active learning methods, namely: Entropy [16], Margin [17], Least
Confidence [17], BADGE [15], GLISTER [18], Coreset [14], BALD [19], and
Adversarial Deepfool [20]. We conduct our study on three FER datasets (FER13,
RAF-DB, and KDEF) and show that surprisingly, simpler methods like Least
Confidence and Margin obtain better results than recently proposed methods
like Coresets and GLISTER. On average, the Least Confidence method shows the
best performance across all three datasets. Additionally, we find that active
learning on FER does indeed suffer from the cold start problem. To address
this issue, we propose a simple yet effective solution: self-supervised pre-
training using the unlabelled data. We select the best-performing active
learning method, Least Confidence, and show that by adding a self-supervised
pre-training step, the cold start problem is reduced. Specifically, for self-
supervised pre-training, we explore BYOL, MOCO, Barlow Twins, SwAV, and SimCLR
and observe that while all of them are effective in reducing the negative
impact of the cold start issue, SimCLR is the most effective. The self-
supervised pre-training step helps the method select more representative
samples at the first cycle and effectively enables better learning at later
cycles. Overall, our proposed solution shows up to 9% improvement over random
sampling and up to 6% improvements over the scenario where the cold start
issue is not addressed. Further ablation studies confirm that the improvements
in performance are not simply due to a better encoder (pre-trained), but
rather because of the fact that better samples are selected from the
unlabelled set due to the added pre-training step.

Our contributions in this work are summarized as follows:

  * •

We study active learning in the context of FER by exploring eight different
active learning methods on three FER datasets.

  * •

We propose a new solution to reduce the cold start problem in active learning
for FER, and show substantial overall improvements in performance.

  * •

To contribute to the field of active learning in the context of affective
computing and to enable reproducibility, we release the code for this work at:
github.com/ShuvenduRoy/ActiveFER.

(a) SimCLR

(b) MoCO v2.

(c) BYOL

(d) SwAV

(e) Barlow Twins

Figure 2: Architecture of different self-supervised training frameworks
explored for the pre-training.

##  II Related Work

In this section, we discuss the related literature in two areas relevant to
this work: (a) active learning and (b) self-supervised learning.

###  II-A Active Learning

The objective of active learning is to utilize a selection criterion for
selecting the most representative samples from an unlabelled set for
annotation. Although active learning is not a new concept, the rise of deep
learning has resulted in a surge in active learning methods since deep
learning methods require large datasets to train, which are not always
available for many domains. Some selection methods in earlier forms of active
learning utilized the concept of uncertainty in the model’s prediction as an
indicator for selecting new samples for labelling. For example, in [16], the
entropy in the model’s prediction on an unlabelled sample was used as the
selection criterion. Two other variants of uncertainty-based sampling
techniques were proposed in [17]. The first criterion (Margin) uses the
difference between the top two predictions as the indicator for selection. A
confident prediction will have a large difference between the highest and
second-highest predictions over the number of classes. The second criterion
(Least Confidence) simply takes the maximum over the class probability as the
indicator. In this criterion, a less confident prediction will have lower
confidence for the predicted class.

More recent methods focus more on the learning progress of the model and
utilize more specific signals as the selection criteria. For instance, [15]
inspects the loss gradient and selects a set of samples with diverse loss
gradients. GLISTER [18] is another method focusing on diverse sampling over
the entire dataset using bi-level optimization. The concept of Coresets [14]
has also been utilized as a selection criterion. BALD [19] uses Bayesian deep
learning to maximize the information between the prediction and model
posterior as an indicator for sampling. The concept of adversarial attacks has
also been utilized to estimate the decision boundary of classes and select
samples that are close to the boundaries [20].

###  II-B Self-supervised Learning

Self-supervised learning (SSL) is one of the most popular unsupervised
representation learning methods that has shown remarkable progress in various
areas. SSL can learn important representations of the data without any
supervision. Most of the earlier SSL methods utilized the concept of pre-text
tasks, where an auxiliary task was defined on the unlabelled data. One example
of such a pre-text task is rotation prediction, where an input unlabelled
image is rotated at a certain degree and the model is tasked with predicting
the angle of rotation. A more recent form of SSL utilizes the concept of
contrastive learning. Contrastive learning in computer vision was popularized
by SimCLR, which utilizes a contrastive loss on two augmentations of an
unlabelled image to maximize their agreement in the embedding space. SimCLR
also utilizes the concept of projection heads, hard augmentations, and
carefully designed training protocols to perform effectively in many
scenarios. Many variants of SimCLR have been since proposed. For instance,
MoCo [21] utilizes a momentum encoder to encode one augmented image, while the
other image is encoded by an online encoder. Later BYOL [22] proposed a
slightly different objective function with respect to MoCO, which predicts the
embedding of one augmented image from the other. SwAV [23] also has a similar
idea with the distinction of predicting a prototype rather than the actual
embedding. Barlow Twins [24] proposed a different loss calculated on the
cross-correlation between the predicted embedding of the two images. This
method explicitly avoided the mode collapse problem of self-supervised
learning while achieving strong performance on downstream tasks.

##  III Method

###  III-A Preliminaries

Let $X_{U}={(x_{i})_{i=1}^{N}}$ be a set of unlabelled samples, where $N$ is
the total number of samples in the unlabelled set, and $n$ be the total
labelling budget, where $n\ll N$. An active learning method first randomly
samples a small subset of $s$ samples from $X_{U}$ and annotates them to form
$X_{L_{S}}={(x_{i},y_{i})_{i=1}^{s}}$. The model is trained with $X_{L_{S}}$
for certain epochs. These two steps are together called a cycle. Over the next
$(c-1)$ cycles, the active learning method samples $(n-s)/(c-1)$ samples per
cycle using a selection criterion, and trains the model with the current and
previously sampled data. In this study, we explore the following active
learning methods.

Entropy. In Entropy, the uncertainty in the prediction of the model is used as
the selection criterion [16]. Let $p(x)=softmax(M_{j}(x))$ be the prediction
of the model, where $M_{j}$ is the trained model at cycle $j$. The entropy
selection criterion ($H(x)$) is represented as:

$H(x)=-\sum_{i}{p(x)_{i}log(p(x)_{i})},$ (1)

where $i$ is the index over the vector dimension of the model’s predictions.
Here, the method chooses the sample with the highest entropy.

Margin. This method considers the difference between the highest two
predictions as the selection criterion ($F(x)$) [17], which is defined as:

$F(x)=p(x)_{m_{1}}-p(x)_{m_{2}}.$ (2)

Here, $m_{1}$ and $m_{2}$ are the largest and second-largest predictions, and
the active learning method selects the sample with the lowest margin.

Least Confidence. Least Confidence is another simple approach where the
prediction confidence is used as the selection criterion [17]. This criterion
($C(x)$) is defined as:

$C(x)=max_{i}p(x)_{i},$ (3)

Here, the active learning method selects the minimum $C(x)$ over all the
unlabeled samples.

BADGE. This approach is one of the more recent active learning selection
methods that take the model’s learning progress into consideration [15]. It
first computes the gradient of the last layer with some pre-defined loss
function. Then, a K-Means++ algorithm is utilized to find the desired number
of centers with diverse loss gradients.

GLISTER. In this recent method, the active learning solution aims to select
samples that are representative of the entire domain [18]. For this, GLISTER
utilizes a bi-level optimization problem where the inner optimization learns
model parameters, and the outer optimization selects a set of unlabeled
samples.

Coreset. In this approach, the aim of the method is to find the samples that
represent or capture the structure of the entire unlabelled dataset [14].
Here, a k-center algorithm is utilized to solve a pre-defined objective
function to find the coresets.

BALD. In this method, the concept of Bayesian deep learning is utilized for
active learning [19]. BALD uses the concept of information maximization
between the prediction and model posterior to select a pool of samples.

Adversarial DeepFool. This active learning method operates by selecting points
that are closer to the decision boundaries [20]. Since calculating the actual
decision boundary in the embedding space is difficult, this method proposes to
use the concept of adversarial attacks to estimate it. For each sample, the
number of adversarial perturbations required to flip the prediction is
considered as the indication of how close it is to the decision boundary.

###  III-B Proposed Solution for the Cold Start Problem

In this section, we present a simple solution to address the cold start
problem in active learning for improved performance in the context of FER. The
proposed solution involves a two-step training protocol. In the first step, we
pre-train the model with the entire unlabeled set $X_{U}$ in a self-supervised
setting to learn the underlying representation of the data. In the second
step, we follow conventional active learning training, where the learned
representation from the first step helps the model to learn a discriminative
representation at the first cycle from the small labelled subset $X_{LS}$,
without overfitting. This approach enables a better selection of
representative samples in later cycles of the active learning training
process, effectively reducing the cold start problem. We explore some recently
proposed self-supervised methods, including SimCLR [25], MoCo v2 [21], BYOL
[22], SwAV [23], and Barlow Twins [24], for the first step. For the second
step, we use the Least Confidence as the active learning component. Below, we
provide a brief overview of each of the self-supervised methods that we
explore to evaluate our proposed solution.

SimCLR [25] is a popular contrastive self-supervised technique responsible for
popularizing contrastive learning in the field of computer vision. The basic
idea behind this method is to learn from positive and negative samples, where
positive samples are variations or transformations of an input sample,
typically generated through augmentations. All other samples are considered
negative with respect to the input sample. By bringing together positive
samples and moving them away from negatives in the embedding space,
contrastive learning allows for effective learning of the underlying
representation of the data. The contrastive loss function for two positive
samples, denoted as $i$ and $j$, is defined by:

$\displaystyle\mathcal{L}_{i,j}=-log\frac{exp(cos(z_{i},z_{j})/\tau)}{\sum_{k=1}^{2N}\mathbf{1}_{[k\neq
i]}exp(cos(z_{i},z_{k})/\tau)},$ (4)

where, $z_{i}$ is the embedding of the encoder, $cos()$ is the cosine
similarity function, and $\tau$ is a temperature parameter. A visual
illustration of the SimCLR method is depicted in Figure 2(a).

MoCo [21] is another popular self-supervised learning technique. Similar to
SimCLR, the basic idea behind this technique is to learn representations that
can differentiate between positive and negative samples. The positive pairs
are again generated by data augmentation, while negative pairs are taken from
a queue of samples that were stored at previous iterations of training. MoCo
maintains two encoders, one ‘online’ and the other ‘momentum’. The online
encoder is updated after processing each minibatch, while the momentum encoder
is updated using a moving average of the online encoder parameters. This
ensures that the momentum encoder is always slightly behind the online
encoder, enabling it to capture information from a larger amount of samples.
The momentum encoder is updated with the following equation:

$\displaystyle\theta_{k}=m*\theta_{k}+(1-m)*\theta_{q},$ (5)

where $\theta_{q}$ and $\theta_{k}$ are the parameters of the online and
momentum encoder and $m$ is the momentum coefficient. MoCo v2 is a simple
extension of MoCo that utilizes the projection head and hard augmentation
concepts introduced in SimCLR. Figure 2(b) depicts the MoCo framework.

BYOL [22] also utilizes two encoders called online and target encoder. Like
MoCo, the target encoder is a moving average of the online encoder. The
objective of BYOL is to train an online encoder to predict the target
encoder’s representation of the same image under different augmentations. The
online encoder is updated with the following loss function:

$\displaystyle\mathcal{L}_{i,j}=2-2\frac{<P,Z_{j}>}{||P||_{2}\cdot||Z_{j}||_{2}},$
(6)

where $Z_{j}$ is the embedding generated by the target encoder, and $P$ is the
prediction generated from the online encoder’s representation. BYOL framework
is depicted in Figure 2(c)

SwAV [23] is another popular self-supervised learning technique, which is a
clustering-based approach. It first generates multiple views on a sample image
by applying augmentations and then predicts its cluster assignment.
Considering each view as a query and other views as keys, SwAV [23] utilizes
contrastive learning on its cluster assignment. This clustering-based approach
is able to learn high-quality representations even when the dataset is highly
diverse or has many classes. The SwAV method is visualized in Figure 2(d).

Barlow Twins [24] proposed a loss function that explicitly avoids the collapse
in self-supervised representation learning. It does so by calculating the
cross-correlation matrix between two augmented images and making this matrix
close to identity. The loss function of Barlow Twins is represented as
follows:

$\displaystyle\mathcal{L}_{BT}=\sum_{i}{(1-C_{ii})}^{2}+\lambda\sum_{i}\sum_{j\neq
i}{C_{ij}}^{2},$ (7)

where $C_{ij}$ is the cross-correlation between $i$th and $j$th images in a
batch. Here, the first term is called the invariance term, and the second term
is called the redundancy reduction term. The Barlow Twins method is shown in
Figure 2(e).

##  IV Experiments and Results

In this section, we describe the experimental setup and the results. First, we
present the implementation details and dataset description. Then we discuss
the results of different active learning methods for FER. Finally, we present
the results of our proposed method and a details study of different aspects of
the method.

###  IV-A Datasets and Implementation Details

The experiments in the paper are conducted on 3 popular expression recognition
datasets: FER13, RAF-DB, and KDEF. These datasets were selected to cover
various aspects of FER datasets, including dataset size (from small to very
large), spatial resolutions (from low to high), and sources (lab condition vs.
in-the-wild). FER13 [26] is an in-the-wild dataset that was collected by the
Google search API. All images in this dataset have an input resolution of
48$\times$48, and it contains seven expression classes with 28K and 7K images
in the training and validation splits, respectively. RAF-DB [27] is another
in-the-wild dataset containing 12K and 3K images for training and validation,
respectively. The images in this dataset are re-scaled to a resolution of
96$\times$96\. KDEF [28] is a smaller dataset that was collected in a lab
environment with comparatively higher-resolution images.

To ensure a fair comparison, all the active learning methods in the experiment
were trained under the same training settings. Specifically, a ResNet-18 model
is trained for seven cycles ($c$) with 40% of the total labelled samples of
the original dataset. The models were trained with an SGD optimizer with a
learning rate of 0.01 and a batch size of 20. The pre-training was done for
400 epochs following the implementation details of SimCLR [25]. The code is
implemented in PyTorch and trained with Nvidia V100 GPU.

Table I: Accuracy of FER13, RAF-DB, and KDEF for different active learning methods averaged over three runs. All experiments are done on seven cycles. Methods | FER13 | RAF-DB | KDEF
---|---|---|---
Random Sampling |  63.15$\pm$0.33 |  73.74$\pm$1.52 |  83.57$\pm$2.85
Entropy [16] |  65.86$\pm$0.33 |  76.81$\pm$1.51 |  80.78$\pm$1.76
Margin [17] | 65.87$\pm$0.62 |  77.64$\pm$0.93 |  83.30$\pm$0.92
Least Confidence [17] |  65.63$\pm$0.28 |  76.63$\pm$0.95 | 86.03$\pm$1.44
BADGE [15] |  64.91$\pm$0.21 |  77.29$\pm$0.59 |  82.00$\pm$5.11
GLISTER [18] |  65.13$\pm$0.2 |  77.02$\pm$0.76 |  83.71$\pm$1.67
CoreSet [14] |  64.99$\pm$0.09 |  75.76$\pm$0.96 |  81.94$\pm$1.75
BALD [19] |  65.58$\pm$0.17 |  74.79$\pm$1.06 |  79.69$\pm$1.42
Adv. DeepFool [20] |  64.81$\pm$0.38 | 78.56$\pm$0.48 |  79.96$\pm$5.21
Average | 65.35 | 76.48 | 82.20

###  IV-B Performance of Existing Active Learning Methods on FER

In this section, we evaluate the performance of various active learning
methods for FER on the three aforementioned datasets. The results of this
study are presented in Table I. We observe three important findings that
summarize the results from this study as follows:

Table II: Performance of different approaches averaged over three runs. Method | FER13 | RAF-DB | KDEF
---|---|---|---
Random Sampling |  63.15$\pm$0.33 |  73.74$\pm$1.52 |  83.57$\pm$2.85
Least Confidence |  65.63$\pm$0.28 |  76.63$\pm$0.95 |  86.03$\pm$1.44
Pre-training + Random Sampling |  64.66$\pm$0.22 |  76.92$\pm$1.01 |  85.96$\pm$2.42
Pre-training + Least Confidence (our approach) | 67.11$\pm$0.70 | 79.56$\pm$0.64 | 92.77$\pm$2.17

(1) Existing methods perform poorly on FER. In general, active learning on FER
does not provide a reasonable improvement over random sampling. Only Least
confidence and GLISTER show improvements over random sampling for all
datasets. The average across the active learning methods shows some
improvement over random sampling for FER13 and RAF-DB datasets (2.30% and
2.74%) but gets reduced performance for KDEF. This signifies the fact that FER
requires more specialized active learning methods to get a reasonable
improvement over random sampling.

(2) Simpler methods outperform more complicated and specialized methods. For
FER, simpler approaches such as Entropy, Margin, or Least Confidence show
improvements over the latest methods like BADGE, GLISTER, and CoreSet. Among
the two methods that show improvements for all datasets (Least Confidence,
GLISTER), Least confidence shows 2.48%, 2.89%, and 2.46% improvements on FER,
RAF-DB, and KDEF, while GLISTER shows 1.98%, 3.28%, and 0.14% improvements.
Thus we can conclude that the Least Confidence method is the best default
choice for FER datasets for active learning.

(3) Active learning does not work well on small datasets. In our experiment on
the smallest dataset, KDEF, we find no improvement for most of the active
learning methods. Apart from Least Confident and GLISTER, all other methods
perform below random sampling. We argue that this poor performance is caused
by the cold start problem. Since KFEF is a small dataset, training starts with
a very small-sized labelled set, thus overfitting without learning
generalizable representations. In the next section, we present the results of
our proposed solution that alleviates the cold start problem and improves the
accuracy of all datasets, including the very small KDEF dataset.

(a) Initial labelled set size.

(b) Number of cycles.

(c) Epochs per cycle.

(d) Total labelling budget.

Figure 3: Sensitivity study on various parameters of the 2-step training
protocol on all three datasets. The light-shadowed area shows the standard
deviation.

###  IV-C Performance of the Proposed Solution

In Table II, we present the performance of the best active learning method for
FER, i.e., Least Confidence, with and without our proposed solution for self-
supervised pre-training. In general, we observe noticeable improvements with
the proposed SSL pre-training in comparison to the baseline, with the highest
improvement being 6.74% on the KDEF dataset. Recall that most existing methods
showed performance degradation on KDEF due to the cold start problem.
Furthermore, the proposed method shows a 9.2% improvement over random sampling
on this dataset. This shows the effectiveness of the proposed approach for
solving the cold start problem and improving performance. On the RAF-DB
dataset, we see a 2.93% improvement with pre-training and a 5.82% improvement
over random sampling. Similarly, for FER13, we observe a 1.48% improvement
with pre-training and 3.96% over random sampling.

###  IV-D Ablation Study

In this section, we further analyze the impact of SSL pre-training on active
learning. While Table II demonstrated the positive impact of pre-training on
the Least Confidence method, to investigate whether this improvement is a
result of the pre-training alone or the fact that pre-training combined with
active learning alleviates the cold start problem, we perform the same pre-
training followed by random sampling. This result is presented in Table II,
where we observe a 2.45%, 2.64%, and 6.81% drop in performance on the three
datasets, respectively. This finding enforces that a pre-trained encoder (in a
self-supervised setting) on its own does not provide much improvement. Rather,
selecting more representative samples in active learning boosts performance.

We also investigate different choices for self-supervised pre-training, which
we discussed in Section III-B. We summarize these results in Table III.
Overall, observations from the table show that pre-training with SimCLR
provides the best results compared to other self-supervised methods. Barlow
Twins shows the next best accuracy for RAF-DB and KDEF. MoCo-v2 shows the
second-best accuracy for FER13. While SimCLR provides 1.48%, 2.93%, and 6.74%
improvements for FER13, RAF-DB, and KDER datasets, Barlow Twins shows 2.73%
and 5.04% improvements for RAF-DB and KDEF datasets, and MoCo v2 demonstrates
1.28% improvements for FER13.

Table III: Performance of Least Confidence method with various pre-training approaches. Pre-training | FER13 | RAF-DB | KDEF
---|---|---|---
None |  65.63$\pm$0.28 |  76.63$\pm$0.95 |  86.03$\pm$1.44
BYOL |  65.91$\pm$0.79 |  78.46$\pm$1.78 |  87.32$\pm$0.73
MoCo v2 |  66.91$\pm$0.92 |  79.11$\pm$1.69 |  90.80$\pm$1.04
Barlow Twins |  66.85$\pm$0.87 |  79.36$\pm$1.34 |  91.07$\pm$0.98
SwAV |  66.02$\pm$0.99 |  78.12$\pm$1.95 |  88.89$\pm$1.79
SimCLR |  67.11$\pm$0.70 |  79.56$\pm$1.64 |  92.77$\pm$2.17

###  IV-E Sensitivity study

In this section, we present a detailed sensitivity study on different hyper-
parameters involved in the entire pipeline, including pre-training and active
learning. More specifically, we conduct experiments on the following factors:
(1) initial labelled set size of active learning, (2) number of active
learning cycles, (3) number of epochs per cycle, (4) total labelling budget,
and (5) optimizer. We show the results for the best performing active learning
method for FER, i.e., Least Confidence, on all three datasets.

####  IV-E1 Sensitivity toward the initial labelled set size

The ‘initial labelled set size ($s$)’ is an important hyper-parameter for
active learning. Since the initial labelled set is selected randomly,
selecting a large value for $s$ reduces the number of samples that can be
selected with active learning at later cycles. On the other hand, choosing
very small values of $s$ can contribute to the cold start problem. Therefore,
we investigate different values of $s$ for FER using the two-step training
solution. In Figure 3(a), we illustrate the sensitivity for different initial
labelled set sizes. The figure shows that choosing small values of $s$ leads
to better performance. For example, both FER13 and RAF-DB show the best
performance when only 5% of samples are selected as the initial labelled set.
For KDEF, the best accuracy is achieved with an initial labelled set size of
15%. We argue that the underlying representations learned by the self-
supervised pre-training are responsible for this phenomenon. Due to the pre-
training step, the model can learn from a small initial labelled set and
select most of the samples in later cycles. Another important trend is the
increased standard deviation when more samples are selected as the initial
labelled set, especially for the KDEF dataset.

####  IV-E2 Sensitivity toward the number of active learning cycles

Another important parameter for active learning training is the total number
of cycles. A large number of cycles provides the opportunity for selecting
more representative samples at later cycles. Nevertheless, increasing the
number of cycles also increases the total training cost, and it is, therefore,
important to find an optimal number for this parameter. The sensitivity toward
different numbers of training cycles is presented in Figure 3(b). We find the
best performance is obtained for FER and RAF-DB when the model is trained for
seven cycles and KDEF for nine cycles.

####  IV-E3 Sensitivity toward training epochs per cycle

We also investigate the optimal number of training epochs per cycle. This is
another important parameter since training for an excessive number of epochs
can cause the model to overfit the data available in that cycle, whereas using
too few can hamper learning for the set of data in that cycle. As a result, we
present a sensitivity study on the number of active learning epochs in Figure
3(c). The observations show that the best accuracies are observed for 150
epochs for both KDEF and RAF-DB datasets. The FER13 shows better performance
for longer training as it obtains the best accuracy with 250 epochs, with 200
also showing a close accuracy.

####  IV-E4 Performance versus different labelling budgets

In Figure 3(d), we show the performance for different amounts of total
labelling budget. In general, more labelled samples results in better accuracy
for almost all settings. However, the change in accuracy is sharper in low
data regimes (e.g. increase from 20% to 25%) compared to the higher ones.

Table IV: Accuracy for different optimizers. Optimizer | FER13 | RAF-DB | KDEF
---|---|---|---
SGD |  67.11$\pm$0.70 |  79.56$\pm$1.64 |  92.77$\pm$2.17
Adam |  63.67$\pm$0.68 |  75.25$\pm$2.97 |  85.48$\pm$2.32

####  IV-E5 Sensitivity toward the optimizer

We also investigate the impact of the choice of optimizer on the final
performance in Table IV. The results in this table show that SGD is
considerably better than Adam optimizer for all the datasets. SGD shows 7.29%,
4.31%, and 3.44% higher accuracy compared to Adam on KDEF, RAF-DB and FER13,
respectively.

##  V Conclusion

In this study, we explored active learning as a solution for reducing the
reliance on large amounts of labelled data to train deep learning models for
FER. First, we implemented and evaluated various active learning methods for
FER and confirmed the presence of a cold start problem. To overcome this issue
and further enhance FER performance, we employed a two-step training protocol.
In the first step, we conducted contrastive self-supervised pre-training using
the entire set of unlabeled data. Our extensive studies showed that the two-
step training protocol alleviates the cold start problem and improves
performance by considerable margins. An extensive ablation study showed the
effectiveness of the proposed pre-training step, and a comprehensive
sensitivity study identified the optimal parameters for each dataset. We hope
that this research will bring attention to this important direction for
reducing the labelling cost, which can help facilitate the development of
improved FER methods.

## Acknowledgements

We would like to thank Bank of Montreal and Mitacs for funding this research.
We are also thankful to SciNet HPC Consortium for helping with the computation
resources.

## Ethical Impact Statement

This study did not include the collection of any new datasets as it relied on
three popular public FER datasets used by most existing works in this domain.
These datasets feature diverse demographic groups and contain no personal
identification information (besides what has been already public or known to
the original authors of these datasets) or offensive material, thereby
avoiding any privacy concerns. Since our work relies on datasets collected
from the internet, we believe good generalizability towards different
demographic groups of people is achievable. However, a detailed study on bias
is required to further analyze this notion. We acknowledge that, like any FER
method, the system developed in this paper has the potential to be used to
analyze the facial expressions of individuals without their consent. As a
result, we find it absolutely imperative for such systems to be used ethically
and responsibly with full compliance with ethical, moral, and legal
guidelines. The training process of each experiment took under 12 hours on a
single Nvidia V100 GPU, a reasonable time-frame that does not pose a
significant carbon footprint.

## References

  * [1] Nikhil Churamani and Hatice Gunes,  “Clifer: Continual learning with imagination for facial expression recognition,”  in IEEE International Conference on Automatic Face and Gesture Recognition, 2020, pp. 322–328.
  * [2] Mojtaba Kolahdouzi, Alireza Sepas-Moghaddam, and Ali Etemad,  “Face trees for expression recognition,”  in IEEE International Conference on Automatic Face and Gesture Recognition, 2021, pp. 1–5.
  * [3] Mojtaba Kolahdouzi, Alireza Sepas-Moghaddam, and Ali Etemad,  “Facetoponet: Facial expression recognition using face topology learning,”  IEEE Transactions on Artificial Intelligence, 2022.
  * [4] Evangelos Sariyanidi, Hatice Gunes, and Andrea Cavallaro,  “Learning bases of activity for facial expression recognition,”  IEEE Transactions on Image Processing, vol. 26, no. 4, pp. 1965–1978, 2017.
  * [5] Dimitrios Kollias and Stefanos Zafeiriou,  “Affect analysis in-the-wild: Valence-arousal, expressions, action units and a unified framework,”  arXiv preprint arXiv:2103.15792, 2021.
  * [6] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin,  “Attention is all you need,”  Advances in Neural Information Processing Systems, vol. 30, 2017\.
  * [7] Shuvendu Roy and Ali Etemad,  “Analysis of semi-supervised methods for facial expression recognition,”  in 10th International Conference on Affective Computing and Intelligent Interaction. IEEE, 2022, pp. 1–8.
  * [8] Shuvendu Roy and Ali Etemad,  “Self-supervised contrastive learning of multi-view facial expressions,”  in International Conference on Multimodal Interaction, 2021, pp. 253–257.
  * [9] Shuvendu Roy and Ali Etemad,  “Spatiotemporal contrastive learning of facial expressions in videos,”  pp. 1–8, 2021.
  * [10] Shuvendu Roy and Ali Etemad,  “Impact of labelled set selection and supervision policies on semi-supervised learning,”  arXiv preprint arXiv:2211.14912, 2022.
  * [11] Pengzhen Ren, Yun Xiao, Xiaojun Chang, Po-Yao Huang, Zhihui Li, Brij B Gupta, Xiaojiang Chen, and Xin Wang,  “A survey of deep active learning,”  ACM computing surveys, vol. 54, no. 9, pp. 1–40, 2021.
  * [12] Li Yao, Yan Wan, Hongjie Ni, and Bugao Xu,  “Action unit classification for facial expression recognition using active learning and svm,”  Multimedia Tools and Applications, vol. 80, no. 16, pp. 24287–24301, 2021.
  * [13] Minhaz Uddin Ahmed, Kim Jin Woo, Kim Yeong Hyeon, Md Rezaul Bashar, and Phill Kyu Rhee,  “Wild facial expression recognition based on incremental active learning,”  Cognitive Systems Research, vol. 52, pp. 212–222, 2018.
  * [14] Ozan Sener and Silvio Savarese,  “Active learning for convolutional neural networks: A core-set approach,”  arXiv preprint arXiv:1708.00489, 2017.
  * [15] Jordan T Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal,  “Deep batch active learning by diverse, uncertain gradient lower bounds,”  arXiv preprint arXiv:1906.03671, 2019.
  * [16] Burr Settles,  “Active learning literature survey,”  2009\.
  * [17] Dan Wang and Yi Shang,  “A new active labeling method for deep learning,”  in International Joint Conference on Neural Networks. IEEE, 2014, pp. 112–119.
  * [18] Krishnateja Killamsetty, Durga Sivasubramanian, Ganesh Ramakrishnan, and Rishabh Iyer,  “Glister: Generalization based data subset selection for efficient and robust learning,”  in AAAI Conference on Artificial Intelligence, 2021, vol. 35, pp. 8110–8118.
  * [19] Yarin Gal, Riashat Islam, and Zoubin Ghahramani,  “Deep bayesian active learning with image data,”  in International Conference on Machine Learning. PMLR, 2017, pp. 1183–1192.
  * [20] Melanie Ducoffe and Frederic Precioso,  “Adversarial active learning for deep networks: a margin based approach,”  arXiv preprint arXiv:1802.09841, 2018.
  * [21] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick,  “Momentum contrast for unsupervised visual representation learning,”  in IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 9729–9738.
  * [22] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al.,  “Bootstrap your own latent-a new approach to self-supervised learning,”  Advances in Neural Information Processing Systems, vol. 33, pp. 21271–21284, 2020.
  * [23] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin,  “Unsupervised learning of visual features by contrasting cluster assignments,”  Advances in Neural Information Processing Systems, vol. 33, pp. 9912–9924, 2020.
  * [24] Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny,  “Barlow twins: Self-supervised learning via redundancy reduction,”  in International Conference on Machine Learning. PMLR, 2021, pp. 12310–12320.
  * [25] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton,  “A simple framework for contrastive learning of visual representations,”  in International Conference on Machine Learning. PMLR, 2020, pp. 1597–1607.
  * [26] Ian J Goodfellow, Dumitru Erhan, Pierre Luc Carrier, Aaron Courville, Mehdi Mirza, Ben Hamner, Will Cukierski, Yichuan Tang, David Thaler, Dong-Hyun Lee, et al.,  “Challenges in representation learning: A report on three machine learning contests,”  in International Conference on Neural Information Processing. Springer, 2013, pp. 117–124.
  * [27] Shan Li, Weihong Deng, and JunPing Du,  “Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild,”  in IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017, pp. 2852–2861.
  * [28] Daniel Lundqvist, Anders Flykt, and Arne Öhman,  “The karolinska directed emotional faces,”  CD ROM from Department of Clinical Neuroscience, Psychology Section, Karolinska Institutet, vol. 91, no. 630, pp. 2–2, 1998.


# How to determine a curve singularity

J. Elias ∗ Joan Elias
Departament de Matemàtiques i Informàtica
Universitat de Barcelona (UB)
Gran Via 585, 08007 Barcelona, Spain<EMAIL_ADDRESS>
###### Abstract.

We characterize the finite codimension sub-${\mathbf{k}}$-algebras of
${\mathbf{k}}[\\![t]\\!]$ as the solutions of a computable finite family of
higher differential operators. For this end, we establish a duality between
such a sub-algebras and the finite codimension ${\mathbf{k}}$-vector spaces of
${\mathbf{k}}[u]$, this ring acts on ${\mathbf{k}}[\\![t]\\!]$ by
differentiation.

∗ Partially supported by PID2019-104844GB-I00
2020 MSC: Primary 13H10; Secondary 14B05; 13H15

##  1\. Introduction

It is well know that the normalization of a curve $X$ is a non-singular curve
$Y$. Serre considers in [26, Chap IV] the opposite direction, he showed how to
construct a curve $X$ from a given non singular curve $Y$ such that this curve
is the normalization of $X$. This idea appears in several different contexts.
For instance, in [17], [23], [18], and the references therein, is studied how
to determine the finite codimension sub-${\mathbf{k}}$-algebras $B$ of
${\mathbf{k}}[t]$. Notice that, in this case, $X=\operatorname{Spec}(B)$ is an
algebraic curve and the affine line $Y=\operatorname{Spec}({\mathbf{k}}[t])$
is its normalization. These sub-algebras are defined recursively on the
codimension by linear and higher differential conditions. Only for low
codimensions explicit conditions are known. Since not all higher differential
conditions define sub-algebras of ${\mathbf{k}}[t]$, it is an open problem the
characterization of families of linear higher differential operators defining
finite codimension sub-${\mathbf{k}}$-algebras of ${\mathbf{k}}[t]$, see [18].

In the search of one-dimensional reduced local rings with locally decreasing
Hilbert function, Roberts constructed such a local rings as connex, finite
codimension sub-${\mathbf{k}}$-algebras of
$\prod_{i=1}^{r}{\mathbf{k}}[t_{i}]$ defined by linear and first order
differentials conditions, [19]. See [11] for the proof of Sally’s conjecture
on the monotony of Hilbert functions of one-dimensional Cohen-Macaulay local
rings.

In this paper we consider the local complete case. We characterize the finite
codimension sub-${\mathbf{k}}$-algebras $B$ of
$\Gamma={\mathbf{k}}[\\![t]\\!]$ as the solutions of a computable finite
codimension ${\mathbf{k}}$-vector space
$B^{\perp}\subset\Delta={\mathbf{k}}[u]$ of higher differential operators,
Theorem 3.9. For this purpose, we establish a Macaulay-like duality between
finite codimension sub-${\mathbf{k}}$-algebras $B$ of $\Gamma$ and finite
codimension ${\mathbf{k}}$-vector subspaces $B^{\perp}$, so-called algebra-
forming vector spaces, of the polynomial ring $\Delta$. The polynomial ring
$\Delta$ acts on $\Gamma$ by differentiation as in Macaulay’s duality, see
[20], [14], [15], [16]. At the end of Section 3 we describe the linear maps
$B_{2}^{\perp}\rightarrow B_{1}^{\perp}$ induced by ${\mathbf{k}}$-algebra
morphisms $B_{1}\rightarrow B_{2}$ between two finite codimension
${\mathbf{k}}$-algebras $B_{1}$, $B_{2}$.

In section 4 we study the algebra-forming vector spaces, showing that such a
condition can be checked effectively, Proposition 4.1. After this we prove
that for any finite codimension $\delta$ ${\mathbf{k}}$-algebra $B$ there
exist a finite filtration of ${\mathbf{k}}$-algebras, so-called standard
filtration of $B$, $B=B_{0}\subset B_{1}\subset\cdots\subset
B_{\delta}=\Gamma$ such that
$\operatorname{dim}_{{\mathbf{k}}}(B_{i+1}/B_{i})=1$ for $i=0,\dots,\delta-1$.
As corollary of this construction we get that we only need to consider
algebra-forming single elements in order to define recursively a finite
codimension ${\mathbf{k}}$-algebras. Moreover, we show how to recover the
standard filtration by considering recursively derivations of the local rings
appearing in the filtration, Corollary 4.6.

Section 5 is devoted to study the inverse system of monomial
${\mathbf{k}}$-algebras and the special case of monomial Gorenstein algebras.
We end the section relating the inverse system of a curve singularity with its
generic plane projection and its saturation.

In the last section we link $B^{\perp}$ with the canonical module of $B$,
Proposition 6.1.

The computations of this paper are performed by using the computer algebra
system Singular, [8].

##  2\. Preliminaries

Let $R$ denote the power series ring ${\mathbf{k}}[\\![x_{1},\dots,x_{n}]\\!]$
over an algebraically closed characteristic zero field ${\mathbf{k}}$ and we
denote by ${\mathfrak{m}}=(x_{1},\cdots,x_{n})$ its maximal ideal.

Let $A$ be a one-dimensional local ring with maximal ideal ${\mathfrak{m}}$.
We denote by ${\operatorname{H\\!F}}_{A}$ the Hilbert function of $A$, i.e.
${\operatorname{H\\!F}}_{A}(i)=\operatorname{Length}_{A}({\mathfrak{m}}^{i}/{\mathfrak{m}}^{i+1})$,
$i\geq 0$. It is well known that ${\operatorname{H\\!F}}^{0}_{A}(i)=e_{0}(A)$,
$i\gg 0$, where $e_{0}(A)$ is the multiplicity of $A$. The first integral of
${\operatorname{H\\!F}}_{A}$ is defined by, $i\geq 0$,

${\operatorname{H\\!F}}^{1}_{A}(i)=\sum_{j=0}^{i}{\operatorname{H\\!F}}_{A}(j)=\operatorname{Length}_{A}(A/{\mathfrak{m}}^{i+1}).$

We write ${\operatorname{H\\!F}}_{A}^{0}={\operatorname{H\\!F}}_{A}$. There
exists an integer $e_{1}(A)$ such that
${\operatorname{H\\!F}}^{1}_{A}(i)=e_{0}(A)(i+1)-e_{1}(A)$ for $i\gg 0$; the
(first) Hilbert polynomial is
$\operatorname{H\\!P}^{1}_{A}(T)=e_{0}(A)(T+1)-e_{1}(A)$. See [22, Chapter
XII] for the basic properties of the Hilbert functions of one-dimensional
Cohen-Macaulay local rings.

A branch $X$ is an irreducible curve singularity of
$({\mathbf{k}}^{n},0)=\operatorname{Spec}(R)$, i.e. $X$ is a one-dimensional,
integral scheme $X=\operatorname{Spec}(R/I)$; we write ${\mathcal{O}}_{X}=R/I$
and $I(X)=I$.

Let
$\nu:\overline{X}=\operatorname{Spec}(\overline{{\mathcal{O}}_{X}})\longrightarrow(X,0)$
be the normalization of $(X,0)$, where
$\overline{{\mathcal{O}}_{X}}\cong{\mathbf{k}}[\\![t]\\!]$ is the integral
closure of ${\mathcal{O}}_{X}$ on its full field of fractions
${\mathrm{tot}}({\mathcal{O}}_{X})$. The singularity order of $X$ is
$\delta(X)=\operatorname{dim}_{{\mathbf{k}}}\left({\mathcal{O}}_{\overline{X}}/{\mathcal{O}}_{X}\right).$
We denote by $\mathcal{C}$ the conductor of the finite extension
$\nu^{*}:{\mathcal{O}}_{X}\hookrightarrow\overline{{\mathcal{O}}_{X}}$ and by
$c(X)$ the dimension of $\overline{{\mathcal{O}}_{X}}/\mathcal{C}$.

Given a set of non-negative integers $1\leq a_{1}<\cdots<a_{n}$ we consider
the monomial curve singularity $X(a_{1},\cdots,a_{n})$ defined by the
parameterization

$\begin{array}[]{cccc}\gamma:&R&\longrightarrow&{\mathbf{k}}[\\![t]\\!]\\\
&x_{i}&\mapsto&t^{a_{i}}\end{array}$

i.e. $I(X(a_{1},\cdots,a_{n}))=\operatorname{Ker}(\gamma)$. If
$gcd(a_{1},\cdots,a_{n})=1$ then the induced map

$\gamma:R/I(X(a_{1},\cdots,a_{n}))\longrightarrow{\mathbf{k}}[\\![t]\\!]$

is the normalization map of
$\mathcal{O}_{X(a_{1},\cdots,a_{n})}=R/I(X(a_{1},\cdots,a_{n}))={\mathbf{k}}[\\![t^{a_{1}},\dots,t^{a_{n}}]\\!]$.

We denote by $D_{X}$ the semigroup of values of $X$: the set of integers
$v_{t}(f)=ord_{t}(t)$ where $f\in{\mathcal{O}}_{X}\setminus\\{0\\}$. It is
easy to see that $\delta(X)=\\#(\mathbb{N}\setminus D_{X})$. If $B$ is a
finite codimension sub-${\mathbf{k}}$-algebra of $\Gamma$ then
$X=\operatorname{Spec}(B)$ is branch. We write $D_{B}=D_{X}$.

Let $\omega_{X}$ be the dualizing module of $X$; we can consider the
composition of ${\mathcal{O}}_{X}$-module morphisms

$\gamma_{X}:\Omega_{X}\longrightarrow\nu_{*}\Omega_{\overline{X}}\cong\nu_{*}\omega_{\overline{X}}\longrightarrow\omega_{X}.$

Let $d:{\mathcal{O}}_{X}\longrightarrow\Omega_{X}$ the universal derivation,
then we have a ${\mathbf{k}}$-linear map $\gamma_{X}d$ that we also denote by
$d:{\mathcal{O}}_{X}\longrightarrow\omega_{X}.$ Recall that the Milnor number
of $X$ is
$\mu(X)=\operatorname{dim}_{{\mathbf{k}}}(\omega_{X}/d{\mathcal{O}}_{X})$,
[5]. Since we only consider branches we have that $\mu(X)=2\delta(X)$, [5,
Proposition 1.2.1]. Notice that $X$ is non-singular iff $\mu(X)=0$ iff
$\delta(X)=0$ iff $c(X)=0$.

We denote by $\pi:Bl(X)\longrightarrow X$ the blowing-up of $X$ on its closed
point. The fiber of the closed point of $X$ has a finite number of closed
points: the so-called points of the first neighborhood of $X$. We can iterate
the process of blowing-up until we get the normalization of $X$, see [24] and
[7]. We denote by $\operatorname{Inf}(X)$ the set of infinitely near points of
$X$. The curve singularity defined by an infinitely point $p$ of $X$ will be
denote by $(X,p)$; we set $(X,0)=X$.

######  Proposition 2.1.

Let $X$ be a branch. Then

$(i)$

$\delta(X)=\sum_{p\in\operatorname{Inf}(X)}e_{i}(X,p)$

$(ii)$ It holds

$e_{0}(X)-1\leq e_{1}(X)\leq\delta(X)\leq\mu(X)$

and $e_{1}(X)\leq\binom{e_{0}(X)}{2}-\binom{n-1}{2}$.

$(iii)$ If $X$ is singular then $\delta(X)+1\leq c(X)\leq 2\delta(X)$, and
$c(X)=2\delta(X)$ if and only if ${\mathcal{O}}_{X}$ is a Gorenstein ring.

###### Proof.

$(i)$ [25]. $(ii)$ [5, Proposition 1.2.4 (i)], [25], [10], [12]. $(iii)$ [26,
Proposition 7, pag. 80], and [2]. ∎

##  3\. Macaulay-like duality

In this section we establish a Macaulay-like duality for the family of
sub-${\mathbf{k}}$-algebras $B$ of $\Gamma={\mathbf{k}}[\\![t]\\!]$ of finite
codimension. For the classical Macaulay’s duality see [20], [14], and for the
generalization to higher dimension of Macaulay’s duality see [15], [16].
Recall that Macaulay’s duality is a particular case of Matlis’ duality, [4].

We write $\Delta={\mathbf{k}}[u]$; $\Gamma$ is a $\Delta$-module with $\Delta$
acting on $\Gamma$ by derivation. This action denoted by $\circ$ is defined by

$\begin{array}[]{ clc}\circ:\Delta\times\Gamma&\longrightarrow&\Gamma\\\
(g,f)&\to&g\circ f=g(\partial_{t})(f)\end{array}$

where $\partial_{t}$ denotes the derivative with respect to $t$. This action
induces a non-singular ${\mathbf{k}}$-bilinear perfect pairing:

(1) $\begin{array}[]{
lclc}\perp:&\Delta\times\Gamma&\longrightarrow&{\mathbf{k}}\\\
&(g,f)&\mapsto&g\perp f=(g\circ f)(0)\end{array}$

######  Definition 3.1.

Given a sub-${\mathbf{k}}$-algebra $B$ of $\Gamma={\mathbf{k}}[\\![t]\\!]$ we
define $B^{\perp}$ as the set of $g\in\Delta$ such that $g\perp f=0$ for all
$f\in B$. Notice that $B^{\perp}$ is a ${\mathbf{k}}$-vector subspace of
$\Delta$, this is, following the classic Macaulay’s duality terminology, the
inverse system of $B$. Given a ${\mathbf{k}}$-vector subspace $V\subset\Delta$
we consider $\operatorname{Ann}(V)\subset\Gamma$ as the set of power series
$f\in\Gamma$ such that $g\perp f=0$ for all $g\in V$.

Let $B$ be a finite codimension sub-${\mathbf{k}}$-algebra of $\Gamma$. Then
we have a non-singular ${\mathbf{k}}$-bilinear perfect pairing:

(2) $\begin{array}[]{
cclc}\perp:&B^{\perp}\times\frac{\Gamma}{B}&\longrightarrow&{\mathbf{k}}\\\
&(g,\overline{f})&\mapsto&g\perp f\end{array}$

We denote by $Perp(B)$ the ${\mathbf{k}}$-vector space of maps

$\begin{array}[]{cclc}g^{\perp}:&B&\longrightarrow&{\mathbf{k}}\\\
&f&\mapsto&g\perp f\end{array}$

for all $g\in\Delta$. These maps are the elements of the dual space of $B$
with finite support: $g^{\perp}({\mathfrak{m}}_{B}^{d})=0$ for
$d>\operatorname{deg}(g)$. We denote by $Der_{{\mathbf{k}}}(B)$ the
${\mathbf{k}}$-vector space of ${\mathbf{k}}$-derivations of $B$. Since
$Der_{{\mathbf{k}}}(B)\cong({\mathfrak{m}}_{B}/{\mathfrak{m}}_{B}^{2})^{*}$,
we can identify $Der_{{\mathbf{k}}}(B)$ with the ${\mathbf{k}}$-vector space
of elements $\sigma$ of the dual space of $B$ such that
$\sigma({\mathfrak{m}}_{B}^{2})=0$.

We have $Der_{{\mathbf{k}}}(B)\subset Perp(B)$, this inclusion is strict. Let
us consider the codimension $8$ algebra
$B={\mathbf{k}}[\\![t^{4},t^{7},t^{17}]\\!]$. The linear map
$(u^{11})^{\perp}:B\longrightarrow{\mathbf{k}}$ is not a derivation since
$t^{11}\in{\mathfrak{m}}_{B}^{2}$ and $(u^{11}){\perp}(t^{11})=11!\neq 0$.

Next step is to characterize the vector ${\mathbf{k}}$-vector subspaces
$B^{\perp}$ of $\Delta$ where $B$ ranges the family of finite codimension
sub-${\mathbf{k}}$-algebras of $\Gamma$. First, we give some properties of
$B^{\perp}$ that we will use along the paper.

Given a polynomial $g=\sum_{i=0}^{d}a_{i}u^{i}\in\Delta$ we denote by
${\rm{Supp}}(g)$ the support of $g$: the finite set of integers $i$ such that
$a_{i}\neq 0$.

######  Proposition 3.2.

Let $B\subset\Gamma$ be a codimension $\delta$ sub-${\mathbf{k}}$-algebra $B$
of $\Gamma$ and let $\mathcal{C}=(t^{c})$ the conductor of the extension
$B\subset\Gamma$. Then

  1. (1)

$\operatorname{dim}_{{\mathbf{k}}}(B^{\perp})=\delta$.

  2. (2)

For all $g\in B^{\perp}$ we have ${\rm{Supp}}(g)\subset[1,c-1]$, and

$u^{[1,e_{0}(B)-1]}=\\{u^{i};i\in[1,e_{0}(B)-1]\\}\subset
B^{\perp}\subset\langle u,u^{2},\dots,u^{c-1}\rangle.$

  3. (3)

The following conditions are equivalent:

    1. (i)

$\delta=0$,

    2. (ii)

$B=\Gamma$,

    3. (iii)

$B^{\perp}=0$,

    4. (iv)

$B^{\perp}\subset\langle u^{2},u^{3},\dots\rangle$.

###### Proof.

$(1)$ Since $\perp$ is a ${\mathbf{k}}$-bilinear perfect pairing we get
$\operatorname{dim}_{{\mathbf{k}}}(B^{\perp})=\delta$, see the equation (2).

$(2)$ Since $B$ is a ${\mathbf{k}}$-algebra we have $1\in B$, so if
$g=\sum_{j\geq 0}a_{i}u^{i}\in B^{\perp}$ then $0=g\perp 1=a_{0}$. Hence
$B^{\perp}\subset\langle u,u^{2},\dots\rangle$. We know that $(t^{c})\subset
B$ so for all $g=\sum_{j\geq 0}a_{i}u^{i}\in B^{\perp}$ we have

$0=g\perp t^{c+i}=(c+i)!a_{c+i}$

$i\geq 0$. Hence, if $g\in B^{\perp}$ then $\operatorname{deg}(g)\leq c-1$.
From this we deduce that $B^{\perp}\subset\langle
u,u^{2},\dots,u^{c-1}\rangle$.

Notice that $v_{t}(f)\geq e_{0}(B)$ for all $f\in B\setminus\\{1\\}$, so given
$i\in[1,e_{0}(B)-1]$ we have $u^{i}\perp f=0$. Hence $u^{i}\in B^{\perp}$ and
then $u^{[1,e_{0}(B)-1]}\subset B^{\perp}$.

$(3)$ The condition of $(i)$ is equivalent to $(ii)$. $(ii)$ trivially implies
$(iii)$ and this implies $(iv)$. If $B^{\perp}\subset\langle
u^{2},u^{3},\dots\rangle$ then $t\in B$, since $B$ is a ${\mathbf{k}}$-algebra
we get $(ii)$. ∎

For all power series $f=\sum_{i\geq 0}b_{i}t^{i}\in\Gamma$ and given a non-
negative integer $s\in\mathbb{N}$ we denote by $[f]_{\leq s}$ the truncated
polynomial $[f]_{\leq s}=\sum_{i\geq 0}^{s}b_{i}t^{i}$.

Let $B$ be a finite codimension sub-${\mathbf{k}}$-algebra of $\Gamma$ with
conductor $c$. Then $B$ is a finitely generated ${\mathbf{k}}$-algebra; let
$f_{1},\dots,f_{r}$ be a system of generators of $B$ as
${\mathbf{k}}$-algebra. We denote by $\natural_{B,d}$, $d\geq c-1$, the finite
set of polynomials $[f_{1}^{l_{1}}\cdots f_{r}^{l_{r}}]_{\leq d}$ with
$l_{i}\geq 0$, $i=1,\dots,r$, and $l_{1}+\cdots+l_{r}\leq d$. We denote by
$W(\\{f_{1},\dots,f_{r}\\},d)\subset\Delta$ the ${\mathbf{k}}$-vector space
generated by the polynomials of $\natural_{B,d}$. Notice that
$W(\\{f_{1},\dots,f_{r}\\},d)+\langle
t^{d+1}\rangle=W(\\{f_{1},\dots,f_{r}\\},d+1)$.

######  Proposition 3.3.

Let $B$ be a finite codimension sub-${\mathbf{k}}$-algebra of $\Gamma$ with
conductor $c$. Then $B^{\perp}$ is the set of $g\in\Delta$ of degree at most
$c-1$ and such that $g\perp h=0$ for all $h\in\natural_{B,c-1}$.

###### Proof.

Let $f_{1},\dots,f_{r}$ be a system of generators of $B$ as
${\mathbf{k}}$-algebra and let $\natural_{B,c-1}$ be the associated set of
polynomials.

If $g\in B^{\perp}$ then $\operatorname{deg}(g)\leq c-1$, Proposition 3.2(2),
so

$0=g\perp(f_{1}^{l_{1}}\cdots f_{r}^{l_{r}})=g\perp[f_{1}^{l_{1}}\cdots
f_{r}^{l_{r}}]_{\leq c-1}.$

Hence $g\perp h=0$ for all $h\in\natural_{B,c-1}$.

Let $g\in\Delta$ be a polynomial with $\operatorname{deg}(g)\leq c-1$ and such
that $g\perp h=0$ for all $h\in\natural_{B,c-1}$. Any $f\in B$ can be written
as

$f=\sum_{l_{1},\dots,l_{r}\in\mathbb{N}}c_{l_{1},\dots,l_{r}}f_{1}^{l_{1}}\cdots
f_{r}^{l_{r}}$

with $c_{l_{1},\dots,l_{r}}\in{\mathbf{k}}$. Since $\operatorname{deg}(g)\leq
c-1$ we have

$g\perp f=\sum_{l_{1},\dots,l_{r}\in\mathbb{N}}c_{l_{1},\dots,l_{r}}(g\perp
f_{1}^{l_{1}}\cdots
f_{r}^{l_{r}})=\sum_{l_{1},\dots,l_{r}\in\mathbb{N}}c_{l_{1},\dots,l_{r}}(g\perp[f_{1}^{l_{1}}\cdots
f_{r}^{l_{r}}]_{\leq c-1})=0,$

so $g\in B^{\perp}$. ∎

######  Remark 3.4.

Notice that Proposition 3.3 shows that the computation of $B^{\perp}$ is
effective. In fact, in the set $\natural_{B,c-1}$ there are involved a finite
number of monomials and we only have to consider polynomials $g$ of degree at
most $c-1$.

######  Remark 3.5.

Although $B^{\perp}$ is a ${\mathbf{k}}$-vector subspace of $\Delta$ for any
sub-${\mathbf{k}}$-algebra $B$ of $\Gamma$, not all $\operatorname{Ann}(V)$ is
a ${\mathbf{k}}$-algebra for a given ${\mathbf{k}}$-vector subspace
$V\subset\Delta$. In fact, let us consider the ${\mathbf{k}}$-vector subspace
$V\subset\Delta$ generated by $u^{2}$. Then $\operatorname{Ann}(V)$ is the set
of $f=\sum_{i\geq 0}a_{i}t^{i}\in\Gamma$ such that $a_{2}=0$. This is not a
${\mathbf{k}}$-algebra because $u^{2}\perp t=0$, so
$t\in\operatorname{Ann}(V)$ and $u^{2}\perp t^{2}=2\neq 0$, so
$t^{2}\notin\operatorname{Ann}(V)$.

######  Definition 3.6.

A finite dimensional ${\mathbf{k}}$-vector subspace $V\subset\Delta$ is so-
called algebra-forming with respect to a ${\mathbf{k}}$-algebra
$B\subset\Gamma$ iff the following conditions hold:

  1. (a)

$g(0)=0$ for all $g\in V$ and,

  2. (b)

for all $f\in B$ such that $g\perp f=0$ for all $g\in V$ it holds $g\perp
f^{2}=0$ for all $g\in V$.

An element $g\in\Delta$ is so-called algebra-forming with respect to $B$ if
$V=\langle g\rangle$ is algebra-forming with respect to $B$.

######  Example 3.7.

Let us consider the codimension $\delta=4$ algebra
$B={\mathbf{k}}[\\![t^{3}+t^{4},t^{5}]\\!]$ of $\Gamma$. The conductor of $B$
is $c=8$. Then $B^{\perp}$ is the set of polynomials $g\in\Delta$ of degree at
most $7$ such that $g\perp f=0$ for
$f\in\natural_{B,c-1}=\\{t^{3}+t^{4},t^{5},t^{6}+2t^{7}\\}$. A simple
computation shows that $B^{\perp}$ is the ${\mathbf{k}}$-vector space
generated by the $4$ linear independent polynomials
$u,u^{2},u^{3}-\frac{1}{4}u^{4},u^{6}-\frac{1}{2.7}u^{7}$. Let us consider

$B_{2}={\mathbf{k}}[\\![t^{3},t^{4},t^{5}]\\!]\subset
B_{3}={\mathbf{k}}[\\![t^{2},t^{3}]\\!],$

then we have $B_{2}=\operatorname{Ann}\langle u^{2}\rangle\cap B_{3}$, i.e.
$u^{2}$ is an algebra-forming element with respect to $B_{2}$.

In the following result we prove that, in fact, if $V\subset\Delta$ is
algebra-forming with respect to a ${\mathbf{k}}$ algebra $B\subset\Gamma$ then
$\operatorname{Ann}(V)\cap B$ is a sub-${\mathbf{k}}$-algebra of $\Gamma$.

######  Proposition 3.8.

Let $V\subset\Delta$ be an algebra-forming ${\mathbf{k}}$-vector subspace with
respect to a ${\mathbf{k}}$-algebra $B\subset\Gamma$. Then
$\operatorname{Ann}(V)\cap B$ is a sub-${\mathbf{k}}$-algebra of $\Gamma$.

###### Proof.

Clearly $C=\operatorname{Ann}(V)\cap B$ is a ${\mathbf{k}}$-vector subspace of
$\Gamma$. Given $f_{1},f_{2}\in C$ we have that $f_{1}+f_{2}\in C$ and from

$f_{1}f_{2}=\frac{1}{2}((f_{1}+f_{2})^{2}-f_{1}^{2}-f_{2}^{2})$

we deduce that $g\perp(f_{1}f_{2})=0$, i.e. $f_{1}f_{2}\in C$. Since $g(0)=0$
for all $g\in V$ we get $1\in C$, so $C$ is a sub-${\mathbf{k}}$-algebra of
$\Gamma$. ∎

The following result is an extension of Macaulay’s duality to finite
codimension sub-${\mathbf{k}}$-algebras $B\subset\Gamma$.

######  Theorem 3.9.

Given a non-negative integers $\delta>0$ and $c\geq\delta+1$, there is a one-
to-one correspondence $\perp$ between the following sets:

  1. (1)

sub-${\mathbf{k}}$-algebras $B$ of $\Gamma$ of codimension $\delta$ as
${\mathbf{k}}$-vector spaces such that the conductor of $B\subset\Gamma$ is
$(t^{c})$,

  2. (2)

algebra forming, with respect to $\Gamma$, ${\mathbf{k}}$-vector subspace
$V\subset\Delta$ of dimension $\delta$, generated by polynomials of degree at
most $c-1$ and such that there is a polynomial $g\in V$ with
$\operatorname{deg}(g)=c-1$.

This correspondence is inclusion reversing: given two
sub-${\mathbf{k}}$-algebras $B_{1}$ and $B_{2}$ of $\Gamma$, $B_{1}\subset
B_{2}$ if and only if $B_{2}^{\perp}\subset B_{1}^{\perp}$.

###### Proof.

Let $B$ be a sub-${\mathbf{k}}$-algebra $B$ of $\Gamma$. Since we have a non-
singular ${\mathbf{k}}$-bilinear pairing:

$\begin{array}[]{
cclc}\perp:&B^{\perp}\times\frac{\Gamma}{B}&\longrightarrow&{\mathbf{k}}\\\
&(g,\overline{f})&\mapsto&g\perp f\end{array}$

we get that $B^{\perp}$ is a ${\mathbf{k}}$-vector subspace of dimension
$\delta$ of $\Delta$. By definition $B^{\perp}$ is algebra-forming with
respect to $\Gamma$. Being $c$ the conductor we have $(t^{c})\subset B$, so
$\operatorname{deg}(g)\leq c-1$ for all $g\in B^{\perp}$ and there exist $g\in
B^{\perp}$ of degree $c-1$.

Let $V$ be an algebra forming, with respect to $\Gamma$, ${\mathbf{k}}$-vector
subspace satisfying the conditions of (2). Let us consider the
${\mathbf{k}}$-algebra $B=\operatorname{Ann}(V)$. From the perfect pairing (1)
we get that the codimension of $B$ in $\Gamma$ is $\delta$. Since $V$ is
generated by polynomials of degree at most $c-1$ we have that $(t^{c})\subset
B$, so the conductor of $B$ is at most $c$. Furthermore, since there is $g\in
V$ with $\operatorname{deg}(g)=c-1$ we deduce that $c$ is the conductor of
$B$.

It is straightforward to prove the inclusion reversing from the definition of
the inverse system $B^{\perp}$. ∎

We end this section by describing the ${\mathbf{k}}$-linear maps
$B_{2}^{\perp}\longrightarrow B_{1}^{\perp}$ induced by ${\mathbf{k}}$-algebra
isomorphisms $B_{1}\longrightarrow B_{2}$ between two finite codimension
${\mathbf{k}}$-algebras $B_{1}$ and $B_{2}$ of $\Gamma$. Let $c$ be an integer
bigger than the conductors of $B_{1}$ and $B_{2}$.

The perfect pairing $(\ref{pairing})$ induce a perfect pairing

$\begin{array}[]{ lclc}\perp:&\Delta_{\leq
c-1}\times\frac{\Gamma}{(t^{c})}&\longrightarrow&{\mathbf{k}}\\\
&(g,\overline{f})&\mapsto&g\perp f=(g\circ f)(0)\end{array}$

where $\Delta_{\leq c-1}$ is the ${\mathbf{k}}$-vector space of polynomials of
degree at most $c-1$. We consider the usual ${\mathbf{k}}$-vector basis of
$\Gamma/(t^{c})$ of the cosets of $t^{i}$, $i=0,\dots,c-1$. Its dual basis is
$\frac{1}{i!}u^{i}$, $i=0,\dots,c-1$, since

$\left(\frac{1}{i!}u^{i}\right)\perp t^{j}=\delta_{i,j}$

$1\leq i,j\leq c-1$.

The ${\mathbf{k}}$-algebra $B_{i}$ has conductor at most $c$ so we can
consider that $B_{i}\subset\Gamma/(t^{c})$, $i=1,2$. On the other hand, from
Proposition 3.2 we have that $B_{i}^{\perp}\subset\Delta_{\leq c-1}$, $i=1,2$.

If $B_{1}$ is isomorphic to $B_{2}$ by $\phi$ then their normalizations are
isomorphic:

$\Gamma=\overline{B_{1}}\overset{\overline{\phi}}{\cong}\overline{B_{2}}=\Gamma.$

This automorphism is determined by a power series $h(t)\in(t)$ such that
$u\perp h\neq 0$ and

$\begin{array}[]{ lclc}\overline{\phi}:&\Gamma&\longrightarrow&\Gamma\\\
&f&\mapsto&f(h).\end{array}$

Then we have an isomorphism of ${\mathbf{k}}$-vector spaces

$\frac{\Gamma}{B_{1}}\overset{\overline{\phi}}{\longrightarrow}\frac{\Gamma}{B_{2}}$

and the perfect pairing induces a ${\mathbf{k}}$-vector isomorphism

$\phi^{*}:B_{2}^{\perp}\longrightarrow B_{1}^{\perp}.$

The matrix $M_{\phi}$ associated to $\phi$ in the basis ${t^{i}}$,
$i=0,\dots,c-1$, is the $c\times c$ matrix whose columns are the coefficients
of $\phi({t}^{i})={h}^{i}$, $i=0,\dots,c-1$, with respect to this basis. Hence
the matrix of $\phi^{*}:B_{2}^{*}=B_{2}^{\perp}\longrightarrow
B_{1}^{*}=B_{1}^{\perp}$ with respect to the basis $\frac{1}{i!}u^{i}$,
$i=0,\dots,c-1$, is the transpose matrix ${}^{\tau}M_{\phi}$ of $M_{\phi}$.

######  Example 3.10.

Let $B_{2}\subset\Gamma$ be a ${\mathbf{k}}$-algebra generated by two elements
$f_{1}$, $f_{2}$ with $v_{t}(f_{1})=2$ and $v_{t}(f_{2})=7$. We may assume
that $f_{1}=t^{2}+$monomials of higher degree. Then $B_{2}$ is of finite
codimension $\delta=3$ and conductor $c=6$.

Since $\Gamma$ is complete there exist a power series $h\in(t)$ such that
$h^{2}=f_{1}$; we write $h=t+h_{2}t^{2}+\dots+h_{5}t^{5}+\dots$. Notice that
$\Gamma={\mathbf{k}}[\\![h]\\!]$.

Let ${\phi}$ the automorphism of $\Gamma$ defined by $h$, i.e.
${\phi}(f)=f(h)$. Then $\phi^{-1}(B_{2})$ is a ${\mathbf{k}}$-algebra $B_{1}$
generated by $f_{1}^{\prime}=t^{2}$ and $f_{2}^{\prime}(h)$ such that
$v_{h}(f_{2}^{\prime})=7$. After a change of generators $B_{1}$ is generated
by $f_{1}^{\prime}=t^{2}$ and $f_{2}^{\prime}=t^{7}$.

The induced isomorphism $\phi:B_{1}\longrightarrow B_{2}$ has the following
$6\times 6$ associated matrix with respect the basis ${t^{i}}$, $i=0,\dots,5$,

$M_{\phi}=\left(\begin{array}[]{cccccc}1&0&0&0&0&0\\\ 0&1&0&0&0&0\\\
0&h_{2}&1&0&0&0\\\ 0&h_{3}&2h_{2}&1&0&0\\\
0&h_{4}&2h_{3}+h_{2}^{2}&3h_{2}&1&0\\\
0&h_{5}&2b_{4}+2h_{2}h_{3}&3h_{3}+3h_{2}^{2}&4h_{2}&1\\\ \end{array}\right)$

Then the matrix of the isomorphism $\phi^{*}:B_{2}^{\perp}\longrightarrow
B_{1}^{\perp}$ with respect to $\frac{1}{i!}u^{i}$, $i=0,\dots,5$, is
$M_{\phi}^{\tau}$. Since $B_{1}$ is the monomial ${\mathbf{k}}$-algebra
${\mathbf{k}}[\\![t^{2},t^{7}]\\!]$, the ${\mathbf{k}}$-vector space
$B_{1}^{\perp}$ is generated by $u,u^{3},u^{5}$. From this we can compute
$B_{2}^{\perp}$ by considering $(^{\tau}M_{\phi})^{-1}$.

##  4\. Algebra-forming vector spaces

The first goal of this section is to characterize the algebra-forming
${\mathbf{k}}$-vector spaces.

######  Proposition 4.1.

Let $B$ be a ${\mathbf{k}}$-sub-algebra of finite codimension of $\Gamma$ with
conductor $c$, and let $f_{1},\dots,f_{s}$ be a system of generators of $B$.
Given an integer $d\geq c-1$, let $h_{1},\dots,h_{m}$ be a system of
generators of $W(\\{f_{1},\dots,f_{s}\\},d)$.

Let $V$ be a dimension $\delta$ ${\mathbf{k}}$-vector subspace of
$(u)\subset\Delta$ generated by polynomials of degree at most $d-1$. Let
$g_{1},\dots,g_{\delta}\in V$ be a basis of $V$.

Then $V$ is algebra-forming with respect to $B$ iff for all $r$-upla
$(\lambda_{1},\dots,\lambda_{m})\in{\mathbf{k}}^{m}$ such that

(3) $\sum_{j=1}^{m}\lambda_{j}(g_{i}\perp h_{j})=0$

for all $i=1,\dots,\delta$, then

(4) $\sum_{j=1}^{m}\lambda_{j}^{2}(g_{i}\perp h_{j}^{2})+2\sum_{j=1,l=1,j\neq
l}^{m}\lambda_{j}\lambda_{j}(g_{i}\perp h_{j}h_{l})=0$

for all $i=1,\dots,\delta$.

###### Proof.

From Proposition 3.2 we have to prove that for all $f\in B$ such that $g\perp
f=0$ for all $g\in V$ we have that $g\perp f^{2}=0$ for all $g\in V$. Since
the polynomials of $V$ are of degree at most $d-1$ we only have to prove that
for all $f\in W=W(\\{f_{1},\dots,f_{s}\\},d)$ such that $g\perp f=0$ for all
$g\in V$, we have that $g\perp f^{2}=0$ for all $g\in V$.

A general element of $W$ can be written as $f=\sum_{j=1}^{m}\lambda_{j}h_{j}$.
Hence the condition $g_{i}\perp f=0$ is equivalent to

$\sum_{j=1}^{m}\lambda_{j}(g_{i}\perp h_{j})=0$

for all $i=1,\dots,\delta$. Similarly, the condition $g_{i}\perp f^{2}=0$ is
equivalent to

$\sum_{j=1}^{m}\lambda_{j}^{2}(g_{i}\perp h_{j}^{2})+2\sum_{j=1,l=1,j\neq
l}^{m}\lambda_{j}\lambda_{j}(g_{i}\perp h_{j}h_{l})=0$

for all $i=1,\dots,\delta$. ∎

######  Remark 4.2.

The set of points
$(\lambda_{1},\dots,\lambda_{m})\in\mathbb{P}^{m-1}_{{\mathbf{k}}}$ satisfying
the identities of (3) form a linear subvariety $L$, and the points satisfying
the identities of (4) defines a subvariety
$Q\subset\mathbb{P}^{m-1}_{{\mathbf{k}}}$ intersection of $\delta$ quadrics.
Hence $V$ is algebra forming with respect to $B$ iff $L\subset Q$. This is a
computable condition.

######  Definition 4.3.

Let $B$ be a sub-${\mathbf{k}}$-algebra of finite codimension $\delta$ of
$\Gamma$ and conductor $c$. Let $D$ be the semigroup of $B$; we write the set
$t^{\mathbb{N}\setminus D_{B}}=\\{t^{i};i\in\mathbb{N}\setminus D_{B}\\}$ as
$g_{1}=t^{c-1},\dots,g_{\delta}=t$. Then we define the so-called standard
filtration of $B$ as follows: $B_{i}$ is the ${\mathbf{k}}$-algebra generated
by B and $g_{1},\dots,g_{i}$ for $i=1,\dots,\delta$; we set $B_{0}=B$. Notice
that $B_{\delta}=\Gamma$ and that we have

$B=B_{0}\subset B_{1}\subset\dots\subset B_{\delta}=\Gamma$

and $\operatorname{dim}_{{\mathbf{k}}}(B_{i+1}/B_{i})=1$,
$i=0,\dots,\delta-1$.

After the definition of standard filtration we only have to consider algebra-
forming elements $g\in\Delta$, with respect a suitable
sub-${\mathbf{k}}$-algebras of $\Gamma$, in order to define a
${\mathbf{k}}$-algebra recursively. The algebra-forming elements are not
unique as the following example shows.

######  Example 4.4.

Let us consider the Example 3.7. The standard filtration of $B$ is:

$B={\mathbf{k}}[\\![t^{3}+t^{4},t^{5}]\\!]\subset
B_{1}={\mathbf{k}}[\\![t^{3}+t^{4},t^{5},t^{7}]\\!]\subset
B_{2}={\mathbf{k}}[\\![t^{3},t^{4},t^{5}]\\!]\subset
B_{3}={\mathbf{k}}[\\![t^{2},t^{3}]\\!]\subset\Gamma.$

The chain of ${\mathbf{k}}$-algebras is defined as follows. The cosets of
$t,t^{2},t^{4},t^{7}$ in $\Gamma/B$ form a basis of $\Gamma/B$ as
${\mathbf{k}}$-vector space. Then $B_{1}$ is the ${\mathbf{k}}$-algebra
generated by $B$ and $t^{7}$, $B_{2}$ is the ${\mathbf{k}}$-algebra generated
by $B_{1}$ and $t^{4}$, $B_{3}$ is the ${\mathbf{k}}$-algebra generated by $B$
and $t^{2}$, and finally $\Gamma$ is the ${\mathbf{k}}$-algebra generated by
$B$ and $t$.

We know that $B^{\perp}$ is a $4$ dimensional ${\mathbf{k}}$-vector space
generated by $u,u^{2},u^{3}-\frac{1}{4}u^{4},u^{6}-\frac{1}{2.7}u^{7}$; we
have $B_{3}=\operatorname{Ann}\langle u\rangle$,
$B_{2}=\operatorname{Ann}\langle u^{2}\rangle\cap B_{3}$,
$B_{1}=\operatorname{Ann}\langle u^{3}-\frac{1}{4}u^{4}\rangle\cap B_{2}$,
$B=\operatorname{Ann}\langle u^{6}-\frac{1}{2.7}u^{7}\rangle\cap B_{1}$. On
the other hand, the ${\mathbf{k}}$-algebra
$C_{1}={\mathbf{k}}[\\![t^{3}+t^{5},t^{4}]\\!]\subset B_{1}$ can be obtained
as

$C_{1}=\operatorname{Ann}\langle u^{3}-\frac{1}{4.5}u^{5}\rangle\cap B_{2},$

i.e. $u^{3}-\frac{1}{4.5}u^{5}$ is an algebra-forming element with respect to
$B_{2}$. Notice that $B_{1}$ and $C_{1}$ are non analytically isomorphic
codimension one ${\mathbf{k}}$-algebras of $B_{2}$ .

Next we show how to build the standard filtration by using derivations.

######  Proposition 4.5.

Let $C\subset B$ be two sub-${\mathbf{k}}$-algebras of $\Gamma$ such that
$\operatorname{dim}_{{\mathbf{k}}}(B/C)=1$. There exist $\alpha\in
Der_{{\mathbf{k}}}(B)$ such that $\operatorname{Ker}(\alpha)=C$.

###### Proof.

If we denote by ${\mathfrak{m}}_{B}$ the maximal ideal of $B$ then
${\mathfrak{m}}_{C}\subset{\mathfrak{m}}_{B}$,
$\operatorname{dim}_{{\mathbf{k}}}({\mathfrak{m}}_{B}/{\mathfrak{m}}_{C})=1$
and ${\mathfrak{m}}_{B}^{2}\subset{\mathfrak{m}}_{C}$. Since we have

$\frac{{\mathfrak{m}}_{C}}{{\mathfrak{m}}_{B}^{2}}\subset\frac{{\mathfrak{m}}_{B}}{{\mathfrak{m}}_{B}^{2}}$

we deduce that there exists a linear form
$\alpha:\frac{{\mathfrak{m}}_{B}}{{\mathfrak{m}}_{B}^{2}}\longrightarrow{\mathbf{k}}$
such that
$\operatorname{Ker}(\alpha)=\frac{{\mathfrak{m}}_{C}}{{\mathfrak{m}}_{B}^{2}}$.
From this we get the claim. ∎

######  Corollary 4.6.

Let $B$ be a sub-${\mathbf{k}}$-algebra of finite codimension $\delta$ of
$\Gamma$. Let us consider the standard filtration of $B$:

$B=B_{0}\subset B_{1}\subset\dots\subset B_{\delta}=\Gamma.$

For all $i=1,\dots,\delta$ there exists a derivation $\partial_{l_{i}}\in
Der_{{\mathbf{k}}}(B_{i})$, $l_{i}\in{\mathfrak{m}}_{B_{i}}$, such that
$\operatorname{Ker}(\partial_{l_{i}})=B_{i}$.

######  Example 4.7.

Let us consider the Example 4.4. The element $u^{\perp}$ corresponds to the
derivation $\partial_{t}$ of $\Gamma$ defined by $t$, so
$B_{3}=\operatorname{Ker}(\partial_{t})$. The maximal ideal of $B_{3}$ is
minimally generated by $t^{2},t^{3}$, the element $(u^{2})^{\perp}$ is the
derivation $\partial_{t^{2}}\in Der_{{\mathbf{k}}}(B_{3})$, so
$B_{2}=\operatorname{Ker}(\partial_{t^{2}})$. The maximal ideal of $B_{2}$ is
minimally generated by $t^{3},t^{4},t^{5}$. The element
$(u^{3}-\frac{1}{4}u^{4})^{\perp}$ is the derivation
$\partial_{t^{3}-\frac{1}{4}t^{4}}\in Der_{{\mathbf{k}}}(B_{2})$, so
$B_{1}=\operatorname{Ker}(\partial_{t^{3}-\frac{1}{4}t^{4}})$. Finally,
$\partial_{t^{7}}\in Der_{{\mathbf{k}}}(B_{1})$ and
$B=\operatorname{Ker}(\partial_{t^{7}})$.

##  5\. Monomial algebras

In this section we first compute the inverse system of a monomial
${\mathbf{k}}$-algebra. After this, we characterize monomial Gorenstein curve
singularities in terms of its inverse system. We end the section relating the
inverse system of a curve singularity with its generic plane projection and
its saturation.

The following result it is easy to deduce from the proof of the second part of
Proposition 3.2(2).

######  Proposition 5.1.

Let $D$ be an additive sub-semigroup of $\mathbb{N}$ with finite complement.
Then $B^{\perp}$ is the ${\mathbf{k}}$-vector space generated by:
$g_{i}=u^{i}$ for $i\in\mathbb{N}\setminus D$.

######  Example 5.2.

Let $B$ be a sub-${\mathbf{k}}$-algebra of ${\mathbf{k}}[\\![t]\\!]$ of
codimension $\delta=1$. Then $B$ is the ${\mathbf{k}}$-algebra
$B={\mathbf{k}}[\\![D]\\!]$ where $D$ is the sub-semigroup of $\mathbb{N}$
generated by $2,3$. Hence $B^{\perp}$ is the ${\mathbf{k}}$-vector space
generated by $u$, i.e. $B$ is the set of power series $f=\sum_{i\geq
0}b_{i}t^{i}\in{\mathbf{k}}[\\![t]\\!]$ with $u\perp f=b_{1}=0$. See [26]
Example b), Section 4 of Chapter IV, and [18, Section 22].

######  Example 5.3.

Assume now that $B$ is sub-${\mathbf{k}}$-algebra of ${\mathbf{k}}[\\![t]\\!]$
of codimension $\delta=2$. Then its semi-group $D_{B}$ is $D_{1}=\langle
2,5\rangle$ or $D_{2}=\langle 3,4\rangle$. In the first case $B$ is generated
as ${\mathbf{k}}$-algebra by $f_{1}=t^{2}+b_{3}t^{3}$ and $f_{2}=t^{5}$. The
conductor is $c=4$. Then $B^{\perp}$ is generated by $g_{1}=u$,
$g_{2}=6b_{3}u^{2}+u^{3}$. In the second case $B$ is the monomial
${\mathbf{k}}$-algebra $B={\mathbf{k}}[\\![D_{2}]\\!]$ so $B^{\perp}$ is the
sub-${\mathbf{k}}$-algebra generated by $g_{1}=u$ and $g_{2}=u^{2}$. The
conductor is $c=5$. See [18, Section 23]. It is known that the algebras of the
first case are all analytically isomorphic to ${\mathbf{k}}[\\![D_{1}]\\!]$.

The inverse system of a monomial Gorenstein ${\mathbf{k}}$-algebra case can be
handled. Let us recall the definition of symmetric semi-group and the
celebrate result of Kunz.

######  Definition 5.4.

We say that a sub-semigroup $D$ of $\mathbb{N}$ such that
$\\#(\mathbb{N}\setminus D)<\infty$ and with conductor $c$ is symmetric if the
condition $t\in D$ is equivalent to $c-1-t\notin D$.

Kunz proved that the ring ${\mathbf{k}}[\\![D]\\!]$ is Gorenstein ring if and
only if $D$ is a symmetric semigroup,[21]. This symmetry is inherited by
$B^{\perp}$.

######  Proposition 5.5.

Let $D$ be a sub-semigroup of $\mathbb{N}$ such that $\\#(\mathbb{N}\setminus
D)<\infty$ and conductor $c$. The following conditions are equivalent:

  1. (1)

${\mathbf{k}}[\\![D]\\!]$ is Gorenstein,

  2. (2)

for all $g\in{\mathbf{k}}[\\![D]\\!]^{\perp}$ it holds
$t^{c-1}g(1/t)\in{\mathbf{k}}[\\![D]\\!]$.

###### Proof.

Since $B={\mathbf{k}}[\\![D]\\!]$ is a monomial ${\mathbf{k}}$-algebra we know
that $B^{\perp}$ is generated by $g=\sum_{i=1}^{c-1}a_{i}u^{i}$ such that
$a_{i}=0$ for $i\in D$, Proposition 5.1. Then the exponents of the non zero
terms of $t^{c-1}g(1/t)$ are $c-1-i$ with $i\notin D$. Then the claim is
equivalent to the symmetry of $D$, i .e. the Gorensteinness of $B$. ∎

######  Example 5.6.

Let $D$ be the semigroup generated by $4,6,$ and $9$. This is a symmetric
semigroup with conductor $c=12$. The algebra $B={\mathbf{k}}[\\![D]\\!]$ is
Gorenstein and isomorphic to ${\mathbf{k}}[\\![x,y,z]\\!]/I$ where
$I=(x^{3}-y^{2},y^{3}-z^{2})$. Then $B^{\perp}$ is generated by the
polynomials $g=a_{1}u+a_{2}u^{2}+a_{3}u^{3}+a_{5}u^{5}+a_{7}u^{7}$,
$a_{i}\in{\mathbf{k}}$. The polynomials
$t^{11}g(1/t)=a_{1}t^{10}+a_{2}t^{9}+a_{3}u^{8}+a_{4}u^{6}+a_{5}u^{4}$ have
all exponents in $D$. The ${\mathbf{k}}$-vector space $B^{\perp}$ is generated
by the following elements
$g_{1}=u,g_{2}=u^{2},g_{3}=u^{3},g_{4}=u^{5},g_{5}=u^{7}$.

Given a finite codimension subalgebra $B$ of $\Gamma$ we consider the curve
singularity $X=\operatorname{Spec}(B)$ defined by $B$. Let $X^{\prime}$ be the
generic plane projection of $X$, [3], and let $\widetilde{X}$ be the
saturation of $X$, [28] and the references therein. We have

${\mathcal{O}}_{X^{\prime}}\subset{\mathcal{O}}_{X}=B\subset{\mathcal{O}}_{\widetilde{X}}\subset\Gamma$

and then

${\mathcal{O}}_{\widetilde{X}}^{\perp}\subset
B^{\perp}\subset{\mathcal{O}}_{X^{\prime}}^{\perp}.$

We have, [9],

$\delta(\widetilde{X})\leq\delta(X)\leq\delta(X^{\prime})\leq(e_{0}(X)-1)\delta(\widetilde{X})-\binom{e_{0}(X)-1}{2}$

From [27, Proposition 1.6, pag. 971] we know that $\widetilde{X}$ is also the
saturation of $X^{\prime}$.

On the other hand $\widetilde{X}$ is a monomial curve singularity. Assume that
the coset of $x_{1}$ in $B$ is $t^{e_{0}}$ with $e_{0}$ the multiplicity of
$B$. Since the rings are complete and the ground field is algebraically
closed, we can assumed it after a suitable election of the uniformization
parameter of $\Gamma$. Let $\\{e_{0};\beta_{1},\dots,\beta_{g}\\}$ be the
characteristic of $X^{\prime}$, [28, Section 3, pag. 993], then
${\mathcal{O}}_{\widetilde{X}}$ is the monomial subalgebra with generators:

$\left\\{\begin{array}[]{ll}t^{e_{0}},&\hbox{}\\\ t^{s_{\nu}n_{\nu+1}\cdots
n_{g}},&\hbox{}m_{\nu}\leq s_{\nu}\leq[m_{\nu+1}/n_{\nu+1}],\nu=1,\dots,g-1\\\
t^{m_{g}+i},&\hbox{}0\leq i\leq e_{0}-1\end{array}\right.$

where $\beta_{\nu}/e_{0}=m_{\nu}/n_{1}\dots n_{\nu}$ is the $\nu$-th
characteristic exponent of $X^{\prime}$, $\nu=1,\dots,g-1$, and
$\gcd(m_{i},n_{i})=1$ for all $i=1,\dots,g$, [28, Section 3, pag. 995].

The facts ${\mathcal{O}}_{\widetilde{X}}^{\perp}\subset B^{\perp}$ and
Proposition 5.2 can be useful in order to simplify the computation of
$B^{\perp}$ as the next example shows.

######  Example 5.7.

Let us consider the ${\mathbf{k}}$-algebra
$B={\mathbf{k}}[\\![t^{6},t^{8}+t^{11},t^{10}+t^{13}]\\!]$; its saturation is
$\widetilde{B}={\mathbf{k}}[\\![t^{6},t^{8},t^{10},t^{11},t^{13},t^{15}]\\!]$,
[6, Example 2.5.1]. The sequence of multiplicities of the resolution of
$X=\operatorname{Spec}(B)$ is $\\{6,2,2,2,2,1,\dots\\}$. We can compute
$\delta(X)$ by computing $e_{1}(C)$ where $C$ ranges the local rings of the
resolution process, in this case we get $\\{8,1,1,1,1,0,\dots\\}$, so
$\delta(X)=12$. The semigroup of $B$ is
$D=\\{0,6,8,10,12,14,16,18,19,20,22\rightarrow\\}$, i.e. the conductor of $D$
is $22$.

On the other hand the semigroup of ${\mathcal{O}}_{\widetilde{X}}$ is
$\\{0,6,8,10\longrightarrow\\}$, its conductor is $10$. Hence
${\mathcal{O}}_{\widetilde{X}}^{\perp}$ is generated by $u^{i}$ with
$i\in\\{1,2,3,4,5,7,9\\}$, and $B^{\perp}$ is the set of polynomials
$g=\sum_{i=0}^{21}a_{i}u^{i}$ such that $a_{6}=0$, $990a_{11}-a_{8}=0$,
$a_{12}=0$, $1716a_{13}-a_{10}=0$, $a_{16}=0$, $4080a_{17}-a_{14}=0$,
$a_{18}=a_{19}=a_{20}=a_{21}=0$.

##  6\. The canonical module

As in the Artin case we can relate the canonical module with the inverse
system. In that case we have that if $I$ is an Artinian ideal then
$I^{\perp}\cong E_{R/I}({\mathbf{k}})\cong\omega_{R/I}$, [4], [14]. In the
case of branches we can determine the ”negative” part of the canonical module.

Let $X$ be a branch of $({\mathbf{k}}^{n},0)$ and $\overline{X}$ its
normalization. We first describe the canonical module $\omega_{X}$ by using
Rosenlicht’s regular differential forms, [26, IV 9], [5, Section 1], see also
[13]. We denote by $\Omega_{\overline{X}}(p)$ the set of meromorphic forms in
$\overline{X}$ with a pole at most in $p=\nu^{-1}(0)$. Then Rosenlicht’s
differential forms are defined as follows: $\omega^{R}_{X}$ is the set of
$\nu_{*}(\alpha)$, $\alpha\in\Omega_{\overline{X}}(p)$, such that for all
$F\in{\mathcal{O}}_{X}$

${\rm res}_{p}(F\alpha)=0.$

Notice that we have a mapping that we also denote by

$d_{R}:{\mathcal{O}}_{X}\longrightarrow\Omega_{X}\longrightarrow\nu_{*}\Omega_{\overline{X}}\hookrightarrow\omega^{R}_{X}.$

In [1, Chap. VIII] it is proved that
$\omega_{X}\stackrel{{\scriptstyle\phi}}{{\cong}}\omega_{X}^{R}$ and
$d_{R}=\phi d$, where $d:{\mathcal{O}}_{X}\longrightarrow\omega_{X}$ is the
map defined in the Section 1. Since ${\mathcal{O}}_{X}$ is a one-dimensional
reduced ring we know that $\omega_{(X,0)}$ is a sub-${\mathcal{O}}_{X}$-module
of $\mathrm{tot}({\mathcal{O}}_{X})$, [4, 3.3.18]. There is a the perfect
pairing, [26, Chapter IV],

$\begin{array}[]{ccccc}\frac{\nu_{*}{\mathcal{O}}_{\overline{X}}}{{\mathcal{O}}_{X}}&\times&\frac{\omega_{(X,0)}}{\nu_{*}\Omega_{\overline{X}}}&\stackrel{{\scriptstyle\eta}}{{\longrightarrow}}&\mathbb{C}\\\
F&\times&\alpha&\longrightarrow&{\mathrm{res}}_{p}(F\alpha)\end{array}$

notice that for all $\lambda\in R$ it holds $\eta(\lambda
F,\alpha)={\mathrm{res}}_{p}(\lambda F\alpha)=\eta(F,\lambda\alpha).$

######  Proposition 6.1.

Let $X$ be a branch of $({\mathbf{k}}^{n},0)$ and $\overline{X}$ its
normalization. Then we have an isomorphism of the $\delta(X)$ dimensional
${\mathbf{k}}$-vector spaces:

$B^{\perp}\overset{\epsilon}{\cong}\frac{\omega_{X}}{\nu_{*}\Omega_{\overline{X}}}$

such that $\epsilon(g)$ is the coset defined by
$\alpha=\sum_{i=0}^{c-1}i!c_{i}t^{-i-1}$, for all
$g=\sum_{i=0}^{c-1}c_{i}u^{i}\in B^{\perp}$.

###### Proof.

We write $B={\mathcal{O}}_{X}$, $\Gamma=\nu_{*}{\mathcal{O}}_{\overline{X}}$,
and $\Omega_{\overline{X}}=\Gamma dt$. Then $\epsilon$ is the composition of
the isomorphisms induced by the above two perfect pairings

$B^{\perp}\overset{\epsilon_{1}}{\cong}\left(\frac{\Gamma}{B}\right)^{*}\overset{\epsilon_{2}}{\cong}\frac{\omega_{X}}{\nu_{*}\Omega_{\overline{X}}}$

Next we describe both morphims $\epsilon_{1},\epsilon_{2}$. Given $g\in
B^{\perp}$ we can write it as

$g=c_{0}+c_{1}u+\dots,c_{c-1}u^{c-1},$

so $\epsilon_{1}(g)$ is the linear form induced by
$\xi:\Gamma^{*}\longrightarrow{\mathbf{k}}$ defined by: if $f=\sum_{i\geq
0}a_{i}t^{i}\in\Gamma$ then

$\xi(f)=\sum_{i=0}^{c-1}i!a_{i}c_{i}.$

On the other hand, every $\alpha\in\omega_{X}$ can be written as
$\alpha=t^{n}h(t)dt$ with $n\in\mathbb{Z}$ and $h(t)\in\Gamma$ an invertible
series. From [13, Proposition 2.6] we get that
$\alpha=\sum_{i\geq-c}e_{i}t^{i}$ such that ${\rm{res}}_{0}(\alpha F)=0$ for
all $f\in B$. Given $f=\sum_{i\geq 0}a_{i}t^{i}\in\Gamma$ we have

${\rm{res}}_{0}(f\alpha)=\sum_{i=0}^{c-1}a_{i}e_{-i-1}$

so $\epsilon_{2}^{-1}(\alpha)$ is the linear form iduced by
$\xi^{\prime}:\Gamma^{*}\longrightarrow{\mathbf{k}}$ defined by:

$\xi^{\prime}(f)=\sum_{i=0}^{c-1}a_{i}e_{-i-1}.$

From this we deduce that $e_{-i-1}=i!c_{i}$ for $i=0,\dots,c-1$. ∎

######  Example 6.2.

[Example 2.7, [13]] Let us consider the monomial curve $X$ with
parametrization $x_{1}=t^{4},x_{2}=t^{7},x_{3}=t^{9}$. We have $c=11$,
$\delta=6$. Then $\omega_{X}$ is the ${\mathbf{k}}$-vector space spanned by
$t^{-11},t^{-7},t^{-6},t^{-4},t^{-3},t^{-2},t^{n},n\geq 0$, and the quotient
$\omega_{X}/\nu_{*}\Omega_{\overline{X}}$ admits as ${\mathbf{k}}$-vector
space base the cosets of $t^{-11},t^{-7},t^{-6},t^{-4},t^{-3},$ $t^{-2}$, and
${\mathcal{O}}_{X}^{\perp}$ is the ${\mathbf{k}}$-vector space with basis
$u,u^{2},u^{3},u^{5},u^{6},u^{10}$.

## References

  * [1] A. Altman and S. Kleiman, _Introduction to Grothendieck duality theory_ , Lecture Notes in Mathematics, Vol. 146, Springer-Verlag, Berlin, 1970.
  * [2] J. Bertin and P. Carbonne, _Semi-groupes d’entiers et application aux branches_ , J. Algebra 49 (1977), no. 1, 81–95.
  * [3] J. Briançon, A. Galligo, and Granger M., _Deformations equisingulieres des germes de courbes gauches reduïtes_ , Mem. Soc. Math. de France 1 (1980), 1–69.
  * [4] W. Bruns and J. Herzog, _Cohen-Macaulay rings, revised edition_ , Cambridge Studies in Advanced Mathematics, vol. 39, Cambridge University Press, 1997.
  * [5] R.O. Buchweitz and G.M. Greuel, _The Milnor number and deformations of complex curve singularities_ , Inv. Math. 58 (1980), 241–281.
  * [6] A. Campillo and J. Castellanos, _Curve singularities. An algebraic and geometric approach_ , ActualitÃ©s MathÃ©matiques, 2005.
  * [7] S. D. Cutkosky, _Resolution of singularities_ , Graduate Studies in Mathematics, vol. 63, American Mathematical Society, Providence, RI, 2004.
  * [8] W. Decker, G.-M. Greuel, G. Pfister, and H. Schönemann, _Singular 4-3-0–A computer algebra system for polynomial computations_, http://www.singular.uni-kl.de, 2022.
  * [9] J. Elias, _An upper bound of the singularity order for the generic projection_ , J. of Pure and App. Math. 53 (1988), 267–270.
  * [10] by same author, _Characterization of the Hilbert-Samuel polynomials of curve singularities_ , Compositio Math. 74 (1990), 135–155.
  * [11] by same author, _The conjecture of Sally on the Hilbert function for curve singularities_ , J. Algebra 160 (1993), no. 1, 42–49.
  * [12] by same author, _On the deep structure of the blowing-up of curve singularities_ , Math. Proc. Cambridge Philos. Soc. 131 (2001), 227–240.
  * [13] by same author, _On the canonical ideals of one-dimensional Cohen-Macaulay local rings_ , Proc. Edinb. Math. Soc. (2) 59 (2016), no. 1, 77–90.
  * [14] by same author, _Inverse systems of local rings_ , Commutative Algebra and its Interactions to Algebraic Geometry. VIASM 2013-2014, Lecture Notes in Mathematics, vol. 2210, Springer, 2018, pp. 119–164.
  * [15] J. Elias and M. E. Rossi, _The structure of the inverse system of Gorenstein K-algebras_ , Advances in Math 314 (2017), 306–327.
  * [16] by same author, _A constructive approach to one-dimensional Gorenstein ${\mathbf{k}}$-algebras_, Trans. Amer. Math. Soc. 374 (2021), no. 7, 4953–4971.
  * [17] E. A. Gorin, _Subalgebras of finite codimension_ , Mat. Zametki 6 (1969), 321–328.
  * [18] R. Grönkvist, E. Leffler, A. Torstensson, and V. Ufnarovski, _Subalgebras in $K$[x] of small codimension_, Appl. Algebra Engrg. Comm. Comput. 33 (2022), no. 6, 751–789.
  * [19] S.K. Gupta and L.G. Roberts, _Cartesian squares and ordinary singularities of curves_ , Comm. in Algebra 11 (1983), no. 2, 127–182.
  * [20] A. Iarrobino and V. Kanev, _Power sums, Gorenstein algebras, and determinantal loci_ , Lecture Notes in Mathematics, vol. 1721, Springer-Verlag, Berlin, 1999, Appendix C by Iarrobino and Steven L. Kleiman.
  * [21] E. Kunz, _The value semigroup of a one-dimensional Gorenstein ring_ , Proc. Amer. Math. Soc. 25 (1970), 748–751.
  * [22] E. Matlis, _$1$ -dimensional Cohen-Macaulay rings_, Lecture Notes in Mathematics, Vol. 327, Springer-Verlag, Berlin-New York, 1973.
  * [23] D. J. Newman, _Point separating algebras of polynomials_ , Amer. Math. Monthly 81 (1974), 496–498.
  * [24] D.G. Northcott, _The neighbourhoods of a local ring._ , J. London Math. Soc. 30 (1955), 360–375.
  * [25] by same author, _The reduction number of a one-dimensional local ring_ , Mathematika 6 (1959), 87–90.
  * [26] J.P. Serre, _Groupes algébriques et corps de classes_ , Publications de l’institut de mathématique de l’université de Nancago, VII. Hermann, Paris, 1959.
  * [27] O. Zariski, _Studies in equisingularities III_ , Am. J. of Math. 90 (1965), 961–1023.
  * [28] by same author, _General theory of saturation and of saturated local rings. III. Saturation in arbitrary dimension and, in particular, saturation of algebroid hypersurfaces_ , Amer. J. Math. 97 (1975), 415–502.