# Disjoint Contrastive Regression Learning for Multi-Sourced Annotations

###### Abstract

Large-scale datasets are important for the development of deep learning
models. Such datasets usually require a heavy workload of annotations, which
are extremely time-consuming and expensive. To accelerate the annotation
procedure, multiple annotators may be employed to label different subsets of
the data. However, the inconsistency and bias among different annotators are
harmful to the model training, especially for qualitative and subjective
tasks. To address this challenge, in this paper, we propose a novel
contrastive regression framework to address the disjoint annotations problem,
where each sample is labeled by only one annotator and multiple annotators
work on disjoint subsets of the data. To take account of both the intra-
annotator consistency and inter-annotator inconsistency, two strategies are
employed. Firstly, a contrastive-based loss is applied to learn the relative
ranking among different samples of the same annotator, with the assumption
that the ranking of samples from the same annotator is unanimous. Secondly, we
apply the gradient reversal layer to learn robust representations that are
invariant to different annotators. Experiments on the facial expression
prediction task, as well as the image quality assessment task, verify the
effectiveness of our proposed framework.

Index Terms—  Contrastive, regression, disjoint annotation

##  1 Introduction

Large-scale labeled datasets are in heavy demand to boost the performance of
deep learning models. Such datasets often require a high cost of annotations,
which are extremely time-consuming and expensive. Moreover, a large-scale
dataset is usually labeled by multiple annotators. Due to individual
preferences, the labeled results are not always consistent among different
annotators, especially for qualitative and subjective tasks. As a result, to
reduce the annotator bias, each sample is often required to be labeled by
multiple annotators, followed by an average or a majority vote to obtain the
final labels, which leads to a heavier cost of annotation. To alleviate the
labeling effort, it is more efficient to assign disjoint subsets of the data
to individual annotators for labeling. Therefore, how to address the challenge
resulting from the inconsistency and bias among different annotators becomes a
tough task.

Some approaches [1, 2, 3, 4, 5, 6] deal with the noisy-label issue or labeling
biases from crowd-sourced annotations. Many advances [7, 8, 9, 10] have been
made in recent years, especially for classification tasks with crowd-sourced
labels. For regression tasks, except for the inconsistency among multiple
annotators, the intra-annotator consistency also needs to be considered, in
which the labeled rankings of samples from the same annotator usually align
well with the rankings of the latent ground truth scores. An example of this
phenomenon is shown in Fig. 1. How to take account of both intra-annotator
consistency and inter-annotator inconsistency for regression tasks is still
underexplored.

Fig. 1: Example of the intra-annotator consistency and inter-annotator
inconsistency phenomenon. Each box represents a sample from the dataset. The
blue color from dark to light represents the score from low to high. After
labeling, the intra-annotator consistency remains, while the inter-annotator
consistency is violated.

In this paper, we present a novel approach for regression tasks from disjoint
annotations with the designed disjoint margin ranking loss and gradient
reversal layer (GRL) [11]. Specifically, We extend the margin ranking loss
based on the facts of disjoint annotations, with both the intra-annotator
consistency and inter-annotator inconsistency adopted, leading to a robust
embedding framework. Besides that, regularization of the batch distribution is
employed to avoid the collapse to naive solutions. To learn annotator
invariant representations, GRL is further adopted in the model training. All
these make up the entire framework of our proposed Disjoint Contrastive
Regression (DCR). A demonstration of the framework is shown in Fig. 2. The
main contributions of this paper are summarized as follows:

  * •

We present a novel contrastive learning framework for regression tasks with
disjoint annotations.

  * •

We extend the margin ranking loss and resolve the intra-annotator consistency
and inter-annotator inconsistency.

  * •

We adopt GRL in the model training for learning annotator invariant
embeddings.

  * •

We conduct experiments on the facial expression prediction task and image
quality assessment task, demonstrating the effectiveness of the proposed DCR.

Fig. 2: The framework of disjoint contrastive regression (DCR). The input data
is first fed to the embedding net $f_{\boldsymbol{\theta}}$. The score is then
predicted by the regression head $g_{\boldsymbol{\phi}}$. The pairwise
distance of the prediction is optimized by the supervision of disjoint ranking
loss. In the meanwhile, the embeddings are fed to a second classification
branch $h_{\boldsymbol{\psi}}$ that aims to classify different annotators. We
adopt the gradient reversal layer (GRL) to learn the robust annotator-
invariant embeddings.

##  2 Related Work

Crowd-Sourced Annotations. The inter-and intra-observer variations among
experts due to intrinsic differences in human annotators widely occur in
computer vision tasks. Many approaches [1, 2, 3, 4, 5, 6, 12] have been
proposed to deal with the inconsistent label problems from the crowd-sourced
annotations. For example, [13] proposes an EM algorithm and a crowd layer to
deal with the crowd-sourcing problems. To solve the noise and inconsistency in
some limited-sized datasets from multiple annotators, [2] proposes a crowd-
sourced representation learning framework that generates a large number of
training instances from only small-sized labeled sets. However, most existing
works focus on classification tasks. How to deal with inconsistent annotations
with numerical labels for regression tasks is still underexplored.

Cross-Dataset Adaptation. Since data from source and target domains have
different distributions, the key challenge for domain adaptation [14, 15, 16,
17, 18, 19, 20] is to address the domain shift between them. Fine-tuning [21,
22], which diminishes the shift between the two domains, is a typical way for
domain adaptation with transfer learning [23, 24]. For instance, [22] proposes
an adaptive fine-tuning approach to find the optimal strategy per image for
the target data. Transfer learning with triplet loss [24] is employed for
bridging the gap between different domains. Adversarial learning is another
essential way for domain adaptation [14, 15, 25, 26, 27]. For example, [15]
presents a partial adversarial domain adaptation approach to alleviate
negative transfer by reducing the weights of outliers and promote positive
transfer. A multi-adversarial domain adaptation approach [14] is proposed to
capture multi-mode structures to ensure fine-grained alignment of
distributions. The domain-symmetric network [25] is presented on a symmetric
design of source and target classifiers with adversarial training. However,
such domain adaptation approaches cannot directly address the inconsistency
annotation problem raised from multi-sourced data.

##  3 Method

###  3.1 Problem Description

Assume the whole dataset $\mathcal{X}$ are labeled by $K$ annotators. Denote
the subset annotated by the $k$-th annotator as $\mathcal{X}_{k}$ and
different subsets are disjoint with each other, i.e.,
$\mathcal{X}_{k}\cap\mathcal{X}_{j}=\O$ if $j\neq k$ and
$\mathcal{X}=\cup_{k=1}^{K}\mathcal{X}_{k}$. Given the set of labeled samples,
we aim to learn a robust embedding network
$f_{\boldsymbol{\theta}}:\boldsymbol{x}\rightarrow\boldsymbol{z}$, as well as
a regression head $g_{\boldsymbol{\phi}}:\boldsymbol{z}\rightarrow y$, based
on the supervision of the disjoint and inconsistent annotation $\tilde{y}$,
where $\boldsymbol{x}$, $\boldsymbol{z}$ and $y$ are the input sample,
representation, and label, respectively.

###  3.2 Preliminary on Margin Ranking Loss

Margin ranking loss is commonly used for contrastive regression problems,
which is defined as follows,

$\displaystyle\mathcal{L}_{\text{rank}}=\frac{1}{N}\sum_{i,j}\left[\gamma-\boldsymbol{C}_{i,j}({y}_{i}-{y}_{j})\right]_{+},$
(1)

where $N$ is the batch size; $y$ is the prediction; $\boldsymbol{C}_{i,j}=1$
if the annotated scores satisfy $\tilde{y}_{i}\geq\tilde{y}_{j}$, otherwise
$\boldsymbol{C}_{i,j}=-1$; $\gamma$ is a pre-defined margin, and $[\cdot]_{+}$
clamps values to be non-negative. By minimizing the margin ranking loss in the
training, the predicted scores should follow the pairwise relative rankings of
the labeled scores.

###  3.3 Disjoint Contrastive Regression (DCR)

There are three drawbacks of the commonly used margin ranking loss. First, it
omits the issue caused by inter-annotator inconsistency. It is harmful to
optimize the relative ranking when two samples are labeled by different
annotators. Second, it can easily collapse to naive solutions by only
calculating the relative order with no absolute scores as reference. Third, it
does not consider the domain shift coming from different subsets. To deal with
the issues above, we firstly extend the margin ranking loss to the proposed
disjoint ranking loss with conditional annotator indicators, to avoid the
direct comparison across annotators. Then, regularization on the batch
predictions is employed to ensure relative ranking following a certainly
designed distribution. Thirdly, we reduce the preference bias across
annotators with the gradient reversal layer (GRL) with adversarial training,
leading to more bias-invariant representations. The proposed disjoint
contrastive regression (DCR) framework is shown in Fig. 2. The details are
demonstrated as follows.

1) Disjoint Margin Ranking Loss. Given a sample $\boldsymbol{x}_{i}$ annotated
by the $k$-th annotator, denote the annotated label as $\tilde{y}_{i}^{k}$,
and the latent unbiased true label as $y_{i}^{*}$. Usually, $y_{i}^{*}$ is not
accessible. Considering regression tasks, due to the preference bias of
different annotators, it is quite common that, for two samples annotated by
different annotators, we have $\tilde{y}_{i}^{k}<\tilde{y}_{j}^{l}$ while the
latent true label is $y_{i}^{*}>y_{j}^{*}$ in some situations. Such
inconsistency among disjoint annotations can be harmful to the model training.
Meanwhile, we assume the ranking of labels within the same annotator usually
keeps consistent. Based on the above two phenomenons, we should alleviate the
inconsistency for inter-annotator labels, and utilize the consistency of
intra-annotator labels for representation learning.

To this end, we extend the margin ranking loss Eq. (1) to the proposed
disjoint margin ranking loss as follows,

$\displaystyle\mathcal{L}_{\text{d-rank}}=\frac{1}{N}\sum_{i,j}\boldsymbol{M}_{i,j}\left[\gamma-\boldsymbol{C}_{i,j}({y}_{i}-{y}_{j})\right]_{+},$
(2)

and $\boldsymbol{M}_{i,j}$ is defined as,

$\displaystyle\boldsymbol{M}_{i,j}=\sum_{k}\boldsymbol{1}_{[i\in
A_{k}]}\boldsymbol{1}_{[j\in A_{k}]},$ (3)

where $\boldsymbol{1}_{[i\in A_{k}]}$ is the indicator function and returns 1
if the sample $i$ is from the $k$-th annotator $A_{k}$, otherwise it returns
0. $\boldsymbol{M}_{i,j}$ is the judgment of whether the sample $i$ and the
sample $j$ are labeled by the same annotator. In other words,
$\boldsymbol{M}_{i,j}$ ensures that the loss is optimized within the same
annotator. As a result, Eq. (2) will generate a penalty if the predicted
pairwise relative ranking is conflicted with the annotated pairwise relative
ranking. With Eq. (2), we can avoid cross-annotator inconsistency and utilize
the intra-annotator consistency at the same time.

2) Regularize on the Relative Ranking. Eq. (2) only compares the relative
order given two samples, which is unstable and easy to collapse to extreme
naive solutions. To address such a problem, we constrain the predictions to
follow the normal distributions $\mathcal{N}(0,1)$ within batch samples, i.e.,

$\mathcal{L}_{\text{reg}}=\left(\frac{1}{N}\sum_{i}{y}_{i}\right)^{2}+\left(\frac{\sum_{i}{y}_{i}^{2}}{N-1}-1\right)^{2},$
(4)

where the first term constrains the distribution to be zero-mean, while the
second term regularizes the variance to be close to 1. With the normal
distribution as a reference, the regularization term stabilizes the training,
which is verified in the experiment section.

3) Invariant Learning with GRL. To further improve the robustness of
regression performance across multiple biased disjoint annotations, bias-
invariant learning is crucial to ensure that the latent representations are
not annotator-dependent. As a result, we adopt one commonly used domain
adaptation strategy with the gradient reversal layer (GRL) [28] for learning
the latent embeddings. In addition to the embedding network
$f_{\boldsymbol{\theta}}:\boldsymbol{x}\rightarrow\boldsymbol{z}$ and the
regression head $g_{\boldsymbol{\phi}}:\boldsymbol{z}\rightarrow y$, we
consider another annotator classifier
$h_{\boldsymbol{\psi}}:\boldsymbol{z}\rightarrow\boldsymbol{p}$, where
$\boldsymbol{p}$ is the predicted probability for classifying annotators after
the soft-max operation. We consider two loss functions,

$\mathcal{L}_{\boldsymbol{\psi}}=-\frac{1}{N}\sum_{i}\boldsymbol{p}_{i}^{*}\log
h_{\boldsymbol{\psi}}\left(f_{\hat{\boldsymbol{\theta}}}(\boldsymbol{x}_{i})\right),$
(5)
$\mathcal{L}_{\boldsymbol{\theta}}=\frac{1}{N}\sum_{i}\boldsymbol{p}_{i}^{*}\log
h_{\hat{\boldsymbol{\psi}}}\left(f_{\boldsymbol{\theta}}(\boldsymbol{x}_{i})\right),$
(6)

where $\boldsymbol{p}^{*}\in\mathbb{R}^{K}$ is the one-hot vector as the
annotator indicator; $\hat{\boldsymbol{\theta}}$ and $\hat{\boldsymbol{\psi}}$
denote the frozen version of the parameters. In Eq. (5), we freeze the
embedding network and learn the classification head to minimize the cross-
entropy, while in Eq. (6), we freeze the classification head and optimize the
embedding network. Such adversarial training strategy learns bias-invariant
representations.

4) Combined Loss for Representation Learning. We summarize the total loss of
the proposed DCR as follows,

$\mathcal{L}=\mathcal{L}_{\text{d-rank}}+\lambda_{1}\mathcal{L}_{\text{reg}}+\lambda_{2}\mathcal{L}_{\boldsymbol{\psi}}+\lambda_{3}\mathcal{L}_{\boldsymbol{\theta}},$
(7)

where $\lambda_{1}\sim\lambda_{3}$ are the weights of individual losses, all
set to 0.2 as default.

##  4 Experiments

###  4.1 Implementation Details

####  4.1.1 Datasets

We evaluate the performance of our method on two regression tasks, namely
facial expression score prediction task with AFEW-AV [29] and Aff-Wild [30]
datasets, and image quality assessment (IQA) task with CSIQ [31] and TID2013
[32, 33] datasets. For all these four datasets, 80% samples are used for
training, while the remaining samples are used for testing. Some examples from
the datasets are shown in Fig. 3. We present the details of these four
datasets as follows.

AFEW-AV is a subset of the Acted Facial Expressions in the Wild (AFEW) [34]
dataset, focusing on 7 basic facial expressions, namely disgust, fear, anger,
sadness, happiness, surprised and neural. Apart from these classification
labels, AFEW-AV provides the discrete valence and arousal scores, ranging from
-10 to 10. This dataset contains 600 “in-the-wild” video clips with variations
of backgrounds and poses of the subjects. For simplicity, we use the
individual frames as distinct samples for the regression task.

Aff-Wild dataset consists of around 300 videos with 200 subjects, including
130 males and 70 females. This dataset also provides two continuous emotion
dimensions, in terms of valence and arousal, which range from -1 to 1.

CSIQ dataset is based on 30 raw clean images. Six different types of
distortions are added to the clean images, resulting in 866 distorted images
in total. The quality score of each sample is also provided.

TID2013 is an expansion of Tampere Image Dataset 2008 (TID2008) [35], based on
LIVE dataset [36], with the increase of distorted level to 5 and type to 24.
Thus, 3000 distorted image samples are generated from 25 original samples.

Fig. 3: Selected examples with their corresponding scores, including the
images and their corresponding scores, from the datasets. The images on the
first line are samples for facial expression score prediction while those on
the second line are for IQA.

####  4.1.2 Training Details

To mimic the annotated scores generated from multiple annotators with
different biases, we randomly shift the mean of provided ground truth scores
for individual annotators. Meanwhile, we also perturb scores locally with the
relative pairwise ranking unchanged. We normalize the scores in the range [-1,
1] for the face expression score prediction task and [0, 1] for the IQA task
as the final annotated scores, respectively. We show an example of the
augmented score distributions from four annotators of the TID2013 dataset in
Fig. 4. We can see that there are large biases among different annotators.

For the face expression prediction task, we apply ResNet-50 [37] as our
backbone and use the stochastic gradient descent (SGD) optimizer with a
momentum of 0.9 and a weight decay of 5$\times 10^{-4}$. For both datasets,
scores of valence are used for regression. To avoid the influence by the
background, for the AFEW-AV dataset, face images are cropped and aligned by
the landmarks provided by the dataset, and then resized to 112 $\times$ 112,
while the detected and aligned face samples in the Aff-Wild dataset are
applied directly. During the training phase, we rotate the input images within
$\pm 45^{\circ}$ for augmentation. We train ResNet-50 on AFEW-AV for 400
epochs with an initial learning rate of $10^{-3}$, and for 800 epochs with the
same initial learning rate on Aff-Wild. For both datasets, we use a batch size
of 512.

For the image quality assessment task, samples are randomly cropped and
resized to 224 $\times$ 224 with a batch size of 64 for training, while the
testing samples are resized to 256 $\times$ 256 and centrally cropped to 224
$\times$ 224\. During the training phase, we augment the input samples within
$\pm 45^{\circ}$ rotation. The same as that in facial expression prediction
tasks, we also apply ResNet-50 as our backbone and use SGD optimizer with a
momentum of 0.9 and a weight decay of $5\times 10^{-4}$. The initial learning
rates for the CSIQ dataset and TID2013 are both $10^{-3}$ for 300 epochs. For
both of the two tasks, one Nvidia RTX 3090 GPU is used for training. The
method is implemented with the Pytorch toolbox [38].

Fig. 4: An example of the score distributions from four annotators of the
TID2013 dataset.

###  4.2 Evaluation Criteria

We adopt three evaluation metrics, namely pairwise ranking accuracy (PRA),
Spearman’s rank order correlation coefficient (SROCC), and Pearson’s linear
correlation coefficient (PLCC). Since annotation is inconsistent for
comparison across different annotators, we evaluate the results on individual
annotators and take the average as the final results.

####  4.2.1 Pairwise Ranking Accuracy (PRA)

PRA compares the relative pairwise ranking of the predicted scores with
annotated labels. For $N$ samples, we have $N(N-1)/2$ pairs in total. We treat
a pair as a correct prediction if the pairwise ranking of the predicted scores
from the two samples follows the same relative ranking as labeled annotations.
Then the PRA is calculated as the total accuracy given all the possible pairs.

####  4.2.2 Spearman’s Rank Order Correlation Coefficient (SROCC)

The SROCC is defined as follows,

$\text{SROCC = $1-\frac{6\sum_{i=1}^{N}d_{i}^{2}}{N(N^{2}-1)}$},$ (8)

where $d_{i}$ denotes the difference between the ranks of $i$-th testing
images in the ground truth and predicted quality scores.

####  4.2.3 Pearson’s Linear Correlation Coefficient (PLCC)

PLCC calculates the linear correlation between the annotation $\tilde{y}$ and
the predicted label $y$. The definition is shown as follow,

$\text{PLCC =
$\frac{\sum_{i=1}^{N}(\tilde{y}_{i}-\mu_{\tilde{y}})\times({y_{i}}-\mu_{{y}})}{\sqrt{\sum_{i=1}^{N}(\tilde{y}_{i}-\mu_{\tilde{y}}^{2})}\times\sqrt{\sum_{i=1}^{N}({y}_{i}-\mu_{{y}})^{2}}}$},$
(9)

where $\mu$ represents the sample mean. The range of PLCC is from -1 to 1, and
higher scores mean better performance.

###  4.3 Main Results

Evaluation on Facial Expression Score Prediction. To validate the
effectiveness of our proposed DCR framework, we compare with the regression
baseline, i.e., a direct regression with mean squared error (MSE) as the loss
function. The results are shown in Table 1, where the top and the bottom part
shows the results on the AFEW-AV and Aff-Wild datasets, respectively.
$\text{DCR}_{[-]}$ represents the DCR method without GRL in the training. The
best results are marked in bold. As expected, DCR achieves the best
performance.

Method |  PRA (%) |  SROCC |  PLCC
---|---|---|---
Baseline |  63.38 |  0.369 |  0.371
DCR[-] |  89.10 |  0.868 |  0.881
DCR |  89.22 |  0.875 |  0.890
Baseline |  62.28 |  0.336 |  0.348
DCR[-] |  83.35 |  0.812 |  0.827
DCR |  84.01 |  0.829 |  0.837
Table 1: Comparison of different approaches on AFEW-AV (top three) and Aff-
Wild (bottom three) datasets. The best performance is marked in bold.

Evaluation on Image Quality Assessment. Two state-of-the-art (SOTA) methods
for blind image quality assessment, including MetaIQA [39] and HyperIQA [40],
are chosen to compare with our proposed approach. The results on the CSIQ and
TID2013 datasets are shown in Table 2. MetaIQA+DCR and HyperIQA+DCR stand for
approaches with the same architecture used in the original methods with the
combination of the proposed DCR framework. We can see that there is a
consistent improvement on all three metrics with DCR adopted in the training.

Method |  PRA (%) |  SROCC |  PLCC
---|---|---|---
MetaIQA [39] |  92.46 |  0.897 |  0.918
MetaIQA+DCR |  93.91 |  0.916 |  0.932
HyperIQA [40] |  90.06 |  0.887 |  0.893
HyperIQA+DCR |  91.01 |  0.891 |  0.903
MetaIQA [39] |  83.40 |  0.807 |  0.829
MetaIQA+DCR |  85.59 |  0.825 |  0.835
HyperIQA [40] |  81.01 |  0.782 |  0.794
HyperIQA+DCR |  81.87 |  0.812 |  0.828
Table 2: Comparison with SOTA methods on the CSIQ (top four) and TID2013
(bottom four) datasets, with the best performance marked in bold.

###  4.4 Ablation Study

Study on the Regularization Loss. To evaluate the effectiveness of the
regularization term in DCR, we conduct experiments on AFEW-AV and TID2013 with
different choices of $\lambda_{1}$ from 0 to 0.3. The results are shown in
Table 3, where the top and the bottom part shows results on AFEW-AV and
TID2013, respectively. From Table 3, we achieve significantly higher
performance with the regularization term than the method without
regularization, i.e., $\lambda_{1}=0.0$, on both the two datasets. This
indicates that the model is easy to collapse to a naive solution without
regularization. The best performance is achieved when $\lambda_{1}$ is around
0.1$\sim$0.2.

$\lambda_{1}$ |  PRA (%) |  SROCC |  PLCC
---|---|---|---
0.0 |  63.15 |  0.311 |  0.323
0.1 |  89.22 |  0.875 |  0.890
0.2 |  88.51 |  0.864 |  0.876
0.3 |  86.14 |  0.842 |  0.853
0.0 |  62.47 |  0.337 |  0.349
0.1 |  71.78 |  0.697 |  0.701
0.2 |  72.36 |  0.706 |  0.712
0.3 |  67.33 |  0.435 |  0.446
Table 3: Regularization loss on AFEW-AV and TID2013 datasets, respectively,
with the best performance marked in bold.

Study on the Number of Annotators. Table 4 shows the comparison of the AFEW-AV
and TID2013 datasets between different numbers of annotators, ranging from one
to four. As expected, the best performance is achieved with a single annotator
since there is no inconsistent label issue with only one annotator. The
performance does not degrade significantly when increasing the number of
annotators, showing that it is still tolerant when combining disjoint
annotations for training in practical situations.

#Annotator |  PRA (%) |  SROCC |  PLCC
---|---|---|---
1 |  89.31 |  0.886 |  0.912
2 |  88.67 |  0.876 |  0.897
3 |  88.98 |  0.868 |  0.887
4 |  89.22 |  0.875 |  0.890
1 |  72.63 |  0.769 |  0.789
2 |  69.06 |  0.705 |  0.711
3 |  70.80 |  0.749 |  0.761
4 |  72.36 |  0.706 |  0.712
Table 4: Results with different number of annotators on AFEW-AV and TID2013
datasets, respectively, with the best performance marked in bold.

Study on the Disjoint Ranking Loss. Table 5 shows the comparison between the
margin ranking loss $\mathcal{L}_{\text{rank}}$ and the disjoint margin
ranking loss $\mathcal{L}_{\text{d-rank}}$ on the AFEW-AV and TID2013
datasets, respectively. Since $\mathcal{L}_{\text{rank}}$ does not consider
the inter-annotator biases, it is expected that $\mathcal{L}_{\text{d-rank}}$
outperforms $\mathcal{L}_{\text{rank}}$ with a large margin, i.e., a $19.42\%$
and $6.72\%$ improvement on the PRA for the AFEW-AV and TID2013 datasets,
respectively.

Loss |  PRA (%) |  SROCC |  PLCC
---|---|---|---
$\mathcal{L}_{\text{rank}}$ |  69.80 |  0.486 |  0.513
$\mathcal{L}_{\text{d-rank}}$ |  89.22 |  0.875 |  0.890
$\mathcal{L}_{\text{rank}}$ |  65.64 |  0.412 |  0.423
$\mathcal{L}_{\text{d-rank}}$ |  72.36 |  0.706 |  0.712
Table 5: Comparison between margin ranking loss $\mathcal{L}_{\text{rank}}$
and disjoint margin ranking loss $\mathcal{L}_{\text{d-rank}}$ on AFEW-AV and
TID2013 datasets, respectively, with the best performance marked in bold.

Study on Variant Margins. We also test the role of margin in our proposed
disjoint ranking loss. We vary the margin $\gamma$ from 0 to 0.2 on the AFEW-
AV and TID2013 datasets, as shown in Table 6. For both of the two datasets, we
achieve the best results when $\gamma=0$, i.e., no margin is used. With the
increase of the margin, the performance keeps degrading. Generally, the margin
is more appropriate in the classification task where samples from different
classes need to be separated as much as possible. However, for regression
tasks, two similar inputs should generate two similar scores, rather than
intentionally enlarging the distance of the predicted scores. As a result,
setting $\gamma=0$ is more suitable for regression tasks.

Margin $\gamma$ |  PRA (%) |  SROCC |  PLCC
---|---|---|---
0.0 |  89.22 |  0.875 |  0.890
0.1 |  88.43 |  0.841 |  0.887
0.2 |  87.35 |  0.857 |  0.885
0.0 |  72.36 |  0.706 |  0.712
0.1 |  71.40 |  0.702 |  0.716
0.2 |  69.64 |  0.679 |  0.688
Table 6: Results with variant margins in our disjoint ranking loss on the
AFEW-AV and TID2013 datasets, respectively, with the best performance marked
in bold.

##  5 Conclusion

In this paper, we propose a novel disjoint contrastive regression (DCR)
framework to address the challenges of the inconsistent label problems due to
the annotation bias. Experimental results with extensive studies demonstrate
the effectiveness of our proposed DCR framework. In our future work, we will
consider the video-based regression tasks with weak labels for disjoint
annotations.

## References

  * [1] Filipe Rodrigues and Francisco Pereira,  “Deep learning from crowds,”  in Proceedings of the AAAI Conference on Artificial Intelligence, 2018, vol. 32.
  * [2] Guowei Xu, Wenbiao Ding, Jiliang Tang, Songfan Yang, Gale Yan Huang, and Zitao Liu,  “Learning effective embeddings from crowdsourced labels: An educational case study,”  in 2019 IEEE 35th International Conference on Data Engineering (ICDE). IEEE, 2019, pp. 1922–1927.
  * [3] Wentao Wang, Guowei Xu, Wenbiao Ding, Yan Huang, Guoliang Li, Jiliang Tang, and Zitao Liu,  “Representation learning from limited educational data with crowdsourced labels,”  IEEE Transactions on Knowledge and Data Engineering, 2020.
  * [4] Ryutaro Tanno, Ardavan Saeedi, Swami Sankaranarayanan, Daniel C Alexander, and Nathan Silberman,  “Learning from noisy labels by regularized estimation of annotator confusion,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 11244–11253.
  * [5] Vijay Keswani, Matthew Lease, and Krishnaram Kenthapadi,  “Towards unbiased and accurate deferral to multiple experts,”  arXiv preprint arXiv:2102.13004, 2021.
  * [6] Melody Guan, Varun Gulshan, Andrew Dai, and Geoffrey Hinton,  “Who said what: Modeling individual labelers improves classification,”  in Proceedings of the AAAI Conference on Artificial Intelligence, 2018, vol. 32.
  * [7] Sheng Liu, Jonathan Niles-Weed, Narges Razavian, and Carlos Fernandez-Granda,  “Early-learning regularization prevents memorization of noisy labels,”  arXiv preprint arXiv:2007.00151, 2020.
  * [8] Jiangchao Yao, Hao Wu, Ya Zhang, Ivor W Tsang, and Jun Sun,  “Safeguarded dynamic label regression for noisy supervision,”  in Proceedings of the AAAI Conference on Artificial Intelligence, 2019, vol. 33, pp. 9103–9110.
  * [9] Dara Bahri, Heinrich Jiang, and Maya Gupta,  “Deep k-nn for noisy labels,”  in International Conference on Machine Learning. PMLR, 2020, pp. 540–550.
  * [10] Görkem Algan and Ilkay Ulusoy,  “Image classification with deep learning in the presence of noisy labels: A survey,”  Knowledge-Based Systems, vol. 215, pp. 106771, 2021.
  * [11] Yaroslav Ganin and Victor Lempitsky,  “Unsupervised domain adaptation by backpropagation,”  in International conference on machine learning. PMLR, 2015, pp. 1180–1189.
  * [12] Gaoang Wang, Lin Chen, Tianqiang Liu, Mingwei He, and Jiebo Luo,  “Dail: Dataset-aware and invariant learning for face recognition,”  in 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 2021, pp. 8172–8179.
  * [13] Wei-Lun Chao, Hexiang Hu, and Fei Sha,  “Cross-dataset adaptation for visual question answering,”  in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 5716–5725.
  * [14] Zhongyi Pei, Zhangjie Cao, Mingsheng Long, and Jianmin Wang,  “Multi-adversarial domain adaptation,”  in Thirty-second AAAI conference on artificial intelligence, 2018\.
  * [15] Zhangjie Cao, Lijia Ma, Mingsheng Long, and Jianmin Wang,  “Partial adversarial domain adaptation,”  in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 135–150.
  * [16] Guoliang Kang, Lu Jiang, Yi Yang, and Alexander G Hauptmann,  “Contrastive adaptation network for unsupervised domain adaptation,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 4893–4902.
  * [17] Han Zhao, Remi Tachet Des Combes, Kun Zhang, and Geoffrey Gordon,  “On learning invariant representations for domain adaptation,”  in International Conference on Machine Learning. PMLR, 2019, pp. 7523–7532.
  * [18] Mei Wang and Weihong Deng,  “Deep visual domain adaptation: A survey,”  Neurocomputing, vol. 312, pp. 135–153, 2018.
  * [19] Yuanjie Shao, Lerenhan Li, Wenqi Ren, Changxin Gao, and Nong Sang,  “Domain adaptation for image dehazing,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 2808–2817.
  * [20] Yanchao Yang and Stefano Soatto,  “Fda: Fourier domain adaptation for semantic segmentation,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 4085–4095.
  * [21] Weifeng Ge and Yizhou Yu,  “Borrowing treasures from the wealthy: Deep transfer learning through selective joint fine-tuning,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1086–1095.
  * [22] Yunhui Guo, Honghui Shi, Abhishek Kumar, Kristen Grauman, Tajana Rosing, and Rogerio Feris,  “Spottune: transfer learning through adaptive fine-tuning,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 4805–4814.
  * [23] Wouter M Kouw and Marco Loog,  “An introduction to domain adaptation and transfer learning,”  arXiv preprint arXiv:1812.11806, 2018.
  * [24] Xiaoxiang Liu, Lingxiao Song, Xiang Wu, and Tieniu Tan,  “Transferring deep representation for nir-vis heterogeneous face recognition,”  in 2016 International Conference on Biometrics (ICB). IEEE, 2016, pp. 1–8.
  * [25] Yabin Zhang, Hui Tang, Kui Jia, and Mingkui Tan,  “Domain-symmetric networks for adversarial domain adaptation,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 5031–5040.
  * [26] Tuan-Hung Vu, Himalaya Jain, Maxime Bucher, Matthieu Cord, and Patrick Pérez,  “Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 2517–2526.
  * [27] Minghao Chen, Shuai Zhao, Haifeng Liu, and Deng Cai,  “Adversarial-learned loss for domain adaptation,”  in Proceedings of the AAAI Conference on Artificial Intelligence, 2020, vol. 34, pp. 3521–3528.
  * [28] Yaroslav Ganin and Victor Lempitsky,  “Unsupervised domain adaptation by backpropagation,”  arXiv preprint arXiv:1409.7495, 2014.
  * [29] Jean Kossaifi, Georgios Tzimiropoulos, Sinisa Todorovic, and Maja Pantic,  “Afew-va database for valence and arousal estimation in-the-wild,”  Image and Vision Computing, vol. 65, pp. 23–36, 2017.
  * [30] Stefanos Zafeiriou, Dimitrios Kollias, Mihalis A Nicolaou, Athanasios Papaioannou, Guoying Zhao, and Irene Kotsia,  “Aff-wild: valence and arousal’in-the-wild’challenge,”  in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2017, pp. 34–41.
  * [31] Eric Cooper Larson and Damon Michael Chandler,  “Most apparent distortion: full-reference image quality assessment and the role of strategy,”  Journal of electronic imaging, vol. 19, no. 1, pp. 011006, 2010\.
  * [32] Nikolay Ponomarenko, Oleg Ieremeiev, Vladimir Lukin, Karen Egiazarian, Lina Jin, Jaakko Astola, Benoit Vozel, Kacem Chehdi, Marco Carli, Federica Battisti, et al.,  “Color image database tid2013: Peculiarities and preliminary results,”  in european workshop on visual information processing (EUVIP). IEEE, 2013, pp. 106–111.
  * [33] Nikolay Ponomarenko, Oleg Ieremeiev, Vladimir Lukin, Lina Jin, Karen Egiazarian, Jaakko Astola, Benoit Vozel, Kacem Chehdi, Marco Carli, Federica Battisti, et al.,  “A new color image database tid2013: Innovations and results,”  in International Conference on Advanced Concepts for Intelligent Vision Systems. Springer, 2013, pp. 402–413.
  * [34] Abhinav Dhall, Roland Goecke, Simon Lucey, and Tom Gedeon,  “Collecting large, richly annotated facial-expression databases from movies,”  IEEE multimedia, vol. 19, no. 03, pp. 34–41, 2012.
  * [35] Nikolay Ponomarenko, Vladimir Lukin, Alexander Zelensky, Karen Egiazarian, Marco Carli, and Federica Battisti,  “Tid2008-a database for evaluation of full-reference visual quality assessment metrics,”  Advances of Modern Radioelectronics, vol. 10, no. 4, pp. 30–45, 2009.
  * [36] Hamid R Sheikh, Muhammad F Sabir, and Alan C Bovik,  “A statistical evaluation of recent full reference image quality assessment algorithms,”  IEEE Transactions on image processing, vol. 15, no. 11, pp. 3440–3451, 2006.
  * [37] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun,  “Deep residual learning for image recognition,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  * [38] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al.,  “Pytorch: An imperative style, high-performance deep learning library,”  Advances in neural information processing systems, vol. 32, pp. 8026–8037, 2019.
  * [39] Hancheng Zhu, Leida Li, Jinjian Wu, Weisheng Dong, and Guangming Shi,  “Metaiqa: Deep meta-learning for no-reference image quality assessment,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 14143–14152.
  * [40] Shaolin Su, Qingsen Yan, Yu Zhu, Cheng Zhang, Xin Ge, Jinqiu Sun, and Yanning Zhang,  “Blindly assess image quality in the wild guided by a self-adaptive hyper network,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 3667–3676.


#  gleam: Galaxy Line Emission & Absorption Modeling

Andra Stroe Clay Fellow  Center for Astrophysics | Harvard & Smithsonian, 60 Garden St., Cambridge, MA 02138, USA  Victor-Nicolae Savu

(Received December 5, 2020; Revised January 26, 2021; Accepted January 27,
2021)

###### Abstract

We present gleam (Galaxy Line Emission & Absorption Modeling), a Python tool
for fitting Gaussian models to emission and absorption lines in large samples
of 1D extragalactic spectra. gleam is tailored to work well in batch mode
without much human interaction. With gleam, users can uniformly process a
variety of spectra, including galaxies and active galactic nuclei, in a wide
range of instrument setups and signal-to-noise regimes. gleam also takes
advantage of multiprocessing capabilities to process spectra in parallel. With
the goal of enabling reproducible workflows for its users, gleam employs a
small number of input files, including a central, user-friendly configuration
in which fitting constraints can be defined for groups of spectra and
overrides can be specified for edge cases. For each spectrum, gleam produces a
table containing measurements and error bars for the detected spectral lines
and continuum, and upper limits for non-detections. For visual inspection and
publishing, gleam can also produce plots of the data with fitted lines
overlaid. In the present paper, we describe gleam’s main features, the
necessary inputs, expected outputs, and some example applications, including
thorough tests on a large sample of optical/infra-red multi-object
spectroscopic observations and integral field spectroscopic data. gleam is
developed as an open-source project hosted at
https://github.com/multiwavelength/gleam and welcomes community contributions.

Astronomy software (1855), Astronomical techniques (1684), Spectroscopy
(1558), Astronomy data analysis (1858), Galaxies (573), Active galactic nuclei
(16), Open source software (1866)

††journal: AJ††software:  gleam (Stroe & Savu, 2020), Matplotlib (Hunter,
2007), Astropy (Astropy Collaboration et al., 2013), LMFIT (Newville et al.,
2019), Numpy (Harris et al., 2020)

##  1 Introduction

One of the main goals of extragalactic astronomy is to understand the cosmic
evolution of galaxies and black holes in the context of large scale structure.
To obtain a comprehensive view of the physical processes driving their
evolution and unveil their spatial distribution, spectroscopic observations of
large samples of galaxies and active galactic nuclei (AGN) at increasingly
high redshift are required. The most efficient way to obtain large samples
($>100$ to hundreds of thousands of sources) covering large volumes is through
simultaneous observations of many objects. As a consequence, multi-object
spectroscopy (MOS) and integral field unit (IFU) spectroscopy have experienced
significant growth since the 1980s.

MOS plays an important role in repositioning mid-size telescopes, with
instruments dedicated exclusively to completing large surveys of galaxies and
quasars, such as LAMOST (Cui et al., 2012), WHT/WEAVE (Dalton et al., 2012),
SDSS-IV (Blanton et al., 2017) and DESI (DESI Collaboration et al., 2016). The
instrument suite of $6-10$-m class optical/infrared telescopes usually
contains MOS and IFU capabilities, e.g. Keck/DEIMOS (Faber et al., 2003),
Keck/MOSFIRE (McLean et al., 2012), VLT/VIMOS (Le Fèvre et al., 2003),
VLT/KMOS (Sharples et al., 2006), Gemini/GMOS (Hook et al., 2004), Subaru/FMOS
(Kimura et al., 2010), MMT/Hectospec (Fabricant et al., 2005), Magellan/IMACS
(Dressler et al., 2011). In the near future, new wide-field ($>1^{\circ}$),
high-multiplex ($>1000$ targets) MOS instruments will be mounted, such as
VLT/MOONS (Cirasuolo et al., 2012) and VISTA/4MOST (de Jong et al., 2012). All
new-generation ground-based optical/infrared telescopes have MOS and IFU
instruments planned, e.g. ELT/MOSAIC (Jagourel et al., 2018), TMT/WFOS (Pazder
et al., 2006) and GMT/GMACS (Pak et al., 2020). Instruments on the flagship
James Webb Space Telescope, including NIRSpec and MIRI, will also have MOS/IFU
capabilities. MOS/IFU techniques have been also routinely used in the radio
and sub-mm regime (e.g. VLA, ALMA, Thompson et al., 1980; Wootten & Thompson,
2009).

As a result of transformational large-scale public surveys and concerted
guaranteed time efforts completed over the past 3 decades, a growing body of
spectroscopic observations have been made available to the community.
Complementing guaranteed time observations and large scale public surveys,
individual investigators have added MOS and IFU observations tailored to
specific extragalactic science goals. Obtaining statistically robust samples
and particular science cases that involve targets distributed across the sky
(e.g. galaxy population studies in galaxy clusters, quasar surveys, high
redshift galaxy surveys, or intra/circum galactic medium absorption-line
surveys) require the combination of data coming from different telescopes and
instruments. Further, the advent of online databases has made access to fully
or partially reduced observations easier (Ginsburg et al., 2019), enabling
individual authors to make use of existing spectroscopic observations for new
science goals, possibly combining data from different telescopes. The sheer
volume of data warrants automated analysis pipelines with minimal human
interaction.

Striving for reproducible results, many authors in the field provide machine-
readable data plus scripts used to obtain the results with the publication
e.g. the Jupyter (Kluyver et al., 2016) notebook used for creating figures or
the CASA script used to reduce and make images from ALMA data. However,
publications mainly focus on the originally intended science case, leaving
byproducts and intermediate results largely unreported (e.g. unreported line
fluxes when the goal was measuring redshifts). In order to incorporate
archival spectroscopic observations in new projects, researchers need to
partially reproduce and build upon the efforts of the original authors.

The first fundamental property encoded by a spectrum is the source’s redshift.
A number of powerful, modern tools assist astronomers in obtaining accurate
redshifts for large samples in an automatic and unsupervised way while also
ensuring the reliability of the results (e.g. EZ, Garilli et al., 2010). Apart
from providing redshifts, the scientific potential of MOS and IFU observations
is realized in extracting the (resolved) physics and chemistry of
extragalactic objects from emission and absorption lines. With a growing body
of literature with tailored science goals, each publication uses heterogeneous
data and methods to measure emission and absorption lines. As the astronomy
community further adopts the Python programming language (e.g. Astropy,
Astropy Collaboration et al., 2013), various interfaces for fitting functions
exist. However, the low-level function fitting packages require individual
authors to write their own bindings to interface between the reduced
astronomical data and the fitting software. With a high level of duplicated
effort in the community to write tailored code to fit spectral lines and with
the high costs associated with sharing and maintaining it, access to data
analysis software entails a great deal of overhead and represents a barrier to
entry for the field of spectroscopy.

gleam 111https://github.com/multiwavelength/gleam (Galaxy Line Emission &
Absorption Modeling, Stroe & Savu, 2020) is a software tool for fitting
Gaussian models to emission and absorption lines in large samples of galaxy
and AGN spectra. gleam has versatile science applications involving large
samples of 1D spectra or IFU observations. For example, gleam is ideally
suited for unveiling the detailed physics and chemistry of galaxies, as
derived from interstellar medium line ratios and stellar absorption lines, for
a variety of samples spanning both cosmic time and environment. gleam can also
aid in the exploration of IFU cubes through spatially resolved physics,
kinematics, and chemistry. Requiring only the source redshifts and with little
to no interaction, the user can analyze large numbers of spectra in a uniform
manner, even with data taken in different conditions, with different
instrument setups, on different telescopes, at a range of signal-to-noise
(S/N) regimes, and for a wide variety of sources. We tested gleam mainly on
optical and infrared spectra; however, we expect it to also work well on radio
and sub-mm spectra.

In this paper, we provide an introduction to the gleam software, focusing on
features contained in the v1.0 release. Section 2 describes the basic
functionality, while Section 3 discusses the necessary input files and
expected outputs. Section 4 covers some example applications and uses. In
Section 5, we present the open-source development model adopted for gleam,
while in Section 6 we discuss possible extensions to the code in the near
future.

##  2 gleam: Galaxy Line Emission and Absorption Modeling

With gleam, the user can process large numbers of sources in batch mode,
taking advantage of the multiprocessing capabilities of modern CPUs.
Optionally, gleam also provides an interactive interface to inspect individual
line fits on a spectral line and source-by-source basis. gleam fits emission
and absorption lines in fully-reduced 1D spectra using per-source
spectroscopic redshift information and fitting constraints from a central
configuration. The central configuration encourages users to define common
fitting constraints for broad groups of spectra and be deliberate when
defining overrides, which helps prevent user errors and facilitates an easier
review of the methods and results by collaborators, referees, and readers.
gleam fits all lines listed in a central line list and can jointly fit lines
located close together. gleam also reports upper limits and identifies lines
without spectral coverage. If so required, the user can provide a file
containing sky bands, sky lines and/or OH lines to be masked and disregarded
during line fitting. At its core, gleam uses the popular LMFIT Python
package222https://lmfit.github.io/lmfit-py/(Newville et al., 2019) to perform
the line fitting and to calculate and report errors on fit parameters. gleam
is also well integrated with Astropy333https://www.astropy.org/(Astropy
Collaboration et al., 2013), which enables the use of units and FITS tables.

As output, gleam creates a FITS table with Gaussian line measurements and
upper limits (as the case may be), including central wavelength, width,
height, and amplitude, as well as estimates for the continuum under the line,
the line flux, luminosity, equivalent width, and velocity width. gleam can
also make plots of the entire spectrum with fitted lines overlaid, as well as
plots for each individual line fitted, using Matplotlib (Hunter, 2007).

gleam follows open-source practices, with planned features to be added to the
living codebase published online on Github at
https://github.com/multiwavelength/gleam. The latest release of gleam can be
installed easily with Python pip.

##  3 The Software

Below, we introduce gleam’s main functionality, features, required inputs, and
outputs. For a full documentation of the code, we encourage the reader to
consult gleam’s Github page at https://github.com/multiwavelength/gleam.

###  3.1 Model fitting

In fitting the spectrum, gleam groups neighboring spectral lines. For each
spectral line group, a user-defined window of the spectrum around the group is
considered for fitting. gleam models each group as the sum of a constant for
the continuum and one Gaussian for each spectral line. The assumption that the
continuum is locally constant might fail if the window is too wide, while a
too narrow window will not have enough line-free spectrum to properly
constrain the continuum. Sections of the spectrum suffering from
contamination, such as areas with sky lines, can also be masked.

The centers of all Gaussian components can be fixed, constrained to user-
defined intervals, or left as free parameters. An initial guess for the
central wavelength of each Gaussian is used to identify the component. This
initial guess is calculated from the user-provided redshift for each spectrum
and the global list of lines at rest-frame wavelengths. To offer the
flexibility to fit emission and absorption lines in a range of galaxy and AGN
spectra, gleam relies on a single prior, the source redshift, for initializing
the line fitting solution. In all but the brightest sources with the highest
S/N spectral lines, a spectroscopic-quality redshift is required.

A fit is accepted when every Gaussian component passes the user-specified S/N
ratio. When, due to noise and the overlap with sky lines, there is
insufficient information in the data to fit the entire model, gleam
iteratively removes Gaussian components in search of an acceptable fit. Any
removed Gaussian components are treated as non-detections and upper limits are
computed for them.

gleam employs LMFIT to perform the fitting (Newville et al., 2019). Through a
non-linear least-squares minimization using the Levenberg-Marquardt method,
LMFIT enables the robust estimation of both model parameters and their errors.

###  3.2 Naming convention

When handling large numbers of spectra coming from different observations,
sometimes from different telescopes, it is important to adopt a consistent
naming convention. gleam helps with this by prescribing a four-part
hierarchical naming convention that allows for easy identification and
grouping of spectra.

Each measured spectrum is uniquely identified by the combination of the
following four properties:

Sample

A label for the parent sample for the source, e.g. name of the parent galaxy
cluster or famous field,

Setup

A label for the telescope, instrument, or mode used for the observation,

Pointing

An identifier for the individual pointing, fiber configurations, or slit
configuration/mask the observation is part of,

SourceNumber

A source number to distinguish a target within a sample, setup, and pointing
combination.

###  3.3 Inputs

gleam uses five kinds of input files to gather information about spectra in
order to compute the properties of its spectral lines:

  * •

A set of 1D spectra,

  * •

Metafiles that provide a reference redshift for each spectrum,

  * •

Configuration file, which specifies choices of spectral lines, fitting
parameters, cosmological parameters and sky masking,

  * •

Line table with the rest-frame wavelengths of the spectral lines of interest,

  * •

(Optional) Sky band catalog, with details of any wavelengths contaminated by
sky absorption/emission.

In a single run, gleam can process spectra that originate from different data
sets and which might have different units for wavelength or flux. It is
therefore highly recommended to include units in the headers of all spectra
files. gleam propagates the units to the results. Line files and sky band
files should also specify units in the file headers to ensure alignment with
the spectra.

The metadata file contains information about individual spectra in the
project, such as the setup and pointing they were observed with, a numeric
identifier, and their redshift. The user may add custom columns to the
metadata file to store other information about the spectra, such as the sky
coordinates, quality flags, source types, etc. The project can have a single
metadata file or multiple ones, as long as spectra are uniquely labeled.
Because gleam does not process the sky coordinates for sources, it cannot
detect when two spectra pertain to the same source and, therefore, will
produce separate independent fits for each input spectrum. It is incumbent
upon the user to reason about which spectrum best fits their science
requirements. When it is appropriate, another approach would be to
combine/stack the relevant observations into a single spectrum before running
gleam.

gleam can uniformly process large numbers of spectra, even with data taken in
different conditions, with different instruments on different telescopes, and
for a wide variety of sources. The configuration file is used to concisely
describe how the different spectra should be processed, so they can be
analyzed together. For easy editing and review, the configuration file for
gleam uses the YAML444https://yaml.org/ format. Taking advantage of the naming
convention and the many reasonable defaults, the user can tailor the analysis
at 3 levels. The global level parameters override the default configuration
for all the spectra. The setup level offers a way to apply configuration
overrides to groups of spectra (named setups). This level can be used to
capture differences between telescopes or instruments, such as the spectral
resolution. At the most granular level, the user can customize parameters for
individual sources. While per-source overrides can help account for some
particular cases (e.g. a small percentage of sources with both narrow and
broad emission lines), they should be used sporadically due to the associated
typing burden, and in the spirit of keeping the results comparable. The model
parameters for each spectrum are computed by stacking the applicable overrides
on top of the default in order: first, the global overrides, then any
applicable per-setup overrides, and, finally, any applicable per-source
overrides.

gleam cannot specify any default for the line table and the instrumental
resolution, so this information needs to appear at some level in the
configuration file. With these two fields, we present a minimal working gleam
configuration example:

⬇

globals:

line_table: line_lists/Main_optical_lines.fits

resolution: 4.4 Angstrom

gleam fits all lines listed in the line table and iteratively eliminates model
components when the data does not yield satisfactory fits for all the lines. A
S/N parameter, which defines the minimum accepted ratio between the estimated
amplitude of a component and its error, separates detections from upper limits
for each spectral line. In some setups, the user may select a starting subset
of the lines and avoid unnecessary trials (e.g. excluding faint lines the user
does not expect to be detected).

(a) A spectrum of a source at $z\sim 0.1$, with a spectrum dominated by AGN
features, including broad emission lines. The spectrum was taken with
MMT/Hectospec, a multi-fiber instrument.

(b) A high S/N spectrum at $z\sim 0.6$ dominated by star formation, taken in
$>1^{\prime\prime}$ seeing conditions with the fiber-bed WHT/AF2 instrument.

Figure 1: gleam emission and absorption line fits, highlighting different
source types and origin telescopes.

(c) A star-formation dominated galaxy at $z\sim 0.17$, with emission lines
slightly overlapping with a sky-contaminated wavelength range. Data was taken
in $\sim 1^{\prime\prime}$ seeing conditions with thin clouds, with a multi-
slit spectrograph (VLT/VIMOS).

(d) A passive galaxy spectrum with absorption and emission features at $z\sim
0.26$. Data was taken in $\sim 1^{\prime\prime}$ seeing with MMT/Hectorspec
instrument.

Figure 1: Continued.

By default, there is no sky masking and the entire spectrum is used. However,
the user may control whether the model should ignore portions of the spectrum
where sky bands may not have been reliably subtracted. These bands are masked
and disregarded for fitting and treated as if no spectral coverage is
available. For example, for a data set where sky subtraction only failed for a
few of the spectra, the configuration may specify the sky band catalog at the
global level but only turn masking on for individual sources (or setups) for
which the sky subtraction is inadequate.

gleam offers users a lot of flexibility in choosing the way line models are
fit to the data. Neighboring Gaussian components can be fit jointly to account
for nearby or blended lines. The user can also define the amount of continuum
to be fitted on either side of a group, which should be large enough to
encompass enough line-free continuum. Over the range specified, the continuum
should be well approximated by a constant. Any unrelated lines that fall
within the selected continuum are automatically masked. The center of each
Gaussian component is first estimated based on the rest-frame wavelength in
the line catalog and on the redshift estimate of the spectrum (listed in the
metadata file). The Gaussian center can be fixed to the initial guess, allowed
to vary within a small range around it, or can vary freely within the spectral
range of data considered when fitting. This final option can lead to lines
being mislabeled or cross-labeled, so it should only be used when the redshift
estimate for the spectrum is so poor that neither of the two other options is
feasible.

To report luminosities based on the fitted models, gleam uses a set of
cosmological parameters: Hubble constant ($H_{0}$), $\Omega_{0}$ and the CMB
temperature. While the default values for these parameters are reasonably
accurate and up to date, some projects may require slightly different values.
The cosmology section overrides one or more of these parameters, and the
resulting cosmology is then used consistently across all the spectra within a
project. This is the only set of overrides that cannot be made on a per-setup
or per-source basis, since doing so could produce results that are not
comparable between sources.

###  3.4 Outputs

For each of the sources in the sample, gleam produces a FITS table with all of
the line fits and upper limits (with units derived from the input data, if
available). Each line fitted is represented in a separate row, with all the
corresponding line fit details contained in different columns. The output
table contains fit parameters and their associated errors (such as the
continuum estimation, central line wavelength, Gaussian height, standard
deviation, and amplitude), line fluxes, luminosities, and equivalent widths.
Each row also contains a flag for spectral coverage (i.e. whether the line is
covered by the input spectrum) and another to indicate detection (whether the
line is detected above the required S/N). Fit values and errors are omitted
when the spectrum does not cover the spectral line. If a line is not detected,
gleam only reports an upper limit in the amplitude column and omits all other
Gaussian fit parameters. The deconvolved full-width-at-half-maximum (FWHM) and
the velocity FWHM are only reported if the line is spectrally resolved.

If plotting is enabled, gleam produces two types of figures. The first type of
figure shows the entire spectrum with zoom-ins on the emission and absorption
line fits (see Figure 1). The second type of plot is focused on each line fit.
Masked sky areas are shaded gray for clarity.

##  4 Example applications

We demonstrate gleam’s capabilities by showcasing two natural applications,
which also served as testbeds during the code development.

###  4.1 A Large, Heterogeneous Sample of Extragalactic Spectra

gleam is well-suited for measuring emission and absorption lines in large,
heterogeneous samples of extragalactic 1D spectra. In Stroe & Sobral (2021),
we thoroughly tested gleam on all the spectroscopy available to us, which
included about 4200 passive galaxies, star-forming galaxies, AGN, and quasars
at redshifts from 0 to $\sim 1$. The data were taken with four different
instruments (VLT/VIMOS, WHT/AF2, Keck/DEIMOS, and MMT/Hectospec) that employ
different techniques to achieve MOS capabilities (slits, fibers), under a
range of sky, weather, and seeing conditions. In this project, the focus was
on measuring optical emission lines, such as [Oii] ($\lambda\,3728$ Å),
H$\beta$, [Oiii] ($\lambda\lambda\,4960,\,5007$ Å), H$\alpha$, [Nii]
($\lambda\lambda\,6550,\,6585$ Å), H$\beta$, and [Sii]
($\lambda\lambda\,6718,\,6733$ Å), with the spectral resolution for all
instruments being sufficient for separating nearby narrow emission lines.

For the entire sample of 4200 sources, a simple, short configuration file was
sufficient, as illustrated below. The fitting constraints, the set of spectral
lines to be fit, and the sky lines to be masked were the same for the bulk of
the sources. With gleam, it was easy to set a different resolution for each
instrumental setup and, when necessary, add, for example, a different
continuum width (which resulted in more stable fits), turn off sky masking
(when data reduction adequately corrected for the sky absorption), or use a
different line table (when an air versus a vacuum wavelength calibration was
applied to the data). We also set overrides for several individual sources.
For example, we fit the full line list when [Nii] was also present in the data
or when lines were blended. The YAML configuration file for this example
application can be found below and demonstrates all the customization options
that gleam provides. Examples of line fits can be found in Figure 1.

⬇

globals:

sky: line_lists/Sky_bands.fits

mask_sky: True

line_table: line_lists/Main_optical_lines.fits

lines:

- OII

- Hb

- OIII4

- OIII5

- Ha

- NII1

- SII1

- SII2

fitting:

SN_limit: 2

tolerance: 26.0 Angstrom

w: 3.0 Angstrom

mask_width: 20.0 Angstrom

cont_width: 70.0 Angstrom

center: constrained

\parsetups:

VIMOS:

resolution: 12.5 Angstrom

fitting:

cont_width: 60 Angstrom

MMT:

resolution: 6 Angstrom

line_table: line_lists/rsvao.fits

mask_sky: False

Keck:

resolution: 1 Angstrom

fitting:

cont_width: 40 Angstrom

WHT_R316R:

resolution: 8.1 Angstrom

WHT_R600R:

resolution: 4.4 Angstrom

\parsources:

…

A115.MMT.Q1_EXT1.194:

lines: all

…

Without plotting, the fitting for 10 spectral lines (with H$\alpha$ +[Nii] and
the [Sii] doublet fit jointly) for the sample of $>4000$ sources could be
completed in less 20 min, using 6 threads on a modern laptop with a 2.9 GHz
6-Core Intel Core i9 processor. With plotting, the process can take up to 3 h.

###  4.2 Integral Field Unit Observations

Very powerful Python wrappers tailored for the analysis of IFU data exist in
the literature (e.g. GIST, Bittner et al., 2019). Their design is tailored to
accomplish complex applications, such as the detailed modeling of absorption
lines in passive galaxies, which require input spectra with good continuum
detections. gleam does not make assumptions on the underlying physics of the
sources, making it a complementary tool for emission-line dominated sources
that do not benefit from high S/N continuum detections.

In Stroe et al. (2020), gleam was used to measure spectral lines in
Gemini/GMOS IFU observations of 5 emission-line dominated cluster galaxies at
$z\sim 0.2$. For analysis with gleam, the IFU cube for each of the sources was
split into individual spaxels. The nature of the project required slight
reinterpretations of the naming convention. The Sample components was set to
the name of the parent galaxy cluster, Pointing was used to specify the
galaxy, while SourceNumber was used to label each spaxel in the IFU.
Coordinates for each spaxel were added to the metadata file to track the
connection between the spaxels and their sky positions with respect to each
galaxy. This is a good example of using the metadata file for storing more
than just the redshift information. With this interpretation of the naming
convention, the YAML configuration for the project could be specified in just
a few lines:

⬇

globals:

mask_sky: False

line_table: line_lists/rsvao.fits

lines:

- Ha

- NII1

- SII1

- SII2

fitting:

SN_limit: 3

tolerance: 26.0 Angstrom

w: 9.0 Angstrom

mask_width: 20.0 Angstrom

cont_width: 70.0 Angstrom

center: constrained

\parsetups:

GMOS:

resolution: 11.4 Angstrom

##  5 Development model

gleam is developed as an open-source project hosted at
https://github.com/multiwavelength/gleam and published under the permissive
BSD-3-Clause License. The authors welcome community contributions in the form
of bug reports and feature suggestions, as well as code contributions under
the same license via the GitHub pull-request system. gleam is written in the
Python programming language, which is a popular choice both within and outside
the astronomical community, with the hope that interested contributors would
find it easy to get started. The project aims to offer an inclusive and
welcoming place for collaboration and has adopted the Astropy Community Code
of Conduct555https://www.astropy.org/code_of_conduct.html.

##  6 Future developments

In the near future, we will explore a number of natural extensions to gleam.

In its first iteration, gleam was envisioned to work out-of-the-box for most
extragalactic science cases, hence the choice of Gaussian models for the
fitting. In the future, we plan to expand the choices for component types with
other models, such as Voigt (e.g. absorption lines towards quasars) or
asymmetric (e.g. Ly$\alpha$ emission) profiles and more complex continuum
models.

We aim to also provide better support for multiple component fits, such as
when both broad and narrow lines are present in an AGN spectrum. In its
iterative refinement of the model, gleam removes Gaussian components in a
sequential fashion. As such, the most complex model that converges is passed
through the S/N criterion to identify detected spectral lines. As evidenced in
Section 4, gleam robustly fits well-separated spectral lines at non-redundant
wavelength separations. For scenarios in which many ($>3$) blended or nearby
lines at lower S/N are jointly fit, sometimes lines are cross-identified.
Incorrect matching/labeling of spectral lines can be avoided by making use of
constraints on the center of each Gaussian component. In the future, a number
of new additions to gleam will ensure successful and correct fits to a wider
variety of science cases. At the moment, line fitting in gleam does not take
into account line ratio predictions from radiative modeling and, as such,
allows for any ratio between spectral lines. The option for a tighter coupling
between line ratios could be desirable for specific science cases or, for
example, low S/N regimes.

Another direction for development would be to investigate other back-ends for
performing the line fitting, e.g. astropy.modeling from Astropy, which was not
available at the time the main code development was occurring for gleam. This
approach would enable a closer integration with the Astropy suite of packages.

Further, as mentioned in Section 4, we tested gleam on a variety of optical
and infrared observations. In the near future, we will test its robustness
when applied to data at other wavelengths, such as radio (e.g. focusing on Hi
observations) and sub-mm observations (e.g. molecular and atomic lines,
especially at high redshift).

The authors are grateful to the referee for their constructive suggestions,
which improved the paper. Andra Stroe gratefully acknowledges the support of a
Clay Fellowship. gleam heavily relies on a number of scientific Python
dependencies, including Astropy, LMFIT, Matplotlib, and NumPy. Its development
makes use of other packages, tools, and services, including git, Poetry, Mypy,
Black, Colorama, pydantic, Click, PyYAML, GitHub, and VSCode. In testing the
software, we made use of observations obtained with the International Gemini
Observatory, with ESO Telescopes at the La Silla Paranal Observatory, with the
William Herschel Telescope, with the W.M. Keck Observatory, and with the MMT
Observatory.

## References

  * Astropy Collaboration et al. (2013) Astropy Collaboration, Robitaille, T. P., Tollerud, E. J., et al. 2013, A&A, 558, A33, doi: 10.1051/0004-6361/201322068
  * Bittner et al. (2019) Bittner, A., Falcón-Barroso, J., Nedelchev, B., et al. 2019, A&A, 628, A117, doi: 10.1051/0004-6361/201935829
  * Blanton et al. (2017) Blanton, M. R., Bershady, M. A., Abolfathi, B., et al. 2017, AJ, 154, 28, doi: 10.3847/1538-3881/aa7567
  * Cirasuolo et al. (2012) Cirasuolo, M., Afonso, J., Bender, R., et al. 2012, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 8446, Ground-based and Airborne Instrumentation for Astronomy IV, ed. I. S. McLean, S. K. Ramsay, & H. Takami, 84460S, doi: 10.1117/12.925871
  * Cui et al. (2012) Cui, X.-Q., Zhao, Y.-H., Chu, Y.-Q., et al. 2012, Research in Astronomy and Astrophysics, 12, 1197, doi: 10.1088/1674-4527/12/9/003
  * Dalton et al. (2012) Dalton, G., Trager, S. C., Abrams, D. C., et al. 2012, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 8446, Ground-based and Airborne Instrumentation for Astronomy IV, ed. I. S. McLean, S. K. Ramsay, & H. Takami, 84460P, doi: 10.1117/12.925950
  * de Jong et al. (2012) de Jong, R. S., Bellido-Tirado, O., Chiappini, C., et al. 2012, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 8446, Ground-based and Airborne Instrumentation for Astronomy IV, ed. I. S. McLean, S. K. Ramsay, & H. Takami, 84460T, doi: 10.1117/12.926239
  * DESI Collaboration et al. (2016) DESI Collaboration, Aghamousa, A., Aguilar, J., et al. 2016, arXiv e-prints, arXiv:1611.00036.  https://arxiv.org/abs/1611.00036
  * Dressler et al. (2011) Dressler, A., Bigelow, B., Hare, T., et al. 2011, PASP, 123, 288, doi: 10.1086/658908
  * Faber et al. (2003) Faber, S. M., Phillips, A. C., Kibrick, R. I., et al. 2003, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 4841, Instrument Design and Performance for Optical/Infrared Ground-based Telescopes, ed. M. Iye & A. F. M. Moorwood, 1657–1669, doi: 10.1117/12.460346
  * Fabricant et al. (2005) Fabricant, D., Fata, R., Roll, J., et al. 2005, PASP, 117, 1411, doi: 10.1086/497385
  * Garilli et al. (2010) Garilli, B., Fumana, M., Franzetti, P., et al. 2010, PASP, 122, 827, doi: 10.1086/654903
  * Ginsburg et al. (2019) Ginsburg, A., Sipőcz, B. M., Brasseur, C. E., et al. 2019, AJ, 157, 98, doi: 10.3847/1538-3881/aafc33
  * Harris et al. (2020) Harris, C. R., Millman, K. J., van der Walt, S. J., et al. 2020, Nature, 585, 357, doi: 10.1038/s41586-020-2649-2
  * Hook et al. (2004) Hook, I. M., Jørgensen, I., Allington-Smith, J. R., et al. 2004, PASP, 116, 425, doi: 10.1086/383624
  * Hunter (2007) Hunter, J. D. 2007, Computing in Science & Engineering, 9, 90, doi: 10.1109/MCSE.2007.55
  * Jagourel et al. (2018) Jagourel, P., Fitzsimons, E., Hammer, F., et al. 2018, in Ground-based and Airborne Instrumentation for Astronomy VII, ed. C. J. Evans, L. Simard, & H. Takami, Vol. 10702, International Society for Optics and Photonics (SPIE), 3162 – 3171, doi: 10.1117/12.2314135
  * Kimura et al. (2010) Kimura, M., Maihara, T., Iwamuro, F., et al. 2010, PASJ, 62, 1135, doi: 10.1093/pasj/62.5.1135
  * Kluyver et al. (2016) Kluyver, T., Ragan-Kelley, B., Pérez, F., et al. 2016, in Positioning and Power in Academic Publishing: Players, Agents and Agendas, ed. F. Loizides & B. Scmidt (Netherlands: IOS Press), 87–90.  https://eprints.soton.ac.uk/403913/
  * Le Fèvre et al. (2003) Le Fèvre, O., Saisse, M., Mancini, D., et al. 2003, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 4841, Instrument Design and Performance for Optical/Infrared Ground-based Telescopes, ed. M. Iye & A. F. M. Moorwood, 1670–1681, doi: 10.1117/12.460959
  * McLean et al. (2012) McLean, I. S., Steidel, C. C., Epps, H. W., et al. 2012, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 8446, Ground-based and Airborne Instrumentation for Astronomy IV, ed. I. S. McLean, S. K. Ramsay, & H. Takami, 84460J, doi: 10.1117/12.924794
  * Newville et al. (2019) Newville, M., Otten, R., Nelson, A., et al. 2019, lmfit/lmfit-py 1.0.0, 1.0.0, Zenodo, doi: 10.5281/zenodo.3588521
  * Pak et al. (2020) Pak, S., DePoy, D. L., Marshall, J. L., et al. 2020, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 11203, Advances in Optical Astronomical Instrumentation 2019, 1120308, doi: 10.1117/12.2547889
  * Pazder et al. (2006) Pazder, J. S., Roberts, S., Abraham, R., et al. 2006, in Ground-based and Airborne Instrumentation for Astronomy, ed. I. S. McLean & M. Iye, Vol. 6269, International Society for Optics and Photonics (SPIE), 644 – 655, doi: 10.1117/12.672712
  * Sharples et al. (2006) Sharples, R., Bender, R., Bennett, R., et al. 2006, nar, 50, 370, doi: http://dx.doi.org/10.1016/j.newar.2006.02.014
  * Stroe et al. (2020) Stroe, A., Hussaini, M., Husemann, B., Sobral, D., & Tremblay, G. 2020, ApJ, 905, L22, doi: 10.3847/2041-8213/abcb04
  * Stroe & Savu (2020) Stroe, A., & Savu, V.-N. 2020, multiwavelength/gleam: Initial release of gleam, v1.0, Zenodo, doi: 10.5281/zenodo.3974969
  * Stroe & Sobral (2021) Stroe, A., & Sobral, D. 2021, ApJ (submitted)
  * Thompson et al. (1980) Thompson, A. R., Clark, B. G., Wade, C. M., & Napier, P. J. 1980, ApJS, 44, 151, doi: 10.1086/190688
  * Wootten & Thompson (2009) Wootten, A., & Thompson, A. R. 2009, IEEE Proceedings, 97, 1463, doi: 10.1109/JPROC.2009.2020572