# Self-Paced Deep Regression Forests
with Consideration of Ranking Fairness

Lili Pan∗, Mingming Meng, Yazhou Ren, Yali Zheng, Zenglin Xu Lili Pan and
Mingming Meng are with the School of Information and Communication
Engineering, University of Electronic Science and Technology of China,
Chengdu, 611731, and Yangtze Delta Region Institute, University of Electronic
Science and Technology of China, Quzhou, 32400, China.Yazhou Ren is with the
School of Computer Science and Engineering, and Yali Zheng is with the School
of Automation Engineering, University of Electronic Science and Technology of
China, Chengdu, 611731, China.Zenglin Xu is with the School of Computer
Science and Engineering, Harbin Institute of Technology, Shenzhen, and also
affiliated with Peng Cheng Lab, Shenzhen, 518055, China.

###### Abstract

Deep discriminative models (DDMs), _e.g._ deep regression forests and deep
decision forests, have been extensively studied recently to solve problems
such as facial age estimation, head pose estimation, _etc._. Due to a shortage
of well-labeled data that does not have noise and imbalanced distribution
problems, learning DDMs is always challenging. Existing methods usually tackle
these challenges through learning more discriminative features or re-weighting
samples. We argue that learning DDMs gradually, from easy to hard, is more
reasonable, for two reasons. First, this is more consistent with the cognitive
process of human beings. Second, noisy as well as underrepresented examples
can be distinguished by virtue of previously learned knowledge. Thus, we
resort to a gradual learning strategy—self-paced learning (SPL). Then, a
natural question arises: can SPL lead DDMs to achieve more robust and less
biased solutions? To answer this question, this paper proposes a new SPL
method: easy and underrepresented examples first, for learning DDMs. This
tackles the fundamental ranking and selection problem in SPL from a new
perspective: fairness. Our idea is fundamental and can be easily combined with
a variety of DDMs. Extensive experimental results on three computer vision
tasks, _i.e._ , facial age estimation, head pose estimation, and gaze
estimation, show our new method gains considerable performance improvement in
both accuracy and fairness. Source code is available at
https://github.com/learninginvision/SPU.

###### Index Terms:

Underrepresented examples, ranking fairness, self-paced learning, deep
regression forests.

##  I Introduction

Deep discriminative models (DDMs) have been extensively studied recently to
solve a variety of computer vision problems, with applications in image
classification, action recognition, and object detection, to name a few. DDMs
use deep neural networks (DNNs) [1, 2, 3, 4] to formulate the input-output
mapping in discriminative models. Due to a shortage of well-labeled training
data, _i.e._ without noise and imbalanced distribution problems, learning DDMs
is particularly challenging. Considerable literature has grown up around the
theme of how to learn DDMs effectively [5, 6, 7].

One typical approach is to learn more discriminative features through rather
deep neural networks and cost-sensitive discriminative functions [8, 9, 10, 5,
11]. The other typical approaches are to reweight training samples according
to estimation errors [12, 13] or gradient directions [14] (_i.e._ meta
learning). Although these approaches can partially alleviate the noise and
imbalanced distribution problem of training data, they are not consistent with
the cognitive process of human beings; the difference is we learn new things
based on previously learned knowledge. In addition, these approaches do not
provide a way to distinguish noisy and underrepresented examples, and thus
cannot fundamentally avoid noisy and biased solutions.

Figure 1: The Motivation of considering underrepresented examples in SPL. (a):
The histogram shows the number of facial images of different ages, and the
average entropy curve indicates the predictive uncertainty. We observe that
the _underrepresented examples_ always have high entropy. (b):
Underrepresented examples are selected only in SPUDRFS at an early pace.
Because the underrepresented examples tend to have relatively large loss, they
would not be selected at an early pace. (c): A new self-paced learning method:
easy and underrepresented examples first. This builds on the intuition that
the underrepresented examples are prone to incur unfair treatment since they
are the “minority” in training data.

Intuitively, learning DDMs from easy to hard may be more reasonable because
already-learned knowledge, in such a process, can be leveraged to learn DDMs.
In addition to this, the noisy and underrepresented examples could be
recognized by virtue of already-learned knowledge. Hence, we resort to a
gradual learning strategy _i.e._ , self-paced learning [15, 16, 17, 18]. So
far, however, there are few studies on learning DDMs in a self-paced manner.
Then, a natural question arises: _can SPL guide DDMs to achieve more robust
and less biased solutions?_

An underlying problem in SPL, which is firstly shown by this study, is that it
assumes the distribution of training data is balanced, and thus the bias of
solutions may be further exacerbated when such an assumption fails. The reason
is that the underrepresented examples, due to _large_ prediction error, would
not be selected for training at early paces, thereby incurring unfair
treatment. This means existing SPL methods have fairness issue.

To address the fairness issue in existing SPL methods as well as answer the
above questions, this paper proposes a new self-paced learning method for
learning DDMs. It tackles the fundamental ranking and selection problems in
SPL from a new perspective: fairness. On one hand, our new method keeps SPL’s
advantage in robustness. On the other hand, it prevents seriously biased
solutions induced by SPL. The major contributions of this work include:

  * •

We for the first time, show that existing SPL methods may lead to seriously
biased solutions. To address this problem, we propose a new SPL method: easy
and underrepresented examples first. This combines with a typical DDM, _i.e._
, deep regression forests (DRFs), can lead to a new model—deep regression
forests with consideration of underrepresented examples (SPUDRFs). The new
model shows considerable performance improvement in both accuracy and
fairness.

  * •

To further promote robustness, in SPUDRFs, we define capped likelihood
function with respect to DRFs’ parameters so as to further exclude noisy
examples.

  * •

To measure regression fairness, we define a fairness metric for regression
problems, which can reflect fairness concerning some sensitive attributes.

  * •

For verification, we validate SPUDRFs on three computer vision tasks: (i)
facial age estimation, (ii) head pose estimation, and (iii) gaze estimation.
Extensive experimental results on the Morph II [19], FG-NET [20], BIWI [21] ,
BU3DFE [22] and MPIIGaze [23] datasets demonstrate the efficacy of our
proposed new self-paced method. On all the above tasks, SPUDRFs almost achieve
state-of-the-art performance.

A preliminary version of this work was published in [24]. This paper
significantly improves [24] in the following aspects. (i) We extend our model
to incorporate capped likelihood, which could further promote robustness. The
robust SPUDRFs can recognize and exclude examples with labeling noise, whereas
the original is unable to do so, and only places more emphasis on reliable
examples. (ii) We extend our model to combine with various weighting schemes,
whereas [24] merely adopts a mixture weighting scheme. (iii) We define the
fairness metric for regression in this work and show obvious fairness
improvement of SPUDRFs against the original SP-DRFs. (iv) We also evaluate
SPUDRFs on a new computer vision task, _i.e._ , gaze estimation, and
demonstrate its validity on the MPIIGaze dataset. (v) Both robustness and
fairness evaluation results are added, along with more comprehensive
discussions.

This paper is organized as follows. Sec. II introduces related work on SPL,
and DDMs for facial age estimation, head pose estimation and gaze estimation.
Sec. III details our proposed SPUDRFs. Sec. IV introduces robust SPUDRFs. Sec.
V defines a fairness metric for regression problem. Sec. VI exhibits and
analyzes the experimental results on three computer vision tasks and five
related datasets. Sec. VII discusses the strengths and potential weaknesses of
this work, followed by Sec. VIII concluding this paper with perspectives.

##  II Related Work

This section reviews SPL methods and DDMs-based facial age estimation, head
pose estimation and gaze estimation.

###  II-A Self-Paced Learning

Self-paced learning, as a gradual learning method, builds on the intuition
that, rather than considering all training samples simultaneously, the
learning should be presented with the training data from easy to difficult
[15, 25]. To date, a great deal of study on self-paced learning (SPL) has been
undertaken, mostly about learning conventional discriminative models, _e.g._
SVM, logistic regressor. There also exists some literature in which SPL is
employed for clustering problems [26, 27, 28].

Learning conventional discriminative models in a self-paced manner exhibits
superiority over traditional methods. In [15], Kumar _et al._ proposed the
original SPL framework in a regime that suggests processing the samples in
order of an easy to hard order. In [16], Zhao _et al._ generalized the hard
weighting scheme in SPL to a soft weighting scheme, which promoted
discrimination accuracy. In [17], Ma _et al._ proposed self-paced co-training,
where SPL is applied for multi-view or multi-modality problems. In [29], Ren
_et al._ introduced capped-norm into the objective function of SPL, so as to
further exclude the interference of noisy examples. In fact, the above work
can be cast as incorporating SPL with conventional classifiers, _e.g._ , SVM,
logistic regressors, _etc._. To incorporate SPL with regression models, in
[30], Han _et al._ learned a mixture of regressors in a self-paced manner, and
added an $\ell_{1,2}$ norm regularizer to avoid poorly conditioned linear sub-
regressors.

In computer vision community, recently some researchers have realized that
learning DDMs in a self-paced manner may lead to more robust solutions. One of
the few studies is [18], where Li _et al._ sought to enhance the learning
robustness of CNNs by SPL. However, they omitted one important problem in
learning discriminative models: the imbalanced distribution of training data.

Our work is inspired by the existing studies [31, 32] which take the class
diversity in sample selection of SPL into consideration. Jiang _et al._ [32]
proposed to ensure the class diversity of samples at the early paces in self-
paced training. Yang _et al._ [31] defined a metric called “complexity of
image category” to measure the number of samples in each category, as well as
jointly classifying difficulty, and used it to select samples. The two methods
mentioned above find that a lack of class diversity in sample selection may
result in seriously biased solutions. However, what causes this problem has
not been studied. This work shows that the cause is fundamentally the
unfairness of sample ranking in SPL, where underrepresented examples may often
have large losses and not be selected at early paces. On the other hand, [31,
32] are only suitable for classification, but not for regression in which the
output targets are continuous and high dimensional. This paper will go further
in this direction, aiming to tackle the fundamental drawback in SPL—ranking
unfairness—and to integrate SPL with DDMs.

###  II-B Facial Age Estimation

Facial age estimation has been extensively studied for a decade in the
computer vision community. In recent years, a large and growing body of
literature [10, 3, 8, 33, 34] has been proposed for age estimation with
varying degrees of success. Ordinal-based approaches [10, 3] are the best-
known and have demonstrated improved results. For example, Niu _et al._
proposed to estimating age through a set of sequential binary queries—each
query refers to a comparison with a predefined age—to exploit the inter-
relationship (ordinal information) among age labels. Furthermore, Gao _et al._
[8] proposed DLDL-v2 to explore the underlying age distribution patterns, so
as to effectively accommodate age ambiguity. Shen _et al._ [33] used VGG-16 to
extract facial features, and mapped the extracted features to age by deep
regression forests (DRFs). Li _et al._ [34] proposed BridgeNet to predict age
by mixing local regression results, where local regressors are learned on
partitioned data space. Deng _et al._ [35] proposed an age classification
method to deal with long-tailed distribution, where a variational margin is
used to minimize the influence of head classes that misleads the prediction of
tailed samples. Overall, these DDMs-based approaches have improved age
estimation performance significantly. However, they plausibly ignore one
problem: the interference with facial images duo to PIE (_i.e._ pose,
illumination and expression) variation, occlusion, misalignment and so forth.
More importantly, these approaches mostly ignore the fact that the training
data is not distributed uniformally.

###  II-C Head Pose Estimation

Head pose estimation has attracted vast attention from the computer vision
community for a long time. Recently, an increasing amount of DDMs-based
methods have been proposed for head pose estimation and have dramatically
boosted estimation accuracy. Of these, Riegler _et al._ [36] utilized
convolutional neural networks (CNNs) to extract patch features within facial
images, and more precise results were achieved. In [4], Huang _et al._
proposed multi-modal deep regression networks, with multi-layer perceptron
(MLP) architecture, to fuse the RGB and depth images for head pose estimation.
In [37], Wang _et al._ proposed a deep coarse-to-fine network to further boost
pose estimation accuracy. In [7], Ruiz _et al._ used a large, synthetically
expanded head pose dataset to train a rather deep multi-loss convolutional
neural networks (CNNs), gaining obvious accuracy improvement. In [38], Kuhnke
_et al._ proposed a domain adaptation method for head pose estimation where
shared label spaces across different domains are assumed. Although accuracy
has improved, handling noisy examples in the above models is not
straightforward. Moreover, the methods mentioned above seldom deal with
imbalanced training data, which may widely exist in visual problems.

###  II-D Gaze Estimation

Appearance-based gaze estimation learns a mapping from facial or eye image to
gaze. Recently, a considerable amount of DDMs based approaches have been
proposed for person-independent gaze estimation. For example, in [39], Zhang
_et al._ used spatial weights VGG-16 to encode face image to gaze direction.
The spatial weights were used to flexibly suppress or enhance features from
different facial regions. In the literature [40], Krafka _et al._ implemented
a CNNs-based gaze tracker by mapping captured images from the left eye, right
eye and face to gaze. The location and size of the head in the original image
are provided for mapping networks. In [41], Fischer _et al._ adopted two
backbone networks to extract features from two eye regions for gaze
regression, and achieved improved robustness. Instead of predicting gaze
directly, in [42], Park _et al._ proposed another method which maps the single
eye image to an intermediate pictorial representation as additional
supervision to simplify 3D gaze direction estimation. Wang _et al._ [43]
proposed a Bayesian adversarial learning approach to address the appearance,
head pose variations in gaze estimation, and overfitting problem. Xiong _et
al._ [44] proposed mixed effect neural networks to combine global and person
specific gaze estimation results together. Meanwhile, other studies show that
facial images can help gaze estimation. In [45, 46, 47], multi-channel
estimation architectures were adopted to utilize facial image and eye images
jointly. These methods outperform the ones that only use eye images as input
for gaze estimation. In addition to person-independent methods, there is some
literature about person-specific gaze adaption methods [48, 49, 50], which
have boosted performance dramatically when a few annotated examples are used
for calibration. These methods indeed improve gaze estimation results;
however, whether we should learn DDMs for gaze estimation in a gradual
learning manner has not been discussed, nor has there been discussion of how
to deal with imbalanced distribution of training data.

##  III Self-Paced Deep Regression Forests with Consideration on
Underrepresented Samples

This section first reviews the basic concepts in DRFs; and introduces the
objective formulation for SPUDRFs, as well as variant weighting strategies and
an underrepresented example augmentation method; and finally details the
optimization algorithm.

###  III-A Preliminaries

Deep regression forests (DRFs), as a deep regression model, connect deep
neural netwoks (DNNs) to regression forests. We start by reviewing the basic
concepts in DRFs [33].

Deep Regression Tree. DRFs usually consist of a number of deep regression
trees, each of which, given input-output pairs
$\left\\{\mathbf{x}_{i},y_{i}\right\\}_{i=1}^{N}$, map extracted features
through DNNs to target output by passing a regression tree. Further, a
regression tree $\mathcal{T}$ consists of split (or decision) nodes
$\mathcal{N}$ and leaf (or prediction) nodes $\mathcal{L}$ [33] (see Fig. 1).
Specifically, each split node $n\in\mathcal{N}$ determines whether a sample
$\mathbf{x}_{i}$ goes to left or right sub-tree; each leaf node
$\ell\in\mathcal{L}$ represents a Gaussian distribution $p_{\ell}(y_{i})$ with
mean $\mu_{l}$ and variance $\sigma^{2}_{l}$—parameters of output distribution
defined for each tree $\mathcal{T}$.

Split Node. Split node represents a split function
$s_{n}(\mathbf{x}_{i};\bm{\Theta}):\mathbf{x}_{i}\rightarrow[0,1]$, where
$\bm{\Theta}$ denotes the parameters of DNNs, as in Fig. 1(c). Conventionally,
$s_{n}(\mathbf{x}_{i};\bm{\Theta})$ is formulated as
$\sigma\left(\mathbf{f}_{\varphi(n)}(\mathbf{x}_{i};\bm{\Theta})\right)$,
where $\sigma(\cdot)$ denotes a sigmoid function while $\varphi(\cdot)$
denotes an index function specifying the $\varphi(n)$-th element of
$\mathbf{f}(\mathbf{x}_{i};\bm{\Theta})$ in correspondence with the split node
$n$, and $\mathbf{f}(\mathbf{x}_{i};\bm{\Theta})$ denotes the extracted
features through DNNs. An example to illustrate the sketch chart of the DRFs
is shown in Fig. 1(c), where $\varphi_{1}$ and $\varphi_{2}$ are two index
functions for two trees. Finally, the probability of the sample
$\mathbf{x}_{i}$ falling into the leaf node $\ell$ is given by:

$\omega_{\ell}(\mathbf{x}_{i}|\bm{\Theta)}=\prod_{n\in\mathcal{N}}s_{n}(\mathbf{x}_{i};\bm{\Theta})^{[\ell\in\mathcal{L}_{n_{\text{left}}}]}\left(1-s_{n}(\mathbf{x}_{i};\bm{\Theta})\right)^{\left[\ell\in\mathcal{L}_{n_{\text{right}}}\right]},$
(1)

where $[\cdot]$ denotes an indicator function conditioned on the argument. In
addition, $\mathcal{L}_{n_{\text{left}}}$ and $\mathcal{L}_{n_{\text{right}}}$
correspond to the sets of leaf nodes owned by the sub-trees
$\mathcal{T}_{n_{\text{left}}}$ and $\mathcal{T}_{n_{\text{right}}}$ rooted at
the left and right children ${n}_{l}$ and ${n}_{r}$ of node $n$.

Leaf Node. For a tree $\mathcal{T}$, each leaf node $\ell\in\mathcal{L}$
defines a Gaussian distribution over output $y_{i}$ conditioned on
$\left\\{\mu_{l},\sigma^{2}_{l}\right\\}$. Since each example $\mathbf{x}_{i}$
has the probability $\omega_{\ell}(\mathbf{x}_{i}|\bm{\Theta)}$ to reach leaf
node $\ell$, considering all leaf nodes, the predictive distribution over
$y_{i}$ shall sum all leaf distribution weighted by
$\omega_{\ell}(\mathbf{x}_{i}|\bm{\Theta)}$:

$p_{\mathcal{T}}(y_{i}|\mathbf{x}_{i};\bm{\Theta},\bm{\pi})=\sum_{\ell\in\mathcal{L}}\omega_{\ell}(\mathbf{x}_{i}|\bm{\Theta)}p_{\ell}(y_{i}),$
(2)

where $\bm{\pi}$ represents the distribution parameters of all leaf nodes
associated with tree $\mathcal{T}$. Note that $\bm{\pi}$ varies along with
tree $\mathcal{T}_{k}$, and thus we rewrite them in terms of $\bm{\pi}_{k}$.

Forests of Regression Trees. Since a forest $\mathcal{F}$ consists of a number
of deep regression trees
$\left\\{\mathcal{T}_{1},...,\mathcal{T}_{k}\right\\}$, the predictive output
distribution shall consider all trees:

$p_{\mathcal{F}}\left(y_{i}|\mathbf{x}_{i},\bm{\Theta},\bm{\Pi}\right)=\frac{1}{K}\sum_{k=1}^{K}p_{\mathcal{T}_{k}}\left(y_{i}|\mathbf{x}_{i};\bm{\Theta},\bm{\pi}_{k}\right),$
(3)

where $K$ is the number of trees and
$\bm{\Pi}=\left\\{\bm{\pi}_{1},...,\bm{\pi}_{K}\right\\}$.
$p_{\mathcal{F}}\left(y_{i}|\mathbf{x}_{i},\bm{\Theta},\bm{\Pi}\right)$
denotes the likelihood that the $i^{th}$ sample has output $y_{i}$.

###  III-B Objective

Figure 2: The average rank of each age group in the $1^{st}$ pace. SP-DRFs
tend to rank the underrepresented examples at the end. In contrast, SPUDRFs
tend to rank the underrepresented examples in the front to ensure they are
selected for training from the very beginning.

Underrepresented Examples. Considering underrepresented examples in SPL is one
of the main contributions of this work. In this work, underrepresented
examples mean “minority”. The underrepresented level could be measured by
predictive uncertainty (_i.e._ entropy). In fact, underrepresented examples
may incur unfair treatment in early paces due to imbalanced data distribution.
Our proposed method tackles the ranking and selection problems in SPL from a
new perspective: _fairness_.

Entropy. Given a sample $\mathbf{x}_{i}$, its predictive uncertainty is
formulated by calculating the entropy of the conditional output distribution
$p_{\mathcal{F}}\left(y_{i}|\mathbf{x}_{i},\bm{\Theta},\bm{\Pi}\right)$:

$H\left[p_{\mathcal{F}}\left(y_{i}|\mathbf{x}_{i},\bm{\Theta},\bm{\Pi}\right)\right]=\frac{1}{K}\sum^{K}_{k=1}H\left[p_{\mathcal{T}_{k}}\left(y_{i}|\mathbf{x}_{i},\bm{\Theta},\bm{\pi}_{k}\right)\right],$
(4)

where $H\left[\cdot\right]$ denotes entropy, and the entropy which corresponds
to the $k^{th}$ tree is:

$H\left[p_{\mathcal{T}_{k}}\left(y_{i}|\mathbf{x}_{i},\bm{\Theta},\bm{\pi}_{k}\right)\right]=\\\
-\int
p_{\mathcal{T}_{k}}\left(y_{i}|\mathbf{x}_{i},\bm{\Theta},\bm{\pi}_{k}\right)\ln
p_{\mathcal{T}_{k}}\left(y_{i}|\mathbf{x}_{i},\bm{\Theta},\bm{\pi}_{k}\right)dy_{i}.$
(5)

Given $\mathbf{x}_{i}$, the larger the entropy
$H\left[p_{\mathcal{F}}\left(y_{i}|\mathbf{x}_{i},\bm{\Theta},\bm{\Pi}\right)\right]$
is, the more uncertain the prediction should be, _i.e._ , the more
underrepresented the sample $\mathbf{x}_{i}$ is.

The predictive distribution
$p_{\mathcal{T}_{k}}\left(y_{i}|\mathbf{x}_{i};\bm{\Theta},\bm{\pi}_{k}\right)$
takes the form
$\sum_{\ell\in\mathcal{L}}\omega_{\ell}(\mathbf{x}_{i}|\bm{\Theta)}p_{\ell}(y_{i})$.
Because the integral of mixture of Gaussians in Eq. (5) is non-trivial, we
resort to Monte Carlo sampling. However, because this has a large
computational cost [51], we turn to use the lower bound of this integral to
approximate the integration:

$H\left[p_{\mathcal{T}_{k}}\left(y_{i}|\mathbf{x}_{i},\bm{\Theta},\bm{\pi}_{k}\right)\right]\approx\frac{1}{2}\sum_{\ell\in\mathcal{L}}\omega_{\ell}(\mathbf{x}_{i}|\mathbf{\Theta})\left[\ln\left(2\pi\sigma_{\ell}^{2}\right)+1\right].$
(6)

Formulation. Let $v_{i}$ denote a latent variable indicating whether the
$i^{th}$ sample is selected $(v_{i}=1)$ or not $(v_{i}=0)$ for training. Our
objective is to jointly maximize the log likelihood function with respect to
DRFs’ parameters $\bm{\Theta}$ and $\bm{\Pi}$, and learn the latent variables
$\mathbf{v}=\left(v_{1},...,v_{N}\right)^{T}$. Fig. 2 shows that the original
self-paced method may miss the underrepresented examples at an early pace,
resulting in ignorance. Considering fairness, we prefer to select the
underrepresented examples, which probably have higher predictive uncertainty
(_i.e._ entropy), particularly at an early pace. Therefore, we maximize the
likelihood function of DRFs coupled with the sample selection term meanwhile
considering ranking fairness,

$\max_{\bm{\Theta},\bm{\Pi},\mathbf{v}}\sum_{i=1}^{N}v_{i}\left\\{\log
p_{\mathcal{F}}\left(y_{i}|\mathbf{x}_{i},\bm{\Theta},\bm{\Pi}\right)+\gamma
H_{i}\right\\}+\lambda\sum_{i=1}^{N}v_{i},$ (7)

where $\lambda$ controls learning pace ($\lambda\geq 0$), $\gamma$ is a trade-
off between loss and uncertainty ($\lambda\geq 0$), $H_{i}$ denotes the
entropy of output distribution
$p_{\mathcal{F}}\left(y_{i}|\mathbf{x}_{i},\bm{\Theta},\bm{\Pi}\right)$. When
$\gamma$ decays to $0$, the objective function is equivalent to the log-
likelihood function with respect to DRFs’ parameters $\bm{\Theta}$ and
$\bm{\Pi}$. In Eq. (7), each sample is weighted by $v_{i}$, and whether $\log
p_{\mathcal{F}}\left(y_{i}|\mathbf{x}_{i},\bm{\Theta},\bm{\Pi}\right)+\gamma
H_{i}>-\lambda$ determines the $i^{th}$ sample is selected. That is, the
sample with high likelihood value or entropy may be selected. The optimal
$v_{i}^{*}$ is:

$\displaystyle v_{i}^{*}=\left\\{\begin{array}[]{ll}1&\textrm{if $\log
p_{\mathcal{F}i}+\gamma H_{i}>-\lambda$}\\\
0&\textrm{otherwise}\end{array}\right.,$ (10)

where $p_{\mathcal{F}}\left(y_{i}|\mathbf{x}_{i},\bm{\Theta},\bm{\Pi}\right)$
is denoted by $p_{\mathcal{F}i}$ for simplicity, and we use this
representation in all following parts. Iteratively increasing $\lambda$ and
decreasing $\gamma$, samples are dynamically involved for training DRFs,
starting with easy and underrepresented examples and ending up with all
samples. Thus, SPUDRFs are prone to achieve more robust and less biased
solutions.

One might argue that the noisy and hard examples tend to have high predictive
uncertainty, with the result that they are selected in the early pace. In
fact, from Eq. (10), we observe that whether a sample is selected is
determined by both its predictive uncertainty and the log likelihood of being
predicted correctly. The noisy and hard examples probably have relatively
large loss _i.e._ low log likelihood $\log p_{\mathcal{F}i}$, and would not be
selected in the early pace.

###  III-C Weighting Schemes

Weighting each sample according to its importance would be more reasonable in
SPUDRFs. In the previous section, we adopted a hard weighting scheme in
SPUDRFs, as defined in Eq. (7), where one sample selected or not is determined
by $v_{i}$. Such a weighting scheme appears to omit the importance of samples.
SPUDRFs can easily incorporate other weighting schemes, including mixture
weighting and soft weighting [52].

Mixture Weighting. Mixture weighting scheme [52] weights selected sample by
its importance, _i.e._ , $0\leq v_{i}\leq 1$. The objective function under
mixture weighting scheme is:

$\max_{\bm{\Theta},\bm{\Pi},\mathbf{v}}\sum_{i=1}^{N}v_{i}\left\\{\log
p_{\mathcal{F}i}+\gamma
H_{i}\right\\}+\zeta\sum_{i=1}^{N}\log\left(v_{i}+\zeta/\lambda\right),$ (11)

where $\zeta$ is a parameter controlling the learning pace. We set
$\zeta=\left(\frac{1}{\lambda^{\prime}}-\frac{1}{\lambda}\right)^{-1}$, and
$\lambda>\lambda^{\prime}>0$ to construct a reasonable soft weighting
formulation. The self-paced regularizer in Eq. (11) is convex with respect to
$v\in\left[0,1\right]$. Then, setting the partial gradient of Eq. (11) with
respect to $v_{i}$ to be zero will lead the following equation:

$\log p_{\mathcal{F}i}+\gamma H_{i}+\frac{\zeta}{v_{i}+\zeta/\lambda}=0.$ (12)

The optimal solution of $v_{i}$ is given by:

$\displaystyle v_{i}^{*}=\left\\{\begin{array}[]{ll}1&\textrm{if $\log
p_{\mathcal{F}i}+\gamma H_{i}\geq-\lambda^{\prime}$}\\\ 0&\textrm{if $\log
p_{\mathcal{F}i}+\gamma H_{i}\leq-\lambda$}\\\ \frac{-\zeta}{\log
p_{\mathcal{F}i}+\gamma
H_{i}}-\zeta/\lambda&\textrm{otherwise}\end{array}\right.$ (16)

If either the log likelihood or the entropy is too large, $v^{*}_{i}$ equals
1. In addition, if the likelihood and entropy are both too small, $v^{*}_{i}$
equals 0. Except for the above two cases, the soft weighting, _i.e._ , the
last line of Eq. (16), is adopted.

Soft Weighting. A soft weighting scheme [52] weights a selected sample with
respect to its output likelihood and entropy. Such a weighting scheme
includes: linear weighting scheme and logarithmic weighting scheme.

Linear weighting. This scheme linearly assigns weights to samples with respect
to output likelihood and entropy. The objective of SPUDRFs under linear
weighting scheme is formulated as:

$\max_{\bm{\Theta},\bm{\Pi},\mathbf{v}}\sum_{i=1}^{N}v_{i}\left\\{\log
p_{\mathcal{F}i}+\gamma
H_{i}\right\\}-\frac{1}{2}\lambda\sum_{i=1}^{N}\left(v_{i}^{2}-2v_{i}\right),$
(17)

where $\lambda>0$, $v_{i}\in[0,1]$. We set the partial gradient of Eq. (17)
with respect to $v_{i}$ to be zero, then the optimal solution for $v_{i}$ is:

$\displaystyle v_{i}^{*}=\left\\{\begin{array}[]{ll}\frac{\log
p_{\mathcal{F}i}+\gamma H_{i}+\lambda}{\lambda}&\textrm{if $\log
p_{\mathcal{F}i}+\gamma H_{i}\geq-\lambda$}\\\
0&\textrm{otherwise}\end{array}.\right.$ (20)

The larger the log likelihood and entropy are, the higher the weight
associated with the $i^{th}$ sample should be.

Logarithmic weighting. This scheme penalizes the output likelihood and entropy
logarithmically. The objective of SPUDRFs under a logarithmic weighting scheme
can be formulated as:

$\max_{\bm{\Theta},\bm{\Pi},\mathbf{v}}\sum_{i=1}^{N}v_{i}\left\\{\log
p_{\mathcal{F}i}+\gamma H_{i}\right\\}-\sum_{i=1}^{N}\left(\zeta
v_{i}-\frac{\zeta^{v_{i}}}{\log\zeta}\right),$ (21)

where $\zeta=1-\lambda$, $0<\lambda<1$ and $v_{i}\in[0,1]$. Similarly, the
optimal solution for $v_{i}$ is

$\displaystyle v_{i}^{*}=\left\\{\begin{array}[]{ll}\frac{\log\left(\zeta-\log
p_{\mathcal{F}i}-\gamma H_{i}\right)}{\log\zeta}&\textrm{if $\log
p_{\mathcal{F}i}+\gamma H_{i}\geq-\lambda$}\\\
0&\textrm{otherwise}\end{array}.\right.$ (24)

###  III-D Underrepresented Example Augmentation

As previously explained, the SPUDRFs method places more emphasis on
underrepresented examples and may achieve less biased solutions. Since the
intrinsic reason for SPL’s biased solutions is the ignorance of
underrepresented examples, we further rebalance training data via distribution
reconstruction. Specifically, we distinguish the underrepresented examples
whose $H_{i}$ are larger than $\beta$, from regular examples at each pace and
augment them. As the number of underrepresented examples increases through
augmentation, the label distribution imbalance problem is alleviated.

###  III-E Optimization

To optimize SPUDRFs, as defined in Eq. (7), we propose a two-step alternative
search strategy (ASS) algorithm: (i) For sample selection, update $\mathbf{v}$
with fixed $\bm{\Theta}$ and $\bm{\Pi}$ (ii) update $\bm{\Theta}$ and
$\bm{\Pi}$ with current fixed sample weights $\mathbf{v}$. In addition, with
fixed $\mathbf{v}$, our DRFs are learned by alternatively updating
$\bm{\Theta}$ and $\bm{\Pi}$. In [33], the parameters $\bm{\Theta}$ for split
nodes (_i.e._ parameters for VGG) are updated through gradient descent since
the loss is differentiable with respect to $\bm{\Theta}$. In comparison, the
parameters $\bm{\Pi}$ for leaf nodes are updated by virtue of variational
bounding [33] when fixing $\bm{\Theta}$.

Algorithm 1 The training process of SPUDRFs.

0: $\mathcal{D}=\left\\{\mathbf{x}_{i},\mathbf{y}_{i}\right\\}_{i=1}^{N}$.

0: Model parameters $\bm{\Pi}$, $\bm{\Theta}$.

1: Initialize $\bm{\Pi}^{0}$, $\bm{\Theta}^{0}$, $\lambda^{0}$, $\gamma^{0}$.

2: while not converged do

3: Update $\mathbf{v}$ by Eq. (10).

4: while not converged do

5: Randomly select a batch from $\mathcal{D}$.

6: Calculate $H_{i}$ for each sample by Eq. (6).

7: Update $\bm{\Theta}$ and $\bm{\Pi}$ to maximize Eq. (7).

8: end while

9: Increase $\lambda$ and decrease $\gamma$.

10: end while

##  IV Robust Self-Paced Deep Regression Forests with Consideration on
underrepresented examples

As has already been discussed in Sec. III, SPL tends to place more emphasis on
reliable examples to achieve more robust solutions. However, SPL’s intrinsic
selection scheme can not exclude noisy examples, especially the examples with
labeling noise. To alleviate this drawback, we introduce capped-likelihood
function in SPUDRFs, which can render an output likelihood with an especially
small value as zero:

$\text{cap}(p_{\mathcal{F}i},\epsilon)=\frac{\max(p_{\mathcal{F}i}-\epsilon,0)}{p_{\mathcal{F}i}-\epsilon}p_{\mathcal{F}i},$
(25)

where $\epsilon$ denotes the threshold and $\epsilon>0$. Given the output
likelihood $p_{\mathcal{F}_{i}}$, the capped likelihood
$p^{c}_{\mathcal{F}_{i}}$ is defined as
$\text{cap}\left(p_{\mathcal{F}_{i}},\epsilon\right)$. Because the log-
likelihood values of the noisy examples are prone to be small, such an
operation sets capped likelihood $p^{c}_{\mathcal{F}_{i}}$ to be negatively
infinite. Thus, noisy examples would not be selected, especially not examples
with labeling noise. SPUDRFs with capped likelihood are defined as robust
SPUDRFs.

$\max_{\bm{\Theta},\bm{\Pi},\mathbf{v}}\sum_{i=1}^{N}v_{i}\left\\{\log
p_{\mathcal{F}i}^{c}+\gamma H_{i}\right\\}+\lambda\sum_{i=1}^{N}v_{i},$ (26)

The robust SPUDRFs can be optimized using the optimization method proposed in
Sec. III-E. Note that, if $p_{\mathcal{F}_{i}}\leq\epsilon$, the capped
likelihood $p^{c}_{\mathcal{F}_{i}}=0$, that is, $\log
p^{c}_{\mathcal{F}_{i}}=-\infty$. Since $\lambda$ is a positive factor,
maximizing Eq. (26) must yield $v_{i}=0$, meaning that the $i^{th}$ sample
would not be selected. By adjusting $\epsilon$, we can exclude a certain ratio
of noisy examples to obtain more robust solutions.

##  V Fairness Metric

In addition to _accuracy_ , fairness should also be an essential metric to
measure the performance of a regression model, particularly when the
regression targets are related to sensitive attributes. For example, in age
estimation, we expect our model to treat the younger and the elder fairly,
_i.e._ , not to result in large MAEs for the former but small MAEs for the
latter. The notion of fairness was originally defined concerning a protected
attribute such as gender, race or age. However, the term fairness has a range
of potential definitions. Here, we adopt a notion of fairness that is for
sensitive features [53]. _Defining a fairness metric for regression is one of
the contributions of this work._

The present studies [54, 55, 56, 57] mostly refer to fairness constraints in
regressions, for example, statistical parity or bounded group loss [54, 56],
but seldom refers to fairness metrics. In this work, we define a new fairness
metric for regression models. To be specific, the test dataset is divided into
$K$ subsets, each of which is denoted by
$\mathbf{D}_{k}=\left\\{\mathbf{x}_{i},y_{i}|y_{i}\in\mathcal{G}_{k}\right\\}$,
and $\mathcal{G}_{k}$ denotes the $k^{th}$ group. A fair model is expected to
have the same performance on all subsets, which can be described
mathematically as:

$\mathbb{E}_{k}\left[L\left(\hat{y},y\right)\right]=\mathbb{E}_{l}\left[L\left(\hat{y},y\right)\right]\quad\forall
k,l\in\left\\{1,2,...,K\right\\},k\neq l.$ (27)

where $\mathbb{E}_{k}\left[\cdot\right]$ denotes the expectation with respect
to the loss $L\left(\cdot\right)$ over group $\mathbf{D}_{k}$, $\hat{y}$ is
the predicted value of the model and $y$ is the real target value. Motivated
by $p\%-$rule [57], which measure classification fairness, we evaluate
fairness between two subsets $\mathbf{D}_{k}$ and $\mathbf{D}_{l}$ as follows:

$f\left(\mathbf{D}_{k},\mathbf{D}_{l}\right)=\min\left(\frac{\mathbb{E}_{k}\left[L\left(\hat{y},y\right)\right]}{\mathbb{E}_{l}\left[L\left(\hat{y},y\right)\right]},\frac{\mathbb{E}_{l}\left[L\left(\hat{y},y\right)\right]}{\mathbb{E}_{k}\left[L\left(\hat{y},y\right)\right]}\right).$
(28)

The loss expectation ratio characterizes the model’s fairness on every two
subsets. A small ratio means the performance on such two subsets is
significantly distinct, reflecting model bias against a particular subset. On
the contrary, a large ratio indicates the losses on $\mathbf{D}_{k}$ and
$\mathbf{D}_{l}$ are similar, indicating the model is relatively fair for
$\mathbf{D}_{k}$ and $\mathbf{D}_{l}$. In particularly, when the ratio is
equal to 1, the model satisfies Eq. (27), and $\mathbf{D}_{k}$ and
$\mathbf{D}_{l}$ are treated fairly.

(a)

(b)

(c)

Figure 3: Example images in the Morph II, FG-NET, BIWI, BU3DFE and MPIIGaze
datasets. (a): The images from the Morph II and FG-NET dataset. (b): The
images from the BIWI and BU3DFE dataset. (c): The images from MPIIGaze
dataset.

Finally, the regression fairness is defined as the expectation of the fairness
between any two subsets,

$\text{FAIR}=\mathbb{E}\left[f\left(\mathbf{D}_{i},\mathbf{D}_{j}\right)\right].$
(29)

In SPUDRFs, when we set $\gamma\textgreater 0$, all samples are sorted by both
likelihood and entropy. As a result, easy and underrepresented samples are
selected first, which means samples from all subsets would be selected at
early paces. Therefore, our model can alleviate the model’s prejudice against
different subsets and improve regression fairness. More extensive evaluation
results can be found in Sec. VI-F.

(a) Morph II

(b) MPIIGaze

Figure 4: Visualization of the SPL process on the Morph II and MPIIGaze
datasets. Left: The typical worst cases at each pace. The images in each panel
are sorted by the increasing paces. The two numbers below each image are the
labels (left) and predicted targets (right). Right: MAE comparison between SP-
DRFs and SPUDRFs at each pace. The MAE bins are sorted by the increasing
paces.

##  VI Experiment

###  VI-A Datasets and Experimental Setup

Dataset. There are five datasets used in our experiments, namely Morph II, FG-
NET, BIWI, BU3DFE and MPIIGaze.

Morph II. The Morph II [19] dataset contains $55,13$4 face images of $13618$
individuals with unbalanced gender and ethnicity distributions. These images
are near-frontal pose, neutral expression, and uniform illumination.

FG-NET. The FG-NET [20] dataset includes $1,002$ color and grey images of $82$
people, with each subject having more than $10$ photos at different ages.
Since all images were taken in an uncontrolled environment, there is a
significant deviation on the lighting, pose, and expression (_i.e._ PIE) of
faces inside the dataset.

BIWI. The BIWI dataset [21] includes $15678$ images collected by a Kinect
sensor device for different persons, and head poses with pitch, yaw, and roll
angles mainly ranging within $\pm 60^{\circ}$, $\pm 75^{\circ}$ and $\pm
50^{\circ}$. These images are from $20$ subjects, including ten males and six
females, where four males have been captured twice with/without glasses.

BU3DFE. The BU-3DFE dataset [22] is collected from $100$ subjects, of whom
$44$ are male, and $56$ are female. Following the work [22], we randomly
rotated the 3D face models to produce $6000$ images with pitch and yaw angles
ranging within $\pm 30^{\circ}$ and $\pm 90^{\circ}$.

MPIIGaze. The MPIIGaze dataset [23] includes $213659$ images from $15$
persons. The number of images for each person is between $1498$ and $34745$.
The normalized gaze angle is in the range of $[-20^{\circ},+5.0^{\circ}]$
degrees in vertical and $[-25^{\circ},+25^{\circ}]$ degrees in horizontal. Due
to massive deviation of image number amongst different persons, similar to
[23], $1500$ images from the left and right eyes of each person were chosen
for final experiment.

Reprocessing and Data Augmentation. MTCNN [58] was used for joint face
detection and alignment on the Morph II and FG-NET datasets. In addition,
following [33], two methods were adopted for data augmentation (i) random
cropping; and (ii) random horizontal flipping. On the BIWI and BU3DFE
datasets, only random cropping was adopted for augmentation. On the MPIIGaze
dataset, two normalized eye images (_i.e._ left and right) were obtained
following the work [23], and only random cropping was used for data
augmentation.

Figure 5: Visualization of leaf node distribution. The $1^{st}$, $3^{rd}$, and
$6^{th}$ paces are chosen for visualization. For SP-DRFs, the Gaussian means
of leaf nodes (denoted by red points in the second row) concentrate in a small
range, resulting in seriously biased solutions. For SPUDRFs, the Gaussian
means of leaf nodes (denoted by orange points in the third row) distribute in
a reasonable range, resulting in lower MAEs.

Parameter Setting. VGG-16 [59] was employed as the backbone network of
SPUDRFs. For MPIIGaze, following [41], two VGG-16 networks were used. In
addition, the pre-trained models were VGG-16 for MPIIGaze, and VGG-Face [9]
for other datasets. When training VGG-16, the batch size were $32$ for Morph
II, BIWI and BU3DFE, $8$ for FG-NET, and $128$ for MPIIGaze. The drop out
ratio was $0.5$. The maximun iterations in each pace was $80k$ for Morph II,
$10k$ for FG-NET, $20k$ for BIWI and BU3DFE, and $5k$ for MPIIGaze. SGD
optimizer was used for BIWI and Adam optimizer was used for other datasets.
The initial learning rate was $2\times 10^{-5}$ for Morph II and BU3DFE,
$1\times 10^{-5}$ for FG-NET, $0.2$ for BIWI, $3\times 10^{-5}$ for MPIIGaze.
We reduced the learning rate ($\texttimes 0.5$) per $10k$ iterations for Morph
II, BIWI and BU3DFE. Here, some hyper-parameter settings are slightly
different from the Caffe version [24].

The hyper-parameters of DRFs were: tree number ($5$), tree depth ($6$), output
unit number of feature learning ($128$). The iterations to update leaf node
predictions was $20$, and the number of images to update leaf node predictions
was the whole training set for MPIIGaze and $50$ times of batch size for other
datasets. In the first pace, $50\%$ samples were selected for training. Here,
$\lambda$ was set to guarantee the first $50\%$ samples with large $\log
p_{\mathcal{F}i}+\gamma H_{i}$ values were involved. $\lambda^{\prime}$ was
set to ensure that $10\%\sim 20\%$ selected samples adopt soft weighting under
a mixture weighing scheme. To promote efficiency, the samples selected in the
previous pace were preserved and the rest were ranked for current sample
selection. In addition, $\gamma$ was initialized to be $15$ for Morph II and
BIWI, and $5$ for FG-NET, BU-3DFE, and MPIIGaze. $\beta$ was used to recognize
underrepresented examples that need to be augmented twice in each pace ($1000$
for BIWI, $400$ for BU3DFE, $2000$ for MPIIGaze). The number of paces was
empirically set to be 10 for Morph II, $6$ for BIWI and MPIIGaze, $3$ for FG-
NET and BU-3DFE. Except for the first pace, an equal proportion of the rest
data was gradually involved in each pace.

Evaluation Details. To evaluate regression accuracy, we used the mean absolute
error (MAE). MAE is defined as
$\sum_{i=1}^{N}\left|\hat{y}_{i}-y_{i}\right|/N$, where $\hat{y_{i}}$
represents the predicted output for the $i^{th}$ sample, and $N$ is the total
number of images for testing. In addition, for age estimation, CS calculates
the percentage of images sorted in the range of
$\left[y_{i}-L,y_{i}+L\right]$:
$CS(L)=\sum_{i=1}^{N}\left[|\hat{y}_{i}-y_{i}|\leq L\right]/N\cdot 100\%$,
where $[\cdot]$ denotes an indicator function and $L$ is the error level.
Further, to measure regression fairness, we adopt our proposed fairness metric
FAIR given in Sec.V.

To calculate the above metrics, for Morph II, BIWI and BU3DFE, each dataset
was divided into a training set ($80\%$) and a testing dataset ($20\%$). The
random division was repeated $5$ times, and the reported MAEs were averaged
over $5$ times. The leave-one-person-out protocol was used for FG-NET [33] and
MPIIGaze [23], where one subject was used for testing and the left subjects
for training.

###  VI-B Validity of SP-DRFs and SPUDRFs

This section validates the original SPL method as well as the new proposed SPL
method for learning DDMs. In the following, SP-DRFs denotes self-paced deep
regression forests, which learn DRFs using the original SPL method.

Self-Paced Learning. Recall that learning DDMs in a gradual learning manner is
more consistent with the cognitive process of human beings, and the noisy
examples can be distinguished by virtue of learned knowledge. That means the
learning can place more emphasis on “reliable” examples. To show this, Fig.
4(a) illustrates the representative face images at each learning pace of SP-
DRFs on the Morph II dataset, sorted by increasing $\lambda$ and decreasing
$\gamma$. The two numbers below each image are the actual age (left) and
predicted age (right). We observe that the training images involved in each
pace become more confusing and noisy step by step, compared to images in early
paces. Since the current model is initialized by the results obtained at the
last pace, the updated model is adaptively calibrated by “reliable” examples
rather than by confusing and noisy ones. Thus, SP-DRFs have improved MAE and
CS compared to DRFs. We observe that SP-DRFs gain improvement by 0.33 in MAE
$\left(2.17\rightarrow 1.84\right)$, by $1.55\%$ in CS
$\left(91.3\%\rightarrow 92.85\%\right)$, as shown in Fig. 6(a).

Some representative eye images in the gradual learning sequence for MPIIGaze
dataset are shown in Fig. 4(b). The images in each panel are sorted by
increasing $\lambda$ and decreasing $\gamma$. The two numbers below each pair
of images are the actual gaze direction (left) and the predicted gaze
direction (right). The same phenomena—the easy examples are prone to be
selected in the early paces, while the confusing and noisy ones are prone to
be selected in the later paces—can be observed. Since the updated model at
each pace is adaptively calibrated by “reliable” examples rather than by
confusing and noisy ones, thus, the MAE associated with each pace decreases
step by step. Finally, the MAE of SP-DRFs decreases to be $4.57$, whereas the
MAE of DRFs is $4.62$ (see Table. II).

Considering Ranking Fairness. As was mentioned in Sec. III, the existing SPL
methods may exacerbate the bias of solutioins. Fig. 5 visualizes the leaf node
distributions of SP-DRFs in the gradual learning process. The Gaussian means
$\mu_{l}$ associated with the $160$ leaf nodes, where each $32$ leaf nodes are
defined for $1$ tree, are plotted in each sub-figures. Three paces, _i.e._ the
$1^{st}$, $3^{rd}$, and $6^{th}$ pace, are randomly chosen for visualization.
For clarity, only pitch and yaw angles are shown. Meanwhile, the leaf node
distributions of SPUDRFs are also visualized in Fig. 5.

In Fig. 5, the comparison between SP-DRFs and SPUDRFs validates our proposed
new SPL method. In SP-DRFs, because the ranking fairness is not considered,
the leaf nodes (red points in the $2^{rd}$ row) are only concentrated in a
small range that most samples are distributed over, thus leading to seriously
biased solutions. The poor MAEs of SP-DRFs can serve as evidence for this. In
contrast, because the ranking fairness is considered in SPUDRFs, the leaf
nodes are distributed over a wide range that could cover underrepresented
examples, thus improving performance. SPUDRFs, in the pitch and yaw
directions, achieve the best performance with MAEs of $0.71$ and $0.69$,
compared to SP-DRFs with MAEs of $1.05$ and $1.23$ ($47.9\%$ and $78.3\%$
improvements).

###  VI-C Comparison with State-of-the-art Methods

We compare SPUDRFs with other state-of-the-art methods on three vision tasks:
age estimation, head pose estimation and gaze estimation.

Results on Age Estimation. The comparison between our method and other
baselines on the Morph II and FG-NET datasets are shown in Fig. 6(a) and Fig.
7(b). The baseline methods include: LSVR [60], RCCA [61], OHRank [62], OR-CNN
[10], Ranking-CNN [3], DRFs [33], DLDL-v2 [8], and PML [35]. The results show
some consistent trends. First, SPUDRFs have superior performance compared to
conventional discriminative models, such as LSVR [60] and OHRank [62]. Second,
SP-DRFs consistently outperform other DDMs. Compared to DRFs, our gains in MAE
are $0.33$ on Morph II and 0.12 on FG-NET, and in CS are $1.55\%$ and
$2.91\%$, respectively. The promotions demonstrate that learning DRFs in a
self-paced manner is more reasonable. Third, SPUDRFs outperform SP-DRFs on
both MAE and CS.

Fig. 6(b) shows the CS curves of SPUDRFs, SP-DRFs and other baseline methods
on the Morph II dataset, and Fig. 7(b) shows the CS curves of different
methods on the FG-NET dataset. On both datasets above, SPUDRFs consistently
outperform other DDMs. We observe, the CS of SPUDRFs reaches $93.34\%$ at
error level $L=5$, significantly outperforming DRFs by $2.04\%$ on the Morph
II dataset. We also observe that SPUDRFs outperform DRFs by $4.09\%$ in CS, on
the FG-NET dataset. The CS increase clearly validates our proposed self-paced
learning method.

Method | MAE$\downarrow$ | CS$\uparrow$
---|---|---
LSVR [60] | 4.31 | 66.2%
RCCA [61] | 4.25 | 71.2%
OHRank [62] | 3.82 | N/A
OR-CNN [10] | 3.27 | 73.0%
Ranking-CNN [3] | 2.96 | 85.0%
DLDL-v2 [8] | 1.97 | N/A
DRFs [33] | 2.17 | 91.3%
PML [35] | 2.15 | N/A
SP-DRFs | 1.84 | 92.85%
SPUDRFs | 1.78 | 93.34%
(a)

(b)

Figure 6: Quantitative comparison with state-of-the-art methods on the Morph
II dataset. Upper: MAE comparison results. Lower: CS comparison results.

Method | MAE$\downarrow$ | CS$\uparrow$
---|---|---
IIS-LDL [63] | 5.77 | N/A
LARR [64] | 5.07 | 68.9%
MTWGP [65] | 4.83 | 72.3%
DIF [66] | 4.80 | 74.3%
OHRank [62] | 4.48 | 74.4%
CAM [67] | 4.12 | 73.5%
DRFs [33] | 2.80 | 84.50%
SP-DRFs | 2.68 | 87.41%
SPUDRFs | 2.64 | 88.59%
(a)

(b)

Figure 7: Quantitative comparison with state-of-the-art methods on the FG-NET
dataset. Upper: MAE comparison results. Lower: CS comparison results.

Results on Head Pose Estimation. Recall that considering underrepresented
examples in SPL is particularly crucial when the training data has an
imbalanced distribution problem. Sec. VI-B has shown the effectiveness of SP-
DRFs and SPUDRFs for head pose estimation tasks on the BIWI dataset. Fig. 5
shows the Gaussian means of learned leaf nodes of SP-DRFs and SPUDRFs on the
BIWI dataset. In SP-DRFs, because the underrepresented examples are neglected,
the learned leaf nodes are distributed over a small range, resulting in
seriously biased solutions. In SPUDRFs, owing to consideration of
underrepresented examples, the learned leaf nodes are distributed over a wide
range, leading to steadily improved MAE results. This section aims to show
that considering ranking fairness in SPL can obtain more reasonable results.

Tab. I shows the comparison results of SP-DRFs and SPUDRFs with recent head
pose estimation approaches. Because SVR [68], RRF [69] and KPLS [70] are all
conventional regression methods, they have inferior performance relative to
MoDRN. SPUDRFs achieve the best performance with an MAE of $0.74$ on the BIWI
dataset and $0.82$ on the BU-3DFE dataset, which is state-of-the-art
performance. Such a significant gain in MAE $(1.13\rightarrow 0.74)$ on BIWI
demonstrates that considering ranking fairness in early paces when training
DRFs can lead to much more reasonable results. The explanation is that, as
illustrated in Fig. 5, the learned leaf nodes of SPUDRFs are distributed over
a wide range that can cover underrepresented examples.

TABLE I: Quantitative comparison with state-of-the-art methods. Left: Comparison results on the BIWI dataset, Right: Comparison results on the BU-3DFE dataset. | Method |  MAE$\downarrow$
---|---
SVR [68] | 3.14
RRF [69] | 3.06
KPLS [70] | 2.88
SAE [71] | 1.94
MoDRN [4] | 1.62
DRFs [33] |  1.33111The reported results are better than our previous work [24] because we used floating point labels in our current experiment but integral labels in [24].
SP-DRFs | 1.13
SPUDRFs | 0.74

| Method |  MAE$\downarrow$
---|---
SVR [68] | 4.21
SAE [71] | 4.14
KPLS [70] | 4.12
RRF [69] | 4.09
MoDRN [4] | 3.86
DRFs [33] | 0.99
SP-DRFs | 0.89
SPUDRFs | 0.82

Results on Gaze Estimation. For gaze estimation, the accuracy comparisons
between our proposed SP-DRFs, SPUDRFs and other baselines on the MPIIGaze
dataset are shown in Tab. II. Both the MAE and standard deviation across all
persons are reported. Note that the standard deviation of RT-GENE [41], Pict-
Gaze [42], and Ordinal Loss [72] are not reported because the original studies
do not provide this information. As shown in Tab. II, we observe that SPUDRFs
outperform all baseline methods in MAE, confirming the effectiveness of our
proposed method. Because RT-GENE [41] directly maps the features extracted by
VGG-16 to gaze through multiple FC layers, it has inferior performance
relative to DRFs $(4.62\rightarrow 4.80)$. For a fair comparison, we chose RT-
GENE with $1$ model. Further, because SPUDRFs method learn DRFs in a self-
paced manner and take into account ranking fairness, it has a further gain in
MAE over DRFs $(4.62\rightarrow 4.45)$. Pict-Gaze [42] regresses an input
image to an intermediate pictorial representation and then regresses the
representation to the gaze direction. Ordinal Loss [72] utilizes ordinal loss
with order regularization to solve the regression problem. The two methods
mentioned above do not take into consideration the differences amongst
samples; they train all examples simultaneously and thus have inferior MAEs.
We observe that the MAE of Pict-Gaze [42] and Ordinal Loss [72] are $4.56$ and
$4.49$ respectively, while ours is $4.45$. It’s noteworthy that SP-DRFs, when
compared with DRFs, only promotes MAE slightly. This is probably due to the
obvious distribution difference between training data and test data in the
leave-one-out setting.

TABLE II: Quantitative comparison with state-of-the-art methods on the MPIIGaze dataset. | Method |  MAE$\downarrow$
---|---
MPIIGaze [23] |  6.30$\pm 1.80$
iTracker [40] |  6.20$\pm 0.85$
GazeNet+ [73] |  5.40$\pm 0.67$
MeNets [44] |  4.90$\pm 0.59$
RT-GENE [41] |  4.80$\pm--$
Pict-Gaze [42] |  4.56$\pm--$
Ordinal Loss [72] |  4.49$\pm--$
DRFs [33] |  4.62$\pm 0.89$
SP-DRFs |  4.57$\pm 0.78$
SPUDRFs | $\textbf{4.45}\pm\textbf{0.84}$

TABLE III: Quantitative Evaluation Results using Different Weighting Schemes.  Weighting Schemes | MORPH | FGNET | BIWI | BU-3DFE | MPIIGaze
---|---|---|---|---|---
SP-DRFs | SPUDRFs | SP-DRFs | SPUDRFs | SP-DRFs | SPUDRFs | SP-DRFs | SPUDRFs | SP-DRFs | SPUDRFs
Hard | 1.85 | 1.78 | 2.68 | 2.66 | 1.24 | 0.76 | 0.94 | 0.84 | 4.58 | 4.45
Linear | 1.84 | 1.80 | 2.69 | 2.66 | 1.18 | 0.79 | 0.92 | 0.82 | 4.57 | 4.46
Log | 1.85 | 1.81 | 2.68 | 2.64 | 1.13 | 0.77 | 0.89 | 0.82 | 4.60 | 4.48
Mixture | 1.84 | 1.80 | 2.79 | 2.70 | 1.26 | 0.74 | 0.93 | 0.86 | 4.58 | 4.45

###  VI-D Different Weighting Schemes

A potential concern for SP-DRFs and SPUDRFs is that different weighting
schemes could affect the estimation performance on a variety of visual tasks.
To evaluate this, we compare SP-DRFs/SPUDRFs under different weighting
schemes, including hard, mixture, and soft weighting. Under all weighting
schemes, $\lambda$ was set as in Sec. VI-A. Under the mixture weighting
scheme, $\lambda^{\prime}$ was set to ensure that $10\%$ selected samples
adopt soft weighting for Morph II, and 20% samples for other datasets.

Table. III shows the comparison results. The performances of SP-DRFs/SPUDRFs
with different weighting schemes are only slightly different. We observe that
the weights for a large proportion of examples are close to 1. For example,
under the log weighting scheme, only $2\%$ of examples have weights below
$0.5$. Under the mixture weighting scheme, the proportion of samples whose
weights are 1 can be set manually. We chose to set this proportion to be
$0.8\sim 0.9$ on different tasks. The MAEs are not guaranteed to be better
than other weighting schemes.

###  VI-E Robust SPUDRFs

The intuition for SPUDRFs to work better on different visual tasks is its
improved robustness, _i.e._ , emphasizing more on “reliable” examples. To
further promote the robustness, we propose robust SPUDRFs in Sec. IV, which
enable SPUDRFs to handle labeling noise. We added noise to labels to test the
validity of robust SPUDRFs on the above datasets. Specifically, we chose
$10\%$ samples in Morph II, BIWI or MPIIGaze datasets, and added Gaussian
noise $\mathcal{N}\left(0,10\right)$ to their labels. To control the
proportion ($0\%\sim 20\%$) of samples whose likelihoods are capped to be 0,
_i.e._ , the portion of samples to be excluded, we set $\epsilon$ at variant
values.

Fig. 8 shows the MAE curves of SPUDRFs across variant capped proportions. We
observe that, when no example’s likelihood is capped at 0, due to the presence
of noise, the corresponding MAE ais large for each dataset. As the capped
proportion grows, the MAE gradually decreases. When the capped proportion
changes to become $10\%$, the MAE in Fig. 8 almost achieves minimal values,
which demonstrate that robust SPUDRFs are capable of excluding noisy examples.
When the capped proportion grows continuously, the MAE changes to become
large. One explanation is that some regular examples may be discarded when the
capped proportion becomes over 10%.

(a)

(b)

(c)

Figure 8: The MAEs of SPUDRFs across different capped proportions evaluated on
three datasets: Morph II, BIWI, and MPIIGaze.

###  VI-F Fairness Improvement

(a)

(b)

Figure 9: Comparison of different methods on the BIWI dataset. In pitch and
yaw directions, SP-DRFs and SPUDRFs outperform DRFs in MAE on most groups, but
the MAEs of SP-DRFs are worse than DRFs on underrepresented groups.

This section discusses how SPUDRFs improve regression fairness on different
visual tasks.

Taking into account of underrepresented examples, SPUDRFs show improved
accuracy for age, pose and gaze estimation, compared to SP-DRFs. In this
section, we show how SPUDRFs further improve regression fairness. In Sec. V,
we define FAIR as a regression fairness metric. To evaluate the regression
fairness of DRFs, SP-DRFs, and SPUDRFs, we divided all the datasets mentioned
above into different subsets and calculated FAIR on them. For Morph II, the
entire data was divided into $7$ groups, _i.e._ ,
$[10,20],[20,30],\ldots,[70,80]$. A similar division was conducted on the FG-
NET dataset. For BIWI and BU-3DFE, the pitch, yaw, and roll directions were
regarded as independent directions, and the interval for each regression group
is $10^{\circ}$. For MPIIGaze, the pitch and yaw directions were regarded as
relevant angles. We set the group interval to be $5^{\circ}$ in each
direction, resulting in a total of 60 groups.

Table. IV shows the FAIR values on all the datasets. The higher the FAIR is,
the more fair the regression model is. Note that SP-DRFs always have inferior
FAIR relative to DRFs. For example, SP-DRFs have a FAIR of $0.43$, while DRFs
have a FAIR of $0.46$ for BIWI, and SP-DRFs have a FAIR of $0.37$ while DRFs
have a FAIR of $0.42$ for FG-NET. The results demonstrate that SP-DRFs tend to
aggravate the bias of solutions. On the other hand, SPUDRFs have significantly
improved FAIR compared to SP-DRFs. We observe SPUDFRs gains improvement by
$5\%$ for FGNET and $27\%$ for BIWI. This is an evidence that SPUDRFs tend to
have more fair estimation results on various visual tasks.

TABLE IV: Regression Fairness Evaluation on Different Datasets. Methods | Morph II | FGNET | BIWI | BU-3DFE | MPIIGaze
---|---|---|---|---|---
DRFs | 0.46 | 0.42 | 0.46 | 0.74 | 0.67
SP-DRFs | 0.44 | 0.37 | 0.43 | 0.72 | 0.67
SPUDRFs | 0.48 | 0.42 | 0.70 | 0.76 | 0.69

Some pose estimation results of DRFs, SP-DRFs, and SPUDRFs for BIWI are shown
in Fig. 9. The estimation accuracy for different pose groups reveals some
consistent trends. First, for most groups, SP-DRFs and SPUDRFs have superior
MAEs over DRFs. In Sec. III-B, we discussed that these gains in fairness are
due to SPL, which can guide DDMs to achieve more reasonable solutions. Second,
SP-DRFs tend to have even worse MAEs for the underrepresented groups than
DRFs, for example, $\left[30^{\circ},60^{\circ}\right]$ in the pitch direction
and $\left[-70^{\circ},-40^{\circ}\right]$ in the yaw direction. This
demonstrates that existing SPL methods have a fatal drawback: the ranking and
selection schemes may incur seriously biased estimation results. Third, also
for underrepresented groups, SPUDRFs gain significant improvement in MAE
compared to DRFs. For example, our gains in MAE are $87.71\%$ and $77.68\%$
from $[-60^{\circ},-50^{\circ}]$ and $[30^{\circ},60^{\circ}]$ in pitch
direction, $80.96\%$ and $89.52\%$ from $[-70^{\circ},-40^{\circ}]$ and
$[60^{\circ},70^{\circ}]$ in yaw direction. This also serves as an evidence
that our proposed self-paced method alleviates the fairness problem in
existing SPL methods.

##  VII Discussion

In the experiments above, we have evaluated SPUDRFs against other baseline
methods on different visual tasks, such as facial age estimation, head pose
estimation and gaze estimation (Sec. VI-C). We also evaluated the SPUDRFs
method under different weighting schemes, its extension with capped likelihood
formulation, and its performance improvement on fairness. On a number of tasks
and datasets, SPUDRFs and SP-DRFs outperform other baseline methods. The
advantage of considering ranking fairness in SPUDRFs is most obvious for BIWI.
For Morph II and BU-3DFE, the performance improvement when considering ranking
fairness in SPUDRFs is also observable.

Learning DDMs in a self-paced manner has some limitations. The most noticeable
one is that, in a leave-one-out setting, SP-DRFs perform only slightly better
than DRFs. We speculate that this is due to the data distribution difference
between training set and test set. Therefore, whether SPL can improve the
performance of DDMs may largely depend on the distribution divergence between
training data and test data.

##  VIII Conclusion

This paper explored how self-paced learning guided deep discriminative models
(DDMs) to obtain robust and less biased solutions on different visual tasks,
_e.g._ facial age estimation, head pose estimation, and gaze estimation. A
novel self-paced method, which considers ranking fairness, was proposed.
Specifically, a new ranking scheme that jointly considers loss and fairness
was proposed in SPL. Such a method was combined with a typical DDM—deep
regression forests (DRFs)—and led to a new model, namely deep regression
forests with consideration on underrepresented examples (SPUDRFs). In
addition, SPUDRFs under different weighting schemes, their extension with
capped likelihood formulation, and their performance improvement on fairness
were discussed. Extensive experiments on three well-known computer vision
tasks demonstrated the efficacy of our proposed new self-paced method. The
future work will include exploring how to incorporate such a method with other
DDMs.

## Acknowledgment

The authors would like to thank Jiabei Zeng of the Institute of Computing
Technology, Chinese Academy on Sciences, for the valuable advice on the gaze
estimation experiments. This work is partially supported by the National Key
R&D Program of China AI2021ZD0112000, National Natural Science Fundation of
China Nos. 62171111, 61806043, 61971106 and 61872068, and the Special Science
Foundation of Quzhou No. 2020D013.

## References

  * [1] S. Chen, C. Zhang, and M. Dong, “Deep age estimation: From classification to ranking,” _IEEE Transactions on Multimedia_ , vol. 20, no. 8, pp. 2209–2222, 2018.
  * [2] C. Yan, C. Lang, W. Tao, X. Du, and Z. Chen, “Age estimation based on convolutional neural network,” in _Pacific Rim Conference on Multimedia_ , 2014, pp. 211–220.
  * [3] S. Chen, C. Zhang, M. Dong, J. Le, and M. Rao, “Using Ranking-CNN for age estimation,” in _CVPR_ , 2017, pp. 742–751.
  * [4] Y. Huang, L. Pan, Y. Zheng, and M. Xie, “Mixture of deep regression networks for head pose estimation,” in _ICIP_. IEEE, 2018, pp. 4093–4097.
  * [5] R. Rothe, R. Timofte, and L. Van Gool, “Deep expectation of real and apparent age from a single image without facial landmarks,” _IJCV_ , vol. 126, no. 2-4, pp. 144–157, 2018.
  * [6] D. Huang, L. Han, and F. De la Torre, “Soft-margin mixture of regressions,” in _CVPR_ , 2017, pp. 4058–4066.
  * [7] N. Ruiz, E. Chong, and J. M. Rehg, “Fine-grained head pose estimation without keypoints,” in _CVPRW_ , 2018, pp. 2074–2083.
  * [8] B.-B. Gao, H.-Y. Zhou, J. Wu, and X. Geng, “Age estimation using expectation of label distribution learning,” in _IJCAI_ , 2018, pp. 712–718.
  * [9] O. M. Parkhi, A. Vedaldi, and A. Zisserman, “Deep face recognition,” 2015.
  * [10] Z. Niu, M. Zhou, L. Wang, X. Gao, and G. Hua, “Ordinal regression with multiple output CNN for age estimation,” in _CVPR_ , 2016, pp. 4920–4928.
  * [11] E. Agustsson, R. Timofte, and L. Van Gool, “Anchored regression networks applied to age estimation and super resolution,” in _ICCV_ , 2017, pp. 1643–1652.
  * [12] Y. Cui, M. Jia, T.-Y. Lin, Y. Song, and S. Belongie, “Class-balanced loss based on effective number of samples,” in _CVPR_ , 2019, pp. 9268–9277.
  * [13] S. Khan, M. Hayat, S. W. Zamir, J. Shen, and L. Shao, “Striking the right balance with uncertainty,” in _CVPR_ , 2019, pp. 103–112.
  * [14] M. Ren, W. Zeng, B. Yang, and R. Urtasun, “Learning to reweight examples for robust deep learning,” _arXiv preprint arXiv:1803.09050_ , 2018.
  * [15] M. P. Kumar, B. Packer, and D. Koller, “Self-paced learning for latent variable models,” in _NeurIPS_ , 2010, pp. 1189–1197.
  * [16] L. Jiang, D. Meng, Q. Zhao, S. Shan, and A. G. Hauptmann, “Self-paced curriculum learning,” in _AAAI_ , 2015, pp. 2694–2700.
  * [17] F. Ma, D. Meng, Q. Xie, Z. Li, and X. Dong, “Self-paced co-training,” in _ICML_ , 2017, pp. 2275–2284.
  * [18] H. Li and M. Gong, “Self-paced convolutional neural networks,” in _IJCAI_ , 2017, pp. 2110–2116.
  * [19] K. Ricanek and T. Tesafaye, “Morph: A longitudinal image database of normal adult age-progression,” in _FG_. IEEE, 2006, pp. 341–345.
  * [20] G. Panis, A. Lanitis, N. Tsapatsoulis, and T. F. Cootes, “Overview of research on facial ageing using the fg-net ageing database,” _Iet Biometrics_ , vol. 5, no. 2, pp. 37–46, 2016.
  * [21] G. Fanelli, M. Dantone, J. Gall, A. Fossati, and L. Van Gool, “Random forests for real time 3d face analysis,” _IJCV_ , vol. 101, no. 3, pp. 437–458, 2013\.
  * [22] L. Pan, J. M. Saragih, and W.-S. Chu, “Mixture of grouped regressors and its application to visual mapping,” _Pattern Recognition_ , vol. 53, pp. 184–194, 2016.
  * [23] X. Zhang, Y. Sugano, M. Fritz, and A. Bulling, “Appearance-based gaze estimation in the wild,” in _CVPR_ , 2015, pp. 4511–4520.
  * [24] L. Pan, S. Ai, Y. Ren, and Z. Xu, “Self-paced deep regression forests with consideration on underrepresented examples,” in _ECCV_ , 2020.
  * [25] D. Meng, Q. Zhao, and L. Jiang, “A theoretical understanding of self-paced learning,” _Information Sciences_ , vol. 414, pp. 319–328, 2017.
  * [26] Z. Huang, Y. Ren, X. Pu, L. Pan, D. Yao, and G. Yu, “Dual self-paced multi-view clustering,” _Neural Networks_ , vol. 140, pp. 184–192, 2021\.
  * [27] Z. Huang, Y. Ren, X. Pu, and L. He, “Non-linear fusion for self-paced multi-view clustering,” in _ACM MM_ , 2021.
  * [28] X. Guo, X. Liu, E. Zhu, X. Zhu, M. Li, X. Xu, and J. Yin, “Adaptive self-paced deep clustering with data augmentation,” _IEEE Transactions on Knowledge and Data Engineering_ , vol. 32, no. 9, pp. 1680–1693, 2019.
  * [29] Y. Ren, X. Yan, Z. Hu, and Z. Xu, “Self-paced multi-task multi-view capped-norm clustering,” in _International Conference on Neural Information Processing_. Springer, 2018, pp. 205–217.
  * [30] L. Han, D. Zhang, D. Huang, X. Chang, J. Ren, S. Luo, and J. Han, “Self-paced mixture of regressions.” in _IJCAI_ , 2017, pp. 1816–1822.
  * [31] J. Yang, X. Wu, J. Liang, X. Sun, M.-M. Cheng, P. L. Rosin, and L. Wang, “Self-paced balance learning for clinical skin disease recognition,” _IEEE transactions on neural networks and learning systems_ , 2019.
  * [32] L. Jiang, D. Meng, S.-I. Yu, Z. Lan, S. Shan, and A. Hauptmann, “Self-paced learning with diversity,” in _NeurIPS_ , 2014, pp. 2078–2086.
  * [33] W. Shen, Y. Guo, Y. Wang, K. Zhao, B. Wang, and A. Yuille, “Deep regression forests for age estimation,” in _CVPR_ , 2018, pp. 2304–2313.
  * [34] W. Li, J. Lu, J. Feng, C. Xu, J. Zhou, and Q. Tian, “Bridgenet: A continuity-aware probabilistic network for age estimation,” in _CVPR_ , 2019, pp. 1145–1154.
  * [35] Z. Deng, H. Liu, Y. Wang, C. Wang, Z. Yu, and X. Sun, “Pml: Progressive margin loss for long-tailed age classification,” in _CVPR_ , 2021, pp. 10 503–10 512.
  * [36] G. Riegler, D. Ferstl, M. Rüther, and H. Bischof, “Hough networks for head pose estimation and facial feature localization,” _Journal of Computer Vision_ , vol. 101, no. 3, pp. 437–458, 2013.
  * [37] Y. Wang, W. Liang, J. Shen, Y. Jia, and L.-F. Yu, “A deep coarse-to-fine network for head pose estimation from synthetic data,” _Pattern Recognition_ , vol. 94, pp. 196–206, 2019.
  * [38] F. Kuhnke and J. Ostermann, “Deep head pose estimation using synthetic images and partial adversarial domain adaption for continuous label spaces,” in _ICCV_ , 2019, pp. 10 164–10 173.
  * [39] X. Zhang, Y. Sugano, M. Fritz, and A. Bulling, “It’s written all over your face: Full-face appearance-based gaze estimation,” in _CVPRW_ , 2017, pp. 51–60.
  * [40] K. Krafka, A. Khosla, P. Kellnhofer, H. Kannan, S. Bhandarkar, W. Matusik, and A. Torralba, “Eye tracking for everyone,” in _CVPR_ , 2016, pp. 2176–2184.
  * [41] T. Fischer, H. J. Chang, and Y. Demiris, “Rt-gene: Real-time eye gaze estimation in natural environments,” in _ECCV_ , 2018, pp. 334–352.
  * [42] S. Park, A. Spurr, and O. Hilliges, “Deep pictorial gaze estimation,” in _ECCV_ , 2018, pp. 721–738.
  * [43] K. Wang, R. Zhao, H. Su, and Q. Ji, “Generalizing eye tracking with bayesian adversarial learning,” in _CVPR_ , 2019, pp. 11 907–11 916.
  * [44] Y. Xiong, H. J. Kim, and V. Singh, “Mixed effects neural networks (menets) with applications to gaze estimation,” in _CVPR_ , 2019, pp. 7743–7752.
  * [45] P. Biswas _et al._ , “Appearance-based gaze estimation using attention and difference mechanism,” in _CVPRW_ , 2021, pp. 3143–3152.
  * [46] Y. Cheng, X. Zhang, F. Lu, and Y. Sato, “Gaze estimation by exploring two-eye asymmetry,” _IEEE Transactions on Image Processing_ , vol. 29, pp. 5259–5272, 2020.
  * [47] Y. Cheng, S. Huang, F. Wang, C. Qian, and F. Lu, “A coarse-to-fine adaptive network for appearance-based gaze estimation,” in _AAAI_ , vol. 34, no. 07, 2020, pp. 10 623–10 630.
  * [48] S. Park, S. D. Mello, P. Molchanov, U. Iqbal, O. Hilliges, and J. Kautz, “Few-shot adaptive gaze estimation,” in _ICCV_ , 2019, pp. 9368–9377.
  * [49] Y. Yu, G. Liu, and J.-M. Odobez, “Improving few-shot user-specific gaze adaptation via gaze redirection synthesis,” in _CVPR_ , 2019, pp. 11 937–11 946.
  * [50] Z. Chen and B. Shi, “Offset calibration for appearance-based gaze estimation via gaze decomposition,” in _WACV_ , 2020, pp. 270–279.
  * [51] M. F. Huber, T. Bailey, H. Durrant-Whyte, and U. D. Hanebeck, “On entropy approximation for gaussian mixture random vectors,” in _IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems_ , 2008, pp. 181–188.
  * [52] L. Jiang, D. Meng, T. Mitamura, and A. G. Hauptmann, “Easy samples first: Self-paced reranking for zero-example multimedia search,” in _ACM MM_ , 2014, pp. 547–556.
  * [53] J. Fitzsimons, A. Al Ali, M. Osborne, and S. Roberts, “A general framework for fair regression,” _Entropy_ , vol. 21, no. 8, pp. 741–753, 2019.
  * [54] A. Agarwal, M. Dudík, and Z. S. Wu, “Fair regression: Quantitative definitions and reduction-based algorithms,” in _ICML_ , 2019, pp. 120–129.
  * [55] R. Berk, H. Heidari, S. Jabbari, M. Kearns, and A. Roth, “Fairness in criminal justice risk assessments: The state of the art,” _Sociological Methods & Research_, vol. 50, no. 1, pp. 3–44, 2021.
  * [56] J. Komiyama, A. Takeda, J. Honda, and H. Shimao, “Nonconvex optimization for regression with fairness constraints,” in _ICML_ , 2018, pp. 2737–2746.
  * [57] M. B. Zafar, I. Valera, M. G. Rogriguez, and K. P. Gummadi, “Fairness constraints: Mechanisms for fair classification,” in _Artificial Intelligence and Statistics_. PMLR, 2017, pp. 962–970.
  * [58] K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, “Joint face detection and alignment using multi-task cascaded convolutional networks,” _IEEE Signal Processing Letters_ , vol. 23, no. 10, pp. 1499–1503, 2016.
  * [59] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” _arXiv 1409.1556_ , 2014.
  * [60] G. Guo, G. Mu, Y. Fu, and T. S. Huang, “Human age estimation using bio-inspired features,” in _CVPR_ , 2009, pp. 112–119.
  * [61] I. Huerta, C. Fernández, and A. Prati, “Facial age estimation through the fusion of texture and local appearance descriptors,” in _ECCV_ , 2014, pp. 667–681.
  * [62] K. Y. Chang, C. S. Chen, and Y. P. Hung, “Ordinal hyperplanes ranker with cost sensitivities for age estimation,” in _CVPR_ , 2011, pp. 585–592.
  * [63] X. Geng, C. Yin, and Z.-H. Zhou, “Facial age estimation by learning from label distributions,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 35, no. 10, pp. 2401–2412, 2013.
  * [64] G. Guo, Y. Fu, C. Dyer, and T. Huang, “Image-based human age estimation by manifold learning and locally adjusted robust regression,” _IEEE Transactions on Image Processing_ , vol. 17, no. 7, pp. 1178–1188, 2008.
  * [65] Z. Yu and D. Y. Yeung, “Multi-task warped gaussian process for personalized age estimation,” in _CVPR_ , 2010, pp. 2622–2629.
  * [66] H. Han, C. Otto, X. Liu, and A. K. Jain, “Demographic estimation from face images: Human vs. machine performance,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 37, no. 6, pp. 1148–1161, 2015.
  * [67] K. Luu, K. Seshadri, M. Savvides, T. D. Bui, and C. Y. Suen, “Contourlet appearance model for facial age estimation,” in _International Joint Conference on Biometrics_ , 2013, pp. 1–8.
  * [68] H. Drucker, C. J. Burges, L. Kaufman, A. J. Smola, and V. Vapnik, “Support vector regression machines,” in _NeurIPS_ , 1997, pp. 155–161.
  * [69] A. Liaw, M. Wiener _et al._ , “Classification and regression by randomforest,” _R news_ , vol. 2, no. 3, pp. 18–22, 2002.
  * [70] M. Al Haj, J. Gonzalez, and L. S. Davis, “On partial least squares in head pose estimation: How to simultaneously deal with misalignment,” in _CVPR_ , 2012, pp. 2602–2609.
  * [71] G. E. Hinton and R. R. Salakhutdinov, “Reducing the dimensionality of data with neural networks,” _science_ , vol. 313, no. 5786, pp. 504–507, 2006\.
  * [72] T. Guo, H. Zhang, B. Yoo, Y. Liu, Y. Kwak, and J.-J. Han, “Order regularization on ordinal loss for head pose, age and gaze estimation,” in _AAAI_ , vol. 35, no. 2, 2021, pp. 1496–1504.
  * [73] X. Zhang, Y. Sugano, M. Fritz, and A. Bulling, “Mpiigaze: Real-world dataset and deep appearance-based gaze estimation,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 41, no. 1, pp. 162–175, 2017\.

|  Lili Pan received her B.Eng. degree in Electronic Engineering, as well as
her M.Eng. and Ph.D. degrees in Information Engineering from University of
Electronic Science and Technology of China (UESTC), China, in 2004, 2007, and
2012, respectively. From 2009 to 2011, she visited the Robotics Institute of
Carnegie Mellon University, USA. She joined the Department of Information
Engineering, UESTC, as a lecturer in 2012. She is currently an associate
professor at UESTC. Her research interests include deep discriminative models,
deep generative models and interpretable machine learning.
---|---
|  Mingming Meng received his B.Eng. degree from University of Electronic
Science and Technology of China (UESTC), China, in 2019. He currently is
pursuing his master’s degree in Information Engineering at UESTC. His research
interests include deep discriminative model, fairness in deep models and
generative adversarial networks.
---|---
|  Yazhou Ren is currently an associate professor with the School of Computer
Science and Engineering, University of Electronic Science and Technology of
China, Chengdu, China. He received the B.Sc. degree in information and
computation science and the Ph.D. degree in computer science from the South
China University of Technology, Guangzhou, China, in 2009 and 2014,
respectively. He visited the Data Mining Laboratory, George Mason University,
USA, from 2012 to 2014. He has published more than 50 peer-reviewed research
articles. His current research interests include clustering, self-paced
learning, and multi-view learning.
---|---
|  Yali Zheng received her PhD from Chongqing University in 2012, and joined
the School of Automation Engineering, University of Electronic Science and
Technology of China in 2013. From 2008 to 2011, she visited he Robotics
Institute of Carnegie Mellon University, USA. Her research interests include
machine learning, pattern recognition and computer vision, especially on graph
matching, deep learning, and their applications in industry fields.
---|---
|  Zenglin Xu received the Ph.D. degree in computer science and engineering
from the Chinese University of Hong Kong. He is currently a full professor in
Harbin Institute of Technology, Shenzhen, and also is affiliated with Peng
Cheng Lab. He has worked at Michigan State University, Cluster of Excellence
at Saarland University, Max Planck Institute for Informatics, Purdue
University, and the University of Electronic Science & Technology of China.
Dr. Xu’s research interests include machine learning and its applications in
computer vision, natural language processing, and health informatics. He
currently serves as an associate editor to Neural Networks and Neurocomputing.
He is the recipient of the outstanding student paper honorable mention of AAAI
2015, the best student paper runner up of ACML 2016, and the 2016 young
researcher award from APNNS.
---|---


# Configurational entropy as a probe of the stability condition of compact
objects

P.S. Koliogiannis<EMAIL_ADDRESS>G.A. Tsalis  C.P. Panos  Ch.C.
Moustakidis<EMAIL_ADDRESS>Department of Theoretical Physics, Aristotle
University of Thessaloniki, 54124 Thessaloniki, Greece

###### Abstract

We systematically extend the statement that the configurational entropy
provides an alternative approach to studying gravitational stability of
compact objects, carried out in the previous work of M. Gleiser and N. Jiang,
Phys. Rev. D 92, 044046 (2015). Inspired by that paper, we try to answer the
crucial question: Is there any one-to-one correspondence between the minimum
of the configurational entropy and the stability point for each realistic
equation of state? In view of the above question, we focus on neutron stars,
quark stars, as well as on the third family of compact stars (hybrid stars),
where a possible phase transition may lead to the existence of twin stars
(stars with equal mass but different radius). In each case, we use a large set
of equations of state investigating the possibility to find correlations
between the stability region obtained from the traditional perturbation
methods to the one obtained by the minimum of configurational entropy. We
found that the suggested prediction of the stability by the minimization of
the configurational entropy, concerning neutron stars and quark stars, does
not have, at least quantitatively, universal validity. However, in several
cases it qualitatively predicts the existence of the stability point. In
conclusion, the configurational entropy, can be considered as an additional
tool which enters into the quiver for studying the stability of compact
objects such as for example neutron and quark stars, but taking into account
that the accuracy of the predictions depends on the specific character of each
equation of state.

information entropy; neutron star; equations of state; stability condition

###### pacs:

04.40.Dg, 26.60.-c, 97.60.Jd, 03.67.-a

##  I Introduction

The concept of the configurational entropy (CE) has been introduced by Gleiser
and Stamatopoulos [1] in order to study the possible relation of the dynamical
and information content of physics models to localized energy configurations
(see also relevant applications in Refs. [2, 3, 4, 5, 6, 7, 8, 9, 10, 11]).
Gleiser and Jiang [3] trying to investigate the possibility to obtain
information about the stability of compact objects from its information-
entropic measure, found that the configurational entropy offers an alternative
way to compute the critical mass for a variety of stellar objects. In
particular, they applied their approach mainly in white dwarfs and boson
stars, but also in neutron stars by applying the case of the Fermi gas
equation of state (EOS). They found that the traditional perturbation methods
correlate well with critical points of the configurational entropy with an
accuracy of a few percent or better [3].

The stability of relativistic stars is a longstanding problem. There are
various methods to examine the stability of a relativistic star (for a
detailed presentation see Ref. [12]). However, in the present work, in
addition to the information-entropic measure, we employ the method based on
the dependence of the gravitational mass $M$ and the corresponding radius $R$
on the central energy density ${\cal E}_{c}$. The stability condition demands
that the mass increases with increasing central energy density $dM/d{\cal
E}_{c}>0$. The extrema in the mass indicates a change in the stability of the
compact star configuration [13, 14].

It is worth noticing that there is not any theoretical argument (or proof) to
relate the stability point to the minimum of the configurational entropy.
However, it is intuitive to expect that since the maximum mass corresponds to
the most compact configuration (maximum mass and minimum radius of a stable
configuration), the corresponding CE will exhibit an extreme value (in this
case a total minimum). The crucial question that needs to be answered is: Is
there any one-to-one correspondence between the minimum of the CE and the
stability point for each realistic EOS? And even more: Is this rule universal
or it depends on the specific character of each EOS? If this is true, then an
additional tool enters into the quiver for studying the stability of compact
objects.

The motivation of the present work is to provide an extended examination of
the results of Ref. [3]. To this end, we apply two analytical solutions of the
Tolman-Oppenheimer-Volkoff (TOV) equations [13] and also a large set of
realistic EOSs in order to study the dependence of the bulk neutron star
properties on the configurational entropy, while inversely relating our focus,
mainly, on the stability conditions. In order to enrich our systematic study,
we also use a set of EOSs corresponding to interacting quark matter (suitable
to describe the properties of a quark star). Finally, we apply a set of EOSs,
with a more complex character, constructed to reproduce properties of a third
family of compact objects, known as twin stars. Thus, we consider that we have
significantly ensured the examination of the findings of Ref. [3], covering a
very large range of cases which corresponds to compact objects.

The paper is organized as follows. In Sec. II we present the basic formalism
of the configurational entropy and we provide the realistic EOSs used in the
present work. In Sec. III the results of the present study are presented and
discussed. Finally, concluding remarks are given in Sec. IV.

##  II Configurational entropy and equations of state

The key quantity to calculate the configurational entropy in momentum space is
the Fourier transform $F({\bf k})$ of the density $\rho(r)={\cal E}(r)/c^{2}$
(where $\mathcal{E}$ is the energy density), originating from the solution of
the TOV equations, that is [3]

$\displaystyle F({\bf k})$ $\displaystyle=$
$\displaystyle\int\int\int\rho(r)e^{-i{\bf k}\cdot{\bf r}}d^{3}{\bf r}.$ (1)

Moreover, the modal fraction $f({\bf k})$ is defined as

$f({\bf k})=\frac{|F({\bf k})|^{2}}{\int|F({\bf k})|^{2}d^{3}{\bf k}},$ (2)

and the normalized one as $\tilde{f}({\bf k})=f({\bf k})/f({\bf k})_{\rm
max}$, where $f({\bf k})_{\rm max}$ is the maximum fraction, which is given in
many cases by the zero mode $k=0$, or by the system’s longest physics mode,
$|k_{min}|=\pi/R$. The above procedure guarantees that $\tilde{f}({\bf k})\leq
1$ for all values of ${\bf k}$. We note here that the system’s longest physics
mode, $|k_{min}|=\pi/R$, is the one that has been used throughout the present
work. Finally, the configurational entropy $S_{C}$, as a functional of
$\tilde{f}({\bf k})$, is given by

$S_{C}[\tilde{f}]=-\int\tilde{f}({\bf k})\ln[\tilde{f}({\bf k})]d^{3}{\bf k}.$
(3)

In detail, we use a set of hadronic EOSs, which have been extensively employed
in the literature for applications in neutron star properties (see Ref. [15]
and references therein). These EOSs have been collected in order to satisfy
the prediction of the maximum observed neutron star mass, that is $M\geq
2M_{\odot}$. In this case, for a specific EOS and for each M-R configuration,
we implement the corresponding density distribution $\rho(r)$. Afterwards, the
calculation of $F(k)$, $f(k)$, and $\tilde{f}(k)$ is performed according to
the presented recipe. It has to be mentioned that it is well established that
the stability point corresponds to the configuration of the maximum mass [13].

Furthermore, we also employ a set of EOSs concerning interacting quark matter
which has been predicted and applied in Ref. [16]. In this case, the pressure
is related to the energy density via the simplified expression [16]

$\displaystyle\frac{P}{4B_{\rm eff}}$ $\displaystyle=$
$\displaystyle\frac{1}{3}\left(\frac{\cal E}{4B_{\rm eff}}-1\right)$
$\displaystyle+$
$\displaystyle\frac{4}{9\pi^{2}}\bar{\lambda}\left(-1+\sqrt{1+\frac{3\pi^{2}}{\bar{\lambda}}\left(\frac{\cal
E}{4B_{\rm eff}}-\frac{1}{4}\right)}\right),$

where $\mathcal{E}$ denotes the energy density and $P$ is the pressure.
Specifically, in the present work we employed the value of $B_{\rm
eff}=150~{}{\rm MeV~{}fm^{-3}}$, while for the dimensionless parameter
$\bar{\lambda}$, we applied the values of $(1,2,5,16)$ [16].

Finally, we employed a more complex EOS, constructed to reproduce properties
of a third family of compact objects, known as twin stars [14, 17]. To be more
specific, we employed the Maxwell construction, suitable to describe phase
transitions in the interior of a compact object and formulated as follows [17]

$\mathcal{E}(P)=\begin{cases}\mathcal{E}(P),&P\leq P_{\rm tr}\\\
\mathcal{E}(P_{{\rm tr}})+\Delta\mathcal{E}+c_{s}^{-2}(P-P_{{\rm tr}}),&P\geq
P_{{\rm tr}}.\end{cases}$ (5)

In the above formula, $c_{\rm s}=\sqrt{{\partial P}/{\partial{\cal E}}}$ is
the speed of sound (in units of the speed of light), and $\Delta\mathcal{E}$
is the magnitude of the energy density jump at the transition point. The
subscript $``{\rm tr}"$ denotes the corresponding quantity at this point. In
the region $P\leq P_{\rm tr}$, we utilized the GRDF-DD2 EOS [18, 19] while in
the region $P\geq P_{\rm tr}$, the value of the speed of sound is fixed at
$c_{\rm s}=1$.

Figure 1: (a) Gravitational mass as a function of the radius for a set of
hadronic EOSs. (b) The corresponding dependence of the gravitational mass as a
function of the central density. (c) The corresponding configurational entropy
as a function of the central density and two analytical solutions of TOV
equations (${\rm a}$ and ${\rm b}$ are constants [3]). The inset figure
indicates the location of the minimization of the CE. The black diamonds
indicate the stability points due to the TM while the open circles correspond
to the minimum of the CE. The hadronic EOSs are presented with the dashed
lines, and the analytical solutions with the solid ones.  Figure 2: (a)
Gravitational mass as a function of the radius for a set of quark star EOSs.
(b) The corresponding dependence of the gravitational mass as a function of
the central density. (c) The corresponding configurational entropy as a
function of the central density. The inset figure indicates the location of
the minimization of the CE. The black diamonds indicate the stability points
due to the TM while the open circles correspond to the minimum of the CE.
Figure 3: (a) Gravitational mass as a function of the radius for a set of
twin-star EOSs. (b) The corresponding dependence of the gravitational mass as
a function of the central density. (c) The corresponding configurational
entropy as a function of the central density. The inset figure indicates the
location of the minimization of the CE. The black diamonds indicate the
stability points due to the TM while the open circles correspond to the
minimum of the CE. In (a) and (b) the twin star EOSs are presented with the
solid lines, where the dashed lines represent the intermediate unstable
region, and in (c) the corresponding EOSs are presented with the dashed lines.

##  III Results and Discussion

In Fig. 1(a) we display the dependence of the gravitational mass on the radius
for the hadronic EOSs, including the Uniform and Tolman VII analytical
solutions for $R=12$ km [20]. The diamonds indicate the stability points due
to the traditional method (TM, which corresponds to the maximum mass
configuration), while the open circles correspond to the minimum of the CE. It
is obvious that in all cases, the location of the stability point does not
coincide with the one corresponding to the minimum of the CE. Moreover, in
some cases, no configuration appears to minimize CE, even at high values of
central density, far from the stability point. In order to further clarify
this point, in Fig. 1(b) we indicate the dependence of the mass on the central
density. The non-coincidence of the corresponding stability points is evident
in this case as well. Finally, in Fig. 1(c) we present the CE as a function of
the central density. In all cases, we found that the configurational entropy
is a decreasing function of the central density, where the appearance of the
minimization is at high-values of density (if it exists). Therefore, the
minimization point of CE and the maximum mass configuration of TM do not
coincide, since the first one is located at high-values of density, far from
those corresponding to the stability point. It has to be noted that the high-
values of density that correspond to the minimization of CE, denote neutron
stars in the instability region. However, there is only one case where the two
mentioned points nearly coincide. This limiting case is the free Fermi gas
which is shown in Fig. 1.

Figure 4: (a) The $MS_{\rm C}\rho_{c}^{-1}$ as a function of the central
density for (a) hadronic neutron stars and (b) quark stars. The black diamonds
indicate the stability points due to the TM while the open circles correspond
to the minimum of the CE.

Quark stars have different configurations compared to neutron stars, since at
the surface while the pressure is vanishing, that is not the case for the
density. This behavior is well reflected on the dependence of the
gravitational mass on the radius. In this case too, for a set of 4 EOSs, the
predictions confirm the discrepancy between TM and CE stability points, as
shown in Fig. 2. Finally, in the scenario of twin stars (hybrid stars), two
stability points exist for the 6 EOSs under consideration. From Fig. 3 the
first stability point coincides with the first minimum of the CE (less than
2$\%$ error from the stability point). However, the latter could be justified,
since at this point a discontinuity on the EOS manifests as a large gap in the
central energy density. Consequently, this minimum can be identified as an
artificial one. Regarding the second minimum of CE, in this case too, it
appears very far from the second stability point (more than two to three times
the central density).

Moreover, in order to further clarify the connection between the minimization
of the CE and the stability point (which corresponds to the maximum mass
configuration), we display in Fig. 4 the dependence of $MS_{\rm
C}\rho_{c}^{-1}$ as a function of the central density $\rho_{c}/ba^{-3}$ for
(a) hadronic neutron stars, and (b) quark stars. It is worth noticing that
while in the case of the Newtonian polytropes the quantity $MS_{\rm
C}\rho_{c}^{-1}$ is nearly constant along the $\rho_{c}/ba^{-3}$ values (see
also Ref. [3]), this rule is not satisfied, at least at low densities, both in
neutron stars and quark stars (the only exception is the case of Fermi gas in
accordance to the prediction of Ref. [3]). We consider that in the majority of
the cases, the more complex dependence of $MS_{\rm C}\rho_{c}^{-1}$ on
$\rho_{c}/ba^{-3}$ is mainly due to the special characteristics of each EOS.
To be more specific, for low values of the density, there is a dramatic
divergence from linearity, thus reflecting the specific feature of each EOS in
this region. On the contrary, at high densities the linearity seems to be
restored. We guess that this is the reason why the two different predictions
of the stability points agree qualitatively but not quantitatively.

It is worth noticing that the existence of a stable configuration is a
property of gravity and does not depend on the EOS. However, the specific
location of the stability point depends on the EOS [21]. In view of the above
comments, ones expects, according to the statement of Ref. [3], the
minimization of the CE to be a consequence of gravity (in the framework of
general relativity). If this is true, then we expect the existence of the
relevant minimum to be independent of the applied EOS. Nonetheless, we found
that this is not a general rule for the case of neutron stars and quark stars.
To be more specific, while there are cases where the CE is minimized, there
are also cases where the CE is monotonically decreasing, with no appearance of
a minimum. In addition, even in the first case scenario, the location of the
minimum does not coincide with the stability point. In spite of the latter,
there is only one exception, which is the free Fermi gas.

##  IV Concluding Remarks

Summarizing, our results lead to the conclusion that the CE can be used
partially as a measure of the stability of compact objects including neutron
stars and quark stars. We could speculate that CE can only qualitatively
predict, in some way, some configurations, which can be correlated to the
stability points. Concluding, although the approach of minimization of the CE
may be fascinating as an alternative way to study the stability point of
neutron stars and quark stars, it is mainly qualitative and not quantitative
in the sense that the results strongly depend on the applied EOS.

Finally, it is necessary to state that, in the present work, we have not
exhausted the investigation of the correlation of CE to the stability
conditions of compact objects. A future perspective is to consider the above
correlation in the case of rapidly rotating or anisotropic stars (where the
pressure is anisotropic). It will be very interesting to investigate and even
more to reveal possible correlation between stability and CE in the mentioned
cases. This can only be done when the corresponding research is carried out in
every detail and covers all possible cases.

## Acknowledgments

The authors would like to thank Dr. Nan Jiang for the correspondence and
useful comments. The implementation of the research was co-financed by Greece
and the European Union (European Social Fund-ESF) through the Operational
Programme «Human Resources Development, Education and Lifelong Learning» in
the context of the Act “Enhancing Human Resources Research Potential by
undertaking a Doctoral Research” Sub-action 2: IKY Scholarship Programme for
PhD candidates in the Greek Universities. All numerical calculations were
performed on a workstation equipped with 2 Intel Xeon Gold 6140 Processors (72
cpu cores in total) provided by the MSc program “Computational Physics” of the
Physics Department, Aristotle University of Thessaloniki.

## References

  * [1] M. Gleiser and N. Stamatopoulos, Phys. Lett. B 713, 304 (2012).
  * [2] M. Gleiser and D. Sowinski, Phys. Lett. B 727, 272 (2013).
  * [3] M. Gleiser and N. Jiang, Phys. Rev. D 92, 044046 (2015).
  * [4] M. Gleiser and D. Sowinski, Phys. Lett. B 747, 125 (2015).
  * [5] N.R.F. Braga, Phys. Lett. B 797, 134919 (2019).
  * [6] N.R.F. Braga and R. da Mata, Phys. Rev. D 101, 105016 (2020).
  * [7] S.H. Alexander, K. Yagi, and N. Yunes, Clas. Quant. Grav. 36, 015010 (2019).
  * [8] R. da Rocha, Phys. Lett. B 823, 136729 (2021).
  * [9] G. Karapetyan, Phys. Lett. B 786, 418 (2018).
  * [10] R.A.C. Correa, P.H.R.S. Moraes, A. de Souza Dutra, W. de Paula, and T. Frederico, Phys. Rev. D 94, 083509 (2016).
  * [11] M. Gleiser, M. Stephens, and D. Sowinski, Phys. Rev. D 97, 096007 (2018).
  * [12] J.M. Bardeen, K.S. Thorne, and D.W. Meltzer, Astrophys. J. 145, 505 (1966).
  * [13] P. Haensel, A.Y. Potekhin, and D.G. Yakovlev, Neutron Stars 1: Equation of State and Structure (Springer-Verlag, New York, 2007).
  * [14] J. Schaffner-Bielich, Compact Star Physics (Cambridge University Press, Cambridge, England,2020).
  * [15] P.S. Koliogiannis and Ch.C. Moustakidis, Phys. Rev. C 101, 015805 (2020).
  * [16] C. Zhang and R.B. Mann, Phys. Rev. D 103, 063018 (2021).
  * [17] M.G. Alford, S. Han, and M. Prakash, Phys. Rev. D 88, 083013 (2013).
  * [18] S. Typel, J. Phys. G: Nucl. Part. Phys. 45, 114001 (17pp) (2018).
  * [19] L. Tsaloukidis, P.S. Koliogiannis, A. Kanakis-Pegios, and Ch.C. Moustakidis, Phys. Rev. D 107, 023012 (2023).
  * [20] D. Kramer, H. Stephani, M.A. MacCallum, and E. Hertl, Exact Solutions of Einstein’s Field Equations (Deutsche Verlag der Wissenschaften, Berlin/Cambridge University Press, Cambridge (1980)).
  * [21] N.K. Glendenning and C. Kettner, Astron. Astrophys. 353, L9 (2000).