# Measuring the Driving Forces of Predictive Performance:
Application to Credit Scoring

Sullivan Hué  Aix Marseille Univ, CNRS, AMSE, Marseille, France. Email:
<EMAIL_ADDRESS>Christophe Hurlin  University of Orléans (LEO) and
Institut Universitaire de France (IUF), Rue de Blois, 45067 Orléans, France.
Email<EMAIL_ADDRESS>Christophe Pérignon  HEC Paris, 1 Rue
de la Libération, 78350 Jouy-en-Josas, France. Email<EMAIL_ADDRESS>Sébastien Saurin  University of Orléans (LEO), Rue de Blois, 45067 Orléans,
France. Email<EMAIL_ADDRESS>
(August 28, 2024
)

###### Abstract

In credit scoring, machine learning models are known to outperform standard
parametric models. As they condition access to credit, banking supervisors and
internal model validation teams need to monitor their predictive performance
and to identify the features with the highest impact on performance. To
facilitate this, we introduce the XPER methodology to decompose a performance
metric (e.g., AUC, $R^{2}$) into specific contributions associated with the
various features of a classification or regression model. XPER is
theoretically grounded on Shapley values and is both model-agnostic and
performance metric-agnostic. Furthermore, it can be implemented either at the
model level or at the individual level. Using a novel dataset of car loans, we
decompose the AUC of a machine-learning model trained to forecast the default
probability of loan applicants. We show that a small number of features can
explain a surprisingly large part of the model performance. Furthermore, we
find that the features that contribute the most to the predictive performance
of the model may not be the ones that contribute the most to individual
forecasts (SHAP). We also show how XPER can be used to deal with heterogeneity
issues and significantly boost out-of-sample performance.

Keywords: Explainability; Credit scoring; Performance metrics; Shapley values

##  1 Introduction

Why is the AUC of a given machine learning model equal to 0.7? Which features
mainly explain this performance? What are the contributions of the different
features to the MSE of a regression model? To answer these questions, we
develop and apply a general methodology, called eXplainable PERformance
(XPER), which measures the marginal contribution of a particular feature to
the predictive performance of a regression or classification model.

Being able to identify the driving forces of the performance of a predictive
model is crucial in the context of credit scoring, where complex and opaque
machine learning models are increasingly being used by banks and fintechs.
Such decomposition is of primary importance both for banking supervisors and
internal model validation teams at banks as they need to understand why a
given model is working or not, and for which types of borrowers. Furthermore,
it also permits to address heterogeneity issues by identifying groups of
individuals for which the features have similar effects on performance. One
can then estimate group-specific models to improve overall performance and
make sure the credit scoring model operates effectively for all borrowers. In
this paper, we propose an application where we use XPER to decompose the AUC
of a scoring model forecasting loan defaults and to identify the features with
the strongest impact on the AUC.

The XPER framework is based on Shapley values (Shapley, (1953)). While the
latter decomposes a payoff among players in a game, XPER values decomposes a
performance metric (e.g., AUC, $R^{2}$) among features in a model. More
precisely, XPER breaks down the difference between a performance metric and a
benchmark value among the various features of the model. Formally, an XPER
value is defined as the weighted average marginal contribution of a given
feature to a performance metric, obtained in a set of coalitions of other
features. For instance, evaluating the XPER value of $x_{1}$ in a three-
feature model implies to assess the incremental performance due to $x_{1}$ by
successively considering four subsets or coalitions of features: one coalition
including no features, only $x_{2}$, only $x_{3}$, and both $x_{2}$ and
$x_{3}$.

The application of the Shapley methodology in the context of model performance
explanation is not trivial. Indeed, many Shapley values can be defined given
the assumptions made on the model, the data, and the features excluded from
the coalitions (see for instance Strumbelj and Kononenko, (2010), Redell,
(2019), Kumar et al., (2020), Sundararajan and Najmi, (2020), and Aas et al.,
(2021)). In this paper, we define XPER values by considering the expectation
of the performance metric with respect to the joint distribution of the target
variable and of the features which are excluded from the coalition. A key
advantage of this definition is that the benchmark value has a meaningful
interpretation: it corresponds to the performance metric that we would obtain
on a hypothetical sample in which the target variable is independent from all
the features included in the model, i.e., a fully misspecified model with
irrelevant features. Moreover, our method does not require to re-estimate the
model (à la Grömping, (2007)) nor to pick ad-hoc values for features excluded
from the coalition (à la Israeli, (2007)).

XPER offers several other advantages. First, as the XPER decomposition is
based on Shapley values, it is theoretically grounded and XPER values satisfy
several desirable axioms. Second, XPER is model-agnostic as it permits to
interpret the predictive performance of any econometric or machine learning
model. Third, it is metric-agnostic as it can break down any performance
metric: e.g., predictive accuracy (AUC, Gini, accuracy), goodness of fit
($R^{2}$), information criterion (AIC, BIC), statistical loss function (MSE,
MAE, Q-like), or economic performance metric (profit-and-loss function).
Fourth, XPER can be implemented either at the global level or at the local
level. At the global level, the XPER value of a given feature measures its
contribution to the performance of the model. At the local level, the XPER
value of a given feature measures its contribution to the model performance
that comes from a given individual. Finally, as the number of coalitions grows
fast with the number of features, we propose two estimation procedures: an
exact one when the number of features remains moderate and an approximated
one, which is a modified version of the Kernel SHAP method of Lundberg and
Lee, (2017).

We apply our methodology in the credit scoring context. Using a novel,
balanced dataset of auto loans provided by an international bank, we
demonstrate the usefulness of XPER for decomposing the AUC and other
performance metrics of a machine-learning credit scoring model. We show that a
small number of features can explain a surprisingly large part of the model
performance. Furthermore, the empirical analysis confirms that XPER values
differ from standard feature importance metrics. More importantly, we show how
to use XPER to boost model performance. To do so, we build homogeneous groups
of individuals by clustering them based on their individual XPER values. By
construction, within a given group, the features tend to have similar effects
on performance (same sign, same strength). We then show that estimating group-
specific models yields to a much higher predictive accuracy than with a one-
fits-all model.

Our paper contributes to the burgeoning literature on eXplainable Artificial
Intelligence (XAI) (Murdoch et al.,, 2019; Molnar,, 2020; Gelman and Vehtari,,
2021). One well-known limitation of AI and machine learning methods comes from
their opacity and lack of explainability. Most of these algorithms are
considered as black boxes in the sense that the corresponding outcomes cannot
be easily explained to final users nor related to the initial features (Sun
and Wang,, 2021). Recently, many explainability methods have been designed to
explain black-box models by measuring which features most affect its
predictions (see Molnar, (2020); Zhao and Hastie, (2021); Liu and Ročková,
(2023)). One of the most impactful XAI methods is the Shapley additive
explanation (SHAP) of Lundberg and Lee, (2017), which distributes the
prediction of a model among its features (see also Sundararajan et al.,
(2017); Lundberg et al., (2018); Sundararajan et al., (2020); Casalicchio et
al., (2019); Bowen and Ungar, (2020); Aas et al., (2021)). While model
predictive performance obviously depends on predictions, the contribution of a
feature to the performance metric also depends on the true value of the target
variable. Our contribution to this literature is to decompose any performance
metric using an XAI approach.

Our paper also contributes to the statistical literature on the decomposition
of performance metrics. Numerous methods have been proposed for the MSE
(Theil,, 1971; Ahlburg,, 1984), for various inequality measures (Bourguignon,,
1979; Shorrocks,, 1980, 1982), and for the $R^{2}$ (see Grömping, (2015) for a
survey and Lindeman et al., (1980), Kruskal, (1987), Chevan and Sutherland,
(1991), Johnson, (2000), Lahaye and Neely, (2020)). Other studies break down
performance metrics using Shapley values (Stufken,, 1992; Israeli,, 2007;
Grömping,, 2007; Redell,, 2019; Casalicchio et al.,, 2019; Williamson and
Feng,, 2020; Williamson et al., 2021a, ; Williamson et al., 2021b, ; Borup et
al.,, 2022). Most of these decompositions are specific to a performance metric
and to a model. Furthermore, some of them imply to re-estimate the model with
different subsets of features, which may lead to omitted variable bias. Others
use ad-hoc values for the features excluded from the coalitions. In contrast,
XPER does not require such an assumption, nor any re-estimation, and is
metric/model agnostic.

The rest of this paper is structured as follows. We introduce our framework of
analysis and the concept of performance metric in Section 2. Section 3
introduces the XPER value decomposition of performance metrics and Section 4
provides some simulation results. Section 5 presents the empirical application
and Section 6 concludes the paper and discusses further applications.

##  2 Framework and performance metrics

We consider a classification or regression problem involving a target
variable, denoted $y$, taking values in $\mathcal{Y}=\\{0,1\\}$ or
$\mathcal{Y}\subseteq\mathbb{R}$ given the problem considered. The $q$-vector
$\mathbf{x}\in\mathcal{X}\subseteq\mathbb{R}^{q}$ refers to the input
features. We denote by $f:\mathbf{x}\rightarrow\hat{y}$ an econometric model
or a machine learning algorithm, where $\hat{y}\in\mathcal{Y}$ is either a
classification output or a regression output, such as $\hat{y}=f(\mathbf{x})$.
We impose no constraint on the model: it may be parametric or not, linear or
not, a weak learner or an ensemble method, etc. For simplicity, for parametric
models we exclude the parameters from the notation, i.e., $f(\mathbf{x})\equiv
f(\mathbf{x};\theta)$. The model is estimated (parametric model) or trained
(machine learning algorithm) once for all on a training sample
$S_{T}=\\{\mathbf{x}_{j},y_{j}\\}^{T}_{j=1}$. The sample size $T$ is
considered as fixed. The trained model, denoted $\hat{f}(.)$, is evaluated on
a test sample
$S_{n}=\\{\mathbf{x}_{i},y_{i},\hat{f}(\mathbf{x}_{i})\\}^{n}_{i=1}$ with a
performance metric (PM). A PM is defined as an assessment measure of the model
predictive performance, e.g., MSE, MAE, or $R^{2}$ for regression models or,
AUC, Brier score, or Gini index, for classification models. More generally,
any information criteria, loss function, or economic performance indicator can
be considered as PM.

######  Definition 1.

A sample performance metric $PM_{n}\in\Theta\subseteq\mathbb{R}$ associated to
the model $\hat{f}(.)$ and the test sample $S_{n}$, is defined as
$PM_{n}=\tilde{G}_{n}(y_{1},...,y_{n};\hat{f}(\mathbf{x_{1}}),...,\hat{f}(\mathbf{x_{n}}))=G_{n}(\mathbf{y};\mathbf{X})$,
where $\mathbf{y}=(y_{1},...,y_{n})^{{}^{\prime}}$ and
$\mathbf{X}=(\mathbf{x}_{1},..,\mathbf{x}_{n})^{{}^{\prime}}$.

For instance, $\Theta=[0,1]$ for AUC or $R^{2}$, and $\Theta=\mathbb{R}^{+}$
for MSE or MAE. We introduce the following three assumptions on the PM.

######  Assumption 1.

The sample PM increases over $\Theta$ with the predictive performance.

Assumption 1 simplifies the interpretation of the PM. For instance, the AUC
satisfies this assumption, as the higher it is, the better the model is at
distinguishing between positive and negative classes. Differently, when
dealing with PMs that are negatively correlated with performance, e.g. MSE, we
need to consider the opposite of the metric.

######  Assumption 2.

The sample PM satisfies the following additive assumption:

$G_{n}(\mathbf{y};\mathbf{X})=\frac{1}{n}\sum_{i=1}^{n}G(y_{i};\mathbf{x}_{i};\hat{\delta}_{n}),$
(1)

where $G(y_{i};\mathbf{x}_{i};\hat{\delta}_{n})$ denotes an individual
contribution to the PM, and $\hat{\delta}_{n}$ is a nuisance parameter which
depends on the test sample $S_{n}$.

For simplicity, we only consider models for which the outcome $y_{i}$ for
instance $i$ only depends on features
$\mathbf{x}_{i}=(x_{i,1},...,x_{i,n})^{{}^{\prime}}$. For regression or
classification models with cross-sectional interactions (e.g., spatial
econometrics model) or time-series dependence, notations have to be adjusted
such that $\hat{y}_{i}=\widehat{f}\left(\mathbf{w}_{i}\right)$, where
$\mathbf{x}_{i}\subseteq\mathbf{w}_{i}$, $\exists j\neq
i:\mathbf{x}_{j}\subseteq\mathbf{w}_{i}$ and/or
$y_{j}\subseteq\mathbf{w}_{i}$. Then, the additive assumption becomes
$G_{n}(\mathbf{y};\mathbf{X})=n^{-1}\sum_{i=1}^{n}G\left(y_{i};\mathbf{w}_{i};\hat{\delta}_{n}\right)$.

######  Assumption 3.

The sample metric $G_{n}(\mathbf{y};\mathbf{X})$ converges to the population
performance metric $\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$,
where $\mathbb{E}_{y,\mathbf{x}}(.)$ refers to the expected value with respect
to the joint distribution of $y$ and $\mathbf{x}$, and $\delta_{0}=\mbox{plim
}\hat{\delta}_{n}$. In addition,
$\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$ exists and is finite.

These assumptions are consistent with a wide range of PMs. In Appendix A, we
detail $G\left(y;\mathbf{x};\delta_{0}\right)$ for standard PMs associated to
regression or classification models. For instance, in the case of a linear
regression model $\hat{f}(\mathbf{x})=\mathbf{x}\hat{\beta}$, when the $R^{2}$
is used as PM, we have:

$\displaystyle
R^{2}=G_{n}(\mathbf{y};\mathbf{X})=\frac{1}{n}\sum_{i=1}^{n}G(y_{i};\mathbf{x}_{i};\hat{\delta}_{n})=1-\frac{\sum_{i=1}^{n}(y_{i}-\mathbf{x}_{i}\hat{\beta})^{2}}{\sum_{j=1}^{n}(y_{j}-\bar{y})^{2}},$
(2)

with
$G(y_{i};\mathbf{x}_{i};\hat{\delta}_{n})=1-\hat{\delta}_{n}^{-1}(y_{i}-\mathbf{x}_{i}\hat{\beta})^{2}$
and $\hat{\delta}_{n}=n^{-1}\sum_{j=1}^{n}(y_{j}-\bar{y})^{2}$. The
corresponding population $R^{2}$ is defined as
$\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))=1-\frac{1}{\sigma_{y}^{2}}\mathbb{E}_{y,\mathbf{x}}\left({(y-\mathbf{x}\hat{\beta})^{2}}\right)$,
with
$G(y;\mathbf{x};\delta_{0})=1-\delta_{0}^{-1}(y-\mathbf{x}\hat{\beta})^{2}$
and $\delta_{0}=\sigma_{y}^{2}$ the variance of the target variable.

##  3 XPER values

###  3.1 Definition

Our objective is to identify the contribution of the model’s features to its
predictive performance, as evaluated by a PM on a given sample. We measure
this contribution through Shapley values (Shapley,, 1953), a method used in
game theory to fairly distribute a payoff $Val(x_{1},...,x_{q})$ among several
players $x_{1},...,x_{q}$. The Shapley value $\phi_{j}$ measures the marginal
impact of a player $x_{j}$ on the payoff by assessing the changes in
$Val(x_{1},...,x_{q})$ when this player is added or not to a coalition of
players already in the game. Denote by $\mathbf{x}^{S}$ the vector of players
included in a coalition $S$ and $\mathbf{x}^{\overline{S}}$ the vector of
excluded players, such that
$\\{\mathbf{x}\\}=\\{\mathbf{x}^{S}\\}\cup\\{\mathbf{x}^{\overline{S}}\\}\cup\\{x_{j}\\}$
and $\mathbf{x}=(x_{1},...,x_{q})$. The Shapley value is then defined as the
weighted average of the marginal contributions associated with all possible
coalitions $S$.

######  Definition 2 (Shapley, (1953)).

The Shapley value contribution of player $x_{j}$ to the payoff is:

$\phi_{j}=\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\left[Val(\mathbf{x}^{S}\cup\\{x_{j}\\})-Val(\mathbf{x}^{S})\right],$
(3) $\omega_{S}=\frac{|S|!\left(q-|S|-1\right)!}{q!},$ (4)

with $Val(.)$ the payoff, $S$ a coalition of players, excluding the player of
interest $x_{j}$, $|S|$ the number of players in the coalition, and
$\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$ the powerset of
the set $\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\}$.

By analogy, we decompose the performance metric
$\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$ (the “payoff”) among
the features $x_{1},...,x_{q}$ (the “players”) of the model
$\hat{f}(\mathbf{x})$ (the “game”). The main difference with the previous
notations is that the payoff
$Val(\mathbf{x};y)=\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$
depends on the joint distribution of the features $x_{1},...,x_{q}$ and the
target variable $y$. Formally, we define XPER as the Shapley value associated
to the performance metric
$\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$ and to the model
$\hat{f}(\mathbf{x})$.

######  Definition 3 (XPER).

The XPER value associated to the feature $x_{j}$ is defined as:

$\phi_{j}=\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\left[\underset{\text{averaging}}{\underbrace{\mathbb{E}_{y,x_{j},\mathbf{x}^{S}}}}\underset{\text{marginalisation}}{\underbrace{\mathbb{E}_{\mathbf{x}^{\overline{S}}}}}\left(G\left(y;\mathbf{x};\delta_{0}\right)\right)-\underset{\text{averaging}}{\underbrace{\mathbb{E}_{y,\mathbf{x}^{S}}}}\underset{\text{marginalisation}}{\underbrace{\mathbb{E}_{x_{j}\mathbf{x}^{\overline{S}}}}}\left(G\left(y;\mathbf{x};\delta_{0}\right)\right)\right],$

with S a coalition of features, excluding the feature of interest $x_{j}$,
$\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$ the powerset of
the set $\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\}$, and $\omega_{S}$
the coalition weight.

The XPER value $\phi_{j}$ measures the weighted average marginal contribution
of the feature $x_{j}$ to the PM over all features coalitions. For each
coalition, the marginal contribution of $x_{j}$ is defined as the difference
between the expected values of (1) the PM obtained while including this
feature in the coalition and (2) the PM obtained while excluding this feature
from the coalition. In Definition 3, the term
$\mathbb{E}_{\mathbf{x}^{\overline{S}}}$ refers to the marginalisation with
respect to the features which are excluded from the coalition $S$. The second
expectation
$\mathbb{E}_{y,x_{j},\mathbf{x}^{S}}\left(G\left(y;\mathbf{x};\delta_{0}\right)\right)$
refers to an averaging effect, i.e., expectation with respect to the joint
distribution of the features $\mathbf{x}^{S}$ included in the coalition with
$x_{j}$, and the target variable $y$.

As an illustration, we consider a three-feature model
$\hat{f}(x_{1},x_{2},x_{3})$ and pick $x_{1}$ as the feature of interest. In
Table 1, we report all the coalitions among the set $\\{x_{2},x_{3}\\}$
(column 1), the associated weights computed according to Equation 4 (column
2), and the marginal contributions (column 3) used to compute the XPER value
$\phi_{1}$. For instance, when considering a coalition with only $x_{1}$, we
first compute the expectation of the PM with respect to the joint distribution
of excluded features $x_{2}$ and $x_{3}$, i.e.,
$\mathbb{E}_{x_{2},x_{3}}(\tilde{G}(y,\hat{f}(x_{1},x_{2},x_{3})))$. This
corresponds to a marginalisation of the PM with respect to the features which
are excluded from the coalition. Then, we consider the expectation of the PM
with respect to the joint distribution of the features included in the
coalition and the target, i.e., $y$ and $x_{1}$ in our example. Thus, we get
an expected value
$\mathbb{E}_{y,x_{1}}\mathbb{E}_{x_{2},x_{3}}(\tilde{G}(y,\hat{f}(x_{1},x_{2},x_{3})))$,
where the first expectation refers to the averaging effect, whereas the second
one refers to the marginalisation effect. The Shapley value is then computed
by summing the weighted differences in the expected PM obtained with or
without the feature of interest, for all the coalitions of other features.

Table 1: Components of $\phi_{1}$ in a three-feature model

$S$ | $\omega_{S}$ | $\mathbb{E}_{y,x_{1},\mathbf{x}^{S}}\mathbb{E}_{\mathbf{x}^{\overline{S}}}\left(G\left(y;\mathbf{x};\delta_{0}\right)\right)-\mathbb{E}_{y,\mathbf{x}^{S}}\mathbb{E}_{x_{1},\mathbf{x}^{\overline{S}}}\left(G\left(y;\mathbf{x};\delta_{0}\right)\right)$
---|---|---
$\left\\{\emptyset\right\\}$ | $1/3$ | $\mathbb{E}_{y,x_{1}}\mathbb{E}_{x_{2},x_{3}}\left(G\left(y;\mathbf{x};\delta_{0}\right)\right)-\mathbb{E}_{y}\mathbb{E}_{x_{1},x_{2},x_{3}}\left(G\left(y;\mathbf{x};\delta_{0}\right)\right)$
$\left\\{x_{2}\right\\}$ | $1/6$ | $\mathbb{E}_{y,x_{1},x_{2}}\mathbb{E}_{x_{3}}\left(G\left(y;\mathbf{x};\delta_{0}\right)\right)-\mathbb{E}_{y,x_{2}}\mathbb{E}_{x_{1},x_{3}}\left(G\left(y;\mathbf{x};\delta_{0}\right)\right)$
$\left\\{x_{3}\right\\}$ | $1/6$ | $\mathbb{E}_{y,x_{1},x_{3}}\mathbb{E}_{x_{2}}\left(G\left(y;\mathbf{x};\delta_{0}\right)\right)-\mathbb{E}_{y,x_{3}}\mathbb{E}_{x_{1},x_{2}}\left(G\left(y;\mathbf{x};\delta_{0}\right)\right)$
$\left\\{x_{2},x_{3}\right\\}$ | $1/3$ | $\mathbb{E}_{y,x_{1},x_{2},x_{3}}\left(G\left(y;\mathbf{x};\delta_{0}\right)\right)-\mathbb{E}_{y,x_{2},x_{3}}\mathbb{E}_{x_{1}}\left(G\left(y;\mathbf{x};\delta_{0}\right)\right)$

Note: This table provides details about the XPER value computation, i.e., the
coalitions (column 1), the associated weights according to Equation 4 (column
2), and the marginal contributions (column 3).

One advantage of our marginalisation-based approach is that there is no need
to re-estimate any sub-model for each subset of features. We consider the
expected value of the PM with respect to the features
$\mathbf{x}^{\overline{S}}$ excluded from the coalitions, while leaving the
model $\hat{f}(\mathbf{x})$ unchanged. An alternative solution would consist
in estimating a submodel for each subset of features. For example, for a model
with three features $x_{1},x_{2},x_{3}$, the computation of the Shapley value
associated to $x_{1}$ would require to estimate four sub-models, namely
without any feature, with $x_{2}$ only, with $x_{3}$ only, and with
$x_{2},x_{3}$, and then to re-estimate the same sub-models while including
$x_{1}$ as an additional feature. This approach was adopted for instance by
Israeli, (2007) to decompose the $R^{2}$ of a linear model. However, re-
estimating a submodel with only a subset of the initial features can lead to
model specification errors, e.g., omitted variable bias in the case of linear
regression. This specification error necessarily distorts the Shapley value
and thus may lead to an unreliable decomposition of the performance metric.

###  3.2 Axioms

The XPER values satisfy different axioms associated to Shapley values. These
axioms are particularly relevant in the context of statistical performance
analysis.

######  Axiom 1 (Efficiency).

The sum of the XPER values $\phi_{j}$, $\forall j=1,...,q$, is equal to the
difference between the performance metric
$\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$ and a benchmark
$\phi_{0}$ such as:

$\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))=\phi_{0}+\sum_{j=1}^{q}\phi_{j},$
(5)

where
$\phi_{0}=\mathbb{E}_{\mathbf{x}}\mathbb{E}_{y}\left(G(y;\mathbf{x};\delta_{0})\right)$
corresponds to the performance metric associated to a population where the
target variable is independent from all features considered in the model.

One of the main advantages of the XPER decomposition is that the benchmark
value $\phi_{0}$ has an insightful interpretation: it corresponds to the PM
that we would obtain on a hypothetical sample in which the target variable $y$
is independent from all model features $\mathbf{x}$, i.e., in a case where the
model $\hat{f}(\mathbf{x})$ is fully misspecified. As an example, for the AUC,
the benchmark $\phi_{0}$ corresponds to the AUC associated to a random
predictor and is equal to $0.5$. For the sensitivity (true positive rate), the
benchmark corresponds to the probability $\Pr(\hat{y}=1)$, for the specificity
(true negative rate) the benchmark is $\Pr(\hat{y}=0)$, etc. Thus, we can
decompose any PM into two parts: (i) a base value $\phi_{0}$ obtained in a
hypothetical case where $y$ and $\mathbf{x}$ would be independent, and (ii) a
component determined by the XPER feature contributions, which depends on their
dependence with the target, i.e., their relevance.

$\underset{\text{population performance
metric}}{\underbrace{\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0})}}=\underset{\text{expected
value under
independence}}{\underbrace{\mathbb{E}_{y}\mathbb{E}_{\mathbf{x}}(G(y;\mathbf{x};\delta_{0})}}+\underset{\text{feature
contributions}}{\underbrace{\sum_{j=1}^{q}\phi_{j}}.}$

The XPER values also satisfy the other main axioms of the Shapley values:

######  Axiom 2 (Symmetry).

If for all subsets
$S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j},x_{k}\right\\})$

$\mathbb{E}_{y,x_{j},\mathbf{x}^{S}}\mathbb{E}_{\mathbf{x}^{\overline{S}}}\left(G\left(y;\mathbf{x},x_{j};\delta_{0}\right)\right)=\mathbb{E}_{y,x_{k},\mathbf{x}^{S}}\mathbb{E}_{\mathbf{x}^{\overline{S}}}\left(G\left(y;\mathbf{x},x_{k};\delta_{0}\right)\right),$

then $\phi_{j}=\phi_{k}$. It means that if two features $x_{j}$ and $x_{k}$
contribute equally to the performance of the model then their effects must be
the same.

######  Axiom 3 (Linearity).

For any two performance metrics $PM_{1}$ and $PM_{2}$

$\phi_{j}(PM_{1}+PM_{2})=\phi_{j}(PM_{1})+\phi_{j}(PM_{2}),$

where $\phi_{j}(PM_{1}+PM_{2})$ refers to the XPER value of feature $x_{j}$
measuring its effect on the performance metric defined as the sum of $PM_{1}$
and $PM_{2}$.

######  Axiom 4 (Null effects).

If for all subsets
$S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$

$\mathbb{E}_{y,x_{j},\mathbf{x}^{S}}\mathbb{E}_{\mathbf{x}^{\overline{S}}}\left(G\left(y;\mathbf{x},x_{j};\delta_{0}\right)\right)=\mathbb{E}_{y,\mathbf{x}^{S}}\mathbb{E}_{x_{j},\mathbf{x}^{\overline{S}}}\left(G\left(y;\mathbf{x},x_{j};\delta_{0}\right)\right),$

then $\phi_{j}=0$. A feature without any impact on the performance of the
model $\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$ has an XPER
value equal to 0.

To illustrate these axioms, consider a linear regression model
$\hat{f}(\mathbf{x}_{i})=\sum_{j=1}^{q}\hat{\beta}_{j}x_{i,j}$ estimated on a
training set, and the $R^{2}$ as PM. For simplicity, we assume that the data
generating process (DGP) associated to the test sample
$S_{n}=\\{\mathbf{x}_{i},y_{i},\hat{f}(\mathbf{x}_{i})\\}^{n}_{i=1}$ satisfies
$\mathbb{V}(\mathbf{x})=diag(\sigma^{2}_{x_{j}}),\forall j=1,...,q$ . Then,
the XPER contribution $\phi_{j}$ of feature $x_{j}$ to the $R^{2}$ is:

$\phi_{j}=\frac{2\hat{\beta}_{j}\sigma_{y,x_{j}}}{\sigma^{2}_{y}},\qquad\quad\forall
j=1,...,q,$ (6)

with $\sigma_{y}^{2}$ the variance of the target variable and
$\sigma_{y,x_{j}}$ its covariance with feature $x_{j}$. See Appendix E.1 for
the proof. Formally, $\phi_{j}$ depends on the estimated parameter
$\hat{\beta}_{j}$ (training set) and the covariance (test set) between $x_{j}$
and the target variable $y$, i.e., $\sigma_{y,x_{j}}$. If the DGP of the
training and test samples are identical, model parameters $\hat{\beta}_{j}$
and covariance $\sigma_{y,x_{j}}$ have the same sign, which means
$\phi_{j}>0$. Otherwise, XPER values may be negative. Similarly, if
$\hat{\beta}_{j}=0$ (i.e., the variable is useless in the model) or if the
feature is uncorrelated with the target variable on the test sample
($\sigma_{y,x_{j}}=0$), then $\phi_{j}=0$. Finally, a feature $x_{j}$ has a
larger contribution to the $R^{2}$ than a feature $x_{s}$ if
$\hat{\beta}_{j}\sigma_{y,x_{j}}>\hat{\beta}_{s}\sigma_{y,x_{s}}$, meaning
that $x_{j}$ is more related to the target variable than $x_{s}$ both in-
sample (through $\hat{\beta}_{j}$) and out-of-sample (through
$\sigma_{y,x_{s}}$).

Most of the time, there is no analytical expression for the XPER values
$\phi_{j}$. Indeed, the function $G\left(y;\mathbf{x};\delta_{0}\right)$ is in
general non-linear in $y$ and $\mathbf{x}$, as the model $\hat{f}(.)$ may be
highly non-linear in $\mathbf{x}$ (e.g., machine learning model) and/or the
performance metric may be non-linear in $y$ and $\mathbf{x}$. One exception is
the case of quadratic loss functions, e.g., MSE, $R^{2}$, etc., associated to
the linear regression model (see Appendix B).

Finally, the XPER values $\phi_{j}$ are theoretical and unobserved quantities.
In Appendix C, we propose two estimation methods for $\phi_{j}$ to accommodate
models with either few features or with a large number of features.

###  3.3 Individual XPER values

The XPER framework can be used to conduct a global analysis of the model
predictive performance through feature contributions $\phi_{j}$, or a local
analysis at the individual level.

######  Definition 4.

(Individual XPER) The individual XPER value $\phi_{i,j}$ associated to
individual $i$ and model feature $j$ satisfies
$\phi_{j}=\mathbb{E}_{y,\mathbf{x}}(\phi_{i,j}(y_{i};\mathbf{x}_{i}))$, where
the random variable $\phi_{i,j}(y_{i};\mathbf{x}_{i})$ corresponds to
individual $i$ and feature $j$ contribution to the performance metric defined
as:

$\phi_{i,j}(y_{i};\mathbf{x}_{i})=\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}w_{S}\left[\mathbb{E}_{\mathbf{x}^{\overline{S}}}\left(G\left(y_{i};x_{i,j},\mathbf{x}_{i}^{S},\mathbf{x}^{\overline{S}};\delta_{0}\right)\right)-\mathbb{E}_{x_{j},\mathbf{x}^{\overline{S}}}\left(G\left(y_{i};x_{j},\mathbf{x}_{i}^{S},\mathbf{x}^{\overline{S}};\delta_{0}\right)\right)\right].$

The individual XPER value can be interpreted as the contribution of individual
$i$ and model feature $j$ to the performance metric. For a given realisation
($y_{i},\mathbf{x}_{i}$), the corresponding individual contribution to the
performance metric can be broken down into:

$\displaystyle
G(y_{i};\mathbf{x}_{i};\delta_{0})=\phi_{i,0}+\sum_{j=1}^{q}\phi_{i,j},$ (7)

where $\phi_{i,j}$ is the realisation of $\phi_{i,j}(y_{i};\mathbf{x}_{i})$
and $\phi_{i,0}$ is the realisation of
$\phi_{i,0}(y_{i})=\mathbb{E}_{\mathbf{x}}(G(y_{i};\mathbf{x};\delta_{0}))$.
The benchmark $\phi_{i,0}$ corresponds to the contribution of an individual to
the performance metric when the target variable $y_{i}$ is independent from
the features $\mathbf{x}_{i}$. Therefore, the difference between the
individual contribution to the performance metric
$G(y_{i};\mathbf{x}_{i};\delta_{0})$ and the individual benchmark $\phi_{i,0}$
is explained by individual XPER values $\phi_{i,j}$.

As an illustration, consider the $R^{2}$ of a linear regression. Then, the
individual XPER value $\phi_{i,j}$ can be expressed as:

$\phi_{i,j}=\sigma^{-2}_{y}\left[\hat{\beta}_{j}(x_{i,j}-\mathbb{E}(x_{j}))A-\hat{\beta}_{j}^{2}(x_{i,j}^{2}-\mathbb{E}(x_{j}^{2}))+\sum_{\begin{subarray}{c}k=1\\\
k\neq
j\end{subarray}}^{q}\hat{\beta}_{k}\hat{\beta}_{j}\sigma_{x_{k},x_{j}}\right].$
(8)

with $A=\left(2y_{i}-\sum_{\begin{subarray}{c}k=1\\\ k\neq
j\end{subarray}}^{q}\hat{\beta}_{k}(x_{i,k}+\mathbb{E}(x_{k}))\right)$. See
Appendix E.2 for the proof. The value $\phi_{i,j}$ measures the contribution
of feature $x_{j}$ to
$G(y_{i};\mathbf{x}_{i};\delta_{0})=1-\sigma_{y}^{-2}(y_{i}-\hat{f}(\mathbf{x}_{i}))^{2}$.
A positive individual XPER value $\phi_{i,j}$ means that incorporating the
information included in feature $x_{j}$ improves model prediction for
individual $i$ compared to the benchmark, hence increases $R^{2}$. Several
comments can be made here. First, if $\hat{\beta}_{j}=0$, i.e., the feature
has no impact on the model outcome, the feature $x_{j}$ does not have any
impact on the $R^{2}$, for all individuals. Second, the closer the realization
of feature $x_{j}$ is to its expected value $\mathbb{E}(x_{j})$, the lower is
the contribution of this feature to the performance metric, for all
individuals. A similar result occurs when $x_{j}^{2}$ is close to its expected
value. Indeed, when the characteristics of an individual are close to the mean
values over the population, his or her contribution to the predictive
performance of the model is also close to the average contribution of other
individuals.

##  4 Simulations

In this section, we conduct a Monte Carlo simulation experiment to illustrate
the XPER methodology and to highlight some of its axioms.

In this experiment, XPER values are used to explain the predictive performance
of a probit regression model, measured by the AUC computed on a test set
$S_{n}$. To understand the mechanisms of the XPER decomposition, we consider a
white-box model for which the predictions are explainable. The DGP is given by
a latent variable model such that $y_{i}=\mathbf{1}(y^{*}_{i}>0)$, where
$\mathbf{1}(.)$ is the indicator function,
$y_{i}^{*}=\mathbf{\omega}_{i}\beta+\varepsilon_{i}$,
$\mathbf{\omega}_{i}=(1:\mathbf{x}^{{}^{\prime}}_{i})$ and $\varepsilon_{i}$
an i.i.d. error term with $\varepsilon_{i}\sim\mathcal{N}(0,1)$. We consider
three i.i.d. features such that
$\mathbf{x}_{i}=(x_{i,1},x_{i,2},x_{i,3})^{{}^{\prime}}\sim\mathcal{N}(\mathbf{0},\mathbf{\Sigma})$
with $diag(\Sigma)=(1.2,1,1)$. The true vector of parameters is
$\beta=(\beta_{0},\beta_{1},\beta_{2},\beta_{3})^{{}^{\prime}}=(0.05,0.5,0.5,0)^{{}^{\prime}}$
with $\beta_{0}$ the intercept.

We simulate $K=5,000$ pseudo-samples
$\\{y_{i}^{s},\mathbf{x}_{i}^{s}\\}_{i=1}^{T+n}$ of size 1,000, for
$s=1,\dots,K$. For each pseudo-sample, we use the first $T=700$ observations
to estimate a probit model and the remaining ones as test set $S_{n}$ to
compute the AUC and the corresponding XPER values according to Equation A1.
For instance, the estimated parameters obtained for the simulation $s=1$ are
equal to
$\\{\hat{\beta}_{0}^{s},\hat{\beta}_{1}^{s},\hat{\beta}_{2}^{s},\hat{\beta}_{3}^{s}\\}=\\{0.0109,0.4943,0.5234,0.0688\\}$
and the AUC is equal to $0.7775$. The associated feature contributions are the
following:

$\displaystyle\underbrace{0.7775}_{AUC}=\underbrace{0.4984}_{\hat{\phi}_{0}}+\underbrace{0.1716}_{\hat{\phi}_{1}}+\underbrace{0.1098}_{\hat{\phi}_{2}}+\underbrace{(-0.0023)}_{\hat{\phi}_{3}}.$

As expected, the estimated benchmark $\hat{\phi}_{0}$ is close to 0.5. As a
reminder, an AUC equal to 0.5 is associated to a random predictor. Hence, the
benchmark $\hat{\phi}_{0}$ corresponds to the AUC of the model
$\hat{f}(\mathbf{x}_{i})=\mathbf{x}_{i}^{{}^{\prime}}\hat{\beta}$ that we
would obtain on a virtual test sample in which the target variable is
independent from all features. The difference between the estimated AUC and
this hypothetical benchmark is explained by the feature contributions. We
verify that these contributions are positive or null for all features. The
feature with the largest variance ($x_{1}$) has also the largest contribution
to the predictive ability of the model ($0.1716/(0.7775-0.4984)\simeq 62\%$).
On the contrary, as the third feature is excluded from the model, its
contribution to the AUC is close to zero. Figure 1 displays the empirical
distributions of AUC, benchmark values $\hat{\phi_{0}}$, and XPER values
associated to features $x_{1}$, $x_{2}$, and $x_{3}$, computed from the $K$
simulations. It confirms the robustness of our analysis, but also illustrates
the possibility to make inference on XPER values by Bootstrap or other
numerical methods.

Figure 1: Empirical distributions of AUC and XPER values

Note: This figure displays the empirical distributions of the AUC and XPER
values on the test sample according to the DGP detailed in Illustration 1. The
solid red lines refer to kernel density estimations.

Table 2: Illustration of AUC XPER values in a three-fold probit model

| $G(y_{i};\mathbf{x}_{i};\hat{\delta}_{n})$ | $\hat{\phi}_{i,0}$ | $\hat{\phi}_{i,1}$ | $\hat{\phi}_{i,2}$ | $\hat{\phi}_{i,3}$ | $y_{i}$ | $\hat{\mathbb{P}}(y_{i}=1|\mathbf{x}_{i})$
---|---|---|---|---|---|---|---
i=1 | 0.9000 | 0.5001 | 0.2450 | 0.1771 | -0.0221 | 1 | 0.6614
i=2 | 1.0067 | 0.5001 | 0.2142 | 0.3203 | -0.0279 | 1 | 0.8785
… | … | … | … | … | … | … | …
i=300 | 0.2533 | 0.4967 | -0.1232 | -0.1237 | 0.0035 | 0 | 0.7616
| 0.7775 | 0.4984 | 0.1716 | 0.1098 | -0.0023 | 0.4967 | 0.4941

Note: This table displays individual contributions to the AUC, individual
benchmarks, and XPER values associated to each feature $x_{j}$, $j=1,2,3$, in
a three-fold probit model. The last row of the table reports average values of
the columns.

The third column of Table 2 displays the individual benchmark. For a binary
classification model, the benchmark
$\hat{\phi}_{i,0}\equiv\hat{\phi}_{i,0}(y_{i})$ only takes two values. In our
simulations, for individuals $y_{i}=1$ (respectively $y_{i}=0$), this value is
equal to $0.5001$ (respectively $0.4967$). Remind that the individual
benchmark corresponds to the contribution to the AUC obtained from a random
predictor for an individual with a target value $y_{i}=1$ or $y_{i}=0$.
Consider the first instance $i=1$ with $y_{i}=1$,
$G(y_{1};\mathbf{x}_{1};\hat{\delta}_{n})=0.9$ and $\hat{\phi}_{1,0}=0.5001$.
As $G(y_{1};\mathbf{x}_{1};\hat{\delta}_{n})>\hat{\phi}_{1,0}$, it means that
the features allow the probit model to better predict the event $y_{1}$ for
this instance than a random predictor.

Finally, columns 4, 5 and 6 report the XPER values associated to features
$x_{1},x_{2},x_{3}$. First, we verify that feature $x_{3}$ has close to no
impact on the AUC for all instances. Second, the heterogeneity of
contributions to the AUC depends on individual characteristics. Remind that at
the global level, the contribution of feature $x_{1}$ is higher than the one
of feature $x_{2}$. However, at the local level, we observe for instance $i=2$
that the contribution of feature $x_{1}$ is smaller than the one of feature
$x_{2}$, i.e., $0.2142<0.3203$. Third, some individual contributions are
negative. For instance, the negative contribution of feature $x_{2}$ for
individual $i=300$ implies that, for this instance, this feature disturbs the
model to predict the true target value $y_{i}=0$.

To further illustrate the XPER methodology, we provide in Appendix F two
additional Monte Carlo simulation experiments illustrating how XPER values can
be used to detect the origin of overfitting.

##  5 Empirical application

###  5.1 Data and model

We implement our methodology on a proprietary database of auto loans provided
by an international bank. For each borrower, we know whether he or she has
eventually defaulted ($y=1$) or not ($y=0$) on the loan. Given the sensitive
nature of the data, we had to randomly under-sample individuals to set the
default rate to an arbitrary 20% level. Besides benefits in terms of
confidentiality, setting a high arbitrary default rate also protects us
against concerns arising from using an unbalanced database. After under-
sampling, our database includes 7,440 borrowers. Besides the default target
variable, we have access to ten features on the loan (funding amount, funded
duration, vehicle price, down-payment) and on the borrower (job tenure, age,
marital status, monthly payment in percentage of income, home ownership
status, credit event). We divide the database into a stratified training (70%)
and test (30%) samples to have the same default rate in both subsamples.

We provide in Table A4 some summary statistics about the features and the
target variable. In our dataset, a typical loan amounts to around 11,500
euros, finances a 13,000 euro car, and lasts for 56 months. A typical borrower
is 45 years old, married, not owning his or her home, has spent nine years in
the same job, experienced no credit event over the past six months, provides
less than a 50% down payment, and allocates 10% of his or her monthly income
to reimburse the car loan. To get a first sense of the role of each feature on
default, we display in Figure A3 their distributions separately for defaulting
and non-defaulting borrowers. This preliminary test indicates that the list of
discriminating feature includes age, credit event, down payment, marital and
ownership status.

Using the training sample, we estimate an XGBoost model to predict default. We
selected this type of model because it is recognised as one of the most
powerful scoring engines (Gunnarsson et al.,, 2021). Another reason for using
an XGBoost is its black-box nature. Indeed, while the XPER methodology is
model-agnostic, we believe it is interesting to assess its usefulness when
used with a particularly complex and opaque algorithm. We select the
hyperparameters of the XGBoost using stratified five-fold cross-validation
based on a balanced accuracy criteria and a random search algorithm (Bergstra
and Bengio,, 2012).

Table 3 displays the values of six performance metrics for the XGBoost model
obtained on the training and test samples. Specifically, (1) the AUC measures
the discriminatory ability of the model, (2) the Brier Score evaluates the
accuracy of probabilities, and (3) the accuracy, Balanced Accuracy (BA),
sensitivity, and specificity assess the correctness of categorical
predictions. As shown in Table 3, the XGBoost has an AUC of 0.7521, a Brier
Score of 0.1433, and an accuracy of 79.53 on the test sample. We observe some
over-fitting in the model as its performances drops slightly from the training
sample to the test sample. Overall, the performance metrics of our model are
comparable to those displayed in Gunnarsson et al., (2021).

Table 3: XGBoost Performances

Sample | Size (%) | AUC | Brier Score | Accuracy | BA | Sensitivity | Specificity
---|---|---|---|---|---|---|---
Training | 70 | 0.8969 | 0.0958 | 86.98 | 72.43 | 48.18 | 96.69
Test | 30 | 0.7521 | 0.1433 | 79.53 | 58.69 | 23.99 | 93.39

###  5.2 XPER decomposition

We display in Figure 2(a) the decomposition of the AUC among the ten features
obtained with the estimation method detailed in Appendix C.2. For ease of
presentation, we express the feature contributions in percentage of the spread
between the AUC and its benchmark (see Equation 5). As shown in Table 3, the
AUC in the test sample is equal to 0.7521, which is significantly better than
the benchmark value of 0.5 obtained for a random predictor. We see that around
40% of this over-performance is coming from funding amount. The second most
contributing feature is job tenure, which accounts for another 18%. It is
interesting to note that with only two features, we can explain more than half
of the performance of the model. Next, we have five features which contribute
each for another 8-10% to the performance. At the other side of the spectrum,
down payment is the feature contributing the least to the AUC. This feature
does not help the model to better predict default than a random predictor as
its XPER value is close to 0. Note that in Appendix G.3, we also analyse the
effect of the various features on the performance for each borrower
individually.

(a) XPER values

(b) PI-based feature contributions

Figure 2: XPER decomposition and Permutation Importance (PI)

Note: This figure displays in Panel (a) the XPER values for the AUC of the
XGBoost model estimated on the test sample and in Panel (b) the feature
contributions for the AUC based on Permutation Importance.

We now compare the XPER performance decomposition to standard feature
contribution methods commonly used in machine learning to assess the impact of
a feature on performance or to explain the output of a black-box model, namely
Permutation Importance (PI), feature importance, and SHAP values.

First, we distinguish between the XPER decomposition of the AUC and the
permutation importance introduced by Breiman, (2001). The latter computes for
a feature $x_{j}$, the decrease in the accuracy of the model when the values
of this feature are randomly reshuffled across instances. As the permutation
breaks the dependency between the target variable and the feature, the
resulting drop in model accuracy indicates how much the model depends on the
feature. In our analysis, the PI results indicate the average decrease in
accuracy when the values are reshuffled to obtain more robust results. As for
the previous methods, we divide each feature contribution by the sum of all
features contributions to compare them to the XPER values. As shown in Figure
2(b), XPER values and PI deliver very distinct results. For instance, the
contribution of the job tenure is twice as important with XPER than with PI,
whereas the contribution of the car price is twice as large with PI than with
XPER. Moreover, unlike with XPER, the age of the customer, home-ownership
status, and monthly payment loan have virtually no impact on the performance
of the model according to PI. Therefore, even if PI also assesses the effect
of the features on the performance of the model it delivers different results
than XPER.

Second, we contrast in Figure 3(a) the XPER values of the AUC and the XGBoost-
based feature importance. The latter computes for a feature $x_{j}$, the
average gain across all splits the feature is used in. For ease of comparison,
we divide each feature contribution by the sum of the ten feature
contributions. As shown in Figure 3(a), the result is rather striking. Indeed,
the two methodologies lead to very different contributions as some dominating
features in a given methodology play a minor role in the other. For instance,
credit event exhibits the highest feature importance but it is only the $9th$
most contributing feature according to XPER. Differently, funding amount plays
a very important role to explain performance but does not contribute much in
terms of feature importance.

(a) XPER vs. feature importance

(b) XPER vs. SHAP

Figure 3: XPER vs. other feature contribution methodologies

Note: This figure compares the XPER values of the AUC with (a) the XGBoost-
based feature importances and (b) the average absolute SHAP values.

Third, we compare in Figure 3(b) the XPER decomposition of the AUC with the
SHAP values introduced by Lundberg and Lee, (2017). In our context, the SHAP
values assess the impact of the different features on the probabilities of
default of each borrower. As it is the norm, we take the average absolute SHAP
values for each feature to assess the feature contribution at the model level.
As before, we divide each feature contribution by the sum of all features
contributions to express them in percentages. In Figure 3(b), we see that SHAP
and XPER provide for some features very different information. For instance,
the car price contribution is more than twice as important for SHAP than for
XPER. Similarly, the funding amount contribution is around 40% for XPER
whereas only 28% for SHAP. We conclude that XPER delivers different,
incremental information over SHAP at the model level.

###  5.3 Using XPER to boost model performance

We now show how XPER can be used to deal with heterogeneity issues and improve
out-of-sample performance. In practice, it is often challenging to estimate a
single model able to correctly estimate the relationship between the target
variable and the features for all individuals in the sample. In this section,
we propose an alternative two-step procedure. First, we build homogeneous
groups of individuals displaying similar XPER values using a clustering
algorithm. Second, within each group, we estimate a group-specific model.
Given the definition of XPER, in each group, the features have a similar
impact on the target variable. This alternative approach is likely to increase
the model performance compared to the one-fits-all model, which has been used
so far in the empirical application.

In the first step, we use the KMedoid method to create two clusters of
individuals with similar XPER values $\hat{\phi}_{i,j}$ on the training
sample. We see in Figure 4 that individuals in group 1 tend to be older,
married, home-owners and ask for a mid-range loan to finance a mid-priced car.
One striking difference is that in group 2, individuals tend to finance cars
with extremely low prices or high prices. Overall, group 1 gathers individuals
displaying a lower risk profile compared to those in group 2. This is
confirmed by the fact that the default rate is much higher in group 2 (52%)
than in group 1 (13%). Moreover, as shown in Figures 7(a) and 7(b) in Appendix
G.4, the feature distributions are very different for defaulters and non-
defaulters in each group. We also see in Figure A8 that in group 2, using the
car price deteriorates the performance of the model as its XPER value is close
to -20% vs. 20% in group 1. Therefore, these results suggest that estimating
group-specific models is likely to boost predictive performance.

In the second step, we estimate using the training dataset one XGBoost model
per group to predict default. Then, we aim to compare the resulting out-of-
sample performance with the one of the one-fits-all model. To do so, we first
assign each individual of the test sample to either group 1 or group 2 using
the clustering rule built on the training sample. We see in Table 4 that the
performance of our strategy (column 2) is significantly higher than the
performance of the initial model (column 1). For instance, the AUC of the
model on the test sample increases by 16 percentage points, from 0.752 to
0.912. Another interesting result is that by using two distinct models, the
bank would be able to correctly detect 64% of the defaulting borrowers against
only 24% with the initial model. Yet, it would still be able to correctly
detect more than 95% of the non-defaulting borrowers.

Finally, we benchmark our strategy with a standard clustering approach based
on the features themselves. Specifically, we use the K-prototype algorithm to
create two clusters of individuals with comparable feature values. As shown in
column 3, the performance of the standard approach is very close to the one of
the one-fits-all model (AUC = 0.744 vs. 0.752), and is much lower than the
performance of the XPER-based models.

Figure 4: Features distribution by group based on XPER values

Note: This figure displays the distribution of the features on the training
sample by group created from individual XPER values using the KMedoids
methodology. For continuous features, we use kernel density estimation. Dark
red refers to the first group and light red to the second group.

Table 4: Model performances

| Initial | Clusters on XPER values | Clusters on features
---|---|---|---
| (1) | (2) | (3)
AUC | 0.752 | 0.912 | 0.744
Brier score | 0.143 | 0.080 | 0.151
Accuracy | 79.53 | 89.11 | 79.53
Balanced Accuracy | 58.69 | 79.74 | 59.11
Sensitivity | 23.99 | 64.13 | 25.11
Specificity | 93.39 | 95.35 | 93.11

##  6 Discussion

We have introduced XPER, a methodology designed to measure the feature
contributions to the performance of any regression or classification model.
XPER is built on Shapley values and interpretability tools developed in
machine learning but with the distinct objective of focusing on model
performance, such as AUC or $R^{2}$, and not on model predictions, $\hat{y}$.

Specifically, XPER breaks down the difference between a performance metric of
the model and a benchmark value. The latter corresponds to the performance
metric that we would obtain on a hypothetical sample in which the target
variable is independent from all the features included in the model. As such,
the XPER decomposition only focuses on the part of the performance that
directly originate from the features. Other advantages of the method include
being implementable either at the model level or at the individual level, not
being plagued by model specification error, as it does not require re-
estimating the model, and having, as a special case, the standard
explainability method in machine learning (SHAP).

We show that identifying the driving forces of the performance of a predictive
model is very useful in practice. In a loan default forecasting application,
XPER appears to be able to efficiently deal with heterogeneity issues and to
significantly boost out-of-sample performance. To do so, we create homogeneous
groups of borrowers by clustering them based on their individual XPER values.
We find that estimating group-specific models yields to a much higher
predictive accuracy than with a one-fits-all model.

Several applications of our method could be envisioned in the future. In
medical applications, XPER could be used to identify the variables that have
the highest impact on the accuracy of a model forecasting, for instance, the
resistance of a virus (Williamson et al., 2021a, ). In finance, our
methodology could identify the main drivers of the financial performance of a
portfolio of assets constructed using a large number of state variables.
Another natural application would be to choose variables during the selection
phase of model development. Indeed, by contrasting the XPER values estimated
in the training and validation sets, one could only keep the variables
maintaining a high-enough level of importance in the validation test.
Furthermore, XPER could go beyond performance. Indeed, as XPER can decompose
any function of both $\hat{y}$ and $y$, one could use it to identify the
features responsible for the lack of algorithmic fairness of a given machine
learning model (Chen et al.,, 2023).

## References

  * Aas et al., (2021) Aas, K., Jullum, M., and Løland, A. (2021).  Explaining individual predictions when features are dependent: More accurate approximations to Shapley values.  Artificial Intelligence, 298:103502.
  * Ahlburg, (1984) Ahlburg, D. A. (1984).  Forecast evaluation and improvement using Theil’s decomposition.  Journal of Forecasting, 3(3):345–351.
  * Bergstra and Bengio, (2012) Bergstra, J. and Bengio, Y. (2012).  Random search for hyper-parameter optimization.  Journal of Machine Learning Research, 12:281–305.
  * Borup et al., (2022) Borup, D., Coulombe, G., Rapach, D. E., Schütte, E. C. M., and Schwenk-Nebbe, S. (2022).  The Anatomy of Out-of-Sample Forecasting Accuracy.  Federal Reserve Bank of Atlanta.
  * Bourguignon, (1979) Bourguignon, F. (1979).  Decomposable Income Inequality Measures.  Econometrica, 47(4):901–920.
  * Bowen and Ungar, (2020) Bowen, D. and Ungar, L. (2020).  Generalized shap: Generating multiple types of explanations in machine learning.  arXiv preprint arXiv:2006.07155.
  * Breiman, (2001) Breiman, L. (2001).  Random forests.  Machine Learning, 45(1):5–32.
  * Casalicchio et al., (2019) Casalicchio, G., Molnar, C., and Bischl, B. (2019).  Visualizing the feature importance for black box models.  In Machine Learning and Knowledge Discovery in Databases, pages 655–670. Springer International Publishing.
  * Chen et al., (2023) Chen, H., Lu, W., Song, R., and Ghosh, P. (2023).  On learning and testing of counterfactual fairness through data preprocessing.  Journal of the American Statistical Association, pages 1–11.
  * Chevan and Sutherland, (1991) Chevan, A. and Sutherland, M. (1991).  Hierarchical partitioning.  The American Statistician, 45(2):90–96.
  * Gelman and Vehtari, (2021) Gelman, A. and Vehtari, A. (2021).  What are the most important statistical ideas of the past 50 years?  Journal of the American Statistical Association, 116(536):2087–2097.
  * Grömping, (2015) Grömping, U. (2015).  Variable importance in regression models.  Wiley interdisciplinary reviews: Computational statistics, 7(2):137–152.
  * Grömping, (2007) Grömping, U. (2007).  Estimators of relative importance in linear regression based on variance decomposition.  The American Statistician, 61(2):139–147.
  * Gunnarsson et al., (2021) Gunnarsson, B. R., vanden Broucke, S., Baesens, B., Óskarsdóttir, M., and Lemahieu, W. (2021).  Deep learning for credit scoring: Do or don’t?  European Journal of Operational Research, 295(1):292–305.
  * Israeli, (2007) Israeli, O. (2007).  A Shapley-based decomposition of the R-square of a linear regression.  Journal of Economic Inequality, 5(2):199–212.
  * Johnson, (2000) Johnson, J. W. (2000).  A heuristic method for estimating the relative weight of predictor variables in multiple regression.  Multivariate Behavioral Research, 35(1):1–19.
  * Kruskal, (1987) Kruskal, W. (1987).  Relative importance by averaging over orderings.  The American Statistician, 41(1):6–10.
  * Kumar et al., (2020) Kumar, I. E., Venkatasubramanian, S., Scheidegger, C., and Friedler, S. A. (2020).  Problems with shapley-value-based explanations as feature importance measures.  International Conference on Machine Learning.
  * Lahaye and Neely, (2020) Lahaye, J. and Neely, C. (2020).  The role of jumps in volatility spillovers in foreign exchange markets: meteor shower and heat waves revisited.  Journal of Business & Economic Statistics, 38(2):410–427.
  * Lindeman et al., (1980) Lindeman, R. H., Merenda, P. F., and Gold, R. Z. (1980).  Introduction to Bivariate and Multivariate Analysis.  Glenview.
  * Liu and Ročková, (2023) Liu, Y. and Ročková, V. (2023).  Variable selection via thompson sampling.  Journal of the American Statistical Association, 118(541):287–304.
  * Lundberg et al., (2018) Lundberg, S. M., Erion, G. G., and Lee, S.-I. (2018).  Consistent individualized feature attribution for tree ensembles.  arXiv preprint arXiv:1802.03888.
  * Lundberg and Lee, (2017) Lundberg, S. M. and Lee, S.-I. (2017).  A unified approach to interpreting model predictions.  Advances in Neural Information Processing Systems, 30.
  * Molnar, (2020) Molnar, C. (2020).  Interpretable machine learning.  Lulu.com.
  * Murdoch et al., (2019) Murdoch, W. J., Singh, C., Kumbier, K., Abbasi-Asl, R., and Yu, B. (2019).  Definitions, methods, and applications in interpretable machine learning.  Proceedings of the National Academy of Sciences, 116(44):22071–22080.
  * Perron and Yamamoto, (2021) Perron, P. and Yamamoto, Y. (2021).  Testing for changes in forecasting performance.  Journal of Business & Economic Statistics, 39(1):148–165.
  * Redell, (2019) Redell, N. (2019).  Shapley decomposition of R-squared in machine learning models.  arXiv preprint arXiv:1908.09718.
  * Shapley, (1953) Shapley, L. (1953).  The value of a player in n-person games.  Contributions to the Theory of Games, 2:307–317.
  * Shorrocks, (1980) Shorrocks, A. F. (1980).  The Class of Additively Decomposable Inequality Measures.  Econometrica, 48(3):613–625.
  * Shorrocks, (1982) Shorrocks, A. F. (1982).  Inequality decomposition by factor components.  Econometrica, 50(1):193–211.
  * Strumbelj and Kononenko, (2010) Strumbelj, E. and Kononenko, I. (2010).  An Efficient Explanation of Individual Classifications using Game Theory.  Journal of Machine Learning Research, 11:1–18.
  * Stufken, (1992) Stufken, J. (1992).  On hierarchical partitioning.  The American Statistician, 46:70–71.
  * Sun and Wang, (2021) Sun, Y. and Wang, L. (2021).  Stochastic tree search for estimating optimal dynamic treatment regimes.  Journal of the American Statistical Association, 116(533):421–432.
  * Sundararajan et al., (2020) Sundararajan, M., Dhamdhere, K., and Agarwal, A. (2020).  The shapley taylor interaction index.  International Conference on Machine Learning, pages 9259–9268.
  * Sundararajan and Najmi, (2020) Sundararajan, M. and Najmi, A. (2020).  The many Shapley values for model explanation.  International Conference On Machine Learning, pages 9269–9278.
  * Sundararajan et al., (2017) Sundararajan, M., Taly, A., and Yan, Q. (2017).  Axiomatic attribution for deep networks.  International Conference on Machine Learning, pages 3319–3328.
  * Theil, (1971) Theil, H. (1971).  Applied Economic Forecasting.  North-Holland, Amsterdam.
  * Williamson and Feng, (2020) Williamson, B. and Feng, J. (2020).  Efficient nonparametric statistical inference on population feature importance using shapley values.  In International Conference on Machine Learning, pages 10282–10291. PMLR.
  * (39) Williamson, B. D., Gilbert, P. B., Carone, M., and Simon, N. (2021a).  Nonparametric variable importance assessment using machine learning techniques.  Biometrics, 77(1):9–22.
  * (40) Williamson, B. D., Gilbert, P. B., Simon, N. R., and Carone, M. (2021b).  A general framework for inference on algorithm-agnostic variable importance.  Journal of the American Statistical Association, pages 1–14.
  * Zhao and Hastie, (2021) Zhao, Q. and Hastie, T. (2021).  Causal interpretations of black-box models.  Journal of Business & Economic Statistics, 39(1):272–281.

## Appendix

##  Appendix A Examples of performance metrics

Table A1: Performance metrics (a) Panel A: Regression models

Metrics | $G_{n}(\mathbf{y},\mathbf{x})$ | $G(y_{i};\mathbf{x}_{i};\hat{\delta}_{n})$ | $\hat{\delta}_{n}$
---|---|---|---
MAE |  $\frac{1}{n}\sum_{i=1}^{n}\left|y_{i}-\hat{f}(\mathbf{x}_{i})\right|$ |  $\left|y_{i}-\hat{f}(\mathbf{x}_{i})\right|$ |  $\emptyset$
MSE |  $\frac{1}{n}\sum_{i=1}^{n}\left(y_{i}-\hat{f}(\mathbf{x}_{i})\right)^{2}$ |  $\left(y_{i}-\hat{f}(\mathbf{x}_{i})\right)^{2}$ |  $\emptyset$
$R^{2}$ |  $1-\frac{\sum_{i=1}^{n}\left(y_{i}-\hat{f}(\mathbf{x}_{i})\right)^{2}}{\sum_{j=1}^{n}(y_{j}-\bar{y})^{2}}$ |  $1-\hat{\delta}_{n}^{-1}\left(y_{i}-\hat{f}(\mathbf{x}_{i})\right)^{2}$ |  $n^{-1}\sum_{j=1}^{n}(y_{j}-\bar{y})^{2}$

(b) Panel B: Classification models

Metrics | $G_{n}(\mathbf{y},\mathbf{x})$ | $G(y_{i};\mathbf{x}_{i};\hat{\delta}_{n})$ | $\hat{\delta}_{n}$
---|---|---|---
Accuracy |  $\frac{1}{n}\sum_{i=1}^{n}\left(y_{i}\hat{f}(\mathbf{x}_{i})+(1-y_{i})(1-\hat{f}(\mathbf{x}_{i}))\right)$ |  $y_{i}\hat{f}(\mathbf{x}_{i})+(1-y_{i})(1-\hat{f}(\mathbf{x}_{i}))$ |  $\emptyset$
BA |  $\frac{1}{n}\sum_{i=1}^{n}\frac{1}{2}\left[\frac{y_{i}\hat{f}(\mathbf{x}_{i})}{\frac{1}{n}\sum_{j=1}^{n}y_{j}}+\frac{(1-y_{i})(1-\hat{f}(\mathbf{x}_{i}))}{\frac{1}{n}\sum_{j=1}^{n}(1-y_{j})}\right]$ |  $\frac{1}{2}\left[\hat{\delta}_{n_{1}}^{-1}\left(y_{i}\hat{f}(\mathbf{x}_{i})\right)+\hat{\delta}_{n_{2}}^{-1}\left((1-y_{i})(1-\hat{f}(\mathbf{x}_{i}))\right)\right]$ |  $\hat{\delta}_{n_{1}}=\frac{1}{n}\sum_{j=1}^{n}y_{j}$
|  |  $\hat{\delta}_{n_{2}}=\frac{1}{n}\sum_{j=1}^{n}(1-y_{j})$
Brier score |  $\frac{1}{n}\sum_{i=1}^{n}\left(y_{i}-\hat{P}(\mathbf{x}_{i})\right)^{2}$ |  $\left(y_{i}-\hat{P}(\mathbf{x}_{i})\right)^{2}$ |  $\emptyset$
Precision |  $\frac{1}{n}\sum_{i=1}^{n}\left(\frac{y_{i}\hat{f}(\mathbf{x}_{i})}{\frac{1}{n}\sum_{j=1}^{n}\hat{f}(\mathbf{x}_{j})}\right)$ |  $\hat{\delta}_{n}^{-1}y_{i}\hat{f}(\mathbf{x}_{i})$ |  $\frac{1}{n}\sum_{j=1}^{n}\hat{f}(\mathbf{x}_{j})$
Sensitivity |  $\frac{1}{n}\sum_{i=1}^{n}\left(\frac{y_{i}\hat{f}(\mathbf{x}_{i})}{\frac{1}{n}\sum_{j=1}^{n}y_{j}}\right)$ |  $\hat{\delta}_{n}^{-1}y_{i}\hat{f}(\mathbf{x}_{i})$ |  $\frac{1}{n}\sum_{j=1}^{n}y_{j}$
Specificity |  $\frac{1}{n}\sum_{i=1}^{n}\left(\frac{(1-y_{i})(1-\hat{f}(\mathbf{x}_{i}))}{\frac{1}{n}\sum_{j=1}^{n}(1-y_{j})}\right)$ |  $\hat{\delta}_{n}^{-1}(1-y_{i})(1-\hat{f}(\mathbf{x}_{i}))$ |  $\frac{1}{n}\sum_{j=1}^{n}(1-y_{j})$
AUC |  $\frac{\sum_{i=1}^{n}\sum_{j=1}^{n}(1-y_{i})y_{j}I(\hat{P}(\mathbf{x}_{i})<\hat{P}(\mathbf{x}_{j}))}{\sum_{j=1}^{n}y_{j}\sum_{j=1}^{n}(1-y_{j})}$ |  $\left((1-y_{i})\times\hat{\delta}_{n_{1}}(\mathbf{x}_{i})\right)\hat{\delta}_{n_{2}}^{-1}$ |  $\hat{\delta}_{n_{1}}(\mathbf{x}_{i})=\frac{1}{n}\sum_{j=1}^{n}y_{j}I(\hat{P}(\mathbf{x}_{i})<\hat{P}(\mathbf{x}_{j}))$
$I(\hat{P}(\mathbf{x}_{i})<\hat{P}(\mathbf{x}_{j}))=\left\\{\begin{array}[]{ll}0&\mbox{ if }\hat{P}(\mathbf{x}_{i})>\hat{P}(\mathbf{x}_{j})\\\ 0.5&\mbox{ if }\hat{P}(\mathbf{x}_{i})=\hat{P}(\mathbf{x}_{j})\\\ 1&\mbox{ if }\hat{P}(\mathbf{x}_{i})<\hat{P}(\mathbf{x}_{j})\end{array}\right.$ |  |  $\hat{\delta}_{n_{2}}=\frac{1}{n^{2}}\sum_{j=1}^{n}y_{j}\sum_{j=1}^{n}(1-y_{j})$

Note: This table displays the expression of sample performance metrics
$G_{n}(\mathbf{y},\mathbf{x})$, individual contribution to the sample
performance metric $G(y_{i};\mathbf{x}_{i};\hat{\delta}_{n})$, and the
corresponding nuisance parameter $\hat{\delta}_{n}$. We distinguish between
the performance metric associated to regression models (Panel A) and those
related to classification models (Panel B).

##  Appendix B Examples of XPER values decomposition

We provide several examples in Table 2(b) of the XPER decomposition of
regression and classification performance metrics. For regression models, we
consider a linear regression model
$\hat{f}(\mathbf{x}_{i})=\sum_{j=1}^{q}\hat{\beta}_{j}x_{i,j}$, where we
assume that the DGP generating the test sample
$S_{n}=\\{\mathbf{x}_{i},y_{i},\hat{f}(\mathbf{x}_{i})\\}^{n}_{i=1}$ satisfies
$\mathbb{E}\left(\mathbf{x}\right)=0_{q}\mbox{ and
}\mathbb{V}(\mathbf{x})=diag(\sigma^{2}_{x_{j}})\mbox{ }\forall j=1,...,q$,
and $\mathbb{E}(y)=0$. We denote by $\sigma_{y}^{2}$ the variance of the
target variable and by $\sigma_{y,x_{j}}$ the covariance between the feature
$x_{j}$ and the target variable. For classification models, we consider any
binary classification model $\hat{f}(\mathbf{x})$, with
$\hat{P}(\mathbf{x})=\hat{\mathbb{P}}(y=1|\mathbf{x})$ the estimated
probability of belonging to class 1 ($y=1$). We denote by
$\sigma_{y,\hat{f}(\mathbf{x})}$ the covariance between the target variable
and the classification output.

Table A2: Examples of XPER values decomposition (a) Panel A: Regression models

Metrics | $\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$ | $\phi_{0}$ | $\phi_{j}$
---|---|---|---
MSE |  $2\sum_{j=1}^{q}\hat{\beta}_{j}\sigma_{y,x_{j}}$ |  $-\sum_{j=1}^{q}\hat{\beta}_{j}^{2}\sigma^{2}_{x_{j}}-\sigma_{y}^{2}$ |  $2\hat{\beta}_{j}\sigma_{y,x_{j}}$
$-\sum_{j=1}^{q}\hat{\beta}_{j}^{2}\sigma^{2}_{x_{j}}-\sigma_{y}^{2}$ |  |
$R^{2}$ |  $\frac{\sigma_{y,\hat{y}}}{\sigma^{2}_{y}}$ |  $-\frac{\sigma_{y,\hat{y}}}{\sigma^{2}_{y}}$ |  $\frac{2\hat{\beta}_{j}\sigma_{y,x_{j}}}{\sigma^{2}_{y}}$

(b) Panel B: Classification models

Metrics | $\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$ | $\phi_{0}$ | $\phi_{j}$
---|---|---|---
Accuracy |  $2\sigma_{y,\hat{f}(\mathbf{x})}+2\mathbb{P}(y=1)\hat{P}(\mathbf{x})$ |  $2\mathbb{P}(y=1)\hat{P}(\mathbf{x})+1-\mathbb{P}(y=1)-\hat{P}(\mathbf{x})$ |  No closed-form
$+1-\mathbb{P}(y=1)-\hat{P}(\mathbf{x})$ |  |
Precision |  $\frac{\sigma_{y,\hat{f}(\mathbf{x})}}{\mathbb{P}(\hat{f}(\mathbf{x})=1)}+\mathbb{P}(y=1)$ |  $\mathbb{P}(y=1)$ |  No closed-form
Sensitivity |  $\frac{\sigma_{y,\hat{f}(\mathbf{x})}}{\mathbb{P}(y=1)}+\mathbb{P}(\hat{f}(\mathbf{x})=1)$ |  $\mathbb{P}(\hat{f}(\mathbf{x})=1)$ |  No closed-form
Specificity |  $\frac{\sigma_{y,\hat{f}(\mathbf{x})}}{\mathbb{P}(y=0)}+\mathbb{P}(\hat{f}(\mathbf{x})=0)$ |  $\mathbb{P}(\hat{f}(\mathbf{x})=0)$ |  No closed-form
AUC |  No closed-form |  $0.5$ |  No closed-form

Note: This table displays the expression of population performance metrics
$\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$, benchmark values
$\phi_{0}$, and XPER values $\phi_{j}$. We distinguish between the performance
metric associated to regression models (Panel A) and those related to
classification models (Panel B). See Appendix E.3, E.4, and E.5 for the proofs
associated to the MSE, the $R^{2}$ and the accuracy.

##  Appendix C Estimation

###  C.1 With a small number of features

In this section, we discuss the estimation procedure for the XPER values
$\phi_{j}$ and $\phi_{i,j}$, based on a test sample
$S_{n}=\\{\mathbf{x}_{i},y_{i},\hat{f}(\mathbf{x}_{i})\\}^{n}_{i=1}$. Below,
we assume that the model has a small number of features, typically $q\leq
10$.111The standard estimation framework is only feasible for a small number
of features $q$. Indeed, the computation of the XPER values $\phi_{j}$
according to definition 3 becomes cumbersome as the number of features
increases. Indeed, it requires to consider $q\times 2^{q-1}$ coalitions of
features, e.g., $5,120$ for $q=10$ and $10,485,760$ for $q=20$, etc. When the
number of features is larger, an approximation of the XPER values is required
and we propose a modified version of the Kernel SHAP method of Lundberg and
Lee, (2017) (see Appendix C.2 for more details).

Under Assumption 1, the XPER value $\phi_{j}$ can be estimated by a weighted
average of individual contributions differences such as:

$\small\hat{\phi}_{j}=\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}w_{S}\left[\frac{1}{n^{2}}\sum_{u=1}^{n}\sum_{v=1}^{n}G\left(y_{v};x_{v,j},\mathbf{x}_{v}^{S},\mathbf{x}_{u}^{\overline{S}};\hat{\delta}_{n}\right)-\frac{1}{n^{2}}\sum_{u=1}^{n}\sum_{v=1}^{n}G\left(y_{v};x_{u,j},\mathbf{x}_{v}^{S},\mathbf{x}_{u}^{\overline{S}};\hat{\delta}_{n}\right)\right],$
(A1)

with $S$ a coalition, i.e., a subset of features, excluding the feature of
interest $x_{j}$, and
$\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$ the powerset of
the set $\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\}$.

Table A3 illustrates the computation of the estimated value $\hat{\phi}_{1}$
associated to feature $x_{1}$ in a model with three features
$(x_{1},x_{2},x_{3})$. For each coalition of features (column 1), we report
the corresponding weight (column 2) along with the estimated marginal
contribution of feature $x_{1}$ to the performance metric (column 3). The
intuition is as follows: the sum over index $u$ refers to the marginalisation
effect, whereas the sum over index $v$ refers to the averaging effect. For a
given instance $v$, we compute its average performance metric by replacing the
features ${x}^{\overline{S}}$ which are not included in the coalition by the
corresponding values observed for all the instances of the test sample
(marginalisation). Then, we compute the average performance metric for all the
instances $v$ (averaging).

Table A3: Computation of the XPER value $\hat{\phi}_{1}$ in a three-feature
model

$S$ | $w_{S}$ | $\frac{1}{n^{2}}\sum_{u=1}^{n}\sum_{v=1}^{n}G\left(y_{v};x_{v,j},\mathbf{x}_{v}^{S},\mathbf{x}_{u}^{\overline{S}};\hat{\delta}_{n}\right)-\frac{1}{n^{2}}\sum_{u=1}^{n}\sum_{v=1}^{n}G\left(y_{v};x_{u,j},\mathbf{x}_{v}^{S},\mathbf{x}_{u}^{\overline{S}};\hat{\delta}_{n}\right)$
---|---|---
$\left\\{\emptyset\right\\}$ | $1/3$ | $\frac{1}{n^{2}}\sum_{u=1}^{n}\sum_{v=1}^{n}G\left(y_{v};x_{v,1},x_{u,2},x_{u,3};\hat{\delta}_{n}\right)-\frac{1}{n^{2}}\sum_{u=1}^{n}\sum_{v=1}^{n}G\left(y_{v};x_{u,1},x_{u,2},x_{u,3};\hat{\delta}_{n}\right)$
$\left\\{x_{2}\right\\}$ | $1/6$ | $\frac{1}{n^{2}}\sum_{u=1}^{n}\sum_{v=1}^{n}G\left(y_{v};x_{v,1},x_{v,2},x_{u,3};\hat{\delta}_{n}\right)-\frac{1}{n^{2}}\sum_{u=1}^{n}\sum_{v=1}^{n}G\left(y_{v};x_{u,1},x_{v,2},x_{u,3};\hat{\delta}_{n}\right)$
$\left\\{x_{3}\right\\}$ | $1/6$ | $\frac{1}{n^{2}}\sum_{u=1}^{n}\sum_{v=1}^{n}G\left(y_{v};x_{v,1},x_{u,2},x_{v,3};\hat{\delta}_{n}\right)-\frac{1}{n^{2}}\sum_{u=1}^{n}\sum_{v=1}^{n}G\left(y_{v};x_{u,1},x_{u,2},x_{v,3};\hat{\delta}_{n}\right)$
$\left\\{x_{2},x_{3}\right\\}$ | $1/3$ | $\frac{1}{n}\sum_{v=1}^{n}G\left(y_{v};x_{v,1},x_{v,2},x_{v,3};\hat{\delta}_{n}\right)-\frac{1}{n^{2}}\sum_{u=1}^{n}\sum_{v=1}^{n}G\left(y_{v};x_{u,1},x_{v,2},x_{v,3};\hat{\delta}_{n}\right)$

  * Note: This table displays details of empirical XPER value computation, i.e., the coalitions (column 1), the associated weights (column 2) and the estimated marginal contributions (column 3).

Similarly to the estimation of features $\phi_{j}$, we can estimate the
individual XPER values $\phi_{i,j}$. For any individual $i$ of the test sample
of $S_{n}$ and any feature $x_{j}$, a consistent estimator of the individual
XPER values $\phi_{i,j}$ is defined as:

$\small\hat{\phi}_{i,j}=\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}w_{S}\left[\frac{1}{n}\sum_{u=1}^{n}G\left(y_{i};x_{i,j},\mathbf{x}_{i}^{S},\mathbf{x}_{u}^{\overline{S}};\hat{\delta}_{n}\right)-\frac{1}{n}\sum_{u=1}^{n}G\left(y_{i};x_{u,j},\mathbf{x}_{i}^{S},\mathbf{x}_{u}^{\overline{S}};\hat{\delta}_{n}\right)\right].$
(A2)

By definition, these individual XPER values satisfy:

$\small\hat{\phi}_{j}=\frac{1}{n}\sum_{i=1}^{n}\hat{\phi}_{i,j}.$ (A3)

###  C.2 With a large number of features

As shown in section 3, the XPER value $\phi_{j}$ relies on every possible
coalition
$S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$ for
each feature. The total number of coalitions required to compute
$\left(\phi_{1},\dots,\phi_{q}\right)$ is equal to $q\times 2^{(q-1)}$
coalitions, with $2^{(q-1)}$ the number of coalitions for feature $j$. As the
total number of coalitions quickly increases with the number of features
($5,120$ for $q=10$ and $10,485,760$ for $q=20$), XPER values turn out to be
cumbersome to compute or estimate in practice. This issue is related to the
general concept of Shapley value and is not specific to our approach.

To overcome this issue, a standard approach in the literature is to
approximate the exact values. Intuitively, the idea is to rely only on a
subset of coalitions to compute XPER values rather than on its total number in
order to reduce the number of computations required. To do so, we propose an
approximation method of XPER values which is based on the recent model-
agnostic approach of Lundberg and Lee, (2017) called Kernel SHAP. The main
advantage of this approach is that it can be used for any application as it
does not depend on the model $f\left(.\right)$ nor on the performance metric
$G_{n}\left(\mathbf{y};\mathbf{X}\right)$. Denote as $S_{k}$ a coalition
randomly drawn for $k=1,\dots,K$, $K<q\times 2^{(q-1)}$ the total number of
coalitions drawn, and
$\tilde{S_{k}}=\\{\mathbf{x}^{\bar{S_{k}}}\\}\cup\\{\mathbf{x}_{j}\\}$ the
vector of features not included in coalition $S_{k}$. Formally, the
approximation of XPER values $\hat{\phi}_{j}$ is based on a subset $K$ of
coalitions $S_{k}$ and on the following model

$G_{n,k}\left(\mathbf{y};\mathbf{X}\right)=\phi_{0}+\sum_{j=1}^{q}\phi_{j}z_{k,j},$
(A4)

where

$G_{n,k}\left(\mathbf{y};\mathbf{X}\right)=\frac{1}{n^{2}}\sum_{u=1}^{n}\sum_{v=1}^{n}G\left(y_{v};\mathbf{x}_{v}^{S_{k}},\mathbf{x}_{u}^{\overline{S}_{k}};\hat{\delta}_{n}\right),$

is a sample performance metric associated to the coalition $S_{k}$, and
$z_{k,j}$ is equal to $1$ if the feature $j$ belongs to the coalition $S_{k}$,
and $0$ otherwise. In matrix notation, we have

$\displaystyle\mathbf{G_{n}\left(\mathbf{y};\mathbf{X}\right)}$
$\displaystyle=\mathbf{Z}\boldsymbol{\phi}$
$\displaystyle\underset{\left(K\times
1\right)}{\mathbf{G_{n}\left(\mathbf{y};\mathbf{X}\right)}}=\begin{pmatrix}G_{n,1}\left(\mathbf{y};\mathbf{X}\right)\\\
\vdots\\\
G_{n,K}\left(\mathbf{y};\mathbf{X}\right)\end{pmatrix},\underset{\left(K\times
q+1\right)}{\mathbf{Z}}$
$\displaystyle=\begin{pmatrix}1&z_{1,1}&{1}&z_{1,q}\\\
\vdots&\vdots&\ddots&\vdots\\\ 1&z_{K,1}&{1}&z_{K,q}\\\
\end{pmatrix},\underset{\left(q+1\times
1\right)}{\boldsymbol{\phi}}=\begin{pmatrix}\phi_{0}\\\ \phi_{1}\\\ \vdots\\\
\phi_{q}\end{pmatrix}.$

This approach relies on the comparison of $K$ values of the performance metric
obtained for $K$ randomly drawn coalitions involving different subset of
features. The features being present in some coalitions and absent in others,
the estimation of this regression allows to approximate the impact of each
feature on the performance metric that would be obtained for the subset $k$ of
features, and thus to approximate estimated XPER values $\hat{\phi}_{j}$.
Moreover, contrary to Equation A1 this approach estimates the impact of every
feature all at once, similarly to parameters’ estimation in a traditional
ordinary least squares regression, and is much faster as it only involves the
calculation of $K$ coalitions rather than $q\times 2^{(q-1)}$.

Recall that in Equation A1, the sum of marginal contribution to the
performance metric is weighted as some coalitions are more informative than
others. To this end, Lundberg and Lee, (2017) propose the Kernel Shap which is
a function that attributes a higher weight to the most informative coalitions,
and which also takes into account the fact that some coalitions are drawn
while others are not. The Kernel Shap function attributes a weight
$\omega_{S_{k}}$ to coalition $S_{k}$ such as:

$\omega_{S_{k}}=\frac{\left(q-1\right)}{\frac{q!}{|S_{k}|!\left(q-|S_{k}|\right)!}|S_{k}|\left(q-|S_{k}|\right)},$
(A5)

where $|S_{k}|$ is the number of features included in the coalition $S^{k}$.
To take into account these weights, Lundberg and Lee, (2017) propose to
estimate Equation A4 by Weighted Least Squares (WLS).222Note that
$S_{k}=\\{\mathbf{x}\\}$ and $S_{k}=\\{\emptyset\\}$ are not considered in the
set of randomly individual coalitions as they lead to infinite weights. The
$K$ coalitions are thus drawn from $2^{q}-2$ coalitions. See Lundberg and Lee,
(2017) for more details on the approximation approach of Shapley values. In
practice, a relatively large value of $K$ allows to obtain accurate
approximations of $\hat{\phi}_{j}$.

Finally, this approach can also be used to approximate individual
contributions to the estimated XPER values based on the following model:

$G_{k}\left(y_{i};\mathbf{x}_{i}\right)=\phi_{i,0}+\sum_{j=1}^{q}\phi_{i,j}z_{k,j},$
(A6)

where

$G_{k}\left(y_{i};\mathbf{x}_{i}\right)=\frac{1}{n}\sum_{u=1}^{n}G\left(y_{i};\mathbf{x}_{i}^{S_{k}},\mathbf{x}_{u}^{\tilde{S}_{k}};\hat{\delta}_{n}\right).$

Equation A6 is then estimated for each observation $i$ of the sample, and the
weight $w_{S_{k}}$ remains the same as defined previously.

##  Appendix D XPER vs. SHAP

In this section, we compare the XPER method with the Shapley additive
explanation (SHAP) method of Lundberg and Lee, (2017). As SHAP has now become
ubiquitous in machine learning, we believe it is important to clearly show the
added value of XPER over SHAP. In the latter, the contribution of a feature
$x_{j}$ to the predicted value $\hat{f}(\mathbf{x}_{i})$ for individual $i$,
denoted $\phi_{i,j}^{SHAP}$, is defined as:

$\displaystyle\phi_{i,j}^{SHAP}=\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}w_{S}\left[\mathbb{E}_{\mathbf{x}^{\overline{S}}}\left(\hat{f}(x_{i,j},\mathbf{x}_{i}^{S},\mathbf{x}^{\overline{S}})\right)-\mathbb{E}_{x_{j},\mathbf{x}^{\overline{S}}}\left(\hat{f}(x_{j},\mathbf{x}_{i}^{S},\mathbf{x}^{\overline{S}})\right)\right],$
(A7)
$\hat{f}(\mathbf{x}_{i})=\phi_{i,0}^{SHAP}+\sum_{j=1}^{q}\phi_{i,j}^{SHAP},$
(A8) $\phi_{i,0}^{SHAP}=\mathbb{E}\left(\hat{f}(\mathbf{x}_{i})\right).$ (A9)

######  Proposition 1.

SHAP is a particular case of XPER where the individual contribution to the
performance metric is equal to the predicted value of the model,
$G\left(y_{i};\mathbf{x}_{i};\delta_{0}\right)=\hat{f}(\mathbf{x}_{i})$.

See Appendix E.7 for the proof. As stated in Proposition 1, we can show that
SHAP is a particular case of XPER where the performance metric does not take
into account the target variable. However, as performance metrics generally
include at least the target variable to compare the predictions to their true
value, SHAP and individual XPER values will differ in most cases. However, one
may still wonder whether SHAP and individual XPER values provide the same
information. If this were to be true, we should find that for a given feature
$x_{j}$ (1) a positive (negative) SHAP value means that this feature has a
positive (negative) effect on the performance, and (2) a large SHAP value
implies that this feature has a strong impact on performance. Below, we show
that none of these statements are true. Indeed, in the former case, depending
on the value of the target variable, a positive XPER value is not necessarily
associated to a positive SHAP value. Intuitively, if the target variable is
positive, a variable can contribute to increase the predicted value of the
model ($\phi_{i,j}^{SHAP}>0$) which can reduce the spread between the
predicted value and the target value ($\phi_{i,j}>0$). However, if the target
variable is negative, a variable which contributes to decrease the predicted
value of the model ($\phi_{i,j}^{SHAP}<0$) can also reduce the prediction
error of the model ($\phi_{i,j}>0$). For instance, when the model only
includes one variable, if the performance metric is defined as
$G(y_{i};\hat{f}(\mathbf{x}_{i});\delta_{0})=-(y_{i}-\hat{f}(\mathbf{x}_{i}))^{2}$,
and if we assume that $\mathbb{E}(\hat{f}(x_{1}))=0$, we can show that:

$\phi_{i,1}=2\hat{\varepsilon}_{i}\phi_{i,1}^{SHAP}+\left(\phi_{i,1}^{SHAP}\right)^{2}+\mathbb{V}\left(\hat{f}(x_{i,1})\right),$
(A10)

where $\hat{\varepsilon}_{i}=y_{i}-\hat{f}(x_{i,1})$ is the prediction error
of the model for individual $i$. See Appendix E.6 for the proof. As we can
see, a positive XPER value can be associated to either a positive or negative
SHAP value. If the prediction error and the SHAP value are both positive
(negative), it means that this variable contributes to reduce the spread
between the predicted value and the target value, which results in a positive
XPER value. Therefore, a positive and a negative SHAP value can both lead to a
positive XPER value. Moreover, as the individual performance and the
predictions of the model does not have the same domain, except in the case
where $G\left(y_{i};\mathbf{x}_{i};\delta_{0}\right)=\hat{f}(\mathbf{x}_{i})$,
the magnitude of SHAP and XPER values can be very different from one to
another.

Although designed at the individual level, the SHAP method is also used to
assess the effect of the features at the global level by taking the mean of
the absolute SHAP values for each feature. We have:

$\phi_{j}^{SHAP}=\mathbb{E}\left(\left|\phi_{i,j}^{SHAP}\right|\right),$ (A11)

where $\phi_{j}^{SHAP}$ refers to the Mean Absolute SHAP values of feature
$j$. This raises again the question of how similar these values are from the
XPER values. One major difference between SHAP and XPER is that SHAP values
cannot be negative at the global level. This difference turns out to be
important as negative XPER values allow to red-flag features that deteriorate
the performance of the model on a given sample. For instance, this could help
to explain the origin of the overfitting of a model as mentioned in Section 2.
Note that according to Equation A8 and A9, it would not be appropriate to take
the mean of SHAP values without taking the absolute value, as we would end up
decomposing a zero:

$\displaystyle\mathbb{E}\left(\hat{f}(\mathbf{x}_{i})\right)$
$\displaystyle=\phi_{i,0}^{SHAP}+\sum_{j=1}^{q}\mathbb{E}\left(\phi_{i,j}^{SHAP}\right),$
$\displaystyle\Leftrightarrow\sum_{j=1}^{q}\mathbb{E}\left(\phi_{i,j}^{SHAP}\right)$
$\displaystyle=\mathbb{E}\left(\hat{f}(\mathbf{x}_{i})\right)-\mathbb{E}\left(\hat{f}(\mathbf{x}_{i})\right)=0.$
(A12)

Moreover, as both methods provide different results at the individual level,
there is no reason to think that they would be equivalent at the global level.
In the empirical section in Section 5, we confirm that the differences between
XPER and SHAP can be substantial in practice.

##  Appendix E Proofs

###  E.1 Proof of Equation 6

######  Lemma 1.

The sum of the weights across all coalitions $S$ is equal to 1,
$\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}=1$.

###### Proof.

According to the definition of $\omega_{S}$ in Equation 4 and knowing that
$\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})=\bigcup_{k=0}^{q-1}\mathcal{P}_{k}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$,
where $\mathcal{P}_{k}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$
refers to the collection of all subsets of size $k$ that can be formed from
the powerset $\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$,
we have:

$\displaystyle\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}$
$\displaystyle=\sum_{S\subseteq\bigcup_{k=0}^{q-1}\mathcal{P}_{k}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\frac{1}{q\times
C^{|S|}_{q-1}}.$

with $C^{|S|}_{q-1}$ the number of $|S|\mbox{-combinations}$ of a set with
$q-1$ elements. As
$\mathcal{P}_{k}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})\cap\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})=\emptyset,\leavevmode\nobreak\
\forall k\neq l$ we derive that:

$\displaystyle\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}$
$\displaystyle=\sum_{k=0}^{q-1}\sum_{S\subseteq\mathcal{P}_{k}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\frac{1}{q\times
C^{|S|}_{q-1}}.$

For each $k$,
$\mathcal{P}_{k}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$ is composed
of $C^{k}_{q-1}$ subsets of size $k$. As
$S\subseteq\mathcal{P}_{k}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\}$,
we know that $|S|=k$, which implies that $C^{|S|}_{q-1}=C^{k}_{q-1}$. Thus,

$\displaystyle\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}=\sum_{k=0}^{q-1}C^{k}_{q-1}\frac{1}{q\times
C^{k}_{q-1}}=\sum_{k=0}^{q-1}\frac{1}{q}=1.$

∎

######  Lemma 2.

Consider a linear regression model
$\hat{f}(\mathbf{x})=\sum_{j=1}^{q}\hat{\beta}_{j}x_{j}$ where we assume that
the DGP of the test sample
$S_{n}=\\{\mathbf{x}_{i},y_{i},\hat{f}(\mathbf{x}_{i})\\}^{n}_{i=1}$ satisfies
$\mathbb{E}\left(\mathbf{x}\right)=\mu_{q}\mbox{ and
}\mathbb{V}(\mathbf{x})=\Sigma$ a positive semi-definite matrix, with
$\Sigma_{k,j}=\sigma_{x_{k},x_{j}}$ the covariance between feature $x_{k}$ and
$x_{j}$. The individual contribution to the $R^{2}$, for a coalition
$S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$, can
be expressed as:

$\displaystyle G(y;\mathbf{x})$
$\displaystyle=1-\sigma^{-2}_{y}\left[y^{2}+\sum_{\begin{subarray}{c}l=1\\\
l\in
S\end{subarray}}^{q}x_{l}^{2}\hat{\beta}_{l}^{2}+\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}x_{k}^{2}\hat{\beta}_{k}^{2}+x_{j}^{2}\hat{\beta}_{j}^{2}+2\sum_{\begin{subarray}{c}1\leq
k<l\leq q\\\ k,l\in
S\end{subarray}}\hat{\beta}_{k}\hat{\beta}_{l}x_{k}x_{l}+2\sum_{\begin{subarray}{c}1\leq
k<l\leq q\\\
k,l\in\overline{S}\end{subarray}}\hat{\beta}_{k}\hat{\beta}_{l}x_{k}x_{l}\right]$
$\displaystyle-\sigma^{-2}_{y}\left[2\sum_{\begin{subarray}{c}k=1\\\ k\in
S\end{subarray}}^{q}\hat{\beta}_{k}\hat{\beta}_{j}x_{k}x_{j}+2\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}\hat{\beta}_{k}\hat{\beta}_{j}x_{k}x_{j}+2\sum_{\begin{subarray}{c}k=1\\\
k\in S\end{subarray}}^{q}\sum_{\begin{subarray}{c}l=1\\\
l\in\overline{S}\end{subarray}}^{q}\hat{\beta}_{k}\hat{\beta}_{l}x_{k}x_{l}\right]$
$\displaystyle-\sigma^{-2}_{y}\left[-2\sum_{\begin{subarray}{c}k=1\\\ k\in
S\end{subarray}}^{q}yx_{k}\hat{\beta}_{k}-2\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}yx_{k}\hat{\beta}_{k}-2yx_{j}\hat{\beta}_{j}\right].$

###### Proof.

Consider a linear regression model
$\hat{f}(\mathbf{x})=\sum_{j=1}^{q}\hat{\beta}_{j}x_{j}$ where we assume that
the DGP of the test sample
$S_{n}=\\{\mathbf{x}_{i},y_{i},\hat{f}(\mathbf{x}_{i})\\}^{n}_{i=1}$ satisfies
$\mathbb{E}\left(\mathbf{x}\right)=\mu_{q}\mbox{ and
}\mathbb{V}(\mathbf{x})=\Sigma$ a positive semi-definite matrix, with
$\Sigma_{k,j}=\sigma_{x_{k},x_{j}}$ the covariance between feature $x_{k}$ and
$x_{j}$.

Reminds that for a linear regression model
$\hat{f}(\mathbf{x})=\sum_{j=1}^{q}\hat{\beta}_{j}x_{j}$, the individual
contribution to the $R^{2}$ is defined as (see Equation 2):

$\displaystyle G(y;\mathbf{x})$
$\displaystyle=1-\frac{(y-\mathbf{x}\hat{\beta})^{2}}{\sigma^{2}_{y}}$
$\displaystyle=1-\sigma^{-2}_{y}\left[y^{2}+\sum_{k=1}^{q}x_{k}^{2}\hat{\beta}_{k}^{2}+2\sum_{1\leq
k<l\leq
q}\hat{\beta}_{k}\hat{\beta}_{l}x_{k}x_{l}-2\sum_{k=1}^{q}yx_{k}\hat{\beta}_{k}\right].$
(A13)

Considering a coalition
$S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$ of
features, the vector of features $\mathbf{x}$ is composed of three sub-
vectors: $\mathbf{x}^{S}$ the vector of features in the coalition $S$,
$\mathbf{x}^{\overline{S}}$ the vector of features apart from the coalition,
and $x_{j}$ the remaining feature of interest, such that
$\mathbf{x}=(\mathbf{x}^{S},\mathbf{x}^{\overline{S}},x_{j})$. Therefore, we
can rewrite Equation A13 as:

$\displaystyle G(y;\mathbf{x})$
$\displaystyle=1-\sigma^{-2}_{y}\left[y^{2}+\sum_{\begin{subarray}{c}l=1\\\
l\in
S\end{subarray}}^{q}x_{l}^{2}\hat{\beta}_{l}^{2}+\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}x_{k}^{2}\hat{\beta}_{k}^{2}+x_{j}^{2}\hat{\beta}_{j}^{2}+2\sum_{\begin{subarray}{c}1\leq
k<l\leq q\\\ k,l\in
S\end{subarray}}\hat{\beta}_{k}\hat{\beta}_{l}x_{k}x_{l}+2\sum_{\begin{subarray}{c}1\leq
k<l\leq q\\\
k,l\in\overline{S}\end{subarray}}\hat{\beta}_{k}\hat{\beta}_{l}x_{k}x_{l}\right]$
$\displaystyle-\sigma^{-2}_{y}\left[2\sum_{\begin{subarray}{c}k=1\\\ k\in
S\end{subarray}}^{q}\hat{\beta}_{k}\hat{\beta}_{j}x_{k}x_{j}+2\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}\hat{\beta}_{k}\hat{\beta}_{j}x_{k}x_{j}+2\sum_{\begin{subarray}{c}k=1\\\
k\in S\end{subarray}}^{q}\sum_{\begin{subarray}{c}l=1\\\
l\in\overline{S}\end{subarray}}^{q}\hat{\beta}_{k}\hat{\beta}_{l}x_{k}x_{l}\right]$
$\displaystyle-\sigma^{-2}_{y}\left[-2\sum_{\begin{subarray}{c}k=1\\\ k\in
S\end{subarray}}^{q}yx_{k}\hat{\beta}_{k}-2\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}yx_{k}\hat{\beta}_{k}-2yx_{j}\hat{\beta}_{j}\right].$

∎

######  Lemma 3.

For a given set of features $\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\}$,
we have:

$\displaystyle
2\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\left(\sum_{\begin{subarray}{c}k=1\\\
k\in S\end{subarray}}^{q}h(x_{k})+\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}g(x_{k})\right)=\sum_{\begin{subarray}{c}k=1\\\
k\neq j\end{subarray}}\left(h(x_{k})+g(x_{k})\right),$

with $h(.)\mbox{ and }g(.)$ some unknown linear or non-linear functions.

###### Proof.

Let consider a quantity $A$ defined as follows:

$\displaystyle
A=2\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\left(\sum_{\begin{subarray}{c}k=1\\\
k\in S\end{subarray}}^{q}h(x_{k})+\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}g(x_{k})\right),$ (A14)

with $h(.)\mbox{ and }g(.)$ some unknown functions. As
$\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})=\bigcup_{l=0}^{q-1}\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$,
where $\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$
refers to the collection of all subsets of size $l$ that can be formed from
the powerset $\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$,
we obtain from Equation A14:

$\displaystyle
A=2\sum_{S\subseteq\bigcup_{l=0}^{q-1}\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\left(\sum_{\begin{subarray}{c}k=1\\\
k\in S\end{subarray}}^{q}h(x_{k})+\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}g(x_{k})\right).$ (A15)

Note that for an even number of features $q$, we have:

$\displaystyle\bigcup_{l=0}^{q-1}\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})=\bigcup_{l=0}^{(q-2)/2}\mathcal{P}_{q-1-l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})\cup\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\}),$
(A16)

whereas for an odd number of features:

$\displaystyle\bigcup_{l=0}^{q-1}\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})=\bigcup_{l=0}^{(q-1)/2}\mathcal{P}_{q-1-l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})\cup\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\}).$
(A17)

Therefore, we now distinguish between the two cases to complete the proof.

When $q$ is even, if we plug the expression of
$\bigcup_{l=0}^{q-1}\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$
given in Equation A16 into Equation A15, then Equation A15 is equivalent to:

$\displaystyle
A=2\sum_{S\subseteq\bigcup_{l=0}^{(q-2)/2}\mathcal{P}_{q-1-l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})\cup\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\left(\sum_{\begin{subarray}{c}k=1\\\
k\in S\end{subarray}}^{q}h(x_{k})+\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}g(x_{k})\right).$ (A18)

As
$\bigcap_{l=0}^{(q-2)/2}\mathcal{P}_{q-1-l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})\cup\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})=\emptyset$,
we can rewrite Equation A18 as follows:

$\displaystyle
A=2\sum_{l=0}^{(q-2)/2}\sum_{S\subseteq\mathcal{P}_{q-1-l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})\cup\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\left(\sum_{\begin{subarray}{c}k=1\\\
k\in S\end{subarray}}^{q}h(x_{k})+\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}g(x_{k})\right).$ (A19)

Note that each coalition
$S\subseteq\mathcal{P}_{q-1-l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})\cup\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$
is either composed of $|S|=q-1-l$ or $|S|=l$ elements obtained from the set of
features $\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\}$ of size $q-1$.
Therefore, according to Lemma 4, all of the coalitions
$S\subseteq\mathcal{P}_{q-1-l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})\cup\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$
have the same weight. We refer to this weight as $\omega_{l}$. Thus, Equation
A19 simplifies to:

$\displaystyle
A=2\sum_{l=0}^{(q-2)/2}\omega_{l}\sum_{S\subseteq\mathcal{P}_{q-1-l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})\cup\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\left(\sum_{\begin{subarray}{c}k=1\\\
k\in S\end{subarray}}^{q}h(x_{k})+\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}g(x_{k})\right).$ (A20)

By construction, each feature
$x_{k}\in\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\}$ is included in half
of the coalitions
$S\subseteq\mathcal{P}_{q-1-l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})\cup\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$.
Note that the set
$\mathcal{P}_{q-1-l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})\cup\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$
is composed of $2\times C^{q-1-l}_{q-1}=2\times C^{l}_{q-1}$ coalitions.
Therefore, each feature $x_{k}$ is included in $C^{l}_{q-1}$ coalitions. Thus,
we can write that:

$\displaystyle\sum_{S\subseteq\mathcal{P}_{q-1-l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})\cup\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\sum_{\begin{subarray}{c}k=1\\\
k\in S\end{subarray}}^{q}h({x}_{k})=\sum_{\begin{subarray}{c}k=1\\\ k\neq
j\end{subarray}}^{q}C^{l}_{q-1}h({x}_{k}).$ (A21)

As each feature $x_{k}\in\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\}$ is
included in half of the coalitions
$S\subseteq\mathcal{P}_{q-1-l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})\cup\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$,
each of them is excluded from the coalitions $S$ half of the time. Therefore,
each feature $x_{k}$ is included in $C^{l}_{q-1}$ coalitions
$\overline{S}\subseteq\mathcal{P}_{q-1-l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})\cup\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$,
such that:

$\displaystyle\sum_{S\subseteq\mathcal{P}_{q-1-l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})\cup\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}g({x}_{k})=\sum_{\begin{subarray}{c}k=1\\\
k\neq j\end{subarray}}^{q}C^{l}_{q-1}g({x}_{k}).$ (A22)

As a consequence, Equation A19 simplifies to:

$\displaystyle
A=2\sum_{l=0}^{(q-2)/2}\omega_{l}C^{l}_{q-1}\left(\sum_{\begin{subarray}{c}k=1\\\
k\neq j\end{subarray}}\left(h(x_{k})+g(x_{k})\right)\right).$ (A23)

Moreover, according to the definition of $\omega_{l}$ in Equation 4, we then
have:

$\displaystyle
A=2\sum_{l=0}^{(q-2)/2}\frac{1}{q}\left(\sum_{\begin{subarray}{c}k=1\\\ k\neq
j\end{subarray}}\left(h(x_{k})+g(x_{k})\right)\right).$ (A24)

Finally, for an even number of features $q$, we obtain:

$\displaystyle
2\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\left(\sum_{\begin{subarray}{c}k=1\\\
k\in S\end{subarray}}^{q}h(x_{k})+\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}g(x_{k})\right)=\sum_{\begin{subarray}{c}k=1\\\
k\neq j\end{subarray}}\left(h(x_{k})+g(x_{k})\right).$ (A25)

Similarly, when $q$ is odd, we can write that:

$\displaystyle
A=2\sum_{l=0}^{(q-1)/2}\omega_{l}\sum_{S\subseteq\mathcal{P}_{q-1-l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})\cup\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\left(\sum_{\begin{subarray}{c}k=1\\\
k\in S\end{subarray}}^{q}h(x_{k})+\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}g(x_{k})\right).$ (A26)

However, when $l=(q-1)/2$, the set
$\mathcal{P}_{q-1-l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})\cup\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$
is only composed of $C^{q-1-l}_{q-1}=C^{l}_{q-1}$ coalitions as
$\mathcal{P}_{(q-1)/2}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})\cup\mathcal{P}_{(q-1)/2}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})=\mathcal{P}_{(q-1)/2}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$.
Therefore, for $l=(q-1)/2$, each feature $x_{k}\in
S\subseteq\mathcal{P}_{q-1-l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})\cup\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$
is included in $C^{l}_{q-1}/2$ coalitions. To take into account this
specificity, we rewrite Equation A26 as follows:

$\displaystyle A$
$\displaystyle=2\sum_{l=0}^{(q-3)/2}\omega_{l}\sum_{S\subseteq\mathcal{P}_{q-1-l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})\cup\mathcal{P}_{l}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\left(\sum_{\begin{subarray}{c}k=1\\\
k\in S\end{subarray}}^{q}h(x_{k})+\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}g(x_{k})\right)$ (A27)
$\displaystyle-2\omega_{(q-1)/2}\sum_{S\subseteq\mathcal{P}_{(q-1)/2}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\left(\sum_{\begin{subarray}{c}k=1\\\
k\in S\end{subarray}}^{q}h(x_{k})+\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}g(x_{k})\right),$ (A28)

which is equal to:

$\displaystyle A$
$\displaystyle=2\sum_{l=0}^{(q-3)/2}\omega_{l}C^{l}_{q-1}\left(\sum_{\begin{subarray}{c}k=1\\\
k\neq
j\end{subarray}}\left(h(x_{k})+g(x_{k})\right)\right)-2w_{|(q-1)/2|}\frac{C^{(q-1)/2}_{q-1}}{2}\left(\sum_{\begin{subarray}{c}k=1\\\
k\neq j\end{subarray}}\left(h(x_{k})+g(x_{k})\right)\right).$ (A29)

According to the definition of $\omega_{l}$ in Equation 4, we then have:

$\displaystyle A$
$\displaystyle=2\sum_{l=0}^{(q-3)/2}\frac{1}{q}\left(\sum_{\begin{subarray}{c}k=1\\\
k\neq
j\end{subarray}}\left(h(x_{k})+g(x_{k})\right)\right)-\left(\sum_{\begin{subarray}{c}k=1\\\
k\neq j\end{subarray}}\left(h(x_{k})+g(x_{k})\right)\right).$ (A30)

Finally, for an odd number of features $q$, we obtain:

$\displaystyle
A=2\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\left(\sum_{\begin{subarray}{c}k=1\\\
k\in S\end{subarray}}^{q}h(x_{k})+\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}g(x_{k})\right)=\sum_{\begin{subarray}{c}k=1\\\
k\neq j\end{subarray}}\left(h(x_{k})+g(x_{k})\right).$ (A31)

As we obtain the same expression for $A$ with an odd and an even number of
features $q$, we conclude that for all $q$:

$\displaystyle
2\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\left(\sum_{\begin{subarray}{c}k=1\\\
k\in S\end{subarray}}^{q}h(x_{k})+\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}g(x_{k})\right)=\sum_{\begin{subarray}{c}k=1\\\
k\neq j\end{subarray}}\left(h(x_{k})+g(x_{k})\right),$ (A32)

with $h(.)\mbox{ and }g(.)$ some unknown functions. ∎

######  Proposition 1.

Consider a linear regression model
$\hat{f}(\mathbf{x})=\sum_{j=1}^{q}\hat{\beta}_{j}x_{j}$ where we assume that
the DGP of the test sample
$S_{n}=\\{\mathbf{x}_{i},y_{i},\hat{f}(\mathbf{x}_{i})\\}^{n}_{i=1}$ satisfies
$\mathbb{E}\left(\mathbf{x}\right)=\mu_{q}\mbox{ and
}\mathbb{V}(\mathbf{x})=\Sigma$ a positive semi-definite matrix, with
$\Sigma_{k,j}=\sigma_{x_{k},x_{j}}$ the covariance between feature $x_{k}$ and
$x_{j}$. Then, the XPER contribution $\phi_{j}$ of feature $x_{j}$ to the
$R^{2}$ is:

$\phi_{j}=\frac{2\hat{\beta}_{j}\sigma_{y,x_{j}}}{\sigma^{2}_{y}},\qquad\quad\forall
j=1,...,q,$ (A33)

with $\sigma_{y}^{2}$ the variance of the target variable and
$\sigma_{y,x_{j}}$ its covariance with feature $x_{j}$.

###### Proof.

Consider a linear regression model
$\hat{f}(\mathbf{x})=\sum_{j=1}^{q}\hat{\beta}_{j}x_{j}$ where we assume that
the DGP of the test sample
$S_{n}=\\{\mathbf{x}_{i},y_{i},\hat{f}(\mathbf{x}_{i})\\}^{n}_{i=1}$ satisfies
$\mathbb{E}\left(\mathbf{x}\right)=\mu_{q}\mbox{ and
}\mathbb{V}(\mathbf{x})=\Sigma$ a positive semi-definite matrix, with
$\Sigma_{k,j}=\sigma_{x_{k},x_{j}}$ the covariance between feature $x_{k}$ and
$x_{j}$.

From Lemma 2 we can derive that, for a coalition
$S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$, we
have:

$\displaystyle\mathbb{E}_{y,\mathbf{x}^{S},x_{j}}\mathbb{E}_{\mathbf{x}^{\overline{S}}}\left(G(y;\mathbf{x};\delta_{0})\right)-\mathbb{E}_{y,\mathbf{x}^{S}}\mathbb{E}_{\mathbf{x}^{\overline{S}},x_{j}}\left(G(y;\mathbf{x};\delta_{0})\right)$
$\displaystyle=\sigma^{-2}_{y}\left[-2\sum_{\begin{subarray}{c}k=1\\\ k\in
S\end{subarray}}^{q}\hat{\beta}_{k}\hat{\beta}_{j}\sigma_{x_{k},x_{j}}+2\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}\hat{\beta}_{k}\hat{\beta}_{j}\sigma_{x_{k},x_{j}}\right]$
$\displaystyle+\sigma^{-2}_{y}\left[2\hat{\beta}_{j}\sigma_{y,x_{j}}\right],$
(A34)

with $\sigma_{y}^{2}$ the variance of the target variable and
$\sigma_{y,x_{j}}$ its covariance with feature $x_{j}$. Thus, according to
Definition 4 and Equation E.1, the XPER contribution $\phi_{j}$ to the $R^{2}$
is equal to:

$\displaystyle\phi_{j}$
$\displaystyle=\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\left(\mathbb{E}_{y,\mathbf{x}^{S},x_{j}}\mathbb{E}_{\mathbf{x}^{\overline{S}}}\left(G(y;\mathbf{x};\delta_{0})\right)-\mathbb{E}_{y,\mathbf{x}^{S}}\mathbb{E}_{\mathbf{x}^{\overline{S}},x_{j}}\left(G(y;\mathbf{x};\delta_{0})\right)\right)$
$\displaystyle=\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\left(\sigma^{-2}_{y}\left[-2\sum_{\begin{subarray}{c}k=1\\\
k\in
S\end{subarray}}^{q}\hat{\beta}_{k}\hat{\beta}_{j}\sigma_{x_{k},x_{j}}+2\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}\hat{\beta}_{k}\hat{\beta}_{j}\sigma_{x_{k},x_{j}}\right]\right)$
$\displaystyle+\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\sigma^{-2}_{y}\left[2\hat{\beta}_{j}\sigma_{y,x_{j}}\right].$
(A35)

As according to Lemma 1 we know that
$\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}=1$,
we obtain:

$\displaystyle\phi_{j}$
$\displaystyle=\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\left(\sigma^{-2}_{y}\left[-2\sum_{\begin{subarray}{c}k=1\\\
k\in
S\end{subarray}}^{q}\hat{\beta}_{k}\hat{\beta}_{j}\sigma_{x_{k},x_{j}}+2\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}\hat{\beta}_{k}\hat{\beta}_{j}\sigma_{x_{k},x_{j}}\right]\right)$
$\displaystyle+\sigma^{-2}_{y}\left[2\hat{\beta}_{j}\sigma_{y,x_{j}}\right].$
(A36)

According to Lemma 3, for
$h(x_{k})=-\hat{\beta}_{k}\hat{\beta}_{j}\sigma_{x_{k},x_{j}}$ and
$g(x_{k})=\hat{\beta}_{k}\hat{\beta}_{j}\sigma_{x_{k},x_{j}}$ we obtain:

$\displaystyle
2\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\left(\sum_{\begin{subarray}{c}k=1\\\
k\in
S\end{subarray}}^{q}-\hat{\beta}_{k}\hat{\beta}_{j}\sigma_{x_{k},x_{j}}+\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}^{q}\hat{\beta}_{k}\hat{\beta}_{j}\sigma_{x_{k},x_{j}}\right)=\sum_{\begin{subarray}{c}k=1\\\
k\neq
j\end{subarray}}\left(-\hat{\beta}_{k}\hat{\beta}_{j}\sigma_{x_{k},x_{j}}+\hat{\beta}_{k}\hat{\beta}_{j}\sigma_{x_{k},x_{j}}\right)=0$
(A37)

Therefore, from Equation A36 and Equation A37 we deduce that the XPER
contribution $\phi_{j}$ of feature $x_{j}$ to the $R^{2}$ is:

$\displaystyle\phi_{j}=\frac{2\hat{\beta}_{j}\sigma_{y,x_{j}}}{\sigma^{2}_{y}}.$

∎

###  E.2 Proof of Equation 8

######  Lemma 4.

The weight associated to a coalition $S$ built from a set of features of size
$q-1$ is equal to the weight of the coalition $\tilde{S}$, where
$|\tilde{S}|=q-1-|S|$, i.e., $\omega_{S}=\omega_{\tilde{S}}$.

###### Proof.

According to Equation 4, $\omega_{S}$ is defined as:

$\displaystyle\omega_{S}=\frac{1}{q\times
C^{|S|}_{q-1}}=\frac{1}{q\times\frac{(q-1)!}{|S|!(q-1-|S|)!}}.$

Similarly, as $|\tilde{S}|=q-1-|S|$, $\omega_{\tilde{S}}$ is expressed as:

$\displaystyle\omega_{\tilde{S}}=\frac{1}{q\times
C^{|\tilde{S}|}_{q-1}}=\frac{1}{q\times
C^{q-1-|S|}_{q-1}}=\frac{1}{q\times\frac{(q-1)!}{(q-1-|S|)!(q-1-(q-1-|S|))!}}=\frac{1}{q\times\frac{(q-1)!}{|S|!(q-1-|S|)!}}=\omega_{S}.$

∎

######  Proposition 2.

Consider a linear regression model
$\hat{f}(\mathbf{x})=\sum_{j=1}^{q}\hat{\beta}_{j}x_{j}$ where we assume that
the DGP of the test sample
$S_{n}=\\{\mathbf{x}_{i},y_{i},\hat{f}(\mathbf{x}_{i})\\}^{n}_{i=1}$ satisfies
$\mathbb{E}\left(\mathbf{x}\right)=\mu_{q}\mbox{ and
}\mathbb{V}(\mathbf{x})=\Sigma$ a positive semi-definite matrix, with
$\Sigma_{k,j}=\sigma_{x_{k},x_{j}}$ the covariance between feature $x_{k}$ and
$x_{j}$. The individual XPER contribution $\phi_{i,j}$ to the $R^{2}$ is:

$\phi_{i,j}=\sigma^{-2}_{y}\left[\hat{\beta}_{j}(x_{i,j}-\mathbb{E}(x_{j}))A-\hat{\beta}_{j}^{2}(x_{i,j}^{2}-\mathbb{E}(x_{j}^{2}))+\sum_{\begin{subarray}{c}k=1\\\
k\neq
j\end{subarray}}^{q}\hat{\beta}_{k}\hat{\beta}_{j}\sigma_{x_{k},x_{j}}\right].$

with $A=\left(2y_{i}-\sum_{\begin{subarray}{c}k=1\\\ k\neq
j\end{subarray}}^{q}\hat{\beta}_{k}(x_{i,k}+\mathbb{E}(x_{k}))\right)$,
$\sigma_{y}^{2}$ the variance of the target variable and
$\sigma_{x_{k},x_{j}}$ the covariance between the feature $x_{k}$ and $x_{j}$.

###### Proof.

Consider a linear regression model
$\hat{f}(\mathbf{x})=\sum_{j=1}^{q}\hat{\beta}_{j}x_{j}$ where we assume that
the DGP of the test sample
$S_{n}=\\{\mathbf{x}_{i},y_{i},\hat{f}(\mathbf{x}_{i})\\}^{n}_{i=1}$ satisfies
$\mathbb{E}\left(\mathbf{x}\right)=\mu_{q}\mbox{ and
}\mathbb{V}(\mathbf{x})=\Sigma$ a positive semi-definite matrix, with
$\Sigma_{k,j}=\sigma_{x_{k},x_{j}}$ the covariance between feature $x_{k}$ and
$x_{j}$.

From Lemma 2, we can derive that for a coalition
$S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})$ and
for an individual $i$ we have:

$\displaystyle\mathbf{E}_{\mathbf{x}^{\overline{S}}}\left(G(y_{i};\mathbf{x}_{i};\delta_{0})\right)-\mathbf{E}_{\mathbf{x}^{\overline{S}},x_{j}}\left(G(y_{i};\mathbf{x}_{i};\delta_{0})\right)$
$\displaystyle=\sigma^{-2}_{y}\left[2y_{i}\hat{\beta}_{j}(x_{i,j}-\mathbb{E}(x_{j}))-\hat{\beta}_{j}^{2}(x_{i,j}^{2}-\mathbb{E}(x_{j}^{2}))\right]$
$\displaystyle+\sigma^{-2}_{y}\left[2\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}\hat{\beta}_{k}\hat{\beta}_{j}\sigma_{x_{k},x_{j}}\right]$
$\displaystyle-\sigma^{-2}_{y}\left[2\hat{\beta}_{j}(x_{i,j}-\mathbb{E}(x_{j}))\left(\sum_{\begin{subarray}{c}k=1\\\
k\in S\end{subarray}}\hat{\beta}_{k}x_{i,k}+\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}\hat{\beta}_{k}\mathbb{E}(x_{k})\right)\right],$
(A38)

with $\sigma_{y}^{2}$ the variance of the target variable and
$\sigma_{x_{k},x_{j}}$ the covariance between the feature $x_{k}$ and $x_{j}$.
Thus, according to Definition 4 and Equation A38, the XPER contribution
$\phi_{i,j}$ to the $R^{2}$ is equal to:

$\displaystyle\phi_{i,j}$
$\displaystyle=\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\left(\mathbf{E}_{\mathbf{x}^{\overline{S}}}\left(G(y_{i};\mathbf{x}_{i};\delta_{0})\right)-\mathbf{E}_{\mathbf{x}^{\overline{S}},x_{j}}\left(G(y_{i};\mathbf{x}_{i};\delta_{0})\right)\right)$
$\displaystyle=\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\sigma^{-2}_{y}\left[2y_{i}\hat{\beta}_{j}(x_{i,j}-\mathbb{E}(x_{j}))-\hat{\beta}_{j}^{2}(x_{i,j}^{2}-\mathbb{E}(x_{j}^{2}))+2\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}\hat{\beta}_{k}\hat{\beta}_{j}\sigma_{x_{k},x_{j}}\right]$
$\displaystyle-\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\sigma^{-2}_{y}\left[2\hat{\beta}_{j}(x_{i,j}-\mathbb{E}(x_{j}))\left(\sum_{\begin{subarray}{c}k=1\\\
k\in S\end{subarray}}\hat{\beta}_{k}x_{i,k}+\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}\hat{\beta}_{k}\mathbb{E}(x_{k})\right)\right].$
(A39)

As according to Lemma 1 we know that
$\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}=1$,
we obtain:

$\displaystyle\phi_{i,j}$
$\displaystyle=\sigma^{-2}_{y}\left[2y_{i}\hat{\beta}_{j}(x_{i,j}-\mathbb{E}(x_{j}))-\hat{\beta}_{j}^{2}(x_{i,j}^{2}-\mathbb{E}(x_{j}^{2}))+2\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}\hat{\beta}_{k}\hat{\beta}_{j}\sigma_{x_{k},x_{j}}\right]$
$\displaystyle-\sigma^{-2}_{y}\left[\hat{\beta}_{j}(x_{i,j}-\mathbb{E}(x_{j}))\times
2\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\left(\sum_{\begin{subarray}{c}k=1\\\
k\in S\end{subarray}}\hat{\beta}_{k}x_{i,k}+\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}\hat{\beta}_{k}\mathbb{E}(x_{k})\right)\right].$
(A40)

According to Lemma 3, for $h(x_{k})=0$ and
$g(x_{k})=\hat{\beta}_{k}\hat{\beta}_{j}\sigma_{x_{k},x_{j}}$ we find that:

$\displaystyle
2\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}\hat{\beta}_{k}\hat{\beta}_{j}\sigma_{x_{k},x_{j}}=\sum_{\begin{subarray}{c}k=1\\\
k\neq j\end{subarray}}^{q}\hat{\beta}_{k}\hat{\beta}_{j}\sigma_{x_{k},x_{j}}$
(A41)

Similarly, for $h(x_{k})=\hat{\beta}_{k}x_{i,k}$ and
$g(x_{k})=\hat{\beta}_{k}\mathbb{E}(x_{k})$:

$\displaystyle
2\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\left(\sum_{\begin{subarray}{c}k=1\\\
k\in S\end{subarray}}\hat{\beta}_{k}x_{i,k}+\sum_{\begin{subarray}{c}k=1\\\
k\in\overline{S}\end{subarray}}\hat{\beta}_{k}\mathbb{E}(x_{i,k})\right)=\sum_{\begin{subarray}{c}k=1\\\
k\neq j\end{subarray}}^{q}\hat{\beta}_{k}(x_{i,k}+\mathbb{E}(x_{k}))$ (A42)

From Equation A40, A41, A42, we obtain:

$\displaystyle\phi_{i,j}$
$\displaystyle=\sigma^{-2}_{y}\left[2y_{i}\hat{\beta}_{j}(x_{i,j}-\mathbb{E}(x_{j}))-\hat{\beta}_{j}^{2}(x_{i,j}^{2}-\mathbb{E}(x_{j}^{2}))+\sum_{\begin{subarray}{c}k=1\\\
k\neq
j\end{subarray}}^{q}\hat{\beta}_{k}\hat{\beta}_{j}\sigma_{x_{k},x_{j}}\right]$
$\displaystyle-\sigma^{-2}_{y}\left[\hat{\beta}_{j}(x_{i,j}-\mathbb{E}(x_{j}))\sum_{\begin{subarray}{c}k=1\\\
k\neq j\end{subarray}}^{q}\hat{\beta}_{k}(x_{i,k}+\mathbb{E}(x_{k}))\right].$
(A43)

Finally, rearranging the terms we obtain:

$\phi_{i,j}=\sigma^{-2}_{y}\left[\hat{\beta}_{j}(x_{i,j}-\mathbb{E}(x_{j}))A-\hat{\beta}_{j}^{2}(x_{i,j}^{2}-\mathbb{E}(x_{j}^{2}))+\sum_{\begin{subarray}{c}k=1\\\
k\neq
j\end{subarray}}^{q}\hat{\beta}_{k}\hat{\beta}_{j}\sigma_{x_{k},x_{j}}\right].$

with $A=\left(2y_{i}-\sum_{\begin{subarray}{c}k=1\\\ k\neq
j\end{subarray}}^{q}\hat{\beta}_{k}(x_{i,k}+\mathbb{E}(x_{k}))\right)$,
$\sigma_{y}^{2}$ the variance of the target variable and
$\sigma_{x_{k},x_{j}}$ the covariance between the feature $x_{k}$ and $x_{j}$.
∎

###  E.3 Proof of the MSE example in Table 1(b)

######  Proposition 3.

Consider a linear regression model
$\hat{f}(\mathbf{x})=\sum_{j=1}^{q}\hat{\beta}_{j}x_{j}$, where
$\mathbb{E}\left(\mathbf{x}\right)=0_{q}\mbox{ and
}\mathbb{V}(\mathbf{x})=diag(\sigma^{2}_{x_{j}}),\forall j=1,...,q$, and
$\mathbb{E}(y)=0$. The contributions $\phi_{j}$ of features $x_{j}$ to the
(opposite of the) MSE satisfy the efficiency axiom such that:

$\underbrace{2\sum_{j=1}^{q}\hat{\beta}_{j}\sigma_{y,x_{j}}-\sum_{j=1}^{q}\hat{\beta}_{j}^{2}\sigma^{2}_{x_{j}}-\sigma_{y}^{2}}_{\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))}=\underbrace{-\sum_{j=1}^{q}\hat{\beta}_{j}^{2}\sigma^{2}_{x_{j}}-\sigma_{y}^{2}}_{\phi_{0}}+\sum_{j=1}^{q}\underbrace{2\hat{\beta}_{j}\sigma_{y,x_{j}}}_{\phi_{j}}.$
(A44)

###### Proof.

Consider a linear regression model
$\hat{f}(\mathbf{x})=\sum_{j=1}^{q}\hat{\beta}_{j}x_{j}$, where
$\mathbb{E}\left(\mathbf{x}\right)=0_{q}\mbox{ and
}\mathbb{V}(\mathbf{x})=diag(\sigma^{2}_{x_{j}}),\forall j=1,...,q$. As shown
in Table 1(b), the individual contribution to the (opposite) MSE is defined as
$G(y;\mathbf{x};\delta_{0}))=-(y-\mathbf{x}\hat{\beta})^{2}$, where
$\mathbf{x}\hat{\beta}=\hat{f}(\mathbf{x})$. Similarly,
$G(y;\mathbf{x};\delta_{0}))$ can be expressed as:

$\displaystyle
G(y;\mathbf{x};\delta_{0}))=-\left[y^{2}+\sum_{j=1}^{q}x_{j}^{2}\hat{\beta}_{j}^{2}+2\sum_{1\leq
j<l\leq
q}\hat{\beta}_{j}\hat{\beta}_{l}x_{j}x_{l}-2\sum_{j=1}^{q}yx_{j}\hat{\beta}_{j}\right].$
(A45)

To complete the proof, we first start by proving that the (opposite) MSE can
be written as:

$\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))=2\sum_{j=1}^{q}\hat{\beta}_{j}\sigma_{y,x_{j}}-\sum_{j=1}^{q}\hat{\beta}_{j}^{2}\sigma^{2}_{x_{j}}-\sigma_{y}^{2}.$
(A46)

Indeed, by taking the expected value of Equation A45 with respect to the joint
distribution of the target variable and the features, we obtain:

$\displaystyle\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$
$\displaystyle=-\mathbb{E}(y^{2})-\sum_{j=1}^{q}\hat{\beta}_{j}^{2}\mathbb{E}(x_{j}^{2})-2\sum_{1\leq
j<l\leq
q}\hat{\beta}_{j}\hat{\beta}_{l}\mathbb{E}(x_{j}x_{l})+2\sum_{j=1}^{q}\hat{\beta}_{j}\mathbb{E}(yx_{j}).$
(A47)

As $\mathbb{E}\left(\mathbf{x}\right)=0_{q}\mbox{ and
}\mathbb{V}(\mathbf{x})=diag(\sigma^{2}_{x_{j}}),\forall j=1,...,q$ we have:

$\displaystyle\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$
$\displaystyle=-\mathbb{E}(y^{2})-\sum_{j=1}^{q}\hat{\beta}_{j}^{2}\sigma^{2}_{x_{j}}+2\sum_{j=1}^{q}\hat{\beta}_{j}\sigma_{y,x_{j}}.$

As we assume that $\mathbb{E}(y)=0$, we have:

$\displaystyle\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$
$\displaystyle=-\sigma^{2}_{y}-\sum_{j=1}^{q}\hat{\beta}_{j}^{2}\sigma^{2}_{x_{j}}+2\sum_{j=1}^{q}\hat{\beta}_{j}\sigma_{y,x_{j}}.$
(A48)

Second, we prove that the benchmark value of the (opposite) MSE can be written
as:

$\phi_{0}=\mathbb{E}_{y}\mathbb{E}_{\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))=-\sigma^{2}_{y}-\sum_{j=1}^{q}\hat{\beta}_{j}^{2}\sigma^{2}_{x_{j}}.$
(A49)

From Equation A45, by taking the expected value with respect to the target
variable and the expected value with respect to the joint distribution of the
features we obtain:

$\displaystyle\phi_{0}=\mathbb{E}_{y}\mathbb{E}_{\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$
$\displaystyle=-\mathbb{E}(y^{2})-\sum_{j=1}^{q}\hat{\beta}_{j}^{2}\mathbb{E}(x_{j}^{2})-2\sum_{1\leq
j<l\leq
q}\hat{\beta}_{j}\hat{\beta}_{l}\mathbb{E}(x_{j}x_{l})+2\sum_{j=1}^{q}\hat{\beta}_{k}\mathbb{E}(y)\mathbb{E}(x_{j}).$
(A50)

As $\mathbb{E}\left(\mathbf{x}\right)=0_{q}\mbox{ and
}\mathbb{V}(\mathbf{x})=diag(\sigma^{2}_{x_{j}}),\forall j=1,...,q,$ we have:

$\displaystyle\phi_{0}=\mathbb{E}_{y}\mathbb{E}_{\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$
$\displaystyle=-\sigma^{2}_{y}-\sum_{j=1}^{q}\hat{\beta}_{j}^{2}\sigma^{2}_{x_{j}}.$
(A51)

Third, we show that the XPER value associated to the feature $x_{j}$ for the
(opposite) MSE can be expressed as:

$\phi_{j}=2\hat{\beta}_{j}\sigma_{y,x_{j}}.$ (A52)

Note that the individual contribution to the $R^{2}$, defined as
$G^{R^{2}}\left(y;\mathbf{x};\delta_{0}\right)=1-\sigma^{-2}_{y}(y-\mathbf{x}\hat{\beta})$
according to Equation 2, can be expressed as
$G^{R^{2}}\left(y;\mathbf{x};\delta_{0}\right)=1+\sigma^{-2}_{y}G\left(y;\mathbf{x};\delta_{0}\right)$,
with $G\left(y;\mathbf{x};\delta_{0}\right)$ the individual contribution to
the MSE. According to Definition 3, the XPER value associated to the feature
$x_{j}$ for the $R^{2}$:

$\displaystyle\phi_{j}^{R^{2}}$
$\displaystyle=\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\left[\mathbb{E}_{y,x_{j},\mathbf{x}^{S}}\mathbb{E}_{\mathbf{x}^{\overline{S}}}\left(1+\sigma^{-2}_{y}G\left(y;\mathbf{x};\delta_{0}\right)\right)-\mathbb{E}_{y,\mathbf{x}^{S}}\mathbb{E}_{\mathbf{x}^{x_{j},\overline{S}}}\left(1+\sigma^{-2}_{y}G\left(y;\mathbf{x};\delta_{0}\right)\right)\right]$
$\displaystyle\phi_{j}^{R^{2}}$
$\displaystyle=\sigma^{-2}_{y}\left[\sum_{S\subseteq\mathcal{P}(\\{\mathbf{x}\\}\setminus\left\\{x_{j}\right\\})}\omega_{S}\left[\mathbb{E}_{y,x_{j},\mathbf{x}^{S}}\mathbb{E}_{\mathbf{x}^{\overline{S}}}\left(G\left(y;\mathbf{x};\delta_{0}\right)\right)-\mathbb{E}_{y,\mathbf{x}^{S}}\mathbb{E}_{\mathbf{x}^{x_{j},\overline{S}}}\left(G\left(y;\mathbf{x};\delta_{0}\right)\right)\right]\right]$
$\displaystyle\phi_{j}^{R^{2}}$
$\displaystyle=\frac{\phi_{j}}{\sigma^{2}_{y}}.$ (A53)

Therefore, from Equation 6 and A53, we obtain:

$\displaystyle\phi_{j}=2\hat{\beta}_{j}\sigma_{y,x_{j}}.$ (A54)

Finally, from Equation A48, A51, and A54, we conclude that:

$\underbrace{2\sum_{j=1}^{q}\hat{\beta}_{j}\sigma_{y,x_{j}}-\sum_{j=1}^{q}\hat{\beta}_{j}^{2}\sigma^{2}_{x_{j}}-\sigma_{y}^{2}}_{\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))}=\underbrace{-\sum_{j=1}^{q}\hat{\beta}_{j}^{2}\sigma^{2}_{x_{j}}-\sigma_{y}^{2}}_{\phi_{0}}+\sum_{j=1}^{q}\underbrace{2\hat{\beta}_{j}\sigma_{y,x_{j}}}_{\phi_{j}}.$
(A55)

∎

###  E.4 Proof of the $R^{2}$ example in Table 1(b)

######  Proposition 4.

Consider a linear regression model
$\hat{y}=\hat{f}(\mathbf{x})=\sum_{j=1}^{q}\hat{\beta}_{j}x_{j}$, where
$\mathbb{E}\left(\mathbf{x}\right)=0_{q}\mbox{ and
}\mathbb{V}(\mathbf{x})=diag(\sigma^{2}_{x_{j}}),\forall j=1,...,q$, and the
features are uncorrelated to the residuals $\hat{\varepsilon}=y-\hat{y}$. The
contributions $\phi_{j}$ of features $x_{j}$ to the $R^{2}$ satisfy the
efficiency axiom such that:

$\displaystyle\underbrace{\frac{\sigma_{y,\hat{y}}}{\sigma^{2}_{y}}}_{\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))}=\underbrace{-\frac{\sigma_{y,\hat{y}}}{\sigma^{2}_{y}}}_{\phi_{0}}+\sum_{j=1}^{q}\underbrace{\frac{2\hat{\beta}_{j}\sigma_{y,x_{j}}}{\sigma^{2}_{y}}}_{\phi_{j}}.$
(A56)

###### Proof.

Consider a linear regression model
$\hat{f}(\mathbf{x})=\sum_{j=1}^{q}\hat{\beta}_{j}x_{j}$, where
$\mathbb{E}\left(\mathbf{x}\right)=0_{q}\mbox{ and
}\mathbb{V}(\mathbf{x})=diag(\sigma^{2}_{x_{j}}),\forall j=1,...,q$. As shown
in Equation 2, the individual contribution to the $R^{2}$ is defined as
$G(y;\mathbf{x};\delta_{0}))=1-\sigma^{2}_{y}(y-\mathbf{x}\hat{\beta})^{2}$,
where $\mathbf{x}\hat{\beta}=\hat{f}(\mathbf{x})$. Similarly,
$G(y;\mathbf{x};\delta_{0}))$ can be expressed as:

$\displaystyle G(y;\mathbf{x};\delta_{0}))$
$\displaystyle=1-\sigma^{-2}_{y}\left[y^{2}+\sum_{j=1}^{q}x_{j}^{2}\hat{\beta}_{j}^{2}+2\sum_{1\leq
j<l\leq
q}\hat{\beta}_{j}\hat{\beta}_{l}x_{j}x_{l}-2\sum_{j=1}^{q}yx_{j}\hat{\beta}_{j}\right].$
(A57)

To complete the proof, we first start by proving that the (opposite) MSE can
be written as:

$\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))=\frac{\sigma_{y,\hat{y}}}{\sigma^{2}_{y}}.$
(A58)

Indeed, by taking the expected value of Equation A57 with respect to the joint
distribution of the target variable and the features, we obtain:

$\displaystyle\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$
$\displaystyle=1-\sigma^{-2}_{y}\left[\mathbb{E}(y^{2})+\sum_{j=1}^{q}\hat{\beta}_{j}^{2}\mathbb{E}(x_{j}^{2})+2\sum_{1\leq
j<l\leq
q}\hat{\beta}_{j}\hat{\beta}_{l}\mathbb{E}(x_{j}x_{l})-2\sum_{j=1}^{q}\hat{\beta}_{j}\mathbb{E}(yx_{j})\right].$
(A59)

We assume that $\mathbb{E}\left(\mathbf{x}\right)=0_{q}\mbox{ and
}\mathbb{V}(\mathbf{x})=diag(\sigma^{2}_{x_{j}}),\forall j=1,...,q$.
Therefore, we have:

$\displaystyle\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$
$\displaystyle=1-\sigma^{-2}_{y}\left[\mathbb{E}(y^{2})+\sum_{j=1}^{q}\hat{\beta}_{j}^{2}\sigma^{2}_{x_{j}}-2\sum_{j=1}^{q}\hat{\beta}_{j}\sigma_{y,x_{j}}\right].$

In a linear model without intercept, we know that $\mathbb{E}(y)=0$,
therefore:

$\displaystyle\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$
$\displaystyle=-\sum_{j=1}^{q}\frac{\hat{\beta}_{j}^{2}\sigma^{2}_{x_{j}}}{\sigma^{2}_{y}}+\sum_{j=1}^{q}\frac{2\hat{\beta}_{j}\sigma_{y,x_{j}}}{\sigma^{2}_{y}}.$
(A60)

Moreover, in a linear regression model, the target variable can be expressed
as $y=\hat{y}+\hat{\varepsilon}$, with $\hat{y}$ the estimated model and
$\hat{\varepsilon}$ the residuals. As the features are uncorrelated from each
other, if we also assume that they are uncorrelated to the residuals we can
show that:

$\sigma_{y,\hat{y}}=\sum_{j=1}^{q}\hat{\beta}_{j}^{2}\sigma^{2}_{x_{j}}=\sum_{j=1}^{q}\hat{\beta}_{j}\sigma_{y,x_{j}},$
(A61)

with $\sigma_{y,\hat{y}}$ the covariance between the target variable and its
prediction. Therefore, the $R^{2}$ as expressed in Equation A60 is also
written as:

$\displaystyle\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$
$\displaystyle=\frac{\sigma_{y,\hat{y}}}{\sigma^{2}_{y}}.$ (A62)

Second, we prove that the benchmark value of $R^{2}$ can be written as:

$\phi_{0}=\mathbb{E}_{y}\mathbb{E}_{\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))=\frac{\sigma_{y,\hat{y}}}{\sigma^{2}_{y}}.$
(A63)

From Equation A45, by taking the expected value with respect to the target
variable and the expected value with respect to joint distribution of the
features, we obtain:

$\displaystyle\phi_{0}=\mathbb{E}_{y}\mathbb{E}_{\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$
$\displaystyle=1-\sigma^{-2}_{y}\left[\mathbb{E}(y^{2})+\sum_{j=1}^{q}\hat{\beta}_{j}^{2}\mathbb{E}(x_{j}^{2})+2\sum_{1\leq
j<l\leq q}\hat{\beta}_{j}\hat{\beta}_{l}\mathbb{E}(x_{j}x_{l})\right]$
$\displaystyle-\sigma^{-2}_{y}\left[-2\sum_{j=1}^{q}\hat{\beta}_{j}\mathbb{E}(y)\mathbb{E}(x_{j})\right].$
(A64)

We assume that $\mathbb{E}\left(\mathbf{x}\right)=0_{q}\mbox{ and
}\mathbb{V}(\mathbf{x})=diag(\sigma^{2}_{x_{j}}),\forall j=1,...,q$.
Therefore, we have:

$\displaystyle\phi_{0}=\mathbb{E}_{y}\mathbb{E}_{\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$
$\displaystyle=-\frac{\sum_{j=1}^{q}\hat{\beta}_{j}^{2}\sigma^{2}_{x_{j}}}{\sigma^{2}_{y}}.$
(A65)

From Equation A61, we can rewrite Equation A65 as:

$\displaystyle\phi_{0}=\mathbb{E}_{y}\mathbb{E}_{\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$
$\displaystyle=-\frac{\sigma_{y,\hat{y}}}{\sigma^{2}_{y}}.$ (A66)

Third, as stated in Proposition 1 the XPER value associated to the feature
$x_{j}$ for the $R^{2}$ can be expressed as:

$\phi_{j}=\frac{2\hat{\beta}_{j}\sigma_{y,x_{j}}}{\sigma^{2}_{y}}.$ (A67)

Finally, from Equation A62, A66, and A67, we conclude that:

$\displaystyle\underbrace{\frac{\sigma_{y,\hat{y}}}{\sigma^{2}_{y}}}_{\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))}=\underbrace{-\frac{\sigma_{y,\hat{y}}}{\sigma^{2}_{y}}}_{\phi_{0}}+\sum_{j=1}^{q}\underbrace{\frac{2\hat{\beta}_{j}\sigma_{y,x_{j}}}{\sigma^{2}_{y}}}_{\phi_{j}}.$
(A68)

∎

###  E.5 Proof of the accuracy example in Table 1(b)

######  Proposition 5.

Consider any binary classification model $\hat{f}(\mathbf{x})$, with
$\hat{P}(\mathbf{x})=\hat{\mathbb{P}}(y=1|\mathbf{x})$ the estimated
probability of belonging to class 1 (y=1). The contributions $\phi_{j}$ of
features $x_{j}$ to the $accuracy$ satisfy the efficiency axiom such that:

$\displaystyle\underbrace{2\sigma_{y,\hat{f}(\mathbf{x})}+2\mathbb{P}(y=1)\hat{P}(\mathbf{x})+1-\mathbb{P}(y=1)-\hat{P}(\mathbf{x})}_{\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))}$
$\displaystyle=\underbrace{2\mathbb{P}(y=1)\hat{P}(\mathbf{x})+1-\mathbb{P}(y=1)-\hat{P}(\mathbf{x})}_{\phi_{0}}$
$\displaystyle+\underbrace{2\sigma_{y,\hat{f}(\mathbf{x})}}_{\sum_{j=1}^{q}\phi_{j}},$
(A69)

with $\hat{P}(\mathbf{x})=\hat{\mathbb{P}}(y=1|\mathbf{x})$ and
$\sigma_{y,\hat{f}(\mathbf{x})}$ the covariance between the target variable
and the classification output.

###### Proof.

Consider any binary classification model $\hat{f}(\mathbf{x})$, with
$\hat{P}(\mathbf{x})=\hat{\mathbb{P}}(y=1|\mathbf{x})$ the estimated
probability of belonging to class 1 (y=1). As shown in Table 1(b), the
individual contribution to the accuracy is defined as:

$G(y;\mathbf{x};\delta_{0}))=y\hat{f}(\mathbf{x})+(1-y)(1-\hat{f}(\mathbf{x})).$
(A70)

To complete the proof, we first start by proving that the accuracy can be
written as:

$\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))=2\sigma_{y,\hat{f}(\mathbf{x})}+2\mathbb{P}(y=1)\hat{P}(\mathbf{x})+1-\mathbb{P}(y=1)-\hat{P}(\mathbf{x}).$
(A71)

Indeed, by taking the expected value of Equation A70 with respect to the joint
distribution of the target variable and the features, we obtain:

$\displaystyle\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$
$\displaystyle=2\mathbb{E}\left(y\hat{f}(\mathbf{x})\right)+1-\mathbb{E}(y)-\mathbb{E}\left(\hat{f}(\mathbf{x})\right)$
$\displaystyle=2\sigma_{y,\hat{f}(\mathbf{x})}+2\mathbb{E}(y)\mathbb{E}\left(\hat{f}(\mathbf{x})\right)+1-\mathbb{E}(y)-\mathbb{E}\left(\hat{f}(\mathbf{x})\right),$
(A72)

with $\sigma_{y,\hat{f}(\mathbf{x})}$ the covariance between the target
variable and the classification output. As the target variable is binary, we
know that $\mathbb{E}(y)=\mathbb{P}(y=1)$ and
$\mathbb{E}\left(\hat{f}(\mathbf{x})\right)=\hat{\mathbb{P}}(y=1|\mathbf{x})=\hat{P}(\mathbf{x})$.
Therefore, we obtain:

$\displaystyle\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$
$\displaystyle=2\sigma_{y,\hat{f}(\mathbf{x})}+2\mathbb{P}(y=1)\hat{P}(\mathbf{x})+1-\mathbb{P}(y=1)-\hat{P}(\mathbf{x}).$
(A73)

Second, from Equation A70, we can see that by taking the expected value with
respect to the target variable and the expected value with respect to joint
distribution of the features, the benchmark value of accuracy can be written
as:

$\displaystyle\phi_{0}=\mathbb{E}_{y}\mathbb{E}_{\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))$
$\displaystyle=2\mathbb{E}\left(y\right)\mathbb{E}\left(\hat{f}(\mathbf{x})\right)+1-\mathbb{E}(y)-\mathbb{E}\left(\hat{f}(\mathbf{x})\right)$
$\displaystyle=2\mathbb{P}(y=1)\hat{P}(\mathbf{x})+1-\mathbb{P}(y=1)-\hat{P}(\mathbf{x}).$
(A74)

Third, according to the axiom 1, we deduce that the sum of the XPER values
associated to the features $x_{j}$ for the accuracy is equal to:

$\displaystyle\sum_{j=1}^{q}\phi_{j}=2\sigma_{y,\hat{f}(\mathbf{x})}.$ (A75)

Finally, from Equation A73, A74 and A75, we conclude that:

$\displaystyle\underbrace{2\sigma_{y,\hat{f}(\mathbf{x})}+2\mathbb{P}(y=1)\hat{P}(\mathbf{x})+1-\mathbb{P}(y=1)-\hat{P}(\mathbf{x})}_{\mathbb{E}_{y,\mathbf{x}}(G(y;\mathbf{x};\delta_{0}))}$
$\displaystyle=\underbrace{2\mathbb{P}(y=1)\hat{P}(\mathbf{x})+1-\mathbb{P}(y=1)-\hat{P}(\mathbf{x})}_{\phi_{0}}$
$\displaystyle+\underbrace{2\sigma_{y,\hat{f}(\mathbf{x})}}_{\sum_{j=1}^{q}\phi_{j}}.$
(A76)

∎