Bbbk openbox NEWBbbk NEWopenbox Bbbk openbox NEWBbbk NEWopenbox

# Recourse under Model Multiplicity via Argumentative Ensembling
(Technical Report)

Junqi Jiang111Equal contribution., Antonio Rago11footnotemark: 1, Francesco
Leofante, Francesca Toni
Department of Computing, Imperial College London, UK
{junqi.jiang, a.rago, f.leofante<EMAIL_ADDRESS>
###### Abstract

Model Multiplicity (MM) arises when multiple, equally performing machine
learning models can be trained to solve the same prediction task. Recent
studies show that models obtained under MM may produce inconsistent
predictions for the same input. When this occurs, it becomes challenging to
provide counterfactual explanations (CEs), a common means for offering
recourse recommendations to individuals negatively affected by models’
predictions. In this paper, we formalise this problem, which we name
_recourse-aware ensembling_ , and identify several desirable properties which
methods for solving it should satisfy. We show that existing ensembling
methods, naturally extended in different ways to provide CEs, fail to satisfy
these properties. We then introduce _argumentative ensembling_ , deploying
computational argumentation to guarantee robustness of CEs to MM, while also
accommodating customisable user preferences. We show theoretically and
experimentally that argumentative ensembling satisfies properties which the
existing methods lack, and that the trade-offs are minimal wrt accuracy.

##  1 Introduction

Model Multiplicity (MM), also known as predictive multiplicity or the Rashomon
Effect, refers to a scenario where multiple, equally performing machine
learning (ML) models may be trained to solve a prediction task Black et al.
(2022b); Breiman (2001); Marx et al. (2020). While the existence of multiple
models that achieve the same accuracy is not a problem per se, recent
literature Marx et al. (2020); Black et al. (2022b) has drawn attention to the
fact that these models may differ greatly in their internals and might thus
produce inconsistent predictions when deployed. Consider the commonly used
scenario of a loan application, where an individual modelled by input
$\mathbf{x}$ with features unemployed status, 33 years of age and low credit
rating applies for a loan. Assume the bank has trained a set of ML models
$\mathcal{M}=\\{M_{1},M_{2},M_{3}\\}$ to predict whether the loan should be
granted or not. Even though each $M_{i}$ may exhibit good performance overall,
their internal differences may lead to conflicts, e.g. if
$M_{1}(\mathbf{x})=M_{2}(\mathbf{x})=0$ (i.e., reject), while
$M_{3}(\mathbf{x})=1$ (i.e., accept).

Ensembling techniques are commonly used to deal with MM scenarios Black et al.
(2022a, b). A standard such technique is naive ensembling Black et al.
(2022a), where the predictions of several models are aggregated to produce a
single outcome that reflects the opinion of the majority of models. For
instance, naive ensembling applied to our running example would result in a
rejection, as a majority of the models agree that the applicant is not
creditworthy. While ensembling methods have been shown to be effective in
practice, their application to consequential decision-making tasks raises some
important challenges.

Indeed, these methods tend to ignore the need to provide avenues for recourse
to users negatively impacted by the models’ outputs, which the ML literature
typically achieves via the provision of _counterfactual explanations_ (CEs)
for the predictions (see Guidotti (2022); Mishra et al. (2021) for recent
overviews). Dealing with MM while also taking CEs into account is non-trivial.
Indeed, standard algorithms designed to generate CEs for single models
typically fail to produce recourse recommendations that are valid across
equally performing models Pawelczyk et al. (2020); Leofante et al. (2023).
This phenomenon may have troubling implications as a lack of robustness may
lead users to question whether a CE is actually explaining the underlying
decision-making task and is not just an artefact of a (subset of) model(s).

Another challenge is that naive ensembling ignores other meta-evaluation
aspects of models like fairness, robustness, and interpretability, while it
has been shown that models under MM can demonstrate substantial differences in
these regards Coston et al. (2021); Rudin (2019); D’Amour et al. (2022) and
users may have strong preferences for some of these aspects, e.g. they may
prefer model fairness to accuracy.

In this paper, we frame the recourse problem under MM formally and propose
different approaches to accommodate recourse as well as user preferences.
After covering related work (§2) and the necessary preliminaries (§3), we make
the following contributions. In §4, we purpose a formalisation of the problem
and several desirable properties for ensembling methods for recourse under MM.
We also consider two natural extensions of naive ensembling to accommodate
generation of CEs and show that they may violate some of the properties we
define. We then propose argumentative ensembling in §5, a novel technique
rooted in computational argumentation (see Atkinson et al. (2017); Baroni et
al. (2018) for overviews). We show that it is able to solve the recourse
problem effectively while also naturally incorporating user preferences, a
problem in which the conflicts between different objectives quickly become
cognitively intractable. We present extensive experiments in §6, where we show
that our framework always provides valid recourse under MM, showing better
evaluations on a number of metrics and without compromising the individual
prediction accuracies. We also demonstrate the usefulness of specifying user
preferences in our framework. We then conclude in §7.

This report is an extended version of Jiang et al. (2024), including
supplementary material. The implementation is publicly available at
https://github.com/junqi-jiang/recourse_under_model_multiplicity.

##  2 Related Work

Model Multiplicity. MM has been shown to affect several dimensions of
trustworthy ML. In particular, among equally accurate models, there could be
different fairness characteristics Wick et al. (2019); Dutta et al. (2020);
Rodolfa et al. (2021); Coston et al. (2021), levels of interpretability Chen
et al. (2018); Rudin (2019); Semenova et al. (2022), model robustness
evaluations D’Amour et al. (2022) and even inconsistent explanations Dong and
Rudin (2019); Fisher et al. (2019); Mehrer et al. (2020); Black et al.
(2022c); Ley et al. (2023); Marx et al. (2023); Leofante et al. (2023).

Recent attempts have been made to address the MM problem. Black et al. (2022b)
suggested candidate models should be evaluated across additional dimensions
other than accuracy (e.g., robustness or fairness evaluation thresholds). They
provided a potential solution based on applying meta-rules to filter out
undesirable models, then use ensemble methods to aggregate them, or randomly
select one of them. Extending these ideas, the selective ensembling method of
Black et al. (2022a) embeds statistical testing into the ensembling process,
such that when the numbers of candidate models predicting an input to the top
two classes are close or equal, which could happen under naive ensembling
strategies like majority voting, an abstention signal can be flagged for
relevant stakeholders. Xin et al. (2022) looked at decision trees and proposed
an algorithm to enumerate all models obtainable under MM; Roth et al. (2023)
instead proposed a model reconciling procedure to resolve the conflicts
between two disagreeing models. Meanwhile, a number of works Marx et al.
(2020); Watson-Daniels et al. (2023); Hsu and Calmon (2022); Semenova et al.
(2022) propose metrics to quantify the extent of MM in prediction tasks.

Counterfactual Explanations and MM. A CE for a prediction of an input by an ML
model is typically defined as another data point that minimally modifies the
input such that the ML would yield a desired classification Tolomei et al.
(2017); Wachter et al. (2017). CEs are often advocated as a means to provide
algorithmic recourse for data subjects, and as such, modern algorithms have
been extended to enforce additional desirable properties such as actionability
Ustun et al. (2019), plausiblity Dhurandhar et al. (2018) and diversity
Mothilal et al. (2020). We refer to Guidotti (2022) for a recent overview.

More central to the MM problem, Pawelczyk et al. (2020) pointed out that CEs
on data manifold are more likely to be robust under MM than minimum-distance
CEs. Leofante et al. (2023) proposed an algorithm for a given ensemble of
feed-forward neural networks to compute robust CEs that are provably valid for
all models in the ensemble. Also related to MM is the line of work that
focuses on the robustness of CEs against model changes, e.g., parameter
updates due to model retraining on the same or slightly shifted data
distribution Upadhyay et al. (2021); Dutta et al. (2022); Black et al.
(2022c); Nguyen et al. (2022); Bui et al. (2022); Ferrario and Loi (2022);
Jiang et al. (2023b); Hamman et al. (2023); Jiang et al. (2023a). These
studies usually aim to generate CEs that are robust across retrained versions
of the same model, which is different from MM where several (potentially
structurally different) models are targeted together.

Computational Argumentation. Argumentation, as understood in AI, inspired by
the seminal Dung (1995), is a set of formalisms for dealing with conflicting
information, as demonstrated in numerous application areas, e.g. online debate
Cabrio and Villata (2013), scheduling Cyras et al. (2019) and judgmental
forecasting Irwin et al. (2022). There have also been a broad range of works
(see Cyras et al. (2021); Vassiliades et al. (2021) for overviews)
demonstrating argumentation’s capability for explaining the outputs of AI
models, e.g. neural networks Potyka (2021); Dejl et al. (2021), Bayesian
classifiers Timmer et al. (2015) and random forests Potyka et al. (2023). To
the best of our knowledge the only work in which argumentation has been
applied to MM specifically is Abchiche-Mimouni et al. (2023), where the
introduced method extracts and selects winning rules from an ensemble of
classifiers, leveraging on argumentation’s strengths in providing
explainability for this process. This differs from our method as we consider
pre-existing CEs computed for single models, rather than using rules as
explanations, and we consider preferences in the ensembling.

##  3 Preliminaries

Given a set of classification _labels_ $\mathcal{L}$, a _model_ is a mapping
$M:\mathbb{R}^{n}\rightarrow\mathcal{L}$; we denote that $M$ classifies an
_input_ $\mathbf{x}\in\mathbb{R}^{n}$ as $\ell$ iff $M(\mathbf{x})=\ell$. In
this paper, we focus on binary classification, i.e. $\mathcal{L}=\\{0,1\\}$,
for simplicity, though our approach is extensible to multi-class problems.
Then, a _counterfactual explanation_ (CE) for $\mathbf{x}$, given $M$, is some
$\mathbf{c}\in\mathbb{R}^{n}\setminus\\{\mathbf{x}\\}$ such that
$M(\mathbf{c})\neq M(\mathbf{x})$, which may be optimised by some distance
metric between the inputs.

A _bipolar argumentation framework_ (BAF) Cayrol and Lagasquie-Schiex (2005)
is a tuple $\langle\mathcal{X},\mathcal{A},\mathcal{S}\rangle$, where
$\mathcal{X}$ is a set of _arguments_ ,
$\mathcal{A}\subseteq\mathcal{X}\times\mathcal{X}$ is a directed relation of
_direct attack_ and $\mathcal{S}\subseteq\mathcal{X}\times\mathcal{X}$ is a
directed relation of _direct support_. Given a BAF
$\langle\mathcal{X},\mathcal{A},\mathcal{S}\rangle$, for any
$\alpha_{1}\in\mathcal{X}$, we refer to
$\mathcal{A}(\alpha_{1})=\\{\alpha_{2}|(\alpha_{2},\alpha_{1})\in\mathcal{A}\\}$
as _$\alpha_{1}$ ’s direct attackers_ and to
$\mathcal{S}(\alpha_{1})=\\{\alpha_{2}|(\alpha_{2},\alpha_{1})\in\mathcal{S}\\}$
as _$\alpha_{1}$ ’s direct supporters_. Then, an _indirect attack_ from
$\alpha_{x}$ on $\alpha_{y}$ is a sequence
$\alpha_{1},r_{1},\ldots,r_{n-1},\alpha_{n}$, where $n\geq 3$,
$\alpha_{1}=\alpha_{x}$, $\alpha_{n}=\alpha_{y}$, $r_{1}\in\mathcal{A}$ and
$r_{i}\in\mathcal{S}$ $\forall i\in\\{2,\ldots,n-1\\}$. Similarly, a
_supported attack_ from $\alpha_{x}$ on $\alpha_{y}$ is a sequence
$\alpha_{1},r_{1},\ldots,r_{n-1},\alpha_{n}$, where $n\geq 3$,
$\alpha_{1}=\alpha_{x}$, $\alpha_{n}=\alpha_{y}$, $r_{n-1}\in\mathcal{A}$ and
$r_{i}\in\mathcal{S}$ $\forall i\in\\{1,\ldots,n-2\\}$. Straightforwardly, a
supported attack on an argument implies there is a direct attack on an
argument.

We will also use notions of acceptability of sets of arguments in BAFs Cayrol
and Lagasquie-Schiex (2005). A set of arguments $X\subseteq\mathcal{X}$, also
called an _extension_ , is said to _set-attack_ any $\alpha_{1}\in\mathcal{X}$
iff there exists an attack (whether direct, indirect or supported) from some
$\alpha_{2}\in X$ on $\alpha_{1}$. Meanwhile, $X$ is said to _set-support_ any
$\alpha_{1}\in\mathcal{X}$ iff there exists a direct support from some
$\alpha_{2}\in X$ on $\alpha_{1}$.222In Cayrol and Lagasquie-Schiex (2005),
set-supports are defined via sequences of supports, which we do not use here.
Then, a set $X\subseteq\mathcal{X}$ _defends_ any $\alpha_{1}\in\mathcal{X}$
iff $\forall\alpha_{2}\in\mathcal{X}$, if $\\{\alpha_{2}\\}$ set-attacks
$\alpha_{1}$ then $\exists\alpha_{3}\in X$ such that $\\{\alpha_{3}\\}$ set-
attacks $\alpha_{2}$. Any set $X\subseteq\mathcal{X}$ is then said to be
_conflict-free_ iff $\nexists\alpha_{1},\alpha_{2}\in X$ such that
$\\{\alpha_{1}\\}$ set-attacks $\alpha_{2}$, and _safe_ iff
$\nexists\alpha_{3}\in\mathcal{X}$ such that $X$ set-attacks $\alpha_{3}$ and
either: $X$ set-supports $\alpha_{3}$; or $\alpha_{3}\in X$. (Note that a safe
set is guaranteed to be conflict-free.) The notion of a set
$X\subseteq\mathcal{X}$ being _d-admissible_ (based on _admissibility_ in Dung
(1995)) is then introduced, requiring that $X$ is conflict-free and defends
all of its elements. This notion is extended to account for safe sets: $X$ is
said to be _s-admissible_ iff it is safe and defends all of its elements.
(Note that an s-admissible set is guaranteed to be d-admissible.) It is also
extended for the notion of _external coherence_ Cayrol and Lagasquie-Schiex
(2005): $X$ is said to be _c-admissible_ iff it is conflict-free, closed for
$\mathcal{S}$ and defends all of its elements. Finally, a set
$X\subseteq\mathcal{X}$ is said to be _d-preferred_ (resp., _s-preferred_ ,
_c-preferred_) iff it is d-admissible (resp., s-admissible, c-preferred) and
maximal wrt set-inclusion.

##  4 Recourse under Model Multiplicity

As mentioned in §1, a common way to deal with MM in practice is to employ
ensembling techniques, where the prediction outcomes of several models are
aggregated to produce a single outcome. Aggregation can be performed in
different ways, as discussed in Black et al. (2022b, a). In the following, we
formalise a notion of naive ensembling, adopted in Black et al. (2022b), and
also known as majority voting, which will serve as a baseline for our
analysis.333It should be noted that in Black et al. (2022b), the case where
there is no majority is not discussed.

######  Definition 1.

Given an input $\mathbf{x}$, a set of models $\mathcal{M}$ and a set of labels
$\mathcal{L}$, we define the set of _top labels_
$\mathcal{L}_{max}\subseteq\mathcal{L}$ as:

$\mathcal{L}_{max}=argmax_{\ell_{i}\in\mathcal{L}}|\\{M_{j}\in\mathcal{M}|M_{j}(\mathbf{x})=\ell_{i}\\}|.$
Then we then use $\mathcal{M}^{n}(\mathbf{x})\in\mathcal{L}_{max}$ to denote
the aggregated classification by _naive ensembling_. In the cases where
$|\mathcal{L}_{max}|>1$, we select $\mathcal{M}^{n}(\mathbf{x})$ from
$\mathcal{L}_{max}$ randomly. With an abuse of notation, we also let
$\mathcal{M}^{n}=\\{M_{j}\in\mathcal{M}|M_{j}(\mathbf{x})=\mathcal{M}^{n}(\mathbf{x})\\}$
denote the set of models that agree on the aggregated classification.

Coming back to our loan example where $M_{1}$ and $M_{2}$ reject the loan
($M_{1}(\mathbf{x})=M_{2}(\mathbf{x})=0$) while $M_{3}$ accepts it
($M_{3}(\mathbf{x})=1$), we obtain $\mathcal{M}^{n}(\mathbf{x})=0$ and
$\mathcal{M}^{n}=\\{M_{1},M_{2}\\}$. Naive ensembling is known to be an
effective strategy to mediate conflicts between models and is routinely used
in practical applications. However, in this paper, we take an additional step
and aim to generate CEs providing recourse for a user that has been impacted
by $\mathcal{M}^{n}(\mathbf{x})$. Recent work by Leofante et al. (2023) has
shown that standard algorithms designed to generate CEs for single models
typically fail to produce recourse recommendations that are robust across
$\mathcal{M}^{n}$. One natural idea to address this would be to extend naive
ensembling to account for CEs. Next, we formalise this idea in terms of
several properties that are important in this setting. We then discuss two
concrete methods extending naive ensembling and analyse them in terms of the
properties we define.

###  4.1 Problem Statement and Desirable Properties

Consider a non-empty set of models
$\mathcal{M}\\!\\!=\\!\\!\\{M_{1},\ldots,M_{m}\\!\\}$ and, for an input
$\mathbf{x}$, assume a set
$\mathcal{C}(\mathbf{x})\\!=\\!\\{\mathbf{c}_{1},\ldots,\mathbf{c}_{m}\\}$
where each $\mathbf{c}_{i}\in\mathcal{C}(\mathbf{x})$ is a CE for
$\mathbf{x}$, given $M_{i}$. In the rest of the paper, wherever it is clear
that we refer to a given $\mathbf{x}$, we use $\mathcal{C}$ and omit its
dependency on $\mathbf{x}$ for readability. Our aim is to solve the problem
outlined below.

Problem: Recourse-Aware Ensembling (RAE)
Input: input $\mathbf{x}$, set $\mathcal{M}$ of models, set $\mathcal{C}$ of
CEs Output: “optimal” set $S\subseteq\mathcal{M}\cup\mathcal{C}$ of models and
CEs.

To characterise optimality, we propose a number of desirable properties for
the outputs of ensembling methods. We refer to these outputs as _solutions_
for RAE. The most basic requirement requires that both models and CEs in the
output are non-empty.

######  Definition 2.

An ensembling method satisfies _non-emptiness_ iff for any given input
$\mathbf{x}$, set $\mathcal{M}$ of models and set $\mathcal{C}$ of CEs, any
solution $S\subseteq\mathcal{M}\cup\mathcal{C}$ is such that
$S\cap\mathcal{M}\neq\emptyset$ and $S\cap\mathcal{C}\neq\emptyset$.

Specifically, non-emptiness ensures that the RAE method returns some models
and some CEs. We then look to ensure that the ensembling method returns a non-
trivial set of models, as formalised by the next property.

######  Definition 3.

An ensembling method satisfies _non-triviality_ iff for any given input
$\mathbf{x}$, set $\mathcal{M}$ of models and set $\mathcal{C}$ of CEs, any
solution $S\subseteq\mathcal{M}\cup\mathcal{C}$ is such that
$|S\cap\mathcal{M}|>1$.

Clearly, the returned models should not disagree amongst themselves on the
classification, which leads to our next requirement.

######  Definition 4.

An ensembling method satisfies _model agreement_ iff for any given input
$\mathbf{x}$, set $\mathcal{M}$ of models and set $\mathcal{C}$ of CEs, any
solution $S\subseteq\mathcal{M}\cup\mathcal{C}$ is such that $\forall
M_{i},M_{j}\in S\cap\mathcal{M}$, $M_{i}(\mathbf{x})=M_{j}(\mathbf{x})$.

The next property, which itself requires model agreement to be satisfied,
checks whether the set of returned models is among the largest of the agreeing
sets of models, a motivating property of naive ensembling.

######  Definition 5.

An ensembling method satisfies _majority vote_ iff it satisfies model
agreement and for any given input $\mathbf{x}$, set $\mathcal{M}$ of models,
set $\mathcal{C}$ of CEs and set $\mathcal{L}$ of labels, any solution
$S\subseteq\mathcal{M}\cup\mathcal{C}$ is such that, letting
$\ell_{i}=M_{j}(\mathbf{x})$ $\forall M_{j}\in S\cap\mathcal{M}$,
$\nexists\ell_{k}\in\mathcal{L}\setminus\\{\ell_{i}\\}$ such that
$|\\{M_{l}\in\mathcal{M}|M_{l}(\mathbf{x})=\ell_{k}\\}|>|\\{M_{l}\in\mathcal{M}|M_{l}(\mathbf{x})=\ell_{i}\\}|$.

Next, we consider the robustness of recourse. Previous work Leofante et al.
(2023) considered a very conservative notion of robustness whereby
explanations are required to be valid for all models in $\mathcal{M}$. While
this might be desirable in some cases, we highlight that satisfying this
property may not always be feasible in practice. We therefore propose a
relaxed notion of robustness, which requires that CEs are valid only for the
models that support them.

######  Definition 6.

An ensembling method satisfies _counterfactual validity_ iff for any given
input $\mathbf{x}$, set $\mathcal{M}$ of models and set $\mathcal{C}$ of CEs,
any solution $S\subseteq\mathcal{M}\cup\mathcal{C}$ is such that $\forall
M_{i}\in S\cap\mathcal{M}$ and $\forall\mathbf{c}_{j}\in S\cap\mathcal{C}$,
$M_{i}(\mathbf{c}_{j})\neq M_{i}(\mathbf{x})$.

While counterfactual validity is a fundamental requirement for any sound
ensembling method, one also needs to ensure that the solutions it generates
are coherent, as formalised below.

######  Definition 7.

An ensembling method satisfies _counterfactual coherence_ iff for any given
input $\mathbf{x}$, set $\mathcal{M}$ of models and set $\mathcal{C}$ of CEs,
any solution $S\subseteq\mathcal{M}\cup\mathcal{C}$, where
$\mathcal{M}=\\{M_{1},\ldots,M_{m}\\}$ and
$\mathcal{C}=\\{\mathbf{c}_{1},\ldots,\mathbf{c}_{m}\\}$, is such that
$\forall i\in\\{1,\ldots,m\\}$, $M_{i}\in S$ iff $\mathbf{c}_{i}\in S$.

Intuitively coherence requires that (i) a CE is returned only if it is
supported by a model and (ii) when the CE is chosen, then its corresponding
model must be part of the support. This ultimately guarantees strong
justification as to why a given recourse is suggested since selected models
and their reasoning (represented by their CEs) are assessed in tandem.

The properties defined above may not be all satisfiable at the same time in
practice, therefore heuristic approaches might be needed. Next, we discuss two
such heuristics to extend naive ensembling towards solving RAE.

###  4.2 Extending Naive Ensembling for Recourse

We now present two strategies that leverage naive ensembling to solve RAE. In
particular, we use the relationship between models in the ensemble
$\mathcal{M}^{n}$ and their corresponding CEs as follows.

######  Definition 8.

Consider an input $\mathbf{x}$, a set $\mathcal{M}$ of models and a set
$\mathcal{C}$ of CEs. Let $\mathcal{M}^{n}\subseteq\mathcal{M}$ be the set of
models obtained by naive ensembling. We define the set of _naive CEs_ as:

$\mathcal{C}^{n}=\\{\mathbf{c}_{i}\in\mathcal{C}\mid
M_{i}\in\mathcal{M}^{n}\\};$
and the set of _valid CEs_ as:

$\mathcal{C}^{v}=\\{\mathbf{c}_{i}\in\mathcal{C}\mid
M_{i}\in\mathcal{M}^{n}\wedge\forall
M_{j}\in\mathcal{M}^{n},M_{j}(\mathbf{c}_{i})\neq M_{j}(\mathbf{x})\\}.$
Then, two possible solutions to RAE are
$S^{n}=\mathcal{M}^{n}\cup\mathcal{C}^{n}$, named _augmented ensembling_ , or
$S^{v}=\mathcal{M}^{n}\cup\mathcal{C}^{v}$, named _robust ensembling_.

Intuitively, augmented ensembling suggests taking all the CEs in $\mathcal{C}$
that correspond to the models in $\mathcal{M}^{n}$. Meanwhile, robust
ensembling extends augmented ensembling by enforcing the additional constraint
that CEs are selected only if they are valid for all models in
$\mathcal{M}^{n}$. We now provide an illustrative example to clarify the
results produced by the two strategies.

######  Example 1.

Consider $\mathcal{M}=\\{M_{1},M_{2},M_{3},M_{4},M_{5}\\}$ and an input
$\mathbf{x}$ such that
$M_{1}(\mathbf{x})=M_{2}(\mathbf{x})=M_{3}(\mathbf{x})=0$ and
$M_{4}(\mathbf{x})=M_{5}(\mathbf{x})=1$. Let
$\mathcal{C}=\\{\mathbf{c}_{1},\mathbf{c}_{2},\mathbf{c}_{3},\mathbf{c}_{4},\mathbf{c}_{5}\\}$
be the set of CEs generated for $\mathbf{x}$, i.e.
$M_{1}(\mathbf{c}_{1})=M_{2}(\mathbf{c}_{2})=M_{3}(\mathbf{c}_{3})=1$, while
$M_{4}(\mathbf{c}_{4})=M_{5}(\mathbf{c}_{5})=0$. Applying naive ensembling to
$\mathcal{M}$ yields $\mathcal{M}^{n}=\\{M_{1},M_{2},M_{3}\\}$ and
$\mathcal{M}^{n}(\mathbf{x})=0$. Then, the set of naive CEs is
$\mathcal{C}^{n}=\\{\mathbf{c}_{1},\mathbf{c}_{2},\mathbf{c}_{3}\\}$, and thus
augmented ensembling gives
$S^{n}=\\{M_{1},M_{2},M_{3},\mathbf{c}_{1},\mathbf{c}_{2},\mathbf{c}_{3}\\}$.
Now, assume that $\mathbf{c}_{1}$ is invalid for $M_{2}$ (i.e.
$M_{2}(\mathbf{c}_{1})=0$), $\mathbf{c}_{2}$ is invalid for $M_{1}$,
$\mathbf{c}_{3}$ is invalid for $M_{2}$, and all three CEs are otherwise valid
for all models in $\mathcal{M}^{n}$. Then, then the set of valid CEs is empty,
i.e. $\mathcal{C}^{v}=\emptyset$, and thus robust ensembling gives
$S^{v}=\\{M_{1},M_{2},M_{3}\\}$.

This example shows that both methods host major drawbacks: augmented
ensembling may produce CEs which are invalid and thus it is not robust to MM,
while robust ensembling is prone to returning no CEs. We now present a
theoretical analysis to assess the extent to which augmented and robust
ensembling are able to satisfy the properties given in Definitions 2 to 7.

######  Theorem 1.

Augmented ensembling satisfies non-emptiness, model agreement, majority vote
and counterfactual coherence. It satisfies non-triviality if
$|\mathcal{M}|\\!>\\!2$. It does not satisfy counterfactual validity.

###### Proof.

By Def. 1, it can be seen by inspection that
$|\mathcal{M}^{n}|\\!\\!>\\!\\!0$. Thus, non-emptiness is satisfied. Again by
inspection of the same definition, $\forall M_{i},M_{j}\in\mathcal{M}^{n}$,
$M_{i}(\mathbf{x})=M_{j}(\mathbf{x})$. Thus, model agreement is satisfied. We
can also see that
$\nexists\ell_{i}\in\mathcal{L}\setminus\\{\mathcal{M}^{n}(\mathbf{x})\\}$
such that
$|\\{M_{j}\in\mathcal{M}|M_{j}(\mathbf{x})=\ell_{i}\\}|>|\mathcal{M}^{n}|$.
Thus, majority vote is satisfied. By Defs. 1 and 8, it can be seen that
$\forall M_{i}\in\mathcal{M}$ and $\forall\mathbf{c}_{i}\in\mathcal{C}$,
$M_{i}\in S$ iff $\mathbf{c}_{i}\in S$. Thus, counterfactual coherence is
satisfied.

Example 1 shows that counterfactual validity is not satisfied by providing a
counterexample.

Finally, the partial satisfaction of non-triviality can be proven by
contradiction: assume $|\mathcal{M}|\\!=\\!n$, $n\\!>\\!2$ but
$|\mathcal{M}^{n}|\\!=\\!1$. By Def. 1, $\mathcal{M}^{n}$ is the largest
subset of $\mathcal{M}$ containing models with the same classification
outcome. However, for binary classification, this implies that the remaining
$n-1$ all agree on the opposite classification, i.e.
$|\mathcal{M}\\!\setminus\\!\mathcal{M}^{n}|\\!>\\!|\mathcal{M}^{n}|$, which
leads to a contradiction. ∎

######  Theorem 2.

Robust ensembling satisfies model agreement, majority vote and counterfactual
validity. It satisfies non-triviality if $|\mathcal{M}|>2$. It does not
satisfy non-emptiness or counterfactual coherence.

###### Proof.

The proofs for model agreement, majority vote and non-triviality are analogous
to those in Theorem 1 and so are omitted.

It can be seen by inspection of Definition 8 that
$\forall\\!M_{i}\\!\in\\!\mathcal{M}^{n}$ and
$\forall\mathbf{c}_{j}\\!\in\\!\mathcal{C}^{v}$,
$M_{i}(\mathbf{c}_{j})\\!\neq\\!M_{i}(\mathbf{x})$. Thus, counterfactual
validity is satisfied.

Example 1 shows that non-emptiness and counterfactual coherence are not
satisfied by providing a counterexample. ∎

These results, summarised in Table 1, demonstrate that there may exist cases
in which both augmented and robust ensembling fail to solve RAE
satisfactorily. This has strong implications on the quality of the results
obtained in practice, as we will show experimentally in §6. Further, these
methods provide no way to take into account users’ preferences over the
models. As previously mentioned (see §1 and §2), there could be different
characteristics among the models in $\mathcal{M}$ in terms of meta-evaluation
aspects, e.g. a model’s fairness, robustness, and simplicity. Depending on the
task, a model’s fairness might be specified as being more important than its
robustness or simplicity. In such cases, it would be desirable to have a
principled way to rank models according to the preference specification, as
promoted in Black et al. (2022b). The combination of these deficiencies
motivates the need for a richer ensembling framework to solve RAE while
incorporating user preferences, given next.

| $\mathcal{M}^{n}\cup\mathcal{C}^{n}$ | $\mathcal{M}^{n}\cup\mathcal{C}^{v}$ | $\mathcal{M}^{a}\cup\mathcal{C}^{a}$
---|---|---|---
non-emptiness | $\checkmark$ |  | $\checkmark$
non-triviality | $\checkmark^{*}$ | $\checkmark^{*}$ | $\checkmark^{*}$
model agreement | $\checkmark$ | $\checkmark$ | $\checkmark$
majority vote | $\checkmark$ | $\checkmark$ |
counterfactual validity |  | $\checkmark$ | $\checkmark$
counterfactual coherence | $\checkmark$ |  | $\checkmark$

Table 1: Augmented ($\mathcal{M}^{n}\cup\mathcal{C}^{n}$) and robust
($\mathcal{M}^{n}\cup\mathcal{C}^{v}$) ensembling, as well as our
argumentative approach ($\mathcal{M}^{a}\cup\mathcal{C}^{a}$, defined in §5),
assessed against the desirable properties defined in §4.1. Satisfaction of a
property is shown by $\checkmark$, while partial satisfaction under given
conditions is shown by $\checkmark^{*}$.

##  5 Argumentative Ensembling

We now present our method for ensembling models and CEs which inherently
supports specifying preferences over models; then we undertake a theoretical
analysis of its properties.

###  5.1 Definition

We start by formalising ways to incorporate the aforementioned preferences
over models. These preferences could be obtained by any information, e.g.
meta-rules over models as suggested in Black et al. (2022b), but we will
assume that they are extracted wrt properties of the models, e.g. their
accuracy or (a metric representing) their simplicity.

######  Definition 9.

Given a set $\mathcal{M}$ of models, a set $\mathcal{P}$ of _properties_ is
such that $\forall\pi\in\mathcal{P}$, $\pi:\mathcal{M}\rightarrow\mathbb{R}$
is a total function.

We then define a preference over the properties such that users can impose a
ranking of priority over them. Here and onwards, for simplicity we use total
orders, denoted $\preceq$, over any set $S$ such that, as usual, for any
$s_{i},s_{j}\in S$, $s_{i}\prec s_{j}$ iff $s_{i}\preceq s_{j}$ and
$s_{i}\not{\succeq}s_{j}$. Also as usual, we say that $s_{i}\simeq s_{j}$ iff
$s_{i}\preceq s_{j}$ and $s_{i}\succeq s_{j}$.

######  Definition 10.

Given a set $\mathcal{P}$ of properties, a _property preference_
$\preceq_{\mathcal{P}}$ is a total order over $\mathcal{P}$.

Model preferences can be defined using property preferences. In the following
example, we define one way for doing so.

######  Example 2.

Consider the same set of models as in Example 1 and a set of properties
$\mathcal{P}\\!\\!=\\!\\!\\{\pi_{1},\pi_{2}\\}$ where $\pi_{1},\pi_{2}$
represent model accuracy and simplicity, resp., with a property preference
$\preceq_{\mathcal{P}}$ such that
$\pi_{1}\\!\\!\succ_{\mathcal{P}}\\!\\!\pi_{2}$ and values for the properties
as follows.

| $M_{1}$ | $M_{2}$ | $M_{3}$ | $M_{4}$ | $M_{5}$
---|---|---|---|---|---
$\pi_{1}$ (accuracy) | 0.85 | 0.87 | 0.86 | 0.86 | 0.87
$\pi_{2}$ (simplicity) | 0 | 0.75 | 1 | 0.5 | 0.75

A simple model preference $\preceq_{\mathcal{M}}$ over $\mathcal{M}$ may be
such that, for any $M_{i},M_{j}\in\mathcal{M}$,
$M_{i}\succ_{\mathcal{M}}M_{j}$ iff: (i) $\pi_{1}(M_{i})>\pi_{1}(M_{j})$; or
(ii) $\pi_{1}(M_{i})=\pi_{1}(M_{j})$ and $\pi_{2}(M_{i})>\pi_{2}(M_{j})$. This
$\preceq_{\mathcal{M}}$ is a total order over $\mathcal{M}$ and results in
$M_{2}\simeq_{\mathcal{M}}M_{5}\succ_{\mathcal{M}}M_{3}\succ_{\mathcal{M}}M_{4}\succ_{\mathcal{M}}M_{1}$.

Other ways to define preferences over models from preferences over properties
of models can be defined, e.g. based on more sophisticated notions of
dominance. We will assume some given notion of total preference over models,
as follows, ignoring how it is obtained.

######  Definition 11.

Given a set $\mathcal{M}$ of models, a _model preference_
$\preceq_{\mathcal{M}}$ over $\mathcal{M}$ is a total order over
$\mathcal{M}$.

How can we incorporate these model preferences into the ensembling, while
still satisfying the properties defined in §4.1? To tackle this problem, we
use bipolar argumentation as follows.444We considered using abstract AFs Dung
(1995), but we found that bipolar AFs are more suitable, given that models and
CEs can be naturally seen as supporting one another.

######  Definition 12.

The _BAF corresponding to_ input $\mathbf{x}$, set $\mathcal{M}$ of models,
set $\mathcal{C}$ of CEs and model preference $\preceq_{\mathcal{M}}\\!$ is
$\langle\mathcal{X},\mathcal{A},\mathcal{S}\rangle$ with:

  * •

$\mathcal{X}=\mathcal{M}\cup\mathcal{C}$;

  * •

$\mathcal{A}\subseteq(\mathcal{M}\times\mathcal{M})\cup(\mathcal{M}\times\mathcal{C})\cup(\mathcal{C}\times\mathcal{M})$
where:

    * –

$\forall M_{i},M_{j}\\!\in\\!\mathcal{M}$, $(M_{i},M_{j})\\!\in\\!\mathcal{A}$
iff $M_{i}(\mathbf{x})\\!\\!\neq\\!\\!M_{j}(\mathbf{x})$,
$M_{i}\\!\succeq_{\mathcal{M}}\\!M_{j}$;

    * –

$\forall M_{i}\\!\in\\!\mathcal{M}$ and $\mathbf{c}_{j}\\!\in\\!\mathcal{C}$
where $M_{i}(\mathbf{c}_{j})\\!=\\!M_{i}(\mathbf{x})$,
$(M_{i},\mathbf{c}_{j})\in\mathcal{A}$ iff $M_{i}\succeq_{\mathcal{M}}M_{j}$
and $(\mathbf{c}_{j},M_{i})\in\mathcal{A}$ iff
$M_{j}\succeq_{\mathcal{M}}M_{i}$;

  * •

$\mathcal{S}\subseteq(\mathcal{M}\times\mathcal{C})\cup(\mathcal{C}\times\mathcal{M})$
where for any $M_{i}\in\mathcal{M}$ and $\mathbf{c}_{j}\in\mathcal{C}$,
$(M_{i},\mathbf{c}_{j}),(\mathbf{c}_{j},M_{i})\in\mathcal{S}$ iff $i=j$.

Here, a model attacks another model if they disagree on the prediction and the
latter is not strictly preferred to the former. This means that models which
are outperformed with regards to preferences must be defended by more-
preferred, agreeing models in order to be considered acceptable. Models and
CEs are treated similarly, with the CEs inheriting the preferences from the
models by which they were generated, and attacks being present between them
when the model considers the CE invalid. This, along with the fact that models
and their CEs support one another, ensures that the models are inherently
linked to their reasoning, in the form of their CEs, and conflicts are drawn
not only when two models’ predictions differ, but also when their reasoning
differs.

Figure 1: BAF for Example 3 where: models’ predictions for the input
$\mathbf{x}$ are given as superscripts, e.g. $M_{1}(\mathbf{x})=0$ but
$M_{4}(\mathbf{x})=1$; reciprocal supports are represented by dual-headed
green arrows labelled with $+$ and standard (reciprocal) attacks are
represented by single-headed (dual-headed, resp.) red arrows labelled with
$-$.

Argumentative ensembling makes use of the set of s-preferred sets of
arguments, referred to as $P_{s}$, in the corresponding BAF
$\langle\mathcal{X},\mathcal{A},\mathcal{S}\rangle$, in order to resolve the
MM problem.

######  Definition 13.

Consider an input $\mathbf{x}$, a set $\mathcal{M}$ of models, a set
$\mathcal{C}$ of CEs, a set $\mathcal{L}$ of labels and a model preference
$\preceq_{\mathcal{M}}$. Let the largest s-preferred sets for the
corresponding BAF $\langle\mathcal{X},\mathcal{A},\mathcal{S}\rangle$ be
defined as:

$X_{max}=argmax_{X\in P_{s}}|X|.$
Then, the solution to RAE by _argumentative ensembling_ is defined as
$S^{a}\in X_{max}$, where $\mathcal{M}^{a}=S^{a}\cap\mathcal{M}$ and
$\mathcal{C}^{a}=S^{a}\cap\mathcal{C}$ and in the case of $|X_{max}|>1$, we
select $S^{a}$ from $X_{max}$ randomly. We also let
$\mathcal{M}^{a}(\mathbf{x})=\ell_{i}\in\mathcal{L}$ where
$M_{j}(\mathbf{x})=\ell_{i}$ $\forall M_{j}\in\mathcal{M}^{a}$.

Note that, alternatively, when $|X_{max}|>1$, we could choose to report all
viable resulting ensembles in $X_{max}$ rather than a random one as defined
above, so that more informed decisions could be made by relevant stakeholders.
We leave this to future work.

The following example demonstrates how quickly the problem, when preferences
are included, can become complex. This is the case even with only five models,
far fewer than usual in MM.

######  Example 3.

The BAF corresponding to input, models and CEs as in Example 2 is
$\langle\mathcal{X},\mathcal{A},\mathcal{S}\rangle$ with (see Fig. 1):
$\mathcal{X}=$ $\\{M_{1},$ $M_{2},$ $M_{3},$ $M_{4},$ $M_{5},$
$\mathbf{c}_{1},$ $\mathbf{c}_{2},$ $\mathbf{c}_{3},$ $\mathbf{c}_{4},$
$\mathbf{c}_{5}\\}$; $\mathcal{A}=$ $\\{(M_{2},M_{4}),$ $(M_{2},M_{5}),$
$(M_{2},\mathbf{c}_{1}),$ $(M_{2},\mathbf{c}_{3}),$ $(M_{3},M_{4}\\!),$
$(M_{3},\mathbf{c}_{4}\\!),$ $(M_{4},M_{1}\\!),$ $(M_{5},M_{1}\\!),$
$(M_{5},M_{2}\\!),$ $(M_{5},M_{3}\\!),$ $(\mathbf{c}_{2},M_{1}\\!)\\!\\}$; and
$\mathcal{S}=$ $\\{(M_{1},\mathbf{c}_{1}),$ $(M_{2},\mathbf{c}_{2}),$
$(M_{3},\mathbf{c}_{3}),$ $(M_{4},\mathbf{c}_{4}),$ $(M_{5},\mathbf{c}_{5}),$
$(\mathbf{c}_{1},M_{1}),$ $(\mathbf{c}_{2},M_{2}),$ $(\mathbf{c}_{3},M_{3}),$
$(\mathbf{c}_{4},M_{4}),$ $(\mathbf{c}_{5},M_{5})\\}$. This leads to $P_{s}=$
$\\{\\{M_{2},$ $\mathbf{c}_{2}\\},$ $\\{M_{4},$ $M_{5},$ $\mathbf{c}_{4},$
$\mathbf{c}_{5}\\}\\}$, and thus $S^{a}=$ $\\{M_{4},$ $M_{5},$
$\mathbf{c}_{4},$ $\mathbf{c}_{5}\\}$ and $\mathcal{M}^{a}(\mathbf{x})=1$.

This example shows how the use of CEs in the ensembling directly results in
the prediction being reversed, relative to the other ensembling methods,
violating majority vote. This is due to the fact that, while the preferences
over the two sets of models are roughly similar, the validity of the CEs for
the models selected by the augmented and robust ensemblings is very poor. This
means that when a CE which is valid for all models is required, some
compromise must be made on the model selection, as we demonstrated.

###  5.2 Theoretical Analysis

We will now undertake a theoretical analysis of argumentative ensembling,
demonstrating some of the desirable behaviours thereof via properties. First,
we consider the properties introduced in §4.1.

######  Theorem 3.

Argumentative ensembling satisfies non-emptiness, model agreement,
counterfactual validity and counterfactual coherence. It satisfies non-
triviality if for some $M_{i}\in\mathcal{M}$, where $\nexists
M_{j}\in\mathcal{M}\setminus\\{M_{i}\\}$ such that
$M_{j}\succ_{\mathcal{M}}M_{i}$, $\exists M_{k}\in\mathcal{M}$ such that
$M_{k}(\mathbf{x})=M_{i}(\mathbf{x})$,
$M_{k}(\mathbf{c}_{i})\\!\\!\neq\\!\\!M_{k}(\mathbf{x})$ and
$M_{i}(\mathbf{c}_{k})\\!\\!\neq\\!\\!M_{i}(\mathbf{x})$. It does not satisfy
majority vote.

###### Proof.

Let us first prove counterfactual coherence. By Def. 12, $\forall
M_{i}\in\mathcal{M}$ and $\forall\mathbf{c}_{j}\in\mathcal{C}$,
$(M_{i},\mathbf{c}_{j}),(\mathbf{c}_{j},M_{i})\in\mathcal{S}$. Thus, there
exists an indirect attack on any $M_{i}\in\mathcal{M}$ iff there exists a
direct attack on $\mathbf{c}_{i}\in\mathcal{C}$. Likewise, there exists an
indirect attack on any $\mathbf{c}_{i}\in\mathcal{C}$ iff there exists a
direct attack on $M_{i}\in\mathcal{M}$. Then, letting
$\mathcal{M}=\\{M_{1},\ldots,M_{m}\\}$ and
$\mathcal{C}=\\{\mathbf{c}_{1},\ldots,\mathbf{c}_{m}\\}$, since we know any
$X\in P_{s}$ is maximal wrt $\mathcal{X}$ by Def. 13, $X$ must be such that
$\forall i\in\\{1,\ldots,m\\}$, $M_{i}\in S$ iff $\mathbf{c}_{i}\in S$.

Let us prove non-emptiness by contradiction. Assume that $\exists X\in P_{s}$
such that $X\cap\mathcal{M}=\emptyset$ or $X\cap\mathcal{C}=\emptyset$. We
know from the above proof that, $\forall X\in P_{s}$,
$X\cap\mathcal{M}\neq\emptyset$ iff $X\cap\mathcal{C}\neq\emptyset$. Then, by
the definition of s-preferred extensions (see §3), it must be the case that
$\forall X\in P_{s}$, $X=\emptyset$. Based on the fact that, by Def. 11,
$\preceq_{\mathcal{M}}$ is a total ordering and thus transitive, this is not
possible as it will always be the case that $\exists M_{i}\in\mathcal{M}$ such
that $\nexists M_{j}\in\mathcal{M}$ where $M_{j}\succ_{\mathcal{M}}M_{i}$.
Thus, by Def. 12, either
$\mathcal{A}(M_{i})\cup\mathcal{A}(\mathbf{c}_{i})=\emptyset$ or
$\forall\alpha_{k}\in\mathcal{A}(M_{i})\cup\mathcal{A}(\mathbf{c}_{i})$,
$M_{i}\in\mathcal{A}(\alpha_{k})$ or
$\mathbf{c}_{i}\in\mathcal{A}(\alpha_{k})$, meaning
$\\{M_{i},\mathbf{c}_{i}\\}$ is either unattacked or is able to defend itself,
therefore would be acceptable in at least one s-preferred extension, and we
have the contradiction.

Let us prove model agreement by contradiction. Assume that $\exists
M_{i},M_{j}\\!\\!\in\\!\mathcal{M}$ such that
$M_{i}(\mathbf{x})\\!\neq\\!M_{j}(\mathbf{x})$ and $\exists
X\\!\\!\in\\!P_{s}$ such that $M_{i},M_{j}\\!\in\\!X$. By Def. 12, it follows
that $\exists(M_{i},M_{j})\\!\in\\!\mathcal{A}$ or
$\exists(M_{j},M_{i})\in\mathcal{A}$, which cannot be the case in an
s-preferred set, which must be conflict-free (see §3), and so we have the
contradiction.

Let us prove counterfactual validity by contradiction. Assume that $\exists
M_{i}\\!\in\\!\mathcal{M}$ and $\exists\mathbf{c}_{j}\\!\in\\!\mathcal{C}$
such that $M_{i}(\mathbf{c}_{j})\\!=\\!M_{i}(\mathbf{x})$ and $\exists
X\\!\in\\!P_{s}$ such that $M_{i},\mathbf{c}_{j}\\!\in\\!X$. By Definition 12,
it can be seen that $\exists(M_{i},\mathbf{c}_{j})\\!\in\\!\mathcal{A}$ or
$\exists(\mathbf{c}_{j},M_{i})\\!\in\\!\mathcal{A}$, which cannot be the case
in an s-preferred set, which, again, must be conflict-free, and so we have the
contradiction.

Let us prove the partial satisfaction of non-triviality by contradiction. From
Def. 13 and the proof for counterfactual coherence above, for
$|\mathcal{M}^{a}|=1$, it must be that $\forall X\in P_{s}$,
$|X\cap\mathcal{M}|=1$. However, from the above assumptions we can see that
for some $M_{i}\in\mathcal{M}$, where $\nexists
M_{j}\in\mathcal{M}\setminus\\{M_{i}\\}$ such that
$M_{j}\succ_{\mathcal{M}}M_{i}$, $\exists M_{k}\in\mathcal{M}$ such that
$M_{k}(\mathbf{x})=M_{i}(\mathbf{x})$, $M_{k}(\mathbf{c}_{i})\neq
M_{k}(\mathbf{x})$ and $M_{i}(\mathbf{c}_{k})\neq M_{i}(\mathbf{x})$. Then, to
avoid a contradiction, it must be that
$\exists\alpha_{l}\in\mathcal{M}\cup\mathcal{C}$ such that
$(\alpha_{l},M_{k})\in\mathcal{A}$ or
$(\alpha_{l},\mathbf{c}_{k})\in\mathcal{A}$, and
$(M_{i},\alpha_{l}),(\mathbf{c}_{i},\alpha_{l})\nin\mathcal{A}$. If
$\alpha_{l}\in\mathcal{M}$, and thus $M_{l}(\mathbf{x})\neq
M_{k}(\mathbf{x})=M_{i}(\mathbf{x})$, then it must be that
$(M_{i},M_{l})\in\mathcal{A}$ (a contradiction, by Def. 12, since
$M_{i}\succeq_{\mathcal{M}}M_{l}$). If $\alpha_{l}\in\mathcal{C}$ and
$M_{l}(\mathbf{x})\neq M_{k}(\mathbf{x})=M_{i}(\mathbf{x})$, then
$(M_{i},M_{l})\in\mathcal{A}$ (a contradiction by the same reasoning).
Finally, if $\alpha_{l}\in\mathcal{C}$ and
$M_{l}(\mathbf{x})=M_{k}(\mathbf{x})=M_{i}(\mathbf{x})$, then either: if
$M_{i}(\mathbf{c}_{l})=M_{i}(\mathbf{x})$ or
$M_{l}(\mathbf{c}_{i})=M_{l}(\mathbf{x})$, then
$(M_{i},\mathbf{c}_{l})\in\mathcal{A}$ (a contradiction); or, otherwise,
$M_{l}\in X^{\prime}$ (which can be checked by repeating the steps for
$M_{k}$, for $M_{l}$ instead). Thus $\exists X^{\prime}\in P_{s}$ where
$|X^{\prime}\cap\mathcal{M}|>1$, and we have the contradiction in all cases.

Finally, Example 3 provides a counterexample which shows that majority vote is
not satisfied. ∎

These results contrast with those for augmented and robust ensembling, as
shown in Table 1. Argumentative ensembling avoids the pitfalls of augmented
and robust ensembling by satisfying non-emptiness, counterfactual validity and
counterfactual coherence. In order to achieve this behaviour, majority vote is
sacrificed as a guarantee. In §6 we will assess the impact of not guaranteeing
majority vote on argumentative ensembling’s accuracy, along with other
metrics.

However, first, we consider the relationship between the different ensembling
methods. For the remainder of the section, we assume as given an input
$\mathbf{x}$, a set $\mathcal{M}$ of models, a set $\mathcal{C}$ of CEs, a
model preference $\preceq_{\mathcal{M}}$ and a corresponding BAF
$\langle\mathcal{X},\mathcal{A},\mathcal{S}\rangle$ with $P_{s}$ the set of
all its s-preferred sets.

######  Theorem 4.

If $\forall M_{i},M_{j}\in\mathcal{M}$, $M_{i}\simeq_{\mathcal{M}}M_{j}$, and
$\forall M_{k}\in\mathcal{M}$ and $\mathbf{c}_{l}\in\mathcal{C}$, where
$M_{k}(\mathbf{x})=M_{l}(\mathbf{x})$, $M_{k}(\mathbf{c}_{l})\neq
M_{k}(\mathbf{c}_{l})$, then augmented, robust and argumentative ensembling
are equivalent.

###### Proof.

If $\forall M_{i},M_{j}\in\mathcal{M}$, $M_{i}\simeq_{\mathcal{M}}M_{j}$, then
it can be seen from Definition 12 that
$(M_{i},M_{j}),(M_{j},M_{i})\in\mathcal{A}$ iff $M_{i}(\mathbf{x})\neq
M_{j}(\mathbf{x})$. Note that, by Definition 8, augmented and robust
ensembling are equivalent since $\forall M_{k}\in\mathcal{M}$ and
$\mathbf{c}_{l}\in\mathcal{C}$, where $M_{k}(\mathbf{x})=M_{l}(\mathbf{x})$,
$M_{k}(\mathbf{c}_{l})\neq M_{k}(\mathbf{c}_{l})$. Also by the assumptions, it
can be seen from Definition 12 that $\forall M_{k}\in\mathcal{M}$ and
$\forall\mathbf{c}_{l}\in\mathcal{C}$,
$(M_{k},\mathbf{c}_{l}),(\mathbf{c}_{l},M_{k})\in\mathcal{A}$ iff
$M_{k}(\mathbf{c}_{l})=M_{k}(\mathbf{x})$ and $M_{k}(\mathbf{x})\neq
M_{l}(\mathbf{x})$ due to the assumptions in the theorem, meaning any attack
is reciprocated and all arguments defend themselves. Then, $\forall
M_{m},M_{n}\in\mathcal{M}$ such that $M_{m}(\mathbf{x})=M_{n}(\mathbf{x})$,
$(M_{m},M_{n})\nin\mathcal{A}$. Thus,
$P_{s}=\\{\\{M_{o}\in\mathcal{M},\mathbf{c}_{o}\in\mathcal{C}|M_{o}(\mathbf{x})=0\\},\\{M_{p}\in\mathcal{M},\mathbf{c}_{p}\in\mathcal{C}|M_{p}(\mathbf{x})=1\\}$.
By Definitions 1, 8 and 13, all forms of ensembling select from the same two
sets of models and CEs in the same manner and are thus equivalent. ∎

We also provide a number of theoretical results concerning the behaviour of
argumentative ensembling, first relating to the preferences. The first result
demonstrates how a completely dominant model wrt the preferences will be
present in all s-preferred sets.

######  Proposition 1.

If $\exists M_{i}\in\mathcal{M}$ such that $\forall M_{j}\in\mathcal{M}$,
$M_{i}\succ_{\mathcal{M}}M_{j}$, then $\forall X\in P_{s}$, $M_{i}\in X$.

###### Proof.

If $\forall M_{j}\in\mathcal{M}$, $M_{i}\succ_{\mathcal{M}}M_{j}$ then, by
Definition 12, $\mathcal{A}(M_{i})=\mathcal{A}(\mathbf{c}_{i})=\emptyset$ and
$\forall M_{j}\in\mathcal{M}$ where $M_{j}(\mathbf{x})\neq M_{i}(\mathbf{x})$,
$M_{i}\in\mathcal{A}(M_{j})$. Similarly, $\forall\mathbf{c}_{k}\in\mathcal{C}$
where $M_{i}(\mathbf{c}_{k})=M_{i}(\mathbf{x})$,
$M_{i}\in\mathcal{A}(\mathbf{c}_{k})$, and so $M_{i}$ indirectly attacks
$M_{k}$. Then, $\nexists X\in P_{s}$ such that $M_{j}\in X$ or $M_{k}\in X$
and thus, $\forall X\in P_{s}$, $M_{i}\in X$. ∎

We also show how, for any two s-preferred sets, there exists some trade-off
between their models wrt the preferences.

######  Lemma 1.

Given two s-preferred sets $X,X^{\prime}\in P_{s}$, $\nexists
M_{i}\in(X\cap\mathcal{M})\setminus X^{\prime}$ such that
$M_{i}\succ_{\mathcal{M}}M_{j}$ $\forall M_{j}\in X^{\prime}$.

Meanwhile, in the non-strict case, a model which is not outperformed by any
other model wrt the preferences will be present in at least one s-preferred
set.

######  Proposition 2.

If $\exists M_{i}\in\mathcal{M}$ such that $\forall M_{j}\in\mathcal{M}$,
$M_{i}\succeq_{\mathcal{M}}M_{j}$, then $\exists X\in P_{s}$ such that
$M_{i}\in X$.

###### Proof.

If $\forall M_{j}\in\mathcal{M}$, $M_{i}\succeq_{\mathcal{M}}M_{j}$ then, by
Definition 12,
$\nexists\alpha_{k}\in\mathcal{A}(M_{i})\cup\mathcal{A}(\mathbf{c}_{i})$ such
that $M_{i}\nin\mathcal{A}(\alpha_{k})$ and
$\mathbf{c}_{i}\nin\mathcal{A}(\alpha_{k})$. Thus, it must be the case that
$\exists X\in P_{s}$ such that $M_{i}\in X$. ∎

We now show that if a model is outperformed wrt the preferences by all other
models, then the outperformed model cannot exist in an s-preferred set unless
it is defended by a more preferred model.

######  Proposition 3.

For any $M_{i}\in\mathcal{M}$, if $M_{j}\succ_{\mathcal{M}}M_{i}$ $\forall
M_{j}\in\mathcal{M}\setminus\\{M_{i}\\}$ and $\exists X\in P_{s}$ such that
$M_{i}\in X$, then, $\forall M_{k}\in\mathcal{M}$ where $M_{k}(\mathbf{x})\neq
M_{i}(\mathbf{x})$, $\exists M_{l}\in X$ such that
$M_{l}\succeq_{\mathcal{M}}M_{k}$.

###### Proof.

By Definition 12, $M_{k}\\!\in\\!\mathcal{A}(M_{i})$ and
$\\{M_{m}\\!\in\\!\mathcal{M}|M_{i}\\!\in\\!\mathcal{A}(M_{m})\\!\vee\\!\mathbf{c}_{i}\\!\in\\!\mathcal{A}(M_{m})\\}=\emptyset$.
Then, given that $\exists X\\!\in\\!P_{s}$ such that $M_{i}\\!\in\\!X$, we
know that, $\forall M_{k}\\!\in\\!\mathcal{A}(M_{i})$ $\exists
M_{l}\\!\in\\!X$ where $M_{l}(\mathbf{x})\\!=\\!M_{i}(\mathbf{x})$ and
$M_{l}\\!\in\\!\mathcal{A}(M_{k})$. Thus, by Definition 12,
$M_{l}\succeq_{\mathcal{M}}M_{k}\succ_{\mathcal{M}}M_{i}$. ∎

We also consider the behaviour of argumentative ensembling wrt the selected
CEs, demonstrating that those from disagreeing models are guaranteed not to be
included in any s-preferred set.

######  Proposition 4.

Any s-preferred set $X\in P_{s}$ is such that
$\nexists\mathbf{c}_{i},\mathbf{c}_{j}\in X\cap\mathcal{C}$ where
$M_{i}(\mathbf{x})\neq M_{j}(\mathbf{x})$.

###### Proof.

Let us prove by contradiction, assuming that
$\exists\mathbf{c}_{i},\mathbf{c}_{j}\in X\cap\mathcal{C}$. Counterfactual
coherence (Theorem 3) requires that $M_{i},M_{j}\in X$. However, by Definition
12, $M_{i}(\mathbf{x})\neq M_{j}(\mathbf{x})$ requires that
$M_{i}\in\mathcal{A}(M_{j})$ or $M_{j}\in\mathcal{A}(M_{i})$, and so we have
the contradiction. ∎

Finally, we show that s-preferred sets of corresponding BAFs in our setting
satisfy all the forms of admissibility for BAFs in Cayrol and Lagasquie-Schiex
(2005).

######  Proposition 5.

Any s-preferred set $X\in P_{s}$ is d-admissible, s-admissible and
c-admissible.

###### Proof.

Trivially, any s-preferred set is s-admissible and thus also d-admissible (see
§3). Then, it can be seen from Definition 12 that
$\mathcal{S}\subseteq(\mathcal{M}\times\mathcal{C})\cup(\mathcal{C}\times\mathcal{M})$,
$i=j$
$\forall(M_{i},\mathbf{c}_{j})\in\mathcal{S}\cap(\mathcal{M}\times\mathcal{C})$
and $k=l$
$\forall(\mathbf{c}_{k},M_{l})\in\mathcal{S}\cap(\mathcal{C}\times\mathcal{M})$.
By counterfactual coherence (Theorem 3), $\forall M_{i}\in\mathcal{M}$ and
$\forall\mathbf{c}_{i}\in\mathcal{C}$, $M_{i}\in X$ iff $\mathbf{c}_{i}\in X$.
Then, since $P_{s}$ contains the sets of $\mathcal{X}$ which are maximal wrt
set-inclusion, any $X^{\prime}\in P_{s}$ must be closed for $\mathcal{S}$ and
thus c-admissible. ∎

##  6 Empirical Evaluation

We now examine the effectiveness of our approach using three real-world
datasets. Specifically, we empirically evaluate the extent to which each of
the ensembling methods introduced in §4.2 and §5 satisfy the desirable
properties defined in §4.1. We also instantiate three variations of
argumentative ensembling by including two different types of model properties
$\mathcal{P}$ and demonstrate the usefulness of incorporating model
preferences into ensembling methods.

###  6.1 Experiment Setup

We apply all ensembling methods on three datasets in the legal and financial
contexts: loan approval (heloc) FICO (2018), recidivism prediction (compas)
Julia Angwin and Kirchner (2016), and credit risk (credit) Hofmann (1994). Due
to neural networks’ sensitivity to randomness at training time, they suffer
severely from MM and are frequently targeted when investigating this research
topic (as discussed in §2). Therefore, even though our method is model-
agnostic, we focus on neural networks for the experiments.

For each dataset, we train-test 150 classifiers with five different hidden
layer sizes using 80% of the dataset (this 80% is train-test split for
training each model; see Appendix A for dataset and training details). The 150
neural networks are trained using different random seeds for parameter
initialisation and different train-test splits (within the train-test 80% of
the dataset), forming a pool of possible models under MM from which we sample
multiple sets $\mathcal{M}$ of models to which we apply our ensembling
methods. We use the remaining 20% of each dataset as test inputs for the
ensembling methods (limited to 500 inputs test set if larger than this).

At each run, we randomly sample, from the model pool, sets $\mathcal{M}$ with
10, 20 or 30 models, then we feed each input to the models to receive their
predicted labels and generate one CE from each model using the nearest
neighbour CEs approach of Brughmans et al. (2023), and finally apply the
ensembling methods. For each size (10, 20, 30), we perform five different
choices of $\mathcal{M}$, and record the mean and standard deviation of the
results (over the five choices of model sets for each size).

As concerns model preferences, we focus on accuracy of the trained classifiers
over the (20%) test inputs and model structure simplicity. For the latter, we
assign the models, from the most complex to the simplest (depending on the
number of neurons in the hidden layers, see Appendix A), scores of {0, 0.25,
0.5, 0.75, 1} such that higher values imply simpler models. Note that multiple
models in $\mathcal{M}$ may have the same simplicity scores as we adopt only
five different model structures to obtain 150 neural networks for each
dataset. Models in $\mathcal{M}$ may have any (near-optimal) test accuracy: in
the experiments each such model has a different accuracy.

| acc. | simp. |  size M/C |  $\mathbf{c}$ val. (fail) | mv | acc. | simp. |  size M/C |  $\mathbf{c}$ val. (fail) | mv | acc. | simp. |  size M/C |  $\mathbf{c}$ val. (fail) | mv
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
| heloc | compas | credit
$|\mathcal{M}|=10$ | .709$\pm$.003 | .856$\pm$.001 | .664$\pm$.008
$S^{n}$ | .709 | .495 | .943/.943 | .657 (.00) | 1.00 | .858 | .464 | .980/.980 | .572 (.00) | 1.00 | .697 | .588 | .817/.817 | .757 (.00) | 1.00
$S^{v}$ | .709 | .495 | .943/.309 | 1.00 (.34) | 1.00 | .858 | .464 | .980/.174 | 1.00 (.51) | 1.00 | .697 | .588 | .817/.457 | 1.00 (.44) | 1.00
$S^{a}$ | .712 | .504 | .499/.499 | 1.00 (.00) | .983 | .859 | .463 | .369/.369 | 1.00 (.00) | .994 | .694 | .600 | .580/.580 | 1.00 (.00) | .953
$S^{a}$-A | .726 | .485 | .357/.357 | 1.00 (.00) | .943 | .864 | .430 | .295/.295 | 1.00 (.00) | .988 | .710 | .593 | .486/.486 | 1.00 (.00) | .825
$S^{a}$-S | .710 | .608 | .462/.462 | 1.00 (.00) | .967 | .860 | .657 | .306/.306 | 1.00 (.00) | .987 | .689 | .626 | .565/.565 | 1.00 (.00) | .925
$S^{a}$-AS | .712 | .528 | .493/.493 | 1.00 (.00) | .980 | .860 | .501 | .360/.360 | 1.00 (.00) | .994 | .696 | .607 | .578/.578 | 1.00 (.00) | .946
$|\mathcal{M}|=20$ | .710$\pm$.003 | .855$\pm$.001 | .663$\pm$.004
$S^{n}$ | .717 | .488 | .940/.940 | .626 (.00) | 1.00 | .859 | .538 | .978/.978 | .544 (.00) | 1.00 | .708 | .571 | .810/.810 | .734 (.00) | 1.00
$S^{v}$ | .717 | .388 | .940/.230 | 1.00 (.37) | 1.00 | .859 | .538 | .978/.111 | 1.00 (.60) | 1.00 | .708 | .571 | .810/.351 | 1.00 (.62) | 1.00
$S^{a}$ | .716 | .466 | .460/.460 | 1.00 (.00) | .984 | .859 | .514 | .331/.331 | 1.00 (.00) | .992 | .691 | .580 | .557/.557 | 1.00 (.00) | .961
$S^{a}$-A | .728 | .432 | .361/.361 | 1.00 (.00) | .950 | .866 | .541 | .235/.235 | 1.00 (.00) | .982 | .709 | .586 | .481/.481 | 1.00 (.00) | .862
$S^{a}$-S | .711 | .551 | .420/.420 | 1.00 (.00) | .966 | .857 | .609 | .304/.304 | 1.00 (.00) | .987 | .684 | .590 | .549/.549 | 1.00 (.00) | .947
$S^{a}$-AS | .715 | .473 | .459/.459 | 1.00 (.00) | .984 | .859 | .555 | .324/.324 | 1.00 (.00) | .990 | .693 | .581 | .556/.556 | 1.00 (.00) | .959
$|\mathcal{M}|=30$ | .710$\pm$.003 | .855$\pm$.001 | .663$\pm$.004
$S^{n}$ | .718 | .512 | .940/.940 | .620 (.00) | 1.00 | .859 | .527 | .976/.976 | .530 (.00) | 1.00 | .710 | .540 | .807/.807 | .727 (.00) | 1.00
$S^{v}$ | .718 | .512 | .940/.205 | 1.00 (.41) | 1.00 | .859 | .527 | .976/.087 | 1.00 (.56) | 1.00 | .710 | .540 | .807/.311 | 1.00 (.70) | 1.00
$S^{a}$ | .716 | .499 | .456/.456 | 1.00 (.00) | .982 | .862 | .519 | .308/.308 | 1.00 (.00) | .990 | .683 | .549 | .551/.551 | 1.00 (.00) | .943
$S^{a}$-A | .729 | .406 | .353/.353 | 1.00 (.00) | .946 | .865 | .532 | .225/.225 | 1.00 (.00) | .983 | .711 | .552 | .441/.441 | 1.00 (.00) | .850
$S^{a}$-S | .712 | .518 | .445/.445 | 1.00 (.00) | .978 | .861 | .567 | .294/.294 | 1.00 (.00) | .988 | .684 | .555 | .546/.546 | 1.00 (.00) | .940
$S^{a}$-AS | .716 | .500 | .456/.456 | 1.00 (.00) | .982 | .862 | .543 | .303/.303 | 1.00 (.00) | .988 | .683 | .549 | .551/.551 | 1.00 (.00) | .944

Table 2: Quantitative evaluations of ensembling methods on three datasets,
heloc, compas, and credit. $|\mathcal{M}|=\\{10,20,30\\}$ stands for results
for different model set sizes, the acc. entries in the rows starting with
$|\mathcal{M}|$ are the average single model accuracies.

Evaluation metrics. Each ensembling method is evaluated against the following
metrics: prediction accuracy over the test set (_acc._), average model
simplicity in the ensemble (_simp._), average size of models and CEs in the
ensemble, measured as percentages of $|\mathcal{M}|$ (_size M/C_), average
validities of ensembled CEs over the ensembled models (_$\mathbf{c}$ val._).
Also, we report the percentage of test inputs for which a method fails to
produce CEs (fail). The results are averaged over all the test inputs except
for the failure cases. We also report the average test set accuracies of the
models in $\mathcal{M}$. Note that the model agreement property (Property 4)
is omitted as it is satisfied by every compared method. To understand how the
violation of the majority vote property affects our method, we measure the
proportion of test inputs for which the predicted label using our method is
the same as that of naive ensembling (_mv_).

Ensembling Methods. We use augmented and robust ensembling as baselines. For
argumentative ensembling, we use four variations with different preferences:
$S^{a}$ ($\mathcal{P}\\!=\\!\emptyset$), $S^{a}$-A
($\mathcal{P}\\!=\\!\\{accuracy\\}$), $S^{a}$-S
($\mathcal{P}\\!=\\!\\{simplicity\\}$) and $S^{a}$-AS
($\mathcal{P}\\!=\\!\\{accuracy$, $simplicity\\}$, where
$accuracy\\!\simeq_{\mathcal{P}}\\!simplicity$). In our implementation of
argumentative ensembling, when $|X_{max}|\\!>\\!1$ (Definition 13), we return
$S^{a}$, which has the same prediction label as naive ensembling. We give
percentages of test inputs with $|X_{max}\\!\\!>\\!\\!1|$ in Table 3 in
Appendix B.

###  6.2 Results and Analyses

We report the results for all experiments in Table 2 (the standard deviations
are presented in Tables 4 to 6 in Appendix C).

Usefulness of preferences. With test accuracy specified as model preference,
$S^{a}$-A shows the best accuracy in all experiments. This validates
Proposition 1, because, assuming that the accuracy for every model in
$\mathcal{M}$ is different, for $S^{a}$-A, there exists a model in
$\mathcal{M}$ that is the most preferred and is included in the ensemble.
Similarly, $S^{a}$-S shows the best simp. scores in all experiments. However,
since simplicity scores are not unique for each model, usually a single most
preferred model does not exist, therefore an optimal simp. evaluation is not
guaranteed. When specifying both properties as model preferences ($S^{a}$-AS),
at least one of the two metrics is improved compared with $S^{a}$.

Desirable properties of ensembling methods. For up to 70% of test inputs,
robust ensembling does not find any CEs ($S^{v}\cap\mathcal{C}=\emptyset$),
confirming its violation of non-emptiness. As $|\mathcal{M}|$ increases,
$S^{v}$ would require finding CEs which are valid for more models, and the
number of CEs found would drop as shown by the results for the size C
evaluations. In contrast, the remaining methods, including argumentative
ensembling, always find non-empty ensembles, the sizes of which are also not
affected by the model set sizes.

$S^{n}$ demonstrates low c Val. scores, showing that, on average, a CE from a
model in the ensemble is only valid for 53.0% to 75.7% of other agreeing
models. Therefore, the violation of counterfactual validity has a significant
impact on results in practice. $S^{v}$ produces valid CEs over models in the
ensemble, however, as previously mentioned, they do not always exist.

For $S^{a}$, we note the same number of models and CEs in the solution set and
100% counterfactual validity, confirming the behaviour predicted by Theorem 3.
Argumentative ensembling shows model and CE ensemble sizes of 22.5% (when
$|\mathcal{M}|=30$) to 58.0% of $|\mathcal{M}|$, meaning that it is non-
trivially more selective than $S^{n}$ and $S^{v}$, only accepting the largest
set of models with similar reasoning local to the test input (validated by
agreement on CEs as required by counterfactual coherence). This results in
comparable test accuracies as $S^{n}$ and $S^{v}$ with guaranteed CE validity.
In fact, mostly, the $S^{a}$-S option has the lowest agreement rate with
majority vote prediction (mv), but it is more accurate than the baselines
using naive ensembling. When no preference is specified, argumentative
ensembling has higher accuracy than majority vote for heloc when
$|\mathcal{M}|=10$ and for compas when $|\mathcal{M}|=10,30$. Thus, we do not
necessarily lose accuracy in satisfying properties besides majority vote.

##  7 Conclusions and Future Work

We have presented a formal study of the problem of providing recourse under
MM. We defined several properties which are desirable in methods for solving
this problem, highlighting deficiencies in extending conservatively the
standard naive ensembling used for MM without recourse. We have then
introduced argumentative ensembling, a novel method for providing recourse
under MM, which leverages computational argumentation to incorporate
robustness guarantees and user preferences over models. We show, by means of a
theoretical analysis, that argumentative ensembling hosts advantages over
other methods, notably in non-emptiness of solutions and validity of CEs,
notwithstanding its ability to handle user preferences. This is, however,
achieved by sacrificing the satisfaction of the property of majority vote, and
so we conducted an empirical analysis with three real-world datasets to
examine the effects of this sacrifice. Our results demonstrate that
argumentative ensembling always finds valid CEs without compromising
prediction accuracy, and shows the usefulness of specifying preferences over
models.

This paper opens up several potentially fruitful directions for future work.
First, it would be interesting to examine the extent to which considering
attacks to or from _sets_ of arguments, rather than single arguments, as in
Nielsen and Parsons (2006); Flouris and Bikakis (2019); Dvorák et al. (2022);
Dimopoulos et al. (2023), may help in MM given the conflicts between agreeing
sets of models. Further, extended AFs Modgil (2009) and value-based AFs Bench-
Capon (2002) may provide useful alternative ways to account for preferences.
We would also like to exploit the explanatory potential of argumentation to
support explainable ensembling, e.g. using sub-graphs as in Fan and Toni
(2014); Zeng et al. (2019). Moreover, in order to support experiments with a
high number of models (beyond the 30 we considered), large-scale argumentation
solvers would be highly desirable. Finally, it would be interesting to assess
the effect which MM has on users’ evaluations of CEs.

## Acknowledgement

Jiang, Rago and Toni were partially funded by J.P. Morgan and by the Royal
Academy of Engineering under the Research Chairs and Senior Research
Fellowships scheme. Leofante is supported by an Imperial College Research
Fellowship grant. Rago and Toni were partially funded by the European Research
Council (ERC) under the European Union’s Horizon 2020 research and innovation
programme (grant agreement No. 101020934). Any views or opinions expressed
herein are solely those of the authors listed.

## References

  * Abchiche-Mimouni et al. [2023] Nadia Abchiche-Mimouni, Leila Amgoud, and Farida Zehraoui.  Explainable ensemble classification model based on argumentation.  In AAMAS 2023, pages 2367–2369, 2023.
  * Atkinson et al. [2017] Katie Atkinson, Pietro Baroni, Massimiliano Giacomin, Anthony Hunter, Henry Prakken, Chris Reed, Guillermo Ricardo Simari, Matthias Thimm, and Serena Villata.  Towards artificial argumentation.  AI Magazine, 38(3):25–36, 2017.
  * Baroni et al. [2018] Pietro Baroni, Dov Gabbay, Massimiliano Giacomin, and Leendert van der Torre, editors.  Handbook of Formal Argumentation.  College Publications, 2018.
  * Bench-Capon [2002] Trevor J. M. Bench-Capon.  Value-based argumentation frameworks.  In NMR 2002, pages 443–454, 2002.
  * Black et al. [2022a] Emily Black, Klas Leino, and Matt Fredrikson.  Selective ensembles for consistent predictions.  In ICLR 2022, 2022.
  * Black et al. [2022b] Emily Black, Manish Raghavan, and Solon Barocas.  Model multiplicity: Opportunities, concerns, and solutions.  In FAccT 2022, pages 850–863, 2022.
  * Black et al. [2022c] Emily Black, Zifan Wang, and Matt Fredrikson.  Consistent counterfactuals for deep models.  In ICLR 2022, 2022.
  * Breiman [2001] Leo Breiman.  Statistical modeling: The two cultures (with comments and a rejoinder by the author).  Statistical science, 16(3):199–231, 2001.
  * Brughmans et al. [2023] Dieter Brughmans, Pieter Leyman, and David Martens.  NICE: an algorithm for nearest instance counterfactual explanations.  Data Mining and Knowledge Discovery, pages 1–39, 2023.
  * Bui et al. [2022] Ngoc Bui, Duy Nguyen, and Viet Anh Nguyen.  Counterfactual plans under distributional ambiguity.  In ICLR 2022, 2022.
  * Cabrio and Villata [2013] Elena Cabrio and Serena Villata.  A natural language bipolar argumentation approach to support users in online debate interactions†.  Argument Comput., 4(3):209–230, 2013.
  * Cayrol and Lagasquie-Schiex [2005] Claudette Cayrol and Marie-Christine Lagasquie-Schiex.  On the acceptability of arguments in bipolar argumentation frameworks.  In ECSQARU 2005, pages 378–389, 2005.
  * Chen et al. [2018] Chaofan Chen, Kangcheng Lin, Cynthia Rudin, Yaron Shaposhnik, Sijia Wang, and Tong Wang.  An interpretable model with globally consistent explanations for credit risk.  CoRR, abs/1811.12615, 2018.
  * Coston et al. [2021] Amanda Coston, Ashesh Rambachan, and Alexandra Chouldechova.  Characterizing fairness over the set of good models under selective labels.  In ICML 2021, pages 2144–2155, 2021.
  * Cyras et al. [2019] Kristijonas Cyras, Dimitrios Letsios, Ruth Misener, and Francesca Toni.  Argumentation for explainable scheduling.  In AAAI 2019, pages 2752–2759, 2019.
  * Cyras et al. [2021] Kristijonas Cyras, Antonio Rago, Emanuele Albini, Pietro Baroni, and Francesca Toni.  Argumentative XAI: A survey.  In IJCAI 2021, pages 4392–4399, 2021.
  * D’Amour et al. [2022] Alexander D’Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D Hoffman, et al.  Underspecification presents challenges for credibility in modern machine learning.  JMLR, 23(1):10237–10297, 2022.
  * Dejl et al. [2021] Adam Dejl, Chloe He, Pranav Mangal, Hasan Mohsin, Bogdan Surdu, Eduard Voinea, Emanuele Albini, Piyawat Lertvittayakumjorn, Antonio Rago, and Francesca Toni.  Argflow: A toolkit for deep argumentative explanations for neural networks.  In AAMAS 2021, pages 1761–1763, 2021.
  * Dhurandhar et al. [2018] Amit Dhurandhar, Pin-Yu Chen, Ronny Luss, Chun-Chen Tu, Pai-Shun Ting, Karthikeyan Shanmugam, and Payel Das.  Explanations based on the missing: Towards contrastive explanations with pertinent negatives.  In NeurIPS 2018, pages 590–601, 2018.
  * Dimopoulos et al. [2023] Yannis Dimopoulos, Wolfgang Dvorák, Matthias König, Anna Rapberger, Markus Ulbricht, and Stefan Woltran.  Sets attacking sets in abstract argumentation.  In NMR 2023, pages 22–31, 2023.
  * Dong and Rudin [2019] Jiayun Dong and Cynthia Rudin.  Variable importance clouds: A way to explore variable importance for the set of good models.  CoRR, abs/1901.03209, 2019.
  * Dung [1995] Phan Minh Dung.  On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n-person games.  Artif. Intell., 77(2):321–358, 1995.
  * Dutta et al. [2020] Sanghamitra Dutta, Dennis Wei, Hazar Yueksel, Pin-Yu Chen, Sijia Liu, and Kush R. Varshney.  Is there a trade-off between fairness and accuracy? A perspective using mismatched hypothesis testing.  In ICML 2020, pages 2803–2813, 2020.
  * Dutta et al. [2022] Sanghamitra Dutta, Jason Long, Saumitra Mishra, Cecilia Tilli, and Daniele Magazzeni.  Robust counterfactual explanations for tree-based ensembles.  In ICML 2022, pages 5742–5756, 17–23 Jul 2022.
  * Dvorák et al. [2022] Wolfgang Dvorák, Matthias König, Markus Ulbricht, and Stefan Woltran.  Rediscovering argumentation principles utilizing collective attacks.  In KR 2022, pages 122–131, 2022.
  * Fan and Toni [2014] Xiuyi Fan and Francesca Toni.  On computing explanations in abstract argumentation.  In ECAI 2014, pages 1005–1006, 2014.
  * Ferrario and Loi [2022] Andrea Ferrario and Michele Loi.  The robustness of counterfactual explanations over time.  IEEE Access, 10:82736–82750, 2022.
  * FICO [2018] FICO.  Explainable machine learning challenge, 2018.
  * Fisher et al. [2019] Aaron Fisher, Cynthia Rudin, and Francesca Dominici.  All models are wrong, but many are useful: Learning a variable’s importance by studying an entire class of prediction models simultaneously.  J. Mach. Learn. Res., 20:177:1–177:81, 2019.
  * Flouris and Bikakis [2019] Giorgos Flouris and Antonis Bikakis.  A comprehensive study of argumentation frameworks with sets of attacking arguments.  Int. J. Approx. Reason., 109:55–86, 2019.
  * Guidotti [2022] Riccardo Guidotti.  Counterfactual explanations and how to find them: literature review and benchmarking.  Data Mining and Knowledge Discovery, pages 1–55, 2022.
  * Hamman et al. [2023] Faisal Hamman, Erfaun Noorani, Saumitra Mishra, Daniele Magazzeni, and Sanghamitra Dutta.  Robust counterfactual explanations for neural networks with probabilistic guarantees.  In ICML 2023, pages 12351–12367, 2023.
  * Hofmann [1994] Hans Hofmann.  Statlog (German Credit Data).  UCI Machine Learning Repository, 1994.
  * Hsu and Calmon [2022] Hsiang Hsu and Flávio P. Calmon.  Rashomon capacity: A metric for predictive multiplicity in classification.  In NeurIPS 2023, pages 28988–29000, 2022.
  * Irwin et al. [2022] Benjamin Irwin, Antonio Rago, and Francesca Toni.  Forecasting argumentation frameworks.  In KR 2022, pages 533–543, 2022.
  * Jiang et al. [2023a] Junqi Jiang, Jianglin Lan, Francesco Leofante, Antonio Rago, and Francesca Toni.  Provably robust and plausible counterfactual explanations for neural networks via robust optimisation.  CoRR, abs/2309.12545, 2023.
  * Jiang et al. [2023b] Junqi Jiang, Francesco Leofante, Antonio Rago, and Francesca Toni.  Formalising the robustness of counterfactual explanations for neural networks.  In AAAI 2023, pages 14901–14909, 2023.
  * Jiang et al. [2024] Junqi Jiang, Antonio Rago, Francesco Leofante, and Francesca Toni.  Recourse under model multiplicity via argumentative ensembling.  In AAMAS 2024, 2024.
  * Julia Angwin and Kirchner [2016] Surya Mattu Julia Angwin, Jeff Larson and Lauren Kirchner.  There’s software used across the country to predict future criminals. and it’s biased against blacks., 2016.
  * Leofante et al. [2023] Francesco Leofante, Elena Botoeva, and Vineet Rajani.  Counterfactual explanations and model multiplicity: a relational verification view.  In KR 2023, pages 763–768, 2023.
  * Ley et al. [2023] Dan Ley, Leonard Tang, Matthew Nazari, Hongjin Lin, Suraj Srinivas, and Himabindu Lakkaraju.  Consistent explanations in the face of model indeterminacy via ensembling.  CoRR, abs/2306.06193, 2023.
  * Marx et al. [2020] Charles T. Marx, Flávio P. Calmon, and Berk Ustun.  Predictive multiplicity in classification.  In ICML 2020, pages 6765–6774, 2020.
  * Marx et al. [2023] Charles Marx, Youngsuk Park, Hilaf Hasson, Yuyang Wang, Stefano Ermon, and Luke Huan.  But are you sure? an uncertainty-aware perspective on explainable AI.  In AISTATS 2023, pages 7375–7391, 2023.
  * Mehrer et al. [2020] Johannes Mehrer, Courtney J Spoerer, Nikolaus Kriegeskorte, and Tim C Kietzmann.  Individual differences among deep neural network models.  Nature communications, 11(1):5725, 2020.
  * Mishra et al. [2021] Saumitra Mishra, Sanghamitra Dutta, Jason Long, and Daniele Magazzeni.  A survey on the robustness of feature importance and counterfactual explanations.  CoRR, abs/2111.00358, 2021.
  * Modgil [2009] Sanjay Modgil.  Reasoning about preferences in argumentation frameworks.  Artif. Intell., 173(9-10):901–934, 2009.
  * Mothilal et al. [2020] Ramaravind Kommiya Mothilal, Amit Sharma, and Chenhao Tan.  Explaining machine learning classifiers through diverse counterfactual explanations.  In FAT 2020, pages 607–617, 2020.
  * Nguyen et al. [2022] Tuan-Duy H. Nguyen, Ngoc Bui, Duy Nguyen, Man-Chung Yue, and Viet Anh Nguyen.  Robust bayesian recourse.  In UAI 2022, pages 1498–1508, 2022.
  * Nielsen and Parsons [2006] Søren Holbech Nielsen and Simon Parsons.  A generalization of dung’s abstract framework for argumentation: Arguing with sets of attacking arguments.  In ArgMAS 2006, pages 54–73, 2006.
  * Pawelczyk et al. [2020] Martin Pawelczyk, Klaus Broelemann, and Gjergji Kasneci.  On counterfactual explanations under predictive multiplicity.  In UAI 2020, pages 809–818, 2020.
  * Potyka et al. [2023] Nico Potyka, Xiang Yin, and Francesca Toni.  Explaining random forests using bipolar argumentation and markov networks.  In AAAI 2023, pages 9453–9460, 2023.
  * Potyka [2021] Nico Potyka.  Interpreting neural networks as quantitative argumentation frameworks.  In AAAI 2021, pages 6463–6470, 2021.
  * Rodolfa et al. [2021] Kit T. Rodolfa, Hemank Lamba, and Rayid Ghani.  Empirical observation of negligible fairness-accuracy trade-offs in machine learning for public policy.  Nat. Mach. Intell., 3(10):896–904, 2021.
  * Roth et al. [2023] Aaron Roth, Alexander Tolbert, and Scott Weinstein.  Reconciling individual probability forecasts.  In FAccT 2023, pages 101–110, 2023.
  * Rudin [2019] Cynthia Rudin.  Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.  Nat. Mach. Intell., 1(5):206–215, 2019.
  * Semenova et al. [2022] Lesia Semenova, Cynthia Rudin, and Ronald Parr.  On the existence of simpler machine learning models.  In FAccT 2022, pages 1827–1858, 2022.
  * Timmer et al. [2015] Sjoerd T. Timmer, John-Jules Ch. Meyer, Henry Prakken, Silja Renooij, and Bart Verheij.  Explaining bayesian networks using argumentation.  In ECSQARU 2015, pages 83–92, 2015.
  * Tolomei et al. [2017] Gabriele Tolomei, Fabrizio Silvestri, Andrew Haines, and Mounia Lalmas.  Interpretable predictions of tree-based ensembles via actionable feature tweaking.  In KDD 2017, pages 465–474, 2017.
  * Upadhyay et al. [2021] Sohini Upadhyay, Shalmali Joshi, and Himabindu Lakkaraju.  Towards robust and reliable algorithmic recourse.  In NeurIPS 2021, pages 16926–16937, 2021.
  * Ustun et al. [2019] Berk Ustun, Alexander Spangher, and Yang Liu.  Actionable recourse in linear classification.  In FAT 2019, pages 10–19, 2019.
  * Vassiliades et al. [2021] Alexandros Vassiliades, Nick Bassiliades, and Theodore Patkos.  Argumentation and explainable artificial intelligence: a survey.  Knowl. Eng. Rev., 36:e5, 2021.
  * Wachter et al. [2017] Sandra Wachter, Brent D. Mittelstadt, and Chris Russell.  Counterfactual explanations without opening the black box: Automated decisions and the GDPR.  Harv. JL & Tech., 31:841, 2017.
  * Watson-Daniels et al. [2023] Jamelle Watson-Daniels, David C. Parkes, and Berk Ustun.  Predictive multiplicity in probabilistic classification.  In AAAI 2023, pages 10306–10314, 2023.
  * Wick et al. [2019] Michael L. Wick, Swetasudha Panda, and Jean-Baptiste Tristan.  Unlocking fairness: a trade-off revisited.  In NeurIPS 2019, pages 8780–8789, 2019.
  * Xin et al. [2022] Rui Xin, Chudi Zhong, Zhi Chen, Takuya Takagi, Margo I. Seltzer, and Cynthia Rudin.  Exploring the whole rashomon set of sparse decision trees.  In NeurIPS 2022, pages 14071–14084, 2022.
  * Zeng et al. [2019] Zhiwei Zeng, Chunyan Miao, Cyril Leung, Zhiqi Shen, and Jing Jih Chin.  Computing argumentative explanations in bipolar argumentation frameworks.  In AAAI 2019, pages 10079–10080, 2019.

## Appendices

##  Appendix A Dataset and Training Details

The neural networks we train for all datasets are with two hidden layers of
sizes {(15, 15), (20, 15), (20, 20), (30, 20), (30, 25)}. The five different
sizes are for obtaining five levels of model simplicity which serves as a toy
property evaluation to demonstrate the usefulness of supporting model
preferences in our framework. We observe no significant changes in test
accuracies across the different model sizes, as the accuracy variations fall
within the range of standard deviation. We use the sklearn implementation of
neural networks for the experiment (https://scikit-
learn.org/stable/modules/generated/
sklearn.neural_network.MLPClassifier.html) and train with a batch size of 64.
All experiments are executed on a standard Windows machine with an Intel Core
i7-12700H CPU with 40GB memory.

The heloc dataset has 9871 data points with 21 continuous features. The 5-fold
cross-validation accuracies for each size are, respectively, .730$\pm$.015,
.738$\pm$.009, .741$\pm$.008, .734$\pm$.007, .736$\pm$.009.

The compas dataset has 6172 data points with 3 discrete and 4 continuous
features. The 5-fold cross-validation accuracies for each size are
.844$\pm$.013, .841$\pm$.013, .845$\pm$.011, .840$\pm$.012, .845$\pm$.012.

There are 1000 data points and 20 categorical features in the credit dataset.
The 5-fold cross-validation accuracies for each size are .709$\pm$.028,
.699$\pm$.007, .705$\pm$.018, .697$\pm$.011, .696$\pm$.021.

##  Appendix B When Multiple Solutions Exist

| multiple. | same. | multiple. | same. | multiple. | same.
---|---|---|---|---|---|---
| heloc | compas | credit
$|\mathcal{M}|=10$ |  |  |
$S^{n}$ | .022$\pm$.003 | .000$\pm$.000 | .005$\pm$.004 | .000$\pm$.000 | .074$\pm$.016 | .000$\pm$.000
$S^{v}$ | .022$\pm$.003 | .000$\pm$.000 | .005$\pm$.004 | .000$\pm$.000 | .074$\pm$.016 | .000$\pm$.000
$S^{a}$ | .164$\pm$.002 | .145$\pm$.004 | .256$\pm$.008 | .248$\pm$.008 | .100$\pm$.015 | .046$\pm$.013
$S^{a}$-A | .000$\pm$.000 | .000$\pm$.015 | .084$\pm$.010 | .083$\pm$.103 | .015$\pm$.029 | .010$\pm$.019
$S^{a}$-S | .097$\pm$.005 | .088$\pm$.026 | .212$\pm$.012 | .207$\pm$.012 | .072$\pm$.025 | .036$\pm$.008
$S^{a}$-AS | .137$\pm$.002 | .122$\pm$.005 | .211$\pm$.005 | .206$\pm$.005 | .089$\pm$.019 | .040$\pm$.010
$|\mathcal{M}|=20$ |  |  |
$S^{n}$ | .009$\pm$.002 | .000$\pm$.000 | .002$\pm$.002 | .000$\pm$.000 | .033$\pm$.006 | .000$\pm$.000
$S^{v}$ | .009$\pm$.002 | .000$\pm$.000 | .002$\pm$.002 | .000$\pm$.000 | .033$\pm$.006 | .000$\pm$.000
$S^{a}$ | .105$\pm$.006 | .095$\pm$.006 | .178$\pm$.052 | .174$\pm$.048 | .057$\pm$.006 | .023$\pm$.006
$S^{a}$-A | .018$\pm$.035 | .016$\pm$.033 | .027$\pm$.054 | .027$\pm$.053 | .014$\pm$.020 | .007$\pm$.012
$S^{a}$-S | .078$\pm$.039 | .072$\pm$.037 | .136$\pm$.028 | .132$\pm$.028 | .047$\pm$.009 | .024$\pm$.011
$S^{a}$-AS | .102$\pm$.009 | .093$\pm$.009 | .146$\pm$.024 | .142$\pm$.022 | .059$\pm$.006 | .024$\pm$.007
$|\mathcal{M}|=30$ |  |  |
$S^{n}$ | .009$\pm$.002 | .000$\pm$.000 | .002$\pm$.001 | .000$\pm$.000 | .022$\pm$.008 | .000$\pm$.000
$S^{v}$ | .009$\pm$.002 | .000$\pm$.000 | .002$\pm$.001 | .000$\pm$.000 | .022$\pm$.008 | .000$\pm$.000
$S^{a}$ | .078$\pm$.013 | .072$\pm$.011 | .133$\pm$.032 | .130$\pm$.032 | .038$\pm$.013 | .019$\pm$.009
$S^{a}$-A | .000$\pm$.000 | .000$\pm$.000 | .035$\pm$.050 | .035$\pm$.050 | .010$\pm$.020 | .005$\pm$.010
$S^{a}$-S | .066$\pm$.019 | .062$\pm$.016 | .093$\pm$.011 | .091$\pm$.011 | .030$\pm$.009 | .012$\pm$.005
$S^{a}$-AS | .076$\pm$.012 | .070$\pm$.012 | .102$\pm$.022 | .098$\pm$.021 | .038$\pm$.012 | .019$\pm$.007
Table 3: The proportion of inputs for which multiple solutions and multiple
solutions with the same prediction label exist.

We report for all ensembling methods the proportion of test inputs for which
multiple solutions exist in Table 3 (_multiple._). In the case of naive
ensembling, whenever multiple solutions are found, they have different
prediction labels. For our argumentative ensembling method, it could be the
case that multiple solutions give the same prediction result in which case the
desirable properties satisfaction is not compromised, we thus also include the
proportion of inputs where this happens (reported as _same._).

Though the number of cases when $|X_{max}|>1$ can be relatively high for our
method, they mostly provide the same prediction results for the input, in
which case the only difference for the end user (individual data subject) is
the recourse they could receive between the multiple solutions. On the other
hand, compared with naive ensembling methods, it is less frequent for our
method to find equally sized ensembles which disagree on the prediction for
the input, observed by $(multiple.-same.)$. As mentioned in §5, in practice,
one could use all viable resulting ensembles to make more informed decisions
about the individual. Therefore, when multiple answers exist, argumentative
ensembling provides extra flexibility for its users.

##  Appendix C Full Evaluation Results

We report the full results of Table 2 including standard deviations in Tables
4 to 6.

| acc. | simp. |  size M/C |  $\mathbf{c}$ val. (fail) | mv.
---|---|---|---|---|---
| heloc
$|\mathcal{M}|=10$ | .709$\pm$.003
$S^{n}$ | .709$\pm$.005 | .495$\pm$.122 | .943$\pm$.003/.943$\pm$.003 | .657$\pm$.020 (.00$\pm$.000) | 1.00$\pm$.000
$S^{v}$ | .709$\pm$.005 | .495$\pm$.122 | .943$\pm$.003/.309$\pm$.028 | 1.00$\pm$.000 (.34$\pm$.059) | 1.00$\pm$.000
$S^{a}$ | .712$\pm$.005 | .504$\pm$.148 | .499$\pm$.016/.499$\pm$.016 | 1.00$\pm$.000 (.00$\pm$.000) | .983$\pm$.004
$S^{a}$-A | .726$\pm$.006 | .485$\pm$.229 | .357$\pm$.048/.357$\pm$.048 | 1.00$\pm$.000 (.00$\pm$.000) | .943$\pm$.015
$S^{a}$-S | .710$\pm$.007 | .608$\pm$.085 | .462$\pm$.035/.462$\pm$.035 | 1.00$\pm$.000 (.00$\pm$.000) | .967$\pm$.026
$S^{a}$-AS | .712$\pm$.005 | .528$\pm$.135 | .493$\pm$.017/.493$\pm$.017 | 1.00$\pm$.000 (.00$\pm$.000) | .980$\pm$.005
$|\mathcal{M}|=20$ | .710$\pm$.003
$S^{n}$ | .717$\pm$.007 | .488$\pm$.132 | .940$\pm$.004/.940$\pm$.004 | .626$\pm$.013 (.00$\pm$.000) | 1.00$\pm$.000
$S^{v}$ | .717$\pm$.007 | .388$\pm$.132 | .940$\pm$.004/.230$\pm$.021 | 1.00$\pm$.000 (.37$\pm$.035) | 1.00$\pm$.000
$S^{a}$ | .716$\pm$.006 | .466$\pm$.144 | .460$\pm$.016/.460$\pm$.016 | 1.00$\pm$.000 (.00$\pm$.000) | .984$\pm$.005
$S^{a}$-A | .728$\pm$.006 | .432$\pm$.149 | .361$\pm$.024/.361$\pm$.024 | 1.00$\pm$.000 (.00$\pm$.000) | .950$\pm$.010
$S^{a}$-S | .711$\pm$.005 | .551$\pm$.094 | .420$\pm$.031/.420$\pm$.031 | 1.00$\pm$.000 (.00$\pm$.000) | .966$\pm$.016
$S^{a}$-AS | .715$\pm$.005 | .473$\pm$.138 | .459$\pm$.016/.459$\pm$.016 | 1.00$\pm$.000 (.00$\pm$.000) | .984$\pm$.005
$|\mathcal{M}|=30$ | .710$\pm$.003
$S^{n}$ | .718$\pm$.004 | .512$\pm$.085 | .940$\pm$.003/.940$\pm$.003 | .620$\pm$.015 (.00$\pm$.000) | 1.00$\pm$.000
$S^{v}$ | .718$\pm$.004 | .512$\pm$.085 | .940$\pm$.003/.205$\pm$.013 | 1.00$\pm$.000 (.41$\pm$.045) | 1.00$\pm$.000
$S^{a}$ | .716$\pm$.004 | .499$\pm$.088 | .456$\pm$.017/.456$\pm$.017 | 1.00$\pm$.000 (.00$\pm$.000) | .982$\pm$.006
$S^{a}$-A | .729$\pm$.005 | .406$\pm$.069 | .353$\pm$.023/.353$\pm$.023 | 1.00$\pm$.000 (.00$\pm$.000) | .946$\pm$.009
$S^{a}$-S | .712$\pm$.005 | .518$\pm$.078 | .445$\pm$.011/.445$\pm$.011 | 1.00$\pm$.000 (.00$\pm$.000) | .978$\pm$.004
$S^{a}$-AS | .716$\pm$.004 | .500$\pm$.088 | .456$\pm$.017/.456$\pm$.017 | 1.00$\pm$.000 (.00$\pm$.000) | .982$\pm$.006
Table 4: Quantitative evaluations of ensembling methods on heloc dataset. | acc. | simp. |  size M/C |  $\mathbf{c}$ val. (fail) | mv
---|---|---|---|---|---
| compas
$|\mathcal{M}|=10$ | .856$\pm$.001
$S^{n}$ | .858$\pm$.001 | .464$\pm$.127 | .980$\pm$.001/.980$\pm$.001 | .572$\pm$.008 (.00$\pm$.000) | 1.00$\pm$.000
$S^{v}$ | .858$\pm$.001 | .464$\pm$.127 | .980$\pm$.001/.174$\pm$.019 | 1.00$\pm$.000 (.51$\pm$.097) | 1.00$\pm$.000
$S^{a}$ | .859$\pm$.004 | .463$\pm$.137 | .369$\pm$.021/.369$\pm$.021 | 1.00$\pm$.000 (.00$\pm$.000) | .994$\pm$.002
$S^{a}$-A | .864$\pm$.003 | .430$\pm$.126 | .295$\pm$.042/.295$\pm$.042 | 1.00$\pm$.000 (.00$\pm$.000) | .988$\pm$.003
$S^{a}$-S | .860$\pm$.002 | .657$\pm$.099 | .306$\pm$.036/.306$\pm$.036 | 1.00$\pm$.000 (.00$\pm$.000) | .987$\pm$.004
$S^{a}$-AS | .860$\pm$.005 | .501$\pm$.126 | .360$\pm$.023/.360$\pm$.023 | 1.00$\pm$.000 (.00$\pm$.000) | .994$\pm$.002
$|\mathcal{M}|=20$ | .855$\pm$.001
$S^{n}$ | .859$\pm$.002 | .538$\pm$.112 | .978$\pm$.002/.978$\pm$.002 | .544$\pm$.004 (.00$\pm$.000) | 1.00$\pm$.000
$S^{v}$ | .859$\pm$.002 | .538$\pm$.112 | .978$\pm$.002/.111$\pm$.021 | 1.00$\pm$.000 (.60$\pm$.066) | 1.00$\pm$.000
$S^{a}$ | .859$\pm$.004 | .514$\pm$.115 | .331$\pm$.018/.331$\pm$.018 | 1.00$\pm$.000 (.00$\pm$.000) | .992$\pm$.006
$S^{a}$-A | .866$\pm$.001 | .541$\pm$.140 | .235$\pm$.027/.235$\pm$.027 | 1.00$\pm$.000 (.00$\pm$.000) | .982$\pm$.008
$S^{a}$-S | .857$\pm$.005 | .609$\pm$.070 | .304$\pm$.030/.304$\pm$.030 | 1.00$\pm$.000 (.00$\pm$.000) | .987$\pm$.004
$S^{a}$-AS | .859$\pm$.004 | .555$\pm$.111 | .324$\pm$.025/.324$\pm$.025 | 1.00$\pm$.000 (.00$\pm$.000) | .990$\pm$.006
$|\mathcal{M}|=30$ | .855$\pm$.001
$S^{n}$ | .859$\pm$.001 | .527$\pm$.094 | .976$\pm$.002/.976$\pm$.002 | .530$\pm$.005 (.00$\pm$.000) | 1.00$\pm$.000
$S^{v}$ | .859$\pm$.001 | .527$\pm$.094 | .976$\pm$.002/.087$\pm$.014 | 1.00$\pm$.000 (.56$\pm$.008) | 1.00$\pm$.000
$S^{a}$ | .862$\pm$.004 | .519$\pm$.097 | .308$\pm$.010/.308$\pm$.010 | 1.00$\pm$.000 (.00$\pm$.000) | .990$\pm$.005
$S^{a}$-A | .865$\pm$.002 | .532$\pm$.142 | .225$\pm$.004/.225$\pm$.004 | 1.00$\pm$.000 (.00$\pm$.000) | .983$\pm$.008
$S^{a}$-S | .861$\pm$.005 | .567$\pm$.080 | .294$\pm$.018/.294$\pm$.018 | 1.00$\pm$.000 (.00$\pm$.000) | .988$\pm$.006
$S^{a}$-AS | .862$\pm$.004 | .543$\pm$.099 | .303$\pm$.016/.303$\pm$.016 | 1.00$\pm$.000 (.00$\pm$.000) | .988$\pm$.007
Table 5: Quantitative evaluations of ensembling methods on compas dataset. | acc. | simp. |  size M/C |  $\mathbf{c}$ val. (fail) | mv
---|---|---|---|---|---
| credit
$|\mathcal{M}|=10$ | .664$\pm$.007
$S^{n}$ | .697$\pm$.019 | .588$\pm$.152 | .817$\pm$.005/.817$\pm$.005 | .757$\pm$.013 (.00$\pm$.000) | 1.00$\pm$.000
$S^{v}$ | .697$\pm$.019 | .588$\pm$.152 | .817$\pm$.005/.457$\pm$.010 | 1.00$\pm$.000 (.44$\pm$.044) | 1.00$\pm$.000
$S^{a}$ | .694$\pm$.011 | .600$\pm$.147 | .580$\pm$.012/.580$\pm$.012 | 1.00$\pm$.000 (.00$\pm$.000) | .953$\pm$.001
$S^{a}$-A | .710$\pm$.003 | .593$\pm$.186 | .486$\pm$.044/.486$\pm$.044 | 1.00$\pm$.000 (.00$\pm$.000) | .825$\pm$.005
$S^{a}$-S | .689$\pm$.010 | .626$\pm$.132 | .565$\pm$.020/.565$\pm$.020 | 1.00$\pm$.000 (.00$\pm$.000) | .925$\pm$.021
$S^{a}$-AS | .696$\pm$.016 | .607$\pm$.148 | .578$\pm$.011/.578$\pm$.011 | 1.00$\pm$.000 (.00$\pm$.000) | .946$\pm$.004
$|\mathcal{M}|=20$ | .663$\pm$.004
$S^{n}$ | .708$\pm$.012 | .571$\pm$.006 | .810$\pm$.004/.810$\pm$.004 | .734$\pm$.010 (.00$\pm$.000) | 1.00$\pm$.000
$S^{v}$ | .708$\pm$.012 | .571$\pm$.006 | .810$\pm$.004/.351$\pm$.030 | 1.00$\pm$.000 (.62$\pm$.039) | 1.00$\pm$.000
$S^{a}$ | .691$\pm$.009 | .580$\pm$.006 | .557$\pm$.008/.557$\pm$.008 | 1.00$\pm$.000 (.00$\pm$.000) | .961$\pm$.001
$S^{a}$-A | .709$\pm$.011 | .586$\pm$.007 | .481$\pm$.046/.481$\pm$.046 | 1.00$\pm$.000 (.00$\pm$.000) | .862$\pm$.005
$S^{a}$-S | .684$\pm$.013 | .590$\pm$.006 | .549$\pm$.012/.549$\pm$.012 | 1.00$\pm$.000 (.00$\pm$.000) | .947$\pm$.010
$S^{a}$-AS | .693$\pm$.008 | .581$\pm$.006 | .556$\pm$.009/.556$\pm$.009 | 1.00$\pm$.000 (.00$\pm$.000) | .959$\pm$.012
$|\mathcal{M}|=30$ | .663$\pm$.004
$S^{n}$ | .710$\pm$.006 | .540$\pm$.040 | .807$\pm$.004/.807$\pm$.004 | .727$\pm$.007 (.00$\pm$.000) | 1.00$\pm$.000
$S^{v}$ | .710$\pm$.006 | .540$\pm$.040 | .807$\pm$.004/.311$\pm$.045 | 1.00$\pm$.000 (.70$\pm$.057) | 1.00$\pm$.000
$S^{a}$ | .683$\pm$.007 | .549$\pm$.039 | .551$\pm$.005/.551$\pm$.005 | 1.00$\pm$.000 (.00$\pm$.000) | .943$\pm$.022
$S^{a}$-A | .711$\pm$.007 | .552$\pm$.048 | .441$\pm$.051/.441$\pm$.051 | 1.00$\pm$.000 (.00$\pm$.000) | .850$\pm$.059
$S^{a}$-S | .684$\pm$.013 | .555$\pm$.034 | .546$\pm$.007/.546$\pm$.007 | 1.00$\pm$.000 (.00$\pm$.000) | .940$\pm$.022
$S^{a}$-AS | .683$\pm$.007 | .549$\pm$.039 | .551$\pm$.005/.551$\pm$.005 | 1.00$\pm$.000 (.00$\pm$.000) | .944$\pm$.023
Table 6: Quantitative evaluations of ensembling methods on credit dataset.