# Inferential models and the decision-theoretic implications of the validity
property

Ryan Martin111Department of Statistics, North Carolina State University;
<EMAIL_ADDRESS>
###### Abstract

Inferential models (IMs) are data-dependent, probability-like structures
designed to quantify uncertainty about unknowns. As the name suggests, the
focus has been on uncertainty quantification for inference, and on
establishing a validity property that ensures the IM is reliable in a specific
sense. The present paper develops an IM framework for decision problems and,
in particular, investigates the decision-theoretic implications of the
aforementioned validity property. I show that a valid IM’s assessment of an
action’s quality, defined by a Choquet integral, will not be too optimistic
compared to that of an oracle. This ensures that a valid IM tends not to favor
actions that the oracle doesn’t also favor, hence a valid IM is reliable for
decision-making too. In a certain special class of structured statistical
models, further connections can be made between the valid IM’s favored actions
and those favored by other more familiar frameworks, from which certain
optimality conclusions can be drawn. An important step in these decision-
theoretic developments is a characterization of the valid IM’s credal set in
terms of confidence distributions, which may be of independent interest.

_Keywords and phrases:_ Choquet integral; confidence distribution; imprecise
probability; fiducial inference; possibility measure.

##  1 Introduction

In scientific applications, often the goal is to learn some feature of a
system under investigation based on observed data. This was the setting
Fisher, (1973) had in mind when developing his widely-used framework for
scientific inference. In other applications, however, the primary focus might
be on decision-making, which can often be carried out successfully without
fully understanding the system under investigation. At least intuitively, if
one were able to reliably learn about the relevant features of the underlying
system, in the sense of Fisher, then good—perhaps even optimal—decisions ought
to be within reach. The goal of the present paper is to investigate this
connection between reliable or valid probabilistic inference and decision-
making in cases where inferences are not based on a precise probability
distribution, Bayesian or otherwise.

Inferential models (IMs) are data-dependent, probability-like structures
designed to provide uncertainty quantification about unknowns. Familiar
constructions of data-dependent probabilities, such as Bayesian (with genuine
or default prior distributions) and fiducial/confidence distributions (Xie and
Singh, 2013; Hannig et al., 2016), are examples of IMs, but so are other less
familiar constructions, such as Dempster–Shafer belief functions (Dempster,
2008; Shafer, 1976; Wasserman, 1990; Denœux and Li, 2018), possibility
measures (Balch, 2012; Hose and Hanss, 2021; Liu and Martin, 2021), or more
general kinds of imprecise probabilities (Augustin et al., 2014; Walley,
1991). In the most general case, the IM’s uncertainty quantification takes the
form of a data-dependent lower and upper probability defined on a class of
assertions/hypotheses about the unknown quantity. Then inference is made based
on the magnitudes of the lower/upper probabilities assigned to various
hypotheses, i.e., if the lower probability is large, then data supports the
hypothesis and, if upper probability is small, then data doesn’t support the
hypothesis. The quality of an IM should be judged based on how reliable its
uncertainty quantification is. A precise notion of reliability, called
validity, was first presented in Martin and Liu, 2015b ; Martin and Liu,
(2013), and later elaborated on in Martin, (2019); Martin, 2021a ; see Section
2.1 below. Roughly, validity requires that the IM’s data-dependent lower and
upper probabilities be suitably calibrated as functions of the data. More
precisely, the lower probabilities assigned to false hypotheses don’t tend to
be too large and, similarly, the upper probabilities assigned to true
hypotheses don’t tend to be too small. Validity also implies that hypothesis
tests and confidence regions derived from the IM exactly control frequentist
error rates, not just approximately or asymptotically.

The developments described above are all focused on the inference problem; to
my knowledge, the decision problem hasn’t yet been considered from the
perspective of valid IMs. To set the scene, a statistical decision problem
comes equipped with a set $\mathscr{A}$ of possible actions along with a loss
function that maps $a\in\mathscr{A}$ and a particular value $\vartheta$ of the
unknown to the penalty/reward that results by taking action $a$ when the state
of nature is $\vartheta$. If, in addition, a data-dependent probability
distribution is available to quantify uncertainty about the unknown, then it’s
common to choose an action that minimizes the corresponding expected loss.
Generalizations of this basic framework to cases where uncertainty is
quantified via an imprecise lower/upper probability have been developed, and
I’ll adopt the same approach here; that is, I’ll rank actions according to a
suitable upper expected loss, a Choquet integral (see Section 2.2). My main
focus, however, is on the decision-theoretic implications of the IM’s validity
property. As stated in the first paragraph above, if the IM reliably solves
the inference problem, then, in some sense, it ought to do the same for the
decision problem. The paper’s first key contribution is in Section 4, where I
show that the valid IM’s upper expected loss tends to be not too small
compared to that of a quasi-oracle who has partial information available about
the state of nature. This is relevant because, if the decision-maker is overly
optimistic about the quality of an action, i.e., his assessment is more
favorable than the quasi-oracle’s, then there’s a chance that he’ll make a
poor decision and incur a non-trivial loss. Theorems 2–3 show, however, that
validity implies the decision-maker’s and quasi-oracle’s assessments of
different actions tend not to be inconsistent and, consequently, the valid IM
is reliable for the decision problem too. That is, existence of an action that
the decision-maker deems to be relatively good and that the quasi-oracle deems
to be relatively poor is a rare event with respect to the sampling
distribution of the data.

An important step in these developments, and perhaps of independent interest,
is a characterization of the set of precise probabilities compatible with the
valid IM’s upper probability. Specifically, using existing results on
connections between the level sets of a possibility contour and calibrated
prediction sets (e.g., Destercke and Dubois, 2014, Proposition 4.1), Theorem 1
establishes that precise probabilities compatible with a valid IM are
confidence distributions. This characterization provides the following helpful
intuition about what a valid IM does: the valid IM’s upper probability
assigned to a hypothesis $A$ is the maximum probability assigned to $A$ by a
confidence distribution. The “maximum” operation implies that the valid IM is
more conservative than any individual confidence distribution, and it’s this
conservatism that makes the IM valid while the individual confidence
distributions are not. For statistical models having a special kind of
additional structure, like the functional models studied in Dawid and Stone,
(1982), the characterization result described above can be strengthened.
What’s relevant about this class of statistical models is that they’re
perfectly suited for Fisher’s fiducial argument, hence they admit a well-
defined fiducial distribution. Then Theorem 4 in Section 5 shows that this
fiducial distribution is where, in a certain sense, the aforementioned
“maximum” is attained. That is, in this special class of models, the fiducial
distribution is “maximal” among all the confidence distributions compatible
with the valid IM. An important decision-theoretic consequence of this
connection is Theorem 5, the paper’s second main contribution, which says
that, under certain conditions on the loss function, the valid IM’s upper
expected loss is minimized at the same action where the fiducial distributions
expected loss is minimized. Combined with the main result in Taraldsen and
Lindqvist, (2013) about decision rules derived from the fiducial distribution,
Theorem 5 implies that actions chosen by applying the minimize (upper)
expected loss principle to valid IMs leads to generally high-quality and, in
some cases, optimal decision rules. Therefore, while validity requires some
degree of conservatism, a valid IM is not so conservative that its recommended
actions are much, if at all, different from those actions recommended by more
familiar statistical frameworks which lack the validity property.

##  2 Background

###  2.1 Inferential models

Let $Y$ be the observable data, taking values in $\mathbb{Y}$, and let

$\mathscr{P}=\\{\mathsf{P}_{Y|\vartheta}:\vartheta\in\Theta\\},$

be the posited statistical model, which is just a family of probability
distributions on $\mathbb{Y}$ indexed by a parameter space $\Theta$. As is
customary, I’ll assume there is a true parameter value, $\theta\in\Theta$,
which is completely unknown. An observation $Y=y$ is made, which provides some
(limited) information about the unknowns, and the goal is to make inference
and/or decisions in the presence of uncertainty about $\theta$.

Inferential models (IMs) are data-dependent, probability-like structures
designed to provide uncertainty quantification about unknowns. To cover the
most general case where the IM output is something different from an
ordinary/precise/additive probability, suppose the IM returns a pair
$(\underline{\Pi}_{y},\overline{\Pi}_{y})$, which I’ll refer to as lower and
upper probabilities, respectively. Mathematically, both $\underline{\Pi}_{y}$
and $\overline{\Pi}_{y}$ are capacities in the sense of Choquet, (1954). But
since one is “lower” and the other is “upper,” the two are necessarily linked
together, so it suffices to describe only the upper probability,
$\overline{\Pi}_{y}$.

Towards this, let $\mathcal{A}\subseteq 2^{\Theta}$ be a collection of subsets
of $\Theta$ that represents all the assertions or hypotheses about $\theta$
can be considered; throughout, both $A$ and “$\theta\in A$” will be called
assertions. Specifically, I require $\mathcal{A}$ to contain $\varnothing$ and
$\Theta$, and be closed under complementation. Since the goal is
“probabilistic inference” on $\theta$, I’ll additionally require $\mathcal{A}$
to be at least as rich as the Borel $\sigma$-algebra on $\Theta$, so that it
contains all the singletons. That way, the IM output is like a “posterior
distribution” in the sense that it assigns (lower and upper) probabilities to
a large collection of interesting hypotheses. In general, a set function
$\gamma:\mathcal{A}\to[0,1]$ is a capacity if $\gamma(\varnothing)=0$,
$\gamma(\Theta)=1$, and $\gamma(A)\leq\gamma(A^{\prime})$ if $A\subseteq
A^{\prime}$. Here I’ll also require the capacity to be sub-additive in the
sense that

$\gamma(A\cup B)\geq\gamma(A)+\gamma(B),\quad\text{for all
$A,B\in\mathcal{A}$, with $A\cap B=\varnothing$}.$

This is a very mild requirement—all the standard upper probabilities models,
e.g., Shafer’s plausibility functions, are sub-additive—so has virtually no
effect the data analyst’s flexibility. For a given sub-additive
$\overline{\Pi}_{y}$, if the corresponding $\underline{\Pi}_{y}$ is defined as

$\underline{\Pi}_{y}(A)=1-\overline{\Pi}_{y}(A^{c}),$ (1)

then it immediately follows that

$\underline{\Pi}_{y}(A)\leq\overline{\Pi}_{y}(A),\quad\text{for all
$A\in\mathcal{A}$},$

hence it makes sense to call these lower and upper probabilities. If the above
inequality is an equality for all $A$, then the IM’s output is an ordinary
probability. In such cases, property (1) reduces to the familiar
complementation rule for ordinary probabilities.

The formal interpretation of lower and upper probability is as follows.
Imagine a situation where assertions about $\theta$ correspond to gambles,
i.e., where agents have the opportunity to buy or sell gambles $\$1(\theta\in
A)$ that pay out $1 if “$\theta\in A$” turns out to be true, and pay out $0
otherwise. Following Walley, (1991) and others, $\underline{\Pi}_{y}(A)$ can
be interpreted as agent’s maximum buying price for the gamble $\$1(\theta\in
A)$ and, similarly, $\overline{\Pi}_{y}(A)$ is his corresponding minimum
selling price. So, if the “real price” is smaller than
$\underline{\Pi}_{y}(A)$ or larger than $\overline{\Pi}_{y}(A)$, then the
agent would buy or sell the gamble, respectively; otherwise, to avoid sure
loss, he neither buys nor sells. When
$\underline{\Pi}_{y}\equiv\overline{\Pi}_{y}$, the buying and selling prices
coincide and the above description reduces to de Finetti’s classical
subjective interpretation of probability.

Nothing prevents this behavioral/gambling from carrying over into the
scientific inference or decision-making setting, and some would even argue
that it should carry over (e.g., Crane, 2018; Shafer, 2021). But since
gambling terminology is rarely used in scientific contexts, I should offer the
following alternative interpretation:

  * •

for a hypothesis $A\subseteq\Theta$ about the unknown $\theta$, the lower
probability $\underline{\Pi}_{y}(A)$ represents the data analyst’s degree of
belief in the truthfulness of the assertion “$\theta\in A$,” given data $y$
and the posited model;

  * •

similarly, the upper probability $\overline{\Pi}_{y}(A)$ the data analyst’s
assessment of how plausible the truthfulness of “$\theta\in A$” is based on
the data $y$ and posited model.

Armed with this, the data analyst would behave very much like the agent in the
gambling situation above, just without a notion of “real price.” That is, if
$\underline{\Pi}_{y}(A)$ is relatively large, then the data strongly supports
$A$, so infer $A$; similarly, if $\overline{\Pi}_{y}(A)$ is small, then the
data is inconsistent with $A$, so infer $A^{c}$. For the many real-life
situations in which $\underline{\Pi}_{y}(A)$ and $\overline{\Pi}_{y}(A)$ are
relatively small and large, respectively, the data just isn’t sufficiently
informative to support either $A$ or $A^{c}$. In such situations, like the
cautious gambler who prefers not participating in a gamble to losing, the data
analyst ought to consider a “don’t know” conclusion (e.g., Dempster, 2008),
switch focus to a more basic assertion, $A^{\prime}$, and/or collect more
informative data.

Beyond simply constructing a data-dependent (precise or imprecise)
probability, Martin, (2019); Martin, 2021a  argued that in order to be a
meaningful quantification of uncertainty, the IM must be valid in the sense
that its inference is reliable. For a given statistical model, $\mathscr{P}$,
and collection of assertions $\mathcal{A}\subseteq 2^{\Theta}$, the IM is said
to be valid if

$\sup_{\theta\in
A}\mathsf{P}_{Y|\theta}\\{\overline{\Pi}_{Y}(A)\leq\alpha\\}\leq\alpha,\quad\text{for
all $\alpha\in[0,1]$ and all $A\in\mathcal{A}$}.$ (2)

Under the present assumptions on $(\underline{\Pi}_{y},\overline{\Pi}_{y})$
and the support $\mathcal{A}$, there’s an equivalent definition of validity in
terms of the lower probability,

$\sup_{\theta\not\in
A}\mathsf{P}_{Y|\theta}\\{\underline{\Pi}_{Y}(A)>1-\alpha\\}\leq\alpha,\quad\text{for
all $\alpha\in[0,1]$ and all $A\in\mathcal{A}$},$

but here I’ll focus on the version in (2). The intuition behind these
conditions is as follows. Based on the interpretation of the lower and upper
probabilities offered above, the data analyst would be drawing an incorrect
inference if the the upper probability of a true assertion was small—or if the
lower probability of a false assertion was large. The validity property
ensures that these undesirable events are rare in the sense of having small
$\mathsf{P}_{Y|\theta}$-probability. Moreover, there is an explicit connection
between the meaning of “small/large” and “rare” since it’s the same $\alpha$
inside and outside of the curly braces in (2). Therefore, statistical methods
that satisfy the validity property are reliable in the sense that error rates
can be easily controlled. From the perspective of prices for gambles, the
validity property (2) ensures that the agent’s minimum selling price,
$\overline{\Pi}_{Y}(A)$, for the gamble $\$1(\theta\in A)$ won’t tend to be
too small when “$\theta\in A$” is true. A similar statement can be made about
the maximum buying price.

The “for all $A\in\mathcal{A}$” part of the validity condition is important
for several reasons. Here I’ll mention only that it allows for marginal
inference on any relevant quantity of interest. Suppose that $\phi(\theta)$ is
some feature of the unknown parameter that’s of interest, e.g., the norm of a
mean vector. Of course, assertions about $\phi(\theta)$ can be expressed as
assertions about $\theta$, i.e., $A=\\{\vartheta:\phi(\vartheta)\in B\\}$, for
$B\subseteq\phi(\Theta)$, and this could be covered by the property (2) for
all practically relevant choices of $\phi$ and $B$. Since it’s known (e.g.,
Fraser, 2011, 2013) that Bayesian and other additive IMs are mis-calibrated
for some (non-linear) functions $\phi$, there is reason to doubt that these
are valid. Indeed, it was shown in Balch et al., (2019) that additive IMs
suffer from what’s called false confidence, which implies they’re not valid in
the sense of (2). For further details about the false confidence phenomenon,
the reader is referred to Martin, (2019); Martin, 2021a . The point is: in
order to achieve validity, it’s necessary that the IM’s output be non-
additive.

How to construct a valid IM and what kind of (non-additive) structure does it
have? The original IM constructions presented in Martin and Liu, (2013);
Martin and Liu, 2015b  were based on user-defined random sets on an auxiliary
variable space. In particular, in the spirit of Fisher’s fiducial inference,
Martin and Liu proposed to express the statistical model,
$\mathsf{P}_{Y|\theta}$, in terms of an association, or data-generating
equation,

$Y=a(\theta,U),\quad U\sim\mathsf{P}_{U},$ (3)

where $a:\Theta\times\mathbb{U}\to\mathbb{Y}$ is a known mapping and $U$ is an
auxiliary variable with known distribution $\mathsf{P}_{U}$ supported
$\mathbb{U}$. The idea is that there are two aspects that influence the
observable data $Y$, namely, the structural part—how the model depends on
$\theta$—and the purely random part. In the formulation (3), these two parts
are controlled by $a$ and by $U$, respectively. A similar setup can be found
in Hannig et al., (2016) and elsewhere. What distinguishes Martin and Liu’s
approach from these others is the handling of uncertainty about the auxiliary
variable $U$ after the data $Y=y$ is observed. Fiducial-like approaches would
“continue to regard” (Dempster, 1963) the unobservable $U$ as a random
variable with distribution $\mathsf{P}_{U}$ after $Y=y$ is observed. However,
there’s no logical basis for such an assumption because, in fact, it’s known
that $U$ solves the deterministic equation $y=a(\theta,U)$, which would
typically be a $\mathsf{P}_{U}$-probability 0 event. In contrast, Martin and
Liu recognize the change in the status of $U$—from a random variable with
distribution $\mathsf{P}_{U}$ before $Y=y$ is observed, to a fixed-but-
unobserved realization after—and, correspondingly change how the uncertainty
about it is quantified. It’s difficult to say what’s the “correct”222I do have
some new ideas pertaining to this point, but I’ll report on these elsewhere.
way to quantify this uncertainty, but, fortunately, the solution is known even
if the logical justification is not. Indeed, Martin and Liu recognized that
valid inference on $\theta$ can be achieved by reliably predicting/guessing
the unobserved value of $U$ associated with the observed $Y=y$, and they
propose to carry out this “reliable” prediction using a random set
$\mathcal{U}$ with distribution $\mathsf{P}_{\mathcal{U}}$ supported on
subsets of $\mathbb{U}$. These random set-based predictions of the unobserved
value of $U$ get mapped to a new and $y$-dependent random set,
$\Theta_{y}(\mathcal{U})$, on the $\theta$-space through (3):

$\Theta_{y}(\mathcal{U})=\bigcup_{u\in\mathcal{U}}\\{\vartheta\in\Theta:y=a(\vartheta,u)\\}.$

That is, if $\mathcal{U}$ represents a set of plausible values for the
unobserved $U$, then any $\vartheta$ values consistent with the observed $Y=y$
and some $u\in\mathcal{U}$ are equally plausible. Under certain mild
conditions on the association (3) and on the random set $\mathcal{U}$, this
leads to

$\overline{\Pi}_{y}(A)=\mathsf{P}_{\mathcal{U}}\\{\Theta_{y}(\mathcal{U})\cap
A\neq\varnothing\\},\quad A\in\mathcal{A}.$

Under some very mild and verifiable conditions on the user-specified random
set $\mathcal{U}$, it can be shown that the $\overline{\Pi}_{y}$ as defined
above achieves the validity condition (2) with $\mathcal{A}=2^{\Theta}$; see,
e.g., Martin and Liu, (2013); Martin and Liu, 2015b  and Martin, (2019);
Martin, 2021a .

Being based on random sets, it turns out that the IM output
$(\underline{\Pi}_{y},\overline{\Pi}_{y})$ described above is a special belief
and plausiblity function pair (e.g., Molchanov, 2005; Nguyen, 2006; Cuzzolin,
2021), possessing a certain continuity property. It was later observed that,
for efficiency reasons, the random set $\mathcal{U}$ should be nested, which
makes the IM output $(\underline{\Pi}_{y},\overline{\Pi}_{y})$ even more
special: it has the mathematical form of a necessity and possibility measure
pair (e.g., Liu and Martin, 2021; Dubois and Prade, 1988; Dubois, 2006). This
connection between valid IMs and possibility measures is important because it
may allow for simpler and more direct constructions. Indeed, a possibility
measure $\overline{\Pi}_{y}$ is determined by its so-called possibility
contour function $\pi_{y}:\Theta\to[0,1]$, given by

$\pi_{y}(\vartheta)=\mathsf{P}_{\mathcal{U}}\\{\Theta_{y}(\mathcal{U})\ni\vartheta\\},\quad\vartheta\in\Theta,$

and the property

$\overline{\Pi}_{y}(A)=\sup_{\vartheta\in A}\pi_{y}(\vartheta),\quad
A\in\mathcal{A}.$ (4)

Consequently, the IM’s $100(1-\alpha)$% plausibility regions can be
conveniently expressed as

$C_{\alpha}(y)=\\{\vartheta:\pi_{y}(\vartheta)>\alpha\\},\quad\alpha\in[0,1].$
(5)

Validity implies that $C_{\alpha}(Y)$ is a nominal $100(1-\alpha)$% confidence
region in the sense that

$\inf_{\theta\in\Theta}\mathsf{P}_{Y|\theta}\\{C_{\alpha}(Y)\ni\theta\\}\geq
1-\alpha.$ (6)

In what follows, I’ll focus exclusively on IMs of this latter type, which are
both valid and whose upper probability $\overline{\Pi}_{y}$ has the
mathematical structure of a possibility measure. All of the examples of valid
IMs presented in, e.g., Martin and Liu, 2015b  are of this type, and the
recent results in Martin, 2021a  strongly suggest that valid IMs can only take
this form, so there are no practical consequences of this restriction.

###  2.2 Decision theory

Decision theory, especially the normative form, is a mathematical framework
analyzing the choices agents make in the presence of uncertainty. In
particular, decision theory aims describe the “optimal” behavior for an agent
faced with decision problem. This requires a way to compare the agent’s
strategies and to (at least partially) rank them by preference. Early efforts,
e.g, Bernoulli, (1954), went directly to considerations of utility and, in
particular, ranking strategies according to their expected utility and, hence,
defining the optimal strategy as one that maximizes expected utility. Other
authors took a different route, by considering general preference orders on
strategies, but reached effectively the same destination. For example, the
celebrated utility theorem of von Neumann and Morgenstern, (1947) says that if
the preference order satisfies certain rationality axioms, then there exists a
utility function such that the ranking of strategies according to preferences
is equivalent to the ranking by expected utility. Similar conclusions, under
different formulations, have been reached by others, e.g., Savage, (1972).
Hence, the maximize expected utility principle. In a statistical decision
problem, it is common to work with loss instead of utility, but the difference
is really only superficial. Basically, if $\ell$ is a loss function, then
$u=-\ell$ is an equivalent utility function. The negation, however, changes
the directions of preference so, for the statistical decision problem, the
maximize-expected-utility principle becomes the minimize-expected-loss
principle.

The remainder of this subsection is devoted to a description of the
statistical decision problem, including introducing the notation that will be
needed in what follows. I don’t have anything to add to the discussions
concerning the appropriateness or suitability of the minimize-expected-loss
principle. My focus, again, is on how the IM’s validity property manifests in
the context of decision-making so, in what follows, I’ll simply adopt the
standard minimize-expected-loss principle—and the appropriate and necessary
generalizations described below—without question.

Let $(\vartheta,a)\mapsto\ell_{a}(\vartheta)$ denote a loss function that
measures the loss incurred by taking an action $a\in\mathscr{A}$ when the
“state of nature” is $\vartheta$. Common examples of loss functions include
squared-error loss, with $\ell_{a}(\vartheta)=\|a-\vartheta\|^{2}$, and 0–1
loss, with

$\ell_{a}(\vartheta)=1(\vartheta\in H,a=1)+1(\vartheta\not\in H,a=0),$

where $H\subset\Theta$ is a (null) hypothesis, and $a=1$ and $a=0$ denote
“reject” and “do not reject” $H$, respectively. I’ll assume throughout that
the loss is non-negative, which is not unreasonable. While negative losses
might make sense in some contexts, these gains would be bounded and so the
loss could be made non-negative by adding a constant. This additive constant
won’t affect judgments of the relative quality of actions.

Of course, an oracle who knows the value of $\theta$ could easily compare the
different possible actions according to their corresponding loss values,
$\ell_{a}(\theta)$. I’ll refer to $a\mapsto\ell_{a}(\theta)$ as the oracle’s
assessment of action $a$. Naturally, the oracle would take the action
$a^{\star}=\arg\min_{a}\ell_{a}(\theta)$ and incur minimal loss. None of us
have oracle powers, however, so the value of $\theta$ is unknown and a
different approach is required. When uncertainty about $\theta$, given $Y=y$,
is quantified via an ordinary probability $Q_{y}$, like a Bayesian posterior
distribution (e.g., Berger, 1985; Ghosh et al., 2006) or a fiducial/confidence
distribution (Schweder and Hjort, 2002, 2016; Xie and Singh, 2013), a typical
strategy is to rank the candidate actions according to their expected loss,
$a\mapsto Q_{y}\ell_{a}$. Then the obvious analogue to the known-$\theta$ case
mentioned above is to select the action that minimizes the posterior expected
loss, $\hat{a}(y)=\arg\min_{a}Q_{y}\ell_{a}$.

Other authors have argued that requiring a single precise probability to
quantify uncertainty puts too much of a burden on the data analyst, and that a
decision-theoretic framework based on more general imprecise probabilities is
better suited for practical applications. Excellent reviews of decision theory
from an imprecise probability perspective can be found in Huntley et al.,
(2014) and Denœux, (2019). At the risk of oversimplification, here I’ll
explain only the key idea that will be relevant in what follows. Given a loss
function like above, if uncertainty is quantified via an imprecise
probability, then it’s only natural to extend the minimize-expected-loss
principle by replacing the expected loss, $Q_{y}\ell_{a}$, with an appropriate
generalization, and then optimizing to find the best action (e.g., Gilboa,
1987; Gilboa and Schmeidler, 1989). It was shown in Gilboa and Schmeidler,
(1994) that an appropriate generalization of the expected loss is obtained via
the Choquet integral; see Appendix C of Troffaes and de Cooman, (2014) for
full details, Section 4.1 of Denœux, (2019) for a summary, and Section 4 below
for the special case where the imprecise probability model is a possibility
measure. Under an imprecise probability framework, there are actually two
versions of “expected loss,” an upper and a lower. Given that the goal is to
choose actions for which the loss is small, and that imprecise probability
frameworks are largely motivated by conservatism, it’s only natural to define
the optimal action in such a framework to be that which minimizes the upper
expected loss. For reasons that will be made clear—see Equations (8) and (11)
below—this strategy is commonly referred to as minimax, and there is extensive
work about this in the literature on imprecise probability (e.g., Huntley et
al., 2014, and the references therein) and in robust Bayesian inference (e.g.,
Berger, 1984, 1985; Vidakovic, 2000). Finally, special decision-theoretic
attention has been paid to “partially consonant” belief models, which include
the (fully consonant) possibility measures considered here; for details, see
Walley, (1987) and Giang and Shenoy, (2011).

##  3 Credal set corresponding to a valid IM

Before jumping in to the decision-theoretic formulation, I’ll first present a
new perspective on how valid IMs and confidence distributions are related. I
believe that this characterization is of independent interest, but it will
also be helpful in what follows. Recall that, as mentioned at the end of
Section 2.1, my focus in the remainder of this paper is on cases where the
valid IM’s output has the form of a necessity and possibility measure pair, so
that $\overline{\Pi}_{y}$ below is determined by a possibility contour,
$\pi_{y}$, via (4).

Since the IM with upper probability $\overline{\Pi}_{y}$ is genuinely non-
additive, there is a collection of probability distributions on $\Theta$ that
are compatible with it. This set of probability measures compatible with
$\overline{\Pi}_{y}$ is called the credal set and is defined as

$\mathscr{C}(\overline{\Pi}_{y})=\\{Q_{y}:\text{$Q_{y}\in\mathbb{Q}$ and
$Q_{y}(A)\leq\overline{\Pi}_{y}(A)$ for all $A\in\mathscr{B}_{\Theta}$}\\},$

where $\mathbb{Q}=\mathbb{Q}(\Theta,\mathscr{B}_{\Theta})$ is the set of all
probability distributions on the measurable space
$(\Theta,\mathscr{B}_{\Theta})$, with $\mathscr{B}_{\Theta}$ the Borel
$\sigma$-algebra on $\Theta$. Though the distributions $Q_{y}$ in
$\mathscr{C}(\overline{\Pi}_{y})$ can depend on $y$, they may not be Bayesian
posterior distributions under any prior. It is possible, however, to give a
practically meaningful characterization of the members of
$\mathscr{C}(\overline{\Pi}_{y})$, which is the main goal of this section.

Recall the IM’s $100(1-\alpha)\%$ plausibility regions for $\theta$ defined in
(5), i.e., $C_{\alpha}(y)=\\{\vartheta:\pi_{y}(\vartheta)>\alpha\\}$. Since
the IM is valid, $\\{C_{\alpha}(y):\alpha\in[0,1]\\}$ is a family of nominal
confidence regions for $\theta$ in the sense that (6) holds. If an
ordinary/precise and necessarily data-dependent probability distribution
$Q_{y}$ satisfies

$Q_{y}\\{C_{\alpha}(y)\\}\geq 1-\alpha,\quad\text{for all $\alpha\in[0,1]$},$
(7)

then I’ll call it a confidence distribution. That is, I’m defining a
confidence distribution to be any probability measure that’s calibrated in the
sense that the probability it assigns to the confidence set $C_{\alpha}(y)$ is
at least the nominal coverage probability. This definition of confidence
distributions is a little different/stronger than that found in, e.g.,
Schweder and Hjort, (2002, 2016) and Xie and Singh, (2013). In these
references, the focus is primarily on the scalar-$\theta$ case, and a
confidence distribution is a data-dependent distribution function whose
upper-$\alpha$ quantile determines a $100(1-\alpha)$% confidence upper limit.
The difference is that the above references define a data-dependent
distribution to be a confidence distribution if its quantiles form nominal
confidence intervals, whereas I’m starting with a given confidence region
$C_{y}(\alpha)$—which could be in more than one dimension—and defining a
distribution to be a confidence distribution if it assigns probability at
least $1-\alpha$ to $C_{\alpha}(y)$ for each $\alpha\in[0,1]$. In the
scalar-$\theta$ case, if (7) holds, then $Q_{y}$ would be a confidence
distribution in the sense of, e.g., Xie and Singh, so this is a
generalization. A similar definition of confidence distributions is given in
Taraldsen, (2021, Sec. 2).

The advantage of this alternative perspective is that it provides some new
insights into what confidence distributions are. In particular, Theorem 1
below establishes that confidence distributions are simply those probability
measures compatible with the possibility measure $\overline{\Pi}_{y}$ from a
valid IM. Therefore, properties developed for the valid IM should have
implications for all compatible confidence distributions, and vice versa.

######  Theorem 1.

For a valid IM whose upper probability $\overline{\Pi}_{y}$ is a possibility
measure, the credal set $\mathscr{C}(\overline{\Pi}_{y})$ is precisely the set
of all confidence distributions satisfying (7).

###### Proof.

An interesting and useful characterization of the set of probabilities
compatible with a possibility measure, summarized in Proposition 4.1 of
Destercke and Dubois, (2014) and the subsequent discussion, is

$Q_{y}\in\mathscr{C}(\overline{\Pi}_{y})\iff\text{$Q_{y}\\{C_{\alpha}(y)\\}\geq
1-\alpha$ for all $\alpha\in[0,1]$}.$

The claim now follows from my definition (7) of a confidence distribution. ∎

This general interpretation of the credal set
$\mathscr{C}(\overline{\Pi}_{y})$ as the set of confidence distributions will
be important in Section 4. I’ll revisit this connection in Section 5 where,
with some additional structure on the statistical model, a much more precise
characterization of the credal set can be given. This, in turn, will allow for
a better understanding of the decision-theoretic properties developed next.

##  4 Inferential models and decisions

###  4.1 Setup

I’ll start here by developing the decision-theoretic framework for IMs. Since
imprecise or non-additive probabilities are often used, even an appropriate
extension of basic notions like “expected loss” requires some care. Recall
that the decision problem comes equipped with a loss function
$\ell_{a}(\vartheta)$ that maps $(a,\vartheta)\in\mathscr{A}\times\Theta$ to
non-negative real numbers. In the familiar Bayesian case, one considers a
posterior expected loss, $Q_{y}\ell_{a}$, of an action $a$, where $Q_{y}$ is
the $y$-dependent posterior distribution on $\Theta$. A similar thing can be
done if the Bayesian posterior is replaced by a fiducial or confidence
distribution. For more general kinds of IMs, those that quantify uncertainty
via an imprecise probability $(\underline{\Pi}_{y},\overline{\Pi}_{y})$, a
different formulation is required.

If one thinks of an upper probability $\overline{\Pi}_{y}$ as an upper bound
on a collection of ordinary probabilities, then its extension to an upper
prevision/expectation is via

$\overline{\Pi}_{y}f=\sup\\{Q_{y}f:Q_{y}\in\mathscr{C}(\overline{\Pi}_{y})\\},$
(8)

where $f$ is an appropriate real-valued function defined on $\Theta$. In light
of Theorem 1, the upper envelope has some further intuition, that is,
$\overline{\Pi}_{y}f$ is largest of the ordinary expected values, $Q_{y}f$,
corresponding to confidence distributions $Q_{y}$ compatible with
$\overline{\Pi}_{y}$. There is an associated lower prevision,
$\underline{\Pi}_{y}f$, which is given by
$\underline{\Pi}_{y}f=-\overline{\Pi}_{y}(-f)$, but this won’t be considered
in what follows.

According to, e.g., Chapter 7 in Troffaes and de Cooman, (2014), extensions of
an upper probability/plausibility to an upper prevision—which is akin to the
jump from probability to expected value via Lebesgue integral in the classical
theory—is carried out via the Choquet integral. As stated in Section 2.1, the
focus here is primarily on IMs whose upper probability $\overline{\Pi}_{y}$ is
a possibility measure. For these, Propositions 7.14 and 15.42, in increasing
generality, establish that the Choquet integral of a non-negative function
$f:\Theta\to\mathbb{R}$ with respect to the possibility measure
$\overline{\Pi}_{y}$, if it exists, is given by

$\overline{\Pi}_{y}f=\int_{0}^{1}\sup\\{f(\vartheta):\pi_{y}(\vartheta)>\alpha\\}\,d\alpha,$
(9)

where $\pi_{y}$ is the possibility contour corresponding to
$\overline{\Pi}_{y}$. The integral in the above display is a Riemann integral,
which exists because the integrand is monotone in $\alpha$. Existence of the
Choquet integral requires that the function $f$ be “previsible” in the sense
of Definition 15.6 in Troffaes and de Cooman, (2014), which is a notion of
integrability with respect to a lower/upper prevision.333Roughly,
previsibility is a growth condition on the function $f$ relative to the upper
probability $\overline{\Pi}$. For example, according to Condition 15.31 in
Troffaes and de Cooman, (2014), $x\overline{\Pi}(\\{|f|\geq x\\})\to 0$ as
$x\to\infty$ and one other similar property implies previsibility of $f$ with
respect to $\overline{\Pi}$. In what follows, I’ll silently assume that all
loss functions are previsible in this sense. Finally, note that the Choquet
integral definition of the upper prevision is equivalent to the previous one
involving a supremum over $\mathscr{C}(\overline{\Pi}_{y})$, so that the
right-hand sides of (8) and (9) are equal.

Since the loss function is assumed to be non-negative, either of the above two
equivalent formulas can be applied directly to define an upper risk, or
expected loss,

$a\mapsto\overline{\Pi}_{y}\ell_{a}.$ (10)

Assuming $\overline{\Pi}_{y}\ell_{a}$ is finite at least for some actions $a$,
this can be used to assess the quality of different actions, relative to the
given loss function and the IM’s data-dependent possibility measure. In
particular, one can select a $y$-dependent “optimal” action as

$\hat{a}(y)=\arg\min_{a}\overline{\Pi}_{y}\ell_{a},$ (11)

which has a minimax connotation, being the action that minimizes the upper
expected loss. The rationale behind the use of upper instead of lower expected
loss is conservatism, i.e., if an action has small upper expected loss, then,
by (8), the corresponding expected loss with respect to any compatible
$Q_{y}\in\mathscr{C}(\overline{\Pi}_{y})$ is also small. Finally, as mentioned
briefly in Section 2.2 and in more detail in Denœux, (2019), the ranking of
actions based on magnitudes of the upper expected loss is consistent with
certain rationality axioms.

###  4.2 Implications of validity

Connections between validity and the no-sure-loss property in the sense of
Walley, (1991, Sec. 6.5.2) were recently made in Cella and Martin, 2021b ;
Cella and Martin, 2021a . But this doesn’t speak directly to the possible
connections between validity and the IM’s assessment of the quality of various
actions. Recall that validity ensures the user won’t make systematically
misleading inferences or, in other words, that incorrect inferences are
controllably rare. In the present context, a poor choice of action may result
if the IM’s assessment of $a$, based on $\overline{\Pi}_{y}\ell_{a}$ in (10),
were very different from the oracle’s assessment $\ell_{a}(\theta)$. That is,
I hope to avoid cases where, for some action $a$, my assessment
$\overline{\Pi}_{y}\ell_{a}$ in (10) of the loss associated with action $a$ is
much more optimistic than the oracle’s, i.e.,
$\overline{\Pi}_{y}\ell_{a}\ll\ell_{a}(\theta)$. Again, such cases are
undesirable because a situation where my assessment of an action is much more
optimistic than the oracle’s creates an unfortunate opportunity for me to
choose a poor action and potentially suffer a non-trivial loss.

To be more precise about what it means for the IM’s assessment,
$\overline{\Pi}_{y}\ell_{a}$, of action $a$ to be “much more optimistic” than
the oracle’s assessment, $\ell_{a}(\theta)$, define the data-dependent local
maximum loss,

$\displaystyle L_{a}(y,\theta)$
$\displaystyle=\sup\\{\ell_{a}(\vartheta):\vartheta\in
C_{\pi_{y}(\theta)}(y)\\}$
$\displaystyle=\sup\\{\ell_{a}(\vartheta):\pi_{y}(\vartheta)>\pi_{y}(\theta)\\},$
(12)

where the second equality follows by the definition (5) of the plausibility
region $C_{\alpha}(y)$. I’ll refer to $L_{a}(y,\theta)$ as the quasi-oracle’s
assessment of $a$, based on data $Y=y$. This corresponds to an oracle who
doesn’t know the exact value of $\theta$ but knows which values $\vartheta$
are at least as $\pi_{y}$-plausible as $\theta$ relative to data $Y=y$. Then
by “much more optimistic” I mean $\overline{\Pi}_{y}\ell_{a}$ being less than
a small multiple of $L_{a}(y,\theta)$, i.e.,

$\overline{\Pi}_{y}\ell_{a}\leq\alpha\,L_{a}(y,\theta),\quad\alpha\in(0,1).$

The proper oracle’s assessment, $\ell_{a}(\theta)$, is out of reach: its
magnitude is always much smaller than that of any reasonable data-driven
assessment. So it’s not meaningful to directly compare $\ell_{a}(\theta)$ and,
say, $\overline{\Pi}_{y}\ell_{a}$. However, it’s easy to see from (12) that
$L_{a}(y,\theta)\geq\ell_{a}(\theta)$, so the quasi-oracle’s assessment is
more conservative than the oracle’s and possibly within reach. Therefore, if
my IM’s assessment, $\overline{\Pi}_{y}\ell_{a}$, of the loss associated with
action $a$ tends not to be much more optimistic than the quasi-oracle’s
assessment, $L_{a}(y,\theta)$, then it also won’t tend to be much more
optimistic than the oracle’s assessment, $\ell_{a}(\theta)$. And not tending
to be much more optimistic than the oracle provides some assurance that poor
actions won’t be favored by the IM.

Below is (to my knowledge) the first result showing the implications of
validity in the context of decision theory. It says that the valid IM’s
assessment $\overline{\Pi}_{Y}\ell_{a}$ tends to be not considerably smaller
than $L_{a}(Y,\theta)$ for all actions $a\in\mathscr{A}$ and for any true
$\theta\in\Theta$.

######  Theorem 2.

Let $\ell_{a}:\Theta\to[0,\infty)$ be a non-negative loss function for each
$a$ in the action space $\mathscr{A}$. For an upper probability
$\overline{\Pi}_{y}$, define the minimum ratio

$R(y,\theta)=\inf_{a\in\mathscr{A}}\frac{\overline{\Pi}_{y}\ell_{a}}{L_{a}(y,\theta)},$

where $L_{a}(y,\theta)$ is as in (12). If the IM with upper probability
$\overline{\Pi}_{Y}$ is valid, then

$\sup_{\theta}\mathsf{P}_{Y|\theta}\\{R(Y,\theta)\leq\alpha\\}\leq\alpha,\quad\text{for
all $\alpha\in[0,1]$}.$

###### Proof.

Define the function

$h_{y,a}(\alpha)=\sup\\{\ell_{a}(\vartheta):\pi_{y}(\vartheta)>\alpha\\},$
(13)

so that the Choquet integral $\overline{\Pi}_{y}\ell_{a}$ is just a Riemann
integral of $h_{y,a}$. It’s clear that $\alpha\mapsto h_{y,a}(\alpha)$ is
decreasing, which implies

$\displaystyle\overline{\Pi}_{y}\ell_{a}$
$\displaystyle=\int_{0}^{1}h_{y,a}(\alpha)\,d\alpha$
$\displaystyle=\Bigl{(}\int_{0}^{\pi_{y}(\theta)}+\int_{\pi_{y}(\theta)}^{1}\Bigr{)}h_{y,a}(\alpha)\,d\alpha$
$\displaystyle\geq\pi(\theta)\,h_{y,a}(\pi_{y}(\theta)).$

This is effectively a Markov inequality for the Choquet integral, a special
case of that presented in Wang, (2011). Note that
$L_{a}(y,\theta)=h_{y,a}(\pi_{y}(\theta))$. Therefore,

$\displaystyle R(Y,\theta)\leq\alpha$
$\displaystyle\iff\overline{\Pi}_{Y}\ell_{a}\leq\alpha
L_{a}(Y,\theta)\;\text{for some $a$}$
$\displaystyle\implies\pi_{Y}(\theta)L_{a}(Y,\theta)\leq\alpha
L_{a}(Y,\theta)\;\text{for some $a$}$
$\displaystyle\iff\pi_{Y}(\theta)\leq\alpha.$

Validity implies the latter event has $\mathsf{P}_{Y|\theta}$-probability
$\leq\alpha$, uniformly in $\theta$. ∎

A simple consequence of Theorem 2, in particular, of the uniformity in actions
$a\in\mathscr{A}$, is that the event $\\{\overline{\Pi}_{Y}\ell_{a}\leq\alpha
L_{a}(Y,\theta)\\}$ has $\mathsf{P}_{Y|\theta}$-probability no more than
$\alpha$ for every action $a\in\mathscr{A}$. This includes data-dependent
actions, in particular, the “best” action
$\hat{a}(y)=\arg\min_{a}\overline{\Pi}_{y}\ell_{a}$, i.e.,

$\sup_{\theta}\mathsf{P}_{Y|\theta}\\{\overline{\Pi}_{Y}\ell_{\hat{a}(Y)}\leq\alpha
L_{\hat{a}(Y)}(Y,\theta)\\}\leq\alpha,\quad\text{for all $\alpha\in[0,1]$}.$

For a quick recap, recall that it would be undesirable if the IM’s assessment
of the quality of $a$ were much more optimistic than the oracle’s or the
quasi-oracle’s—it would put the decision-maker at risk of suffering non-
trivial loss. This is especially true for the “best” action, $a=\hat{a}(y)$,
the one that the decision-maker is likely to take. Theorem 2 shows that
validity of the IM implies that such undesirable cases are rare events with
respect to the true distribution of $Y$. Therefore, validity provides some
assurance that the IM’s data-driven assessment of action $a$ is not
inconsistent with that of the oracle or quasi-oracle, hence the IM helps the
decision-maker mitigate risk. This can be compared to the notion of “safety”
in Grünwald, (2018), e.g., his Theorem 4.

It turns out that the bound in Theorem 2 is rather conservative, at least in
the special cases considered in Section 4.3. One explanation for this
conservatism is that the theorem assumes almost nothing. The formulation
described below assumes a bit more about both the decision problem and the IM,
which leads to a stronger conclusion about the magnitude of
$\overline{\Pi}_{Y}\ell_{a}$ relative to a version of the oracle’s assessment.
Towards this, some more notation and terminology is needed, which I provide
next. I’ll also focus here on the case of a scalar $\theta$ since I’ve not yet
been able to work out the technical details in the more general case; see the
discussion below, and Appendix A.

First, I’ll need some special—but apparently fairly general—structure in the
possibility contour $\pi_{y}$, namely, that

  * •

$\pi_{y}$ is convex increasing to its maximum value 1;

  * •

constant equal to 1 on an interval (perhaps just a singleton);

  * •

and convex decreasing thereafter.

I’ve not found an existing name for such a property so, for lack of any better
options, I’ll refer to this as directional convexity, i.e., convex in both
directions away from the mode(s).444An appropriate generalization beyond the
scalar case being considered here is to assume convexity away from the mode(s)
in all directions. Both the possibility contours plotted in Section 4.3 below
are directionally convex, as are most555The only exceptions I’m aware of are
the discrete data problems (e.g., Martin et al., 2012) where a “most
efficient” possibility contour is used, which typically has some tiny local
modes and jump discontinuities. The recent work in Balch, (2020) was able to
remove these tiny modes, so only the presence of jump discontinuities would
need to be addressed, but I won’t attempt this here. of the others I’ve seen
in the literature.

Second, a more conservative version of the quasi-oracle’s assessment of the
quality of different actions is required. Intuitively, while the quasi-oracle
has much less information than the oracle who knows $\theta$, being able to
restrict attention to those $\vartheta$ values that are at least as
$\pi_{y}$-plausible as $\theta$ is also rather unrealistic. Therefore, even
comparing $\overline{\Pi}_{Y}\ell_{a}$ to $L_{a}(Y,\theta)$, with
$L_{a}(Y,\theta)\gg\ell_{a}(\theta)$, is unfair. A more conservative
comparison could be made if the quasi-oracle’s assessment was inflated by
optimizing over a larger set that includes some $\vartheta$ values that are
only “almost” as $\pi_{y}$-plausible as $\theta$. In particular, define

$\widetilde{L}_{a}(y,\theta)=\sup\\{\ell_{a}(\vartheta):\pi_{y}(\vartheta)>\tfrac{1}{2}\pi_{y}(\theta)\\}.$

Clearly, $\widetilde{L}_{a}(y,\theta)\geq L_{a}(y,\theta)$, so this modified
quasi-oracle assessment is a more conservative threshold. Then the IM’s
assessment, $\overline{\Pi}_{y}\ell_{a}$, not being “much more optimistic”
than the modified quasi-oracle’s assessment, $\widetilde{L}_{a}(y,\theta)$, is
expressed as

$\overline{\Pi}_{y}\ell_{a}\leq\alpha\,\widetilde{L}_{a}(y,\theta),\quad\alpha\in[0,1].$

Analogous to Theorem 2, the following result says that the IM’s assessment
being much more optimistic than the modified quasi-oracle’s, in the sense
described above, is a rare event relative to $\mathsf{P}_{Y|\theta}$.

######  Theorem 3.

Suppose that $\vartheta\mapsto\ell_{a}(\vartheta)$ is convex for each $a$ and
that the IM’s possibility contour $\pi_{y}$ is directionally convex in the
sense described above. Define the minimum ratio

$\widetilde{R}(y,\theta)=\inf_{a\in\mathscr{A}}\frac{\overline{\Pi}_{y}\ell_{a}}{\widetilde{L}_{a}(y,\theta)}.$
(14)

If the IM with upper probability $\overline{\Pi}_{Y}$ is valid, then

$\sup_{\theta}\mathsf{P}_{Y|\theta}\\{\widetilde{R}(Y,\theta)\leq\alpha\\}\leq\alpha,\quad\text{for
all $\alpha\in[0,1]$}.$

###### Proof.

Reinterpreting the Choquet integral $\overline{\Pi}_{y}\ell_{a}$ as an
ordinary expectation of $h_{y,a}(\alpha)$, defined in (13) with respect to
$\alpha\sim{\sf Unif}(0,1)$ leads to

$\displaystyle\overline{\Pi}_{y}\ell_{a}\geq\int_{0}^{\pi_{y}(\theta)}h_{y,a}(\alpha)\,d\alpha=\pi_{y}(\theta)\,\mathsf{E}\\{h_{y,a}(\alpha)\mid\alpha\leq\pi_{y}(\theta)\\}.$

Lemma 1 in Appendix A shows that, under the stated assumptions, $\alpha\mapsto
h_{y,a}(\alpha)$ is convex for each $y$ and $a$. Then it follows from Jensen’s
inequality that

$\overline{\Pi}_{y}\ell_{a}\geq\pi_{y}(\theta)\,h_{y,a}\bigl{(}\tfrac{\pi_{y}(\theta)}{2}\bigr{)}=\pi_{y}(\theta)\,\widetilde{L}_{a}(y,\theta).$

The remainder of the proof proceeds exactly like that of Theorem 2. That is,
$\widetilde{R}(Y,\theta)\leq\alpha$ implies $\pi_{Y}(\theta)\leq\alpha$, and
the latter event has probability not exceeding $\alpha$. ∎

As with Theorem 2, the take-away message is that validity helps to prevent the
IM’s assessment of an action from being overly optimistic compared to the
(modified) quasi-oracle’s assessment, and this helps to protect the decision-
maker from loss. The focus here was on scalar-$\theta$ problems but nothing
about the above proof depends on this. The challenge to extending beyond the
scalar case is proving the claim that $\alpha\mapsto h_{y,a}(\alpha)$ is
convex. I expect that this property holds more generally, at least under some
reasonable conditions, but, as of now, I’ve only been able to establish
convexity for the scalar case; see Appendix A for further details and
discussion on this.

Finally, it’s possible to say more about the decision-theoretic implications
of validity, e.g., how the IM’s suggested decision $\hat{a}(y)$ relates to the
“optimal” decision, but this requires some additional structure in the
statistical model; see Section 5 below.

###  4.3 Examples

####  4.3.1 Location model

Consider the case where $\mathsf{P}_{Y|\theta}$ is a location model, that is,

$Y=\theta+U,\quad U\sim\mathsf{P}_{U},$

where $\mathsf{P}_{U}$ a known distribution on $\mathbb{R}$, with distribution
and density functions $F$ and $f$, respectively. For technical reasons, I’ll
assume that $f$ is bounded and unimodal, strictly decreasing moving away from
the mode in either direction. Also, I’ll focus here on the scalar case, but
the case where $Y=(Y_{1},\ldots,Y_{n})$ is a vector of iid samples, each with
the same scalar location parameter $\theta$, can be handled similarly,
following a preliminary conditioning argument to reduce the problem to the
case analyzed here; see Section 5.

The IM construction is relatively straightforward for this problem; see Martin
and Liu, (2013). Here I’ll proceed using the more “modern” formulation in Liu
and Martin, (2021) that focuses directly on the possibility contours, rather
than the random set formulation in the earlier literature. Define the
possibility contour on the auxiliary space as follows:

$\pi(u)=\mathsf{P}_{U}\\{f(U)\leq f(u)\\},\quad u\in\mathbb{R}.$ (15)

This defines a possibility measure, $\overline{\Pi}$, on $\mathbb{R}$, through
(4). One important feature of this possibility measure is that it forms a
credal set $\mathscr{C}(\overline{\Pi})$ that contains $\mathsf{P}_{U}$, i.e.,
$\mathsf{P}_{U}(B)\leq\overline{\Pi}(B)$ for all measurable sets $B$. Another
important feature, which I’ll not discuss in detail, is that this particular
construction of $\overline{\Pi}$, using the density $f$ of $\mathsf{P}_{U}$ in
(15), is the maximally specific possibility measure corresponding to
$\mathsf{P}_{U}$ (e.g., Dubois et al., 2004, Theorem 3.2), hence is optimal in
a certain sense.

Next, by the extension principle of Zadeh, (1978), for the observed data
$Y=y$, the corresponding possibility contour on the parameter space is given
by

$\pi_{y}(\vartheta)=\pi(y-\vartheta),\quad\vartheta\in\mathbb{R}.$

As this is the first example, to keep things sufficiently simple that the
calculations can be done analytically, I’ll focus on the case where the
auxiliary variable distribution $\mathsf{P}_{U}$ is symmetric around 0 in the
sense that $f(-u)=f(u)$. Then the possibility contours can be simplified and,
in particular,

$\pi_{y}(\vartheta)=2\\{1-F(|y-\vartheta|)\\},\quad\vartheta\in\mathbb{R}.$

This defines an IM whose upper probability $\overline{\Pi}_{y}$ is given by
the supremum in (4). This is equivalent to the “optimal” IM construction
discussed in, e.g., Martin and Liu, (2013), based on the use of a nested and
symmetric random set $\mathcal{U}$. Moreover, it’s easy to check that the IM’s
$100(1-\alpha)$% plausibility intervals are

$C_{\alpha}(y)=\bigl{[}y-F^{-1}(1-\tfrac{\alpha}{2}),\,y+F^{-1}(1-\tfrac{\alpha}{2})\bigr{]},\quad\alpha\in[0,1].$

To define a decision problem, consider the case of “point estimation” where
the action space $\mathscr{A}$ is the parameter space, and the goal is to
select an action $\hat{a}(y)$, depending on data $y$, that’s best in the sense
of minimizing $a\mapsto\overline{\Pi}_{y}\ell_{a}$ for a suitable loss
function $\ell_{a}$. As is customary, I’ll use the squared error loss, i.e.,
$\ell_{a}(\vartheta)=(a-\vartheta)^{2}$. It’ll be clear in what follows that
the upper risk $\overline{\Pi}_{y}\ell_{a}$ is well-defined only if the
density $f$ has sufficiently thin tails that its variance $\mathsf{V}(U)$ is
finite, so I’ll assume this throughout what follows.

Figure 1: Plot of the possibility contour $\pi_{y}(\vartheta)$, for $y=0$,
when $\mathsf{P}_{U}$ is a Student-t distribution with 3 degrees of freedom;
dashed line shows the loss $\ell_{a}(\vartheta)$, when $a=2$.

Figure 1 shows both the contour function $\pi_{y}(\vartheta)$ and the loss
function $\ell_{a}(\vartheta)$ for specific values of $y$ and $a$, with $F$ a
Student-t distribution function. Note, in particular, that $\pi_{y}$ is
directionally convex and, hence, on any level set of $\pi_{y}$, the loss
function attains its maximum at the boundary. Using this observation, and
applying the Choquet integral formula in (9), I get

$\displaystyle\overline{\Pi}_{y}\ell_{a}$
$\displaystyle=\int_{0}^{1}\sup\bigl{[}(\vartheta-a)^{2}:2\\{1-F(|\vartheta-y|)\\}>\alpha\bigr{]}\,d\alpha$
$\displaystyle=\int_{0}^{1}\max\bigl{[}\\{y-a-F^{-1}(1-\tfrac{\alpha}{2})\\}^{2},\\{y-a+F^{-1}(1-\tfrac{\alpha}{2})\\}^{2}\bigr{]}\,d\alpha$
$\displaystyle=\int_{0}^{1}\bigl{\\{}|y-a|+F^{-1}(1-\tfrac{\alpha}{2})\bigr{\\}}^{2}\,d\alpha$
$\displaystyle=\int_{-\infty}^{\infty}\bigl{(}|y-a|+|u|\bigr{)}^{2}\,f(u)\,du$
$\displaystyle=(y-a)^{2}+2|y-a|\,\mathsf{E}|U|+\mathsf{V}(U).$

For comparison, there is a standard fiducial/confidence distribution for the
location problem, which is also the Bayesian posterior distribution under the
default, flat prior for $\theta$. That posterior, denoted by $Q_{y}^{\star}$,
has density function

$q_{y}^{\star}(\vartheta)=f(\vartheta-y),$

and it’s easy to check that the corresponding fiducial expected loss is

$Q_{y}^{\star}\ell_{a}=(y-a)^{2}+\mathsf{V}(U).$

Note that $Q_{y}^{\star}\ell_{a}\leq\overline{\Pi}_{y}\ell_{a}$ for all $a$,
with equality if and only if $a$ equals $y$. Indeed, the two expected loss
functions are equal at their common minimizer $\hat{a}(y)=y$. This minimizer
is Pitman’s optimal equivariant estimator, the solution suggested by any
reasonable approach to estimation of a location parameter. As I show in
Section 5, it’s not a coincidence that the valid IM’s suggested action based
on the minimum upper risk principle aligns with that of other methods in this
location model.

It’s a good sign that the IM’s choice of a best action, the one that minimizes
upper expected loss, aligns with the standard solution in this problem. Beyond
that, it’s of interest to compare the magnitude of IM’s assessment,
$\overline{\Pi}_{Y}\ell_{a}$, to that of the (modified) quasi-oracle’s
assessment, $\widetilde{L}_{a}(Y,\theta)$. Theorem 3 says that the former
won’t be too small compared to the latter, in the sense that
$\widetilde{R}(Y,\theta)$, defined in (14), is stochastically no smaller than
${\sf Unif}(0,1)$ as a function of $Y\sim\mathsf{P}_{Y|\theta}$. Figure 2
shows the distribution function of $\widetilde{R}(Y,\theta)$, i.e.,

$\alpha\mapsto\mathsf{P}_{Y|\theta}\\{\widetilde{R}(Y,\theta)\leq\alpha\\},\quad\alpha\in[0,1],$
(16)

and the fact that this curve is below the diagonal confirms the result in
Theorem 3. For comparison, Figure 2 also shows the distribution function of
the random variable

$\inf_{a\in\mathcal{A}}\frac{Q_{Y}^{\star}\ell_{a}}{\widetilde{L}_{a}(Y,\theta)},$

where $Q_{Y}^{\star}$ is the fiducial/confidence/Bayes posterior distribution
described above. As is clear from the plot, the claim in Theorem 3 does not
hold for the assessment based on the additive $Q_{Y}^{\star}$. That is, the
fiducial/Bayesian IM’s assessment $Q_{Y}^{\star}\ell_{a}$ tends to be a bit
too optimistic compared to the quasi-oracle’s. Whether violation of the
particular property in Theorem 3 has any practical consequences deserves
further investigation. The take-away message is that the IM solution is
sufficiently conservative in the sense that it provides provably valid
inference in the sense of (2), which implies that its assessment of actions is
not too optimistic in the sense of Theorem 3, yet efficient enough to clearly
identify the optimal action for the decision problem.

Figure 2: The black line shows the distribution function of
$\widetilde{R}(Y,\theta)$, as defined in (14), for the symmetric location
model IM described in Section 4.3.1. The gray line shows the distribution
function for the version of $\widetilde{R}(Y,\theta)$ for the fiducial/Bayes
IM. The diagonal line corresponds to the ${\sf Unif}(0,1)$ distribution
function.

There is nothing really special about the symmetric location problem, except
that symmetry makes the calculations more manageable. Here I’ll give a short
description of what happens in the case where the auxiliary variable
distribution is not symmetric. All of the calculations that follow are based
on Monte Carlo approximations.

Suppose that $U$ has a skew-normal distribution (e.g., Azzalini, 1985) with
slant parameter $\kappa$, so that the density is

$f(u)=\phi(u)\,\Phi(\kappa u),\quad u\in\mathbb{R},$

where $\phi$ and $\Phi$ denote the standard normal density and distribution
functions, respectively. Here I take $\kappa=3$, so that the distribution is
skewed to the right. I can follow the same approach to define a $u$-space
possibility contour $\pi$ and then setting
$\pi_{y}(\vartheta)=\pi(y-\vartheta)$ as before. A plot of the latter
possibility contour is shown in Figure 3(a), based on data $y=0$, and the
effect of the asymmetric density is apparent. for the decision problem, again
I’ll consider the squared error loss, $\ell_{a}(\vartheta)=(a-\vartheta)^{2}$.
Figure 3(b) shows the IM’s upper expected loss,
$a\mapsto\overline{\Pi}_{y}\ell_{a}$, along with the analogous fiducial
expected loss $a\mapsto Q_{y}^{\star}\ell_{a}$. Note that the shape of these
curves are very similar, in particular, the minimizers are very close, but the
IM’s curve is strictly above the fiducial’s, as expected based on the
construction and the interpretation as an “upper” expected loss.

(a) Possibility contour (and loss)

(b) (Upper) expected loss

Figure 3: Skew-normal example with $y=0$. Panel (a) shows the possibility
contour $\pi_{y}(\vartheta)$ for the skew-normal illustration; overlaid is the
squared error loss function $\ell_{a}(\cdot)$ with $a=-2$. Panel (b) shows the
IM (solid) and fiducial (dashed) expected losses.

####  4.3.2 Binomial model

Consider a binomial model, $\mathsf{P}_{Y|\theta}={\sf Bin}(n,\theta)$, where
the number of trials, $n$, is known but the success probability,
$\theta\in[0,1]$, is unknown. There are a number of different strategies
available to constructing a binomial IM, the most recent being in Balch,
(2020). Here I’ll follow a simpler strategy, which has some close connections
to the classical Clopper–Pearson confidence interval construction, that I
believe was first presented in Martin and Liu, (2014, Sec. 4.3). This binomial
IM construction was reviewed and recast in more modern terms recently in
Cahoon and Martin, (2021), so I’ll not reproduce the details here. Roughly
speaking, the starting point is the association

$F_{\theta}(Y-1)\leq U<F_{\theta}(U),\quad U\sim{\sf Unif}(0,1),$

where $F_{\theta}$ is the binomial distribution function. A suitable (nested)
random set $U$ is introduced to predict the unobserved value of $U$, which is
pushed through the association, with $Y=y$, leading to a new $y$-dependent
random set in the $\theta$-space. The end result is a possibility contour

$\pi_{y}(\vartheta)=1-\max\\{0,2B_{y+1,n-y}(\vartheta)-1\\}-\max\\{0,1-2B_{n-y+1,y}(\vartheta)\\},$

where $B_{a,b}$ is the ${\sf Beta}(a,b)$ distribution function. Figure 4(a)
shows a plot of this possibility contour for the case $n=18$ and $y=7$. While
it appears from the plot that the edges of the plateau are rounded, this is
not the case; there are corners, so this function is directionally convex in
the sense described above. The aforementioned plateau is around the maximum
likelihood estimator, $\hat{\theta}=y/n\approx 0.39$.

(a) Possibility contour (and loss)

(b) (Upper) expected loss

Figure 4: Binomial example with $n=18$ and $y=7$. Panel (a) shows the
possibility contour $\pi_{y}(\vartheta)$; dashed line shows the weighted
squared error loss function $\ell_{a}(\vartheta)$, when $a=0.2$. Panel (b)
shows the (upper) expected loss functions,
$a\mapsto\overline{\Pi}_{y}\ell_{a}$ (solid) and $a\mapsto
Q_{y}^{\star}\ell_{a}$ (dashed), for the weighted squared error loss function.

For a decision problem, we can consider estimation of $\theta$ with respect to
a weighted squared error loss function

$\ell_{a}(\vartheta)=\frac{(a-\vartheta)^{2}}{\vartheta(1-\vartheta)},\quad\vartheta\in[0,1].$

This loss function is overlaid in Figure 4(a), for the case $a=0.2$. For the
IM’s assessment, $\overline{\Pi}_{y}\ell_{a}$, of an action $a$ relative to
the weighted squared error loss, no closed-form expression is available. This
can, however, be readily evaluated numerically. A plot of that function
$a\mapsto\overline{\Pi}_{y}\ell_{a}$, based on data $n=18$ and $y=7$, is shown
in Figure 4(b). For a comparison, I’ll also consider a fiducial solution to
this binomial problem. There are several different fiducial solutions
available (e.g., Murph et al., 2021), all of which give similar numerical
answers, at least for moderate to large $n$. Here I’ll take the version where
the fiducial distribution has a beta density

$q_{y}^{\star}(\vartheta)={\sf Beta}(\vartheta\mid
y+\tfrac{1}{2},n-y+\tfrac{1}{2}).$

It’s also straightforward to numerically evaluate the fiducial expected loss
$a\mapsto Q_{y}^{\star}\ell_{a}$ and a plot of that function is also displayed
in Figure 4(b). It is apparent that these two curves are roughly the same
shape, with very similar minimizers, but the IM’s expected loss curve is
uniformly larger than the fiducial’s.

Is the difference in magnitudes of any consequence? Of course, there’s no
closed-form expression for the modified quasi-oracle’s assessment
$\widetilde{L}_{a}(y,\theta)$ either, so only numerical calculations are
possible. The goal is to approximate the distribution function of
$\widetilde{R}(Y,\theta)$, defined in (14), as a function of $Y\sim{\sf
Bin}(n,\theta)$. For this experiment, I consider $n=50$ and $\theta=0.3$.
Figure 5 shows the distribution function in (16) for the IM with possibility
contour as described above. According to Theorem 3, this distribution function
should be below that of ${\sf Unif}(0,1)$, which is confirmed by comparing the
black line to the dashed diagonal line. Also shown on the plot is the
distribution function for a version of $\widetilde{R}(Y,\theta)$ that replaces
the IM’s assessment, $\overline{\Pi}_{Y}\ell_{a}$, in the numerator with the
expected loss, $Q_{y}^{\star}\ell_{a}$, for the fiducial solution described
above. Note that the curve corresponding the fiducial distribution passes
above the diagonal line, so it doesn’t satisfy the property in Theorem 3. The
fiducial solution’s violation of the property in Theorem 3 is more severe in
this case than in the location model example above but, again, it remains to
understand what, if any, practical consequences there might be as a result of
this violation. In any case, the IM solution in this binomial problem is
provably valid in the sense of (2), satisfies the property in Theorem 3, and
its “best” action is effectively the same as the fiducial’s. So there’s no
reason not to prefer the valid IM solution in this example.

Figure 5: The black line shows the distribution function of
$\widetilde{R}(Y,\theta)$, as defined in (14), for the binomial IM described
in Section 4.3.2. The gray line shows the distribution function for the
version of $\widetilde{R}(Y,\theta)$ for the fiducial IM. The diagonal line
corresponds to the ${\sf Unif}(0,1)$ distribution function.

##  5 Special case: functional models

###  5.1 Setup

In this section, I follow up on some of the insights presented in Section 3
above. In particular, with some additional structure, a more precise
characterization of the contents of $\mathscr{C}(\overline{\Pi}_{y})$ can be
given. This is perhaps of independent interest, but it will also shed light on
the new decision-theoretic developments in Section 4. The key observation is
that the possibility measure constructed from a valid IM has some special
features, at least when the statistical model is suitably structured. Below I
will introduce this simplifying structure, which covers many practically
useful models, and then proceed to give a more detailed characterization of
the credal set’s contents.

As discussed in Section 2.1 above, suppose the statistical model is re-
expressed as

$Y=a(\theta,U),\quad U\sim\mathsf{P}_{U},$

where $\mathsf{P}_{U}$ is a known probability distribution on $\mathbb{U}$;
see Equation (3). Let $y$ denote the observed value of $Y$. In what follows,
I’ll assume that the equation “$y=a(\vartheta,u)$” defines continuously
differentiable bijection between $\vartheta$ and $u$ for a given $y$. That is,
let $\theta_{y}(u)$ and $u_{y}(\vartheta)$ be such that

$a(\theta_{y}(u),u)=a(\vartheta,u_{y}(\vartheta)),\quad\text{for all
$(u,\vartheta)$},$

for each fixed $y$. In other words, $u_{y}$ and $\theta_{y}$ are inverse maps,
i.e., $u_{y}=\theta_{y}^{-1}$. This is just the functional model setup in
Dawid and Stone, (1982), which is similar to the fiducial perspective taken in
Taraldsen and Lindqvist, (2013). Both of these contain the structural models
of Fraser, (1968), where each of $y$, $\theta$, and $u$ can be identified with
elements in a group of transformations. Then the association (3) is often
rewritten more succinctly as “$y=\theta u$,” where the concatenation “$\theta
u$” on the right corresponds to composition of the two transformations,
$\theta$ and $u$. For technical reasons, it is necessary to assume that the
distribution $\mathsf{P}_{U}$ of $U$ has a density, $f$, with respect to
Lebesgue measure, and that $f$ is not constant on sets with positive
$\mathsf{P}_{U}$-probability.

The above structure is rather restrictive, but often the original statistical
model can be transformed into one that meets these conditions. The main
obstacle is that a one-to-one connection between $\theta$ and $u$ for a given
$y$ requires the dimensions of the two quantities to be the same. However, in
the location model with $n$ iid samples, the baseline association (3) would
have a scalar $\theta$ and a $n$-vector $u$. To side-step this problem, it is
often possible to apply an initial dimension-reduction step to $u$ via a
conditioning. This is described in detail in Chapter 2 of Fraser, (1968) for
the structural/group model case, as well as in Dawid and Stone, (1982), which
is similar to the fiducial perspective taken in Dawid and Stone, (1982) and
Taraldsen and Lindqvist, (2013); a slightly different approach is taken in
Martin and Liu, 2015a . Suffice it to say, for the structural/functional
models in consideration here, it is possible to convert the problem into one
in which the simple one-to-one relationship holds. For the iid location model,
if $\mathsf{P}_{U}$ is normal, then a satisfactory dimension reduction can be
achieved through sufficiency. More generally, in the location model, the
residuals are ancillary statistics and conditioning on the vector of residuals
leads to an effective reduction in dimension. Even more generally, for models
with a group structure, there is a maximal invariant statistic which can be
conditioned on. The point is just that, while the simple one-to-one
association is rather restrictive, many problems can be transformed into one
that satisfies this.

###  5.2 Inferential model constructions

Here I present the construction of two different IMs: a fiducial IM in the
form of a precise probability and an IM based on the construction in Liu and
Martin, (2021) of a data-dependent possibility measure; the latter is valid in
the sense of Section 2.1 while the former is not. The structure of the
functional model allows me to draw a clear connection between these two
different solutions to the inference/decision problem.

I’ll start with the fiducial IM. In the present case, there is a well-defined
fiducial distribution construction, a generalization of the Fisher’s ideas, as
presented in Fraser, (1968) and subsequently in, e.g., Dawid and Stone, (1982)
and Taraldsen and Lindqvist, (2013). Indeed, modern presentations of fiducial
inference suggest constructing a $y$-dependent probability $Q_{y}^{\star}$ on
$\Theta$ based on the formula

$Q_{y}^{\star}(A)=\mathsf{P}_{U}\\{\theta_{y}(U)\in A\\},\quad
A\subseteq\Theta.$ (17)

Since $\mathsf{P}_{U}$ has a density function $f$ with respect to Lebesgue
measure, so does $Q_{y}^{\star}$, and it’s given by the transformation formula

$q_{y}^{\star}(\vartheta)=f(u_{y}(\vartheta))\,J(\vartheta;u_{y}),$

where $J(\vartheta;u_{y})$ is the transformation’s Jacobian, i.e.,

$J(\vartheta;u_{y})=\det\Bigl{|}\frac{\partial
u_{y}(\vartheta)}{\partial\vartheta}\Bigr{|}.$

Alternatively, also from (3), the recently proposed construction in Liu and
Martin, (2021) would produce output in the form of a lower and upper
probability pair, $(\underline{\Pi}_{y},\overline{\Pi}_{y})$, where
$\overline{\Pi}_{y}$ is determined by

$\overline{\Pi}_{y}(A)=\sup_{\vartheta\in A}\pi_{y}(\vartheta),\quad
A\subseteq\Theta,$

with the possibility contour function

$\pi_{y}(\vartheta)=\mathsf{P}_{U}\\{f(U)\leq
f(u_{y}(\vartheta))\\},\quad\vartheta\in\Theta.$

This IM, is valid according to the general theory in, e.g., Section 2.1. For
convenience later on, define the function

$\eta(z)=\mathsf{P}_{U}\\{f(U)<z\\},\quad z\in[0,\sup f].$ (18)

Since the density $f$ is assumed to be non-constant on sets of positive
$\mathsf{P}_{U}$-probability, $\eta$ is strictly increasing and, hence, has an
inverse $\eta^{-1}$. With this notation, the IM’s plausibility regions, which
are needed in what follows, can be expressed as

$C_{\alpha}(y)=\\{\vartheta:\pi_{y}(\vartheta)>\alpha\\}=\\{\vartheta:f(u_{y}(\vartheta))>\eta^{-1}(\alpha)\\}.$
(19)

This latter form for $C_{\alpha}(y)$ will be important to verifying the key
identity (20) below.

###  5.3 Connections

Despite having the same starting point, namely (3), these two approaches seem
entirely different. It turns out, however, that there is a close connection
through the valid IM’s family of plausibility regions,
$\\{C_{\alpha}(y):\alpha\in[0,1]\\}$. First note that, by definition,

$\overline{\Pi}_{y}\\{C_{\alpha}(y)^{c}\\}=\sup_{\vartheta\not\in
C_{\alpha}(y)}\pi_{y}(\vartheta)=\sup_{\vartheta:\pi_{y}(\vartheta)\leq\alpha}\pi_{y}(\vartheta)=\alpha.$

Second, using the special structure of the model formulation above and, in
particular, a change-of-variables from $\vartheta\to u$, I get

$\displaystyle Q_{y}^{\star}\\{C_{\alpha}(y)^{c}\\}$
$\displaystyle=\int_{\pi_{y}(\vartheta)\leq\alpha}q_{y}^{\star}(\vartheta)\,d\vartheta$
$\displaystyle=\int_{f(u_{y}(\vartheta))\leq\eta^{-1}(\alpha)}f(u_{y}(\vartheta))J(\vartheta;u_{y})\,d\vartheta$
$\displaystyle=\int_{f(u)\leq\eta^{-1}(\alpha)}f(u)\,du$
$\displaystyle=\mathsf{P}_{U}\\{f(U)\leq\eta^{-1}(\alpha)\\}$
$\displaystyle=\eta(\eta^{-1}(\alpha))$ $\displaystyle=\alpha,$

where the range-of-integration change in the second inequality follows from
(19), using the invertible function $\eta$ defined in (18). Consequently,

$Q_{y}^{\star}\\{C_{\alpha}(y)^{c}\\}=\overline{\Pi}_{y}\\{C_{\alpha}(y)^{c}\\},\quad\text{for
all $\alpha\in[0,1]$}.$ (20)

This is important for several reasons. First, it implies that the fiducial
distribution, $Q_{y}^{\star}$, is a confidence distribution in the sense
described in Section 3 which, by Theorem 1, implies that
$Q_{y}^{\star}\in\mathscr{C}(\overline{\Pi}_{y})$. Second, the fact that both
assign the same numerical probability/plausibility values to a wide class of
assertions, namely, $\\{C_{\alpha}(y)^{c}:\alpha\in[0,1]\\}$ implies that the
fiducial distribution, $Q_{y}^{\star}$, must be the “most extreme” of those
members of $\mathscr{C}(\overline{\Pi}_{y})$. Moreover, at least intuitively,
the other members in the credal set should be those distributions that are
dominated by $Q_{y}^{\star}$ in some specific sense. I’ll refer to this
special $Q_{y}^{\star}$ as the maximally diffuse confidence distribution in
$\mathscr{C}(\overline{\Pi}_{y})$.

To make this intuitive notion of dominance more precise, some additional
notation and terminology is needed. Following Bergmann, (1991), suppose that
$\Theta$ is equipped with a partial order, denoted by $\preceq$. Then a
function $g:\Theta\to\mathbb{R}$ is said to be “increasing” (with respect to
$\preceq$) if $\vartheta\preceq\vartheta^{\prime}$ implies $g(\vartheta)\leq
g(\vartheta^{\prime})$. Similarly, a set $A\subseteq\Theta$ is said to be
“increasing” (with respect to $\preceq$) if the indicator function
$\vartheta\mapsto 1\\{\vartheta\in A\\}$ is increasing in the sense described
above.

Returning to the statistical problem at hand, recall that the family of
confidence regions is nested. From this nestedness property, a (data-
dependent) partial order on $\Theta$ can be defined. That is, for two points
$\vartheta_{1}$ and $\vartheta_{2}$ in $\Theta$, I’ll say that
“$\vartheta_{1}\preceq\vartheta_{2}$” if

$\text{there exists an $\alpha$ such that $\vartheta_{1}\in C_{\alpha}(y)$ but
$\vartheta_{2}\not\in C_{\alpha}(y)$}.$ (21)

In words, the points in $\Theta$ are ordered according to how small the
confidence level, $\alpha$, needs to be for the $100(1-\alpha)$% confidence
region, $C_{\alpha}(y)$, to contain them. Next, for two distributions on
$\Theta$, say $Q$ and $Q^{\prime}$, define the stochastic order relation
$Q\preceq_{\text{st}}Q^{\prime}$, relative to the aforementioned partial order
“$\preceq$” on $\Theta$, as

$Q(A)\leq Q^{\prime}(A),\quad\text{for all $A$ increasing with respect to
$\preceq$}.$

This proves the following characterization of the valid IM’s credal set,
$\mathscr{C}(\overline{\Pi}_{y})$.

######  Theorem 4.

Let $(\underline{\Pi}_{y},\overline{\Pi}_{y})$ be the realization of a valid
IM based on data $Y=y$, with maximally diffuse confidence distribution
$Q_{y}^{\star}$. Let $\preceq$ denote the ($y$-dependent) partial order
determined by (21) and $\preceq_{\text{st}}$ the corresponding stochastic
order. Then

$Q_{y}\in\mathscr{C}(\overline{\Pi}_{y})\iff
Q_{y}\preceq_{\text{st}}Q_{y}^{\star}.$

For a quick illustration, reconsider the (symmetric) location model,
$Y=\theta+U$, from Section 4.3.1. Since
$\pi_{y}(\vartheta)=2\\{1-F(|y-\vartheta|)\\}$, where $F$ is the distribution
function for the unobservable $U$, it’s easy to check that

$C_{\alpha}(y)=\bigl{[}y-F^{-1}(1-\tfrac{\alpha}{2}),\,y+F^{-1}(1-\tfrac{\alpha}{2})\bigr{]},\quad\alpha\in[0,1].$

Under standard conditions on $F$—e.g., a special case of Example 2.1 in Dawid
and Stone, (1982)—the fiducial distribution for $\theta$, given $Y=y$, has the
density function

$q_{y}^{\star}(\vartheta)=-\tfrac{\partial}{\partial\vartheta}F(y-\vartheta)=f(\vartheta-y),\quad\vartheta\in\Theta.$

This is also the Bayesian solution under the default, flat prior for $\theta$.
It’s well known that the Bayes/fiducial solution in this case is a confidence
distribution, and corresponds to the maximally diffuse element in the credal
set $\mathscr{C}(\overline{\Pi}_{y})$. Furthermore, any distribution that’s
symmetric and “appropriately more concentrated” than $Q_{y}^{\star}$ would be
contained in $\mathscr{C}(\overline{\Pi}_{y})$. For example, scaled versions
of $Q_{y}^{\star}$, having density

$q_{y}(\vartheta)=s^{-1}f(s^{-1}(\vartheta-y)),\quad\text{for $s\in[0,1]$},$

belong to the credal set, even the special case “$s=0$,” corresponding to a
point mass distribution concentrated at $y$. It may also contain other
symmetric distributions.

Beyond providing some understanding of the credal set’s contents, Theorem 4
also has some decision-theoretic implications. The following theorem and the
subsequent discussion make this claim precise.

######  Theorem 5.

Let $(\underline{\Pi}_{y},\overline{\Pi}_{y})$ be a valid IM based on data
$Y=y$, with maximally diffuse confidence distribution $Q_{y}^{\star}$, and let
$\preceq$ denote the $y$-dependent partial order determined by (21). Let
$(a,\vartheta)\mapsto\ell_{a}(\vartheta)$ be a convex, non-negative loss
function such that there exists an action $\hat{a}=\hat{a}(y)$ for which
$\vartheta\mapsto\ell_{\hat{a}}(\vartheta)$ is increasing with respect to
$\preceq$. Then

$\inf_{a\in\mathscr{A}}\overline{\Pi}_{y}\ell_{a}=\overline{\Pi}_{y}\ell_{\hat{a}}=Q_{y}^{\star}\ell_{\hat{a}}=\inf_{a\in\mathscr{A}}Q_{y}^{\star}\ell_{a},$
(22)

and, therefore, $\hat{a}=\hat{a}(y)$ is the minimum (upper) expected loss
action for both the valid IM and the confidence distribution.

###### Proof.

The proof proceeds in three steps, one for each of the three equalities in
(22), starting with the second equality. By definition,
$\overline{\Pi}_{y}\ell_{\hat{a}}=\sup Q_{y}\ell_{\hat{a}}$, where the
supremum is over all $Q_{y}$ in the credal set
$\mathscr{C}(\overline{\Pi}_{y})$. But since the credal set is determined by
the stochastic partial order $Q_{y}\preceq_{\text{st}}Q_{y}^{\star}$, and
$\vartheta\mapsto\ell_{\hat{a}}(\vartheta)$ is increasing with respect to the
same partial order, it follows from Theorem 3(b) in Bergmann, (1991) that the
supremum is attained at the stochastic upper bound,
$Q_{y}^{\star}\ell_{\hat{a}}$.

Next, for the third equality. The expected loss, $a\mapsto
Q_{y}^{\star}\ell_{a}$, is minimized at an action $a$ such that the level sets
of $\ell_{a}$ most closely align with those of the confidence/fiducial
density. But the latter level sets, recall, are the confidence regions that
determine the orders $\preceq$ and $\preceq_{\text{st}}$. Since
$\vartheta\mapsto\ell_{\hat{a}}(\vartheta)$ is increasing with respect to
$\preceq$, its level sets exactly align with the confidence regions and,
hence, with the level sets of the confidence/fiducial density. Therefore, the
minimum is attained at $\hat{a}$.

Finally, for the first equality, note that, again by definition,
$\overline{\Pi}_{y}\ell_{a}$ can’t be smaller than $Q_{y}^{\star}\ell_{a}$ for
any $a$. So, since equality is attained at $\hat{a}$, and since this is the
minimizer of $a\mapsto Q_{y}^{\star}\ell_{a}$, it must also be the minimizer
of $a\mapsto\overline{\Pi}_{y}\ell_{a}$. ∎

I don’t expect the reader to be familiar with a condition like that on the
loss function in Theorem 5, so some further remarks are in order.

  * •

First, the simplest example of this is the symmetric location problem, with
squared error loss, presented in Section 4.3.1 above. There it was shown
directly, by evaluating the risks $\overline{\Pi}_{y}\ell_{a}$ and
$Q_{y}^{\star}\ell_{a}$ and showing that they’re both minimized at the same
point, $\hat{a}(y)=y$. For the case when the error distribution was not
symmetric, the valid IM and fiducial suggest different actions under squared
error loss. However, with a different loss function that incorporates the
shape of the error distribution, a connection between the valid IM and
fiducial actions could be made. While the location model is indeed a very
special case, note that virtually every fixed-dimensional inference problem
takes this form when the data are sufficiently informative. That is, Le Cam’s
deep insights about limiting experiments (e.g., van der Vaart, 2002) suggest
that all “regular” problems boil down to normal locations problem in the
limit. Therefore, a characterization result for location models goes a long
way toward understanding more general problems.

  * •

Second, where might a loss function that satisfies the conditions of Theorem 5
come from? In the symmetric location problem, squared error loss would be
appropriate because it, too, is symmetric. More generally, in a multivariate
location model where the error distribution is elliptically symmetric, a
generalized squared error loss, one that incorporates the error distributions
dispersion matrix, would achieve the theorem’s “increasing” condition. But
what about beyond the symmetric location model? For a decision problem whose
ingredients can be identified with elements of a group, a reasonable strategy
is to define

$\ell_{a}(\vartheta)=k\\{1-\pi(\vartheta^{-1}a)\\},$

for $k:[0,1]\to[0,\infty)$ strictly increasing, such as $k(x)=-c\log(1-x)$,
for a constant $c>0$. For an illustration, reconsider the skew-normal location
model from Section 4.3.1. Following the above strategy, I’ll define a loss
function

$\ell_{a}(\vartheta)=-\tfrac{1}{5}\log\pi(a-\vartheta).$ (23)

Figure 6 shows this loss function, for $a=-1$ and $a=0$, overlaid on the same
possibility contour displayed in Figure 3(a) with $y=0$. This plot
demonstrates that the level sets of the loss function, with $a=0$, are the
same as those of the possibility contour. Therefore, the loss $\ell_{a}$
defined above, with $a=0$, is increasing in the sense considered in Theorem 5.
Consequently, for a general data point $y$, the action corresponding to the
minimum (upper) expected loss is $\hat{a}(y)=y$.

  * •

Third, the “increasing with respect to $\preceq$” is just a sufficient
condition for connecting the suggested actions from the valid IM and fiducial
distribution based on the minimum (upper) expected loss principle. More
generally, one can apply that principle with different loss functions,
including those which are not “increasing” in this sense, and the resulting
action would still satisfy this principle; it’s just that it may be different
from—perhaps even better than—the rule one would get from applying the same
principle to the fiducial distribution.

Figure 6: Same possibility contour $\pi_{y}(\vartheta)$ (solid) as in Figure
3(a), for $y=0$. Overlaid is the loss function $\ell_{a}$ defined in (23) with
$a=-1$ (dotted) and $a=0$ (dashed). Gray lines illustrate that the level sets
of the loss and the possibility contour are the same.

The message in Taraldsen and Lindqvist, (2013) is that fiducial methods, which
inherently have some desirable confidence properties, will also provide good
frequentist decision procedures with some degree of generality. Here the
message is similar but arguably goes further. First, IMs are specifically
designed to achieve valid inference about unknowns, which is stronger than the
confidence property achieved by fiducial methods (e.g., Martin, 2021a ; Martin
et al., 2021). Second, the conservativeness that’s built in to the IM to
achieve this validity property appears in the decision decision-making context
through its assessment of the quality of different actions, and Theorem 2
ensures that the IM’s assessment will not be too optimistic compared to a
suitable oracle’s assessment. This gives the decision-maker some comfort that
the IM’s minimum (upper) expected loss rule is driven by signal rather than
noise. Third, despite its conservativism, the conditions of Theorem 5 define a
range of problems, wide enough to cover almost all problems in an asymptotic
limit, in which the IM’s suggested action based on the minimum (upper)
expected loss principle is the same as the one considered to be “optimal”
based on other criteria. And beyond those problems covered by the previous
theorem, like the skewed-normal or binomial examples above, the solution based
on the valid IM’s upper probability may have some advantages which deserve
further investigation.

##  6 Conclusion

To date, and to my knowledge, inferential models (IMs) have been used
exclusively for inference—no formal theory on decision-making had been
investigated in the literature. This paper aims to fill this gap by, first,
developing a framework for decision-making under uncertainty based on an IM’s
upper probabilities and, second, investigating the decision-theoretic
implications of the IM’s validity property. For the first, I follow previous
suggestions made in the imprecise probability literature to define an upper
expected loss via the Choquet integral of the loss with respect to the IM’s
upper probability, so that different actions can be compared using a natural
variation of the minimize-expected-loss principle. For the second, it was
shown in Theorems 2–3 that validity of the IM implies that its assessment,
$\overline{\Pi}_{Y}\ell_{a}$, of an action $a$ would, in a specific sense, not
tend to be inconsistent with the quasi-oracle’s assessment, thus giving the IM
a sense of reliability in the decision-making context which, in turn, provides
some protection to the decision-maker. While the validity property requires
some degree of conservatism, the numerical examples and the developments in
Section 5 strongly suggest that the IM’s suggested actions are high-quality,
in some cases, optimal from a purely frequentist point of view. Specifically,
Theorem 5 shows that, for a certain class of statistical decision problem, the
valid IM’s suggested action agrees with that of the fiducial distribution,
where the latter is known to be optimal in many cases where an “optimal”
decision rule is known to exist.

In my numerical illustrations, computation of the Choquet integral was
straightforward, whether using Monte Carlo methods or quadrature, as was the
optimization to obtain the minimum (upper) expected loss rule, $\hat{a}$.
However, in higher dimensions, the same computations become more difficult.
For a recent survey on some of the general computational tools available in
the imprecise probability literature for evaluating Choquet integrals, see
Troffaes and Hable, (2014). That reference also describes a simplification in
the special case where the upper probability is a possibility measure. A
thorough investigation into how well these general algorithms work for the
specific case of integration with respect to the valid IM’s lower/upper
probability output is still needed.

The numerical examples in Section 4.3 demonstrated that the fiducial or
confidence distribution’s assessment of the action quality does not satisfy
the property in the conclusion of Theorem 3, while the valid IM’s assessment
does. A relevant question is if this might serve as the basis for an analogue
of the false confidence theorem (Balch et al., 2019) in the context of
decision-making. Two things are needed to achieve this: (a) better
understanding of the quasi-oracle’s assessment, i.e., why a failure to satisfy
the conclusion of Theorem 3 may lead to poor practical decision-making
performance, and (b) a proof that non-valid IM’s can’t satisfy that property.

An important notion in frequentist statistical decision theory is
admissibility. For a decision rule $\hat{a}:\mathbb{Y}\to\mathscr{A}$, define
the (frequentist) expected loss

$r(\theta;\hat{a})=\int\ell_{\hat{a}(y)}(\theta)\,\mathsf{P}_{Y|\theta}(dy).$

Then $\hat{a}$ is inadmissible if there exists another decision rule
$\tilde{a}$ such that $r(\theta;\tilde{a})\leq r(\theta;\hat{a})$ for all
$\theta$, with strict inequality for some $\theta$; otherwise, $\hat{a}$ is
admissible. Unfortunately, the valid IM’s recommended action, defined in (11),
is not admissible in general. To see this, consider the $d$-dimensional normal
mean problem. Under loss $\ell_{a}(\vartheta)=\|a-\vartheta\|^{2}$, the IM’s
recommended action is $\hat{a}(y)=y$, which corresponds to the least squares,
maximum likelihood, fiducial, and flat-prior Bayes estimator, was shown by
Stein, (1956) to be inadmissible when $d\geq 3$; see, also, Brown, (1971).
This is not especially surprising since validity requires that the IM be
“unbiased” in a certain sense which, in the present context implies the IM’s
recommended action be an unbiased estimator in the usual sense. But the
familiar bias-variance tradeoff suggests that $r(\cdot;\hat{a})$ can be
reduced by choosing a biased $\hat{a}$, e.g., like a proper-prior Bayes rule
that’s biased and admissible. To introduce the appropriate “bias” into the IM
construction, it’s natural to consider incorporation of suitable (partial)
prior information, e.g., sparsity. A notion of validity under partial prior
information is currently being developed (see Martin, 2021b ), and an
extension of the decision-theoretic results presented here to that case, along
with admissibility considerations, are part of my ongoing investigations.

## Acknowledgments

This work is supported by the U.S. National Science Foundation, grant
SES–2051225.

##  Appendix A Additional technical details

In the proof of Theorem 3 presented in Section 4.2, an important step was
postponed. The relevant result is stated and proved below.

######  Lemma 1.

As in the statement of Theorem 3, suppose that the loss function
$\vartheta\mapsto\ell_{a}(\vartheta)$ is convex and non-negative for each
action $a$, and the possibility contour $\vartheta\mapsto\pi_{y}(\vartheta)$
is directionally convex. Then the function

$\alpha\mapsto
h_{y,a}(\alpha):=\sup\\{\ell_{a}(\vartheta):\pi_{y}(\vartheta)>\alpha\\},\quad\alpha\in[0,1],$

is convex.

###### Proof.

For simplicity, drop the subscripts $(y,a)$ on the $h$ function. Two key
observations are as follows:

  * •

the level sets $\\{\vartheta:\pi_{y}(\vartheta)>\alpha\\}$ of $\pi_{y}$ are
intervals;

  * •

and the supremum in the definition of $h(\alpha)$ is attained at the one or
both of the endpoints of the aforementioned interval.

Suppose first that the supremum on each of those $\alpha$-indexed intervals is
unique, i.e., that the loss evaluated at one endpoint is strictly larger than
the loss at the other. Define

$\theta_{\alpha}=\arg\max\\{\ell_{a}(\vartheta):\pi_{y}(\vartheta)>\alpha\\}.$

Since both $\ell_{a}$ and $\pi_{y}$ are continuous, the function
$\alpha\mapsto\theta_{\alpha}$ is also continuous, with $\theta_{\alpha}$
converging monotonically from either the left or right to left- or right-most
mode, respectively, as $\alpha\to 1$. Also, again by continuity,
$\pi_{y}(\theta_{\alpha})=\alpha$. If it happens that the loss function takes
the same value on both endpoints of one of the level sets of the possibility
contour, then the choice of $\theta_{\alpha}$ could be made so that
$\alpha\mapsto\theta_{\alpha}$ is continuous.

Take arbitrary $\alpha_{1},\alpha_{1}\in[0,1]$ and $w_{1}\in[0,1]$, and then
set $w_{2}=1-w_{1}$. The goal is to show that

$h(w_{1}\alpha_{1}+w_{2}\alpha_{2})\leq
w_{1}h(\alpha_{1})+w_{2}h(\alpha_{2}),$

where, with the notation defined above, $h(\alpha)=\ell_{a}(\theta_{\alpha})$,
for $\alpha\in[0,1]$. Let $\bar{\alpha}=w_{1}\alpha_{1}+w_{2}\alpha_{2}$ and
$\bar{\theta}_{\alpha}=w_{1}\theta_{\alpha_{1}}+w_{2}\theta_{\alpha_{2}}$.
Since $\pi_{y}$ is assumed to be directionally convex, it follows that

$\pi_{y}(\bar{\theta}_{\alpha})\leq
w_{1}\pi_{y}(\theta_{\alpha_{1}})+w_{2}\pi_{y}(\theta_{\alpha_{2}})=\bar{\alpha}.$

This implies $\bar{\theta}_{\alpha}$ is not in the interior of
$\\{\vartheta:\pi_{y}(\vartheta)>\bar{\alpha}\\}$ and, in turn, that
$\ell_{a}(\bar{\theta}_{\alpha})$ is no less than
$\ell_{a}(\theta_{\bar{\alpha}})$. From this, and convexity of the loss, it
follows that

$\displaystyle h(\bar{\alpha})$
$\displaystyle=\ell_{a}(\theta_{\bar{\alpha}})$
$\displaystyle\leq\ell_{a}(\bar{\theta}_{\alpha})$ $\displaystyle\leq
w_{1}\ell_{a}(\theta_{\alpha_{1}})+w_{2}\ell_{a}(\theta_{\alpha_{2}})$
$\displaystyle=w_{1}h(\alpha_{1})+w_{2}h(\alpha_{2}),$

which proves the convexity claim. ∎

The challenge to extending the above convexity argument to more than one
dimension is that the path $\alpha\mapsto\theta_{\alpha}$ is generally not a
line. Figure 7 shows an example of this. Then $\bar{\theta}_{\alpha}$ is not
necessarily on this path, which makes it difficult to establish that
$\ell_{a}(\bar{\theta}_{\alpha})$ is no less than
$\ell_{a}(\theta_{\bar{\alpha}})$. This was a crucial step in the proof of
Lemma 1 above, so if it could still be made when the aforementioned path is
not a line, then the convexity result and, hence, the claim in Theorem 3 would
hold for the general vector-$\theta$ case.

Figure 7: Contour plot showing level sets of $\pi_{y}$ (gray) and of
$\ell_{a}$ (dotted). The solid black line is the path
$\alpha\mapsto\theta_{\alpha}$, which is a curve in this example.

## References

  * Augustin et al., (2014) Augustin, T., Walter, G., and Coolen, F. P. A. (2014).  Statistical inference.  In Introduction to Imprecise Probabilities, Wiley Ser. Probab. Stat., pages 135–189. Wiley, Chichester.
  * Azzalini, (1985) Azzalini, A. (1985).  A class of distributions which includes the normal ones.  Scand. J. Statist., 12(2):171–178.
  * Balch, (2012) Balch, M. S. (2012).  Mathematical foundations for a theory of confidence structures.  Internat. J. Approx. Reason., 53(7):1003–1019.
  * Balch, (2020) Balch, M. S. (2020).  New two-sided confidence intervals for binomial inference derived using Walley’s imprecise posterior likelihood as a test statistic.  Internat. J. Approx. Reason., 123:77–98.
  * Balch et al., (2019) Balch, M. S., Martin, R., and Ferson, S. (2019).  Satellite conjunction analysis and the false confidence theorem.  Proc. Royal Soc. A, 475(2227):2018.0565.
  * Berger, (1984) Berger, J. O. (1984).  The robust Bayesian viewpoint.  In Robustness of Bayesian Analyses, volume 4 of Stud. Bayesian Econometrics, pages 63–144. North-Holland, Amsterdam.  With comments and with a reply by the author.
  * Berger, (1985) Berger, J. O. (1985).  Statistical Decision Theory and Bayesian Analysis.  Springer-Verlag, New York, second edition.
  * Bergmann, (1991) Bergmann, R. (1991).  Stochastic orders and their application to a unified approach to various concepts of dependence and association.  In Stochastic Orders and Decision Under Risk (Hamburg, 1989), volume 19 of IMS Lecture Notes Monogr. Ser., pages 48–73. Inst. Math. Statist., Hayward, CA.
  * Bernoulli, (1954) Bernoulli, D. (1954).  Exposition of a new theory on the measurement of risk.  Econometrica, 22:23–36.
  * Brown, (1971) Brown, L. D. (1971).  Admissible estimators, recurrent diffusions, and insoluble boundary value problems.  Ann. Math. Statist., 42:855–903.
  * Cahoon and Martin, (2021) Cahoon, J. and Martin, R. (2021).  Generalized inferential models for censored data.  Internat. J. Approx. Reason., 137:51–66.
  * (12) Cella, L. and Martin, R. (2021a).  Valid inferential models for prediction in supervised learning problems.  In Cano, A., De Bock, J., Miranda, E., and Moral, S., editors, Proceedings of the Twelveth International Symposium on Imprecise Probability: Theories and Applications, volume 147 of Proceedings of Machine Learning Research, pages 72–82. PMLR.  Extended version available at https://researchers.one/articles/21.12.00002.
  * (13) Cella, L. and Martin, R. (2021b).  Validity, consonant plausibility measures, and conformal prediction.  Internat. J. Approx. Reason., to appear; https://researchers.one/articles/20.01.00010.
  * Choquet, (1954) Choquet, G. (1953–1954).  Theory of capacities.  Ann. Inst. Fourier, Grenoble, 5:131–295 (1955).
  * Crane, (2018) Crane, H. (2018).  The fundamental principle of probability: Resolving the replication crisis with skin in the game.  Researchers.One, https://researchers.one/articles/18.08.00013.
  * Cuzzolin, (2021) Cuzzolin, F. (2021).  The Geometry of Uncertainty.  Artificial Intelligence: Foundations, Theory, and Algorithms. Springer, Cham.
  * Dawid and Stone, (1982) Dawid, A. P. and Stone, M. (1982).  The functional-model basis of fiducial inference.  Ann. Statist., 10(4):1054–1074.  With discussion.
  * Dempster, (1963) Dempster, A. P. (1963).  Further examples of inconsistencies in the fiducial argument.  Ann. Math. Statist., 34:884–891.
  * Dempster, (2008) Dempster, A. P. (2008).  The Dempster–Shafer calculus for statisticians.  Internat. J. Approx. Reason., 48(2):365–377.
  * Denœux, (2019) Denœux, T. (2019).  Decision-making with belief functions: a review.  Internat. J. Approx. Reason., 109:87–110.
  * Denœux and Li, (2018) Denœux, T. and Li, S. (2018).  Frequency-calibrated belief functions: review and new insights.  Internat. J. Approx. Reason., 92:232–254.
  * Destercke and Dubois, (2014) Destercke, S. and Dubois, D. (2014).  Special cases.  In Introduction to Imprecise Probabilities, Wiley Ser. Probab. Stat., pages 79–92. Wiley, Chichester.
  * Dubois, (2006) Dubois, D. (2006).  Possibility theory and statistical reasoning.  Comput. Statist. Data Anal., 51(1):47–69.
  * Dubois et al., (2004) Dubois, D., Foulloy, L., Mauris, G., and Prade, H. (2004).  Probability-possibility transformations, triangular fuzzy sets, and probabilistic inequalities.  Reliab. Comput., 10(4):273–297.
  * Dubois and Prade, (1988) Dubois, D. and Prade, H. (1988).  Possibility Theory.  Plenum Press, New York.
  * Fisher, (1973) Fisher, R. A. (1973).  Statistical Methods and Scientific Inference.  Hafner Press, New York, 3rd edition.
  * Fraser, (1968) Fraser, D. A. S. (1968).  The Structure of Inference.  John Wiley & Sons Inc., New York.
  * Fraser, (2011) Fraser, D. A. S. (2011).  Is Bayes posterior just quick and dirty confidence?  Statist. Sci., 26(3):299–316.
  * Fraser, (2013) Fraser, D. A. S. (2013).  Discussion: “Confidence distribution, the frequentist distribution estimator of a parameter: a review” [mr3047496].  Int. Stat. Rev., 81(1):42–48.
  * Ghosh et al., (2006) Ghosh, J. K., Delampady, M., and Samanta, T. (2006).  An Introduction to Bayesian Analysis.  Springer, New York.
  * Giang and Shenoy, (2011) Giang, P. H. and Shenoy, P. P. (2011).  A decision theory for partially consonant belief functions.  Internat. J. Approx. Reason., 52(3):375–394.
  * Gilboa, (1987) Gilboa, I. (1987).  Expected utility with purely subjective nonadditive probabilities.  J. Math. Econom., 16(1):65–88.
  * Gilboa and Schmeidler, (1989) Gilboa, I. and Schmeidler, D. (1989).  Maxmin expected utility with nonunique prior.  J. Math. Econom., 18(2):141–153.
  * Gilboa and Schmeidler, (1994) Gilboa, I. and Schmeidler, D. (1994).  Additive representations of non-additive measures and the Choquet integral.  Ann. Oper. Res., 52:43–65.
  * Grünwald, (2018) Grünwald, P. (2018).  Safe probability.  J. Statist. Plann. Inference, 195:47–63.
  * Hannig et al., (2016) Hannig, J., Iyer, H., Lai, R. C. S., and Lee, T. C. M. (2016).  Generalized fiducial inference: a review and new results.  J. Amer. Statist. Assoc., 111(515):1346–1361.
  * Hose and Hanss, (2021) Hose, D. and Hanss, M. (2021).  A universal approach to imprecise probabilities in possibility theory.  Internat. J. Approx. Reason., 133:133–158.
  * Huntley et al., (2014) Huntley, N., Hable, R., and Troffaes, M. C. M. (2014).  Decision making.  In Introduction to Imprecise Probabilities, Wiley Ser. Probab. Stat., pages 190–206. Wiley, Chichester.
  * Liu and Martin, (2021) Liu, C. and Martin, R. (2021).  Inferential models and possibility measures.  Handbook of Bayesian, Fiducial, and Frequentist Inference, to appear; arXiv:2008.06874.
  * Martin, (2019) Martin, R. (2019).  False confidence, non-additive beliefs, and valid statistical inference.  Internat. J. Approx. Reason., 113:39–73.
  * (41) Martin, R. (2021a).  An imprecise-probabilistic characterization of frequentist statistical inference.  https://researchers.one/articles/21.01.00002.
  * (42) Martin, R. (2021b).  Towards a theory of valid inferential models under partial prior information.  https://researchers.one/articles/21.05.00001.
  * Martin et al., (2021) Martin, R., Balch, M., and Ferson, S. (2021).  Response to the comment ‘Confidence in confidence distributions!’.  Proc. R. Soc. A., 477:20200579.
  * Martin et al., (2012) Martin, R., Ermini Leaf, D., and Liu, C. (2012).  Optimal inferential models for a Poisson mean.  arXiv:1207.0105.
  * Martin and Liu, (2013) Martin, R. and Liu, C. (2013).  Inferential models: a framework for prior-free posterior probabilistic inference.  J. Amer. Statist. Assoc., 108(501):301–313.
  * Martin and Liu, (2014) Martin, R. and Liu, C. (2014).  A note on p-values interpreted as plausibilities.  Statist. Sinica, 24(4):1703–1716.
  * (47) Martin, R. and Liu, C. (2015a).  Conditional inferential models: combining information for prior-free probabilistic inference.  J. R. Stat. Soc. Ser. B. Stat. Methodol., 77(1):195–217.
  * (48) Martin, R. and Liu, C. (2015b).  Inferential Models: Reasoning with Uncertainty, volume 147 of Monographs on Statistics and Applied Probability.  CRC Press, Boca Raton, FL.
  * Molchanov, (2005) Molchanov, I. (2005).  Theory of Random Sets.  Probability and Its Applications (New York). Springer-Verlag London Ltd., London.
  * Murph et al., (2021) Murph, A., Hannig, J., and Williams, J. P. (2021).  Introduction to generalized fiducial inference.  Handbook of Bayesian, Fiducial, and Frequentist Inference, to appear.
  * Nguyen, (2006) Nguyen, H. T. (2006).  An Introduction to Random Sets.  Chapman & Hall/CRC, Boca Raton, FL.
  * Savage, (1972) Savage, L. J. (1972).  The Foundations of Statistics.  Dover Publications, Inc., New York, revised edition.
  * Schweder and Hjort, (2002) Schweder, T. and Hjort, N. L. (2002).  Confidence and likelihood.  Scand. J. Statist., 29(2):309–332.
  * Schweder and Hjort, (2016) Schweder, T. and Hjort, N. L. (2016).  Confidence, Likelihood, Probability, volume 41 of Cambridge Series in Statistical and Probabilistic Mathematics.  Cambridge University Press, New York.
  * Shafer, (1976) Shafer, G. (1976).  A Mathematical Theory of Evidence.  Princeton University Press, Princeton, N.J.
  * Shafer, (2021) Shafer, G. (2021).  Testing by betting: a strategy for statistical and scientific communication.  J. Roy. Statist. Soc. Ser. A, 184(2):407–478.
  * Stein, (1956) Stein, C. (1956).  Inadmissibility of the usual estimator for the mean of a multivariate normal distribution.  In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 1954–1955, vol. I, pages 197–206, Berkeley and Los Angeles. University of California Press.
  * Taraldsen, (2021) Taraldsen, G. (2021).  Joint confidence distributions.  https://doi.org/10.13140/RG.2.2.33079.85920.
  * Taraldsen and Lindqvist, (2013) Taraldsen, G. and Lindqvist, B. H. (2013).  Fiducial theory and optimal inference.  Ann. Statist., 41(1):323–341.
  * Troffaes and de Cooman, (2014) Troffaes, M. C. M. and de Cooman, G. (2014).  Lower Previsions.  Wiley Series in Probability and Statistics. John Wiley & Sons, Ltd., Chichester.
  * Troffaes and Hable, (2014) Troffaes, M. C. M. and Hable, R. (2014).  Computation.  In Introduction to Imprecise Probabilities, Wiley Ser. Probab. Stat., pages 329–337. Wiley, Chichester.
  * van der Vaart, (2002) van der Vaart, A. (2002).  The statistical work of Lucien Le Cam.  Ann. Statist., 30(3):631–682.  Dedicated to the memory of Lucien Le Cam.
  * Vidakovic, (2000) Vidakovic, B. (2000).  $\Gamma$-minimax: a paradigm for conservative robust Bayesians.  In Robust Bayesian analysis, volume 152 of Lect. Notes Stat., pages 241–259. Springer, New York.
  * von Neumann and Morgenstern, (1947) von Neumann, J. and Morgenstern, O. (1947).  Theory of Games and Economic Behavior.  Princeton University Press, Princeton, N. J.  2d ed.
  * Walley, (1987) Walley, P. (1987).  Belief function representations of statistical evidence.  Ann. Statist., 15(4):1439–1465.
  * Walley, (1991) Walley, P. (1991).  Statistical Reasoning with Imprecise Probabilities, volume 42 of Monographs on Statistics and Applied Probability.  Chapman & Hall Ltd., London.
  * Wang, (2011) Wang, R.-S. (2011).  Some inequalities and convergence theorems for Choquet integrals.  J. Appl. Math. Comput., 35(1-2):305–321.
  * Wasserman, (1990) Wasserman, L. A. (1990).  Belief functions and statistical inference.  Canad. J. Statist., 18(3):183–196.
  * Xie and Singh, (2013) Xie, M. and Singh, K. (2013).  Confidence distribution, the frequentist distribution estimator of a parameter: a review.  Int. Stat. Rev., 81(1):3–39.
  * Zadeh, (1978) Zadeh, L. A. (1978).  Fuzzy sets as a basis for a theory of possibility.  Fuzzy Sets and Systems, 1(1):3–28.