monthyeardate[]

# Belief Bias Identification

Pedro Gonzalez-Fernandez Department of Microeconomics and Public Economics,
Maastricht University, The Netherlands. CONTACT:
<EMAIL_ADDRESS>Maastricht University

(April, 2024)

###### Abstract

This paper proposes a unified theoretical model to identify and test a
comprehensive set of probabilistic updating biases within a single framework.
The model achieves separate identification by focusing on the updating of
belief distributions, rather than classic point-belief measurements. Testing
the model in a laboratory experiment reveals significant heterogeneity at the
individual level: All tested biases are present, and each participant exhibits
at least one identifiable bias. Notably, motivated-belief biases (optimism and
pessimism) and sequence-related biases (gambler’s fallacy and hot hand
fallacy) are identified as key drivers of biased inference. Moreover, at the
population level, base rate neglect emerges as a persistent influence. This
study contributes to the belief-updating literature by providing a
methodological toolkit for researchers examining links between different
conflicting biases, or exploring connections between updating biases and other
behavioural phenomena.

00footnotetext: For their invaluable support, I would like to thank Elias
Tsakas, Thomas Meissner and Matthias Wibral. For helpful comments, I thank
Peiran Jiao, Michael Thaler and seminar participants at Maastricht University.
I acknowledge funding from the Maastricht Graduate School of Business and
Economics (GSBE) for primary data collection 2023 No. G.23.5168.

##  1 Introduction

Over the last decades, there have been substantial efforts to identify and
categorize belief-updating biases, both within Economics and Psychology
(Edwards,, 1968; Tversky and Kahneman,, 1974; Grether,, 1980). Scholars have
typically studied these biases separately. This has made it more challenging
to tell if someone’s updating behaviour can be attributed to one bias or
another. Despite recent shifts towards a more comprehensive structure of these
biases (Benjamin,, 2019; Bordalo et al., 2023a, ; Bordalo et al., 2023b, ),
there is still a lack of a unifying testable model that can tell multiple
conflicting biases apart.

The essential reason why such a model would be desirable is that the
identification of belief-updating biases can be confounded when some potential
biases are not taken into consideration. For example, imagine a person who
consistently updates her beliefs “too much” in the face of new information.
Traditional models are able to tell whether this behaviour is due to this
person neglecting her prior beliefs (base rate neglect), or interpreting too
much out of the information she receives (overinference). However, other
potential biases could also explain her updating behaviour. If she has
preferences over different outcomes, is she updating her beliefs too much
because she generally overinterprets information, or because she cares about
the outcome (optimism/pessimism)? Or may she instead be jumping to conclusions
because the information she receives is confirming her previous beliefs
(confirmation bias)? Could it also be that she has very little doubt about the
outcome that she believes to be more likely to happen (overconfidence)? By
excluding some of these biases from the model, we may wrongly attribute her
behaviour to a different bias than the one(s) she is actually exhibiting. This
implies that some of the biases which happen to be present in the literature
may only seem to be relevant because we lack a more complete model. For
instance, base-rate neglect might appear to be driving behaviour because, say,
confirmation bias and overconfidence are unaccounted for.

Addressing confounded belief-updating biases is important for two related
purposes. First, identifying these biases separately is relevant because the
actions that people may take under different updating biases could be
completely different even if they lead to the same posterior. For example,
someone suffering from confirmation bias may keep voting for the same
political party after receiving social media information that aligns with her
previous beliefs, while someone who simply neglects prior information might be
more likely to change her mind if reminded of the actual base rate. Secondly,
once confoundedness is taken care of, such a model would be able to shed light
onto which specific biases are more common and which biases drive inference
the most.

Integrating multiple belief biases into one testable model poses one important
methodological challenge. Namely, that the way we usually measure beliefs may
not suffice to separably identify a full range of potentially confounded
biases. The vast majority of the literature measures point beliefs to study
biased updating (Benjamin,, 2019). This approach is simple and might be enough
when the number of biases to take into consideration is small. But with the
inclusion of more biases, the amount of flexibility we have to distinguish one
updating pattern from another is reduced.

This paper constructs a theoretical model that allows to separably identify
multiple conflicting belief-updating biases, using belief
distributions111While Jiao et al., (2020) utilize the idea of using belief
distributions to simultaneously incorporate belief biases their model is
unsuitable for probabilistic biases (relies on normally distributed beliefs),
incorporates a reduced range of biases and is not tested in an experimental
setting. to measure prior and posterior beliefs. One can understand belief
distributions as an individual holding beliefs over multiple outcomes, or
being uncertain about her own beliefs when there are only two states (in a
similar sense to cognitive uncertainty (Enke and Graeber,, 2023)). Either way,
the central idea is that belief distributions give us more leeway to better
distinguish updating biases.

The theoretical framework I propose adopts an approach where an agent observes
a series of Bernoulli trials. The agent’s prior beliefs are assumed to follow
a beta distribution and are updated via Bayesian inference. Since the beta
distribution is conjugate to the binomial distribution, the posterior of the
Bayesian agent will also result in a beta distribution222This updating setting
is comparable to the one employed in the ambiguity aversion literature (see
Abdellaoui et al., (2021) as an example) or cognitive uncertainty (Enke and
Graeber,, 2023).. To integrate different biases, the model accounts for
deviations from Bayesian updating, utilizing distorted likelihoods and priors.
These distortions yield non-Bayesian posterior beta distributions, allowing
for the identification of core biases such as over/under inference (Khaw et
al.,, 2021; Augenblick et al.,, 2021; Ba et al.,, 2022) or base rate
neglect/over use (Benjamin et al.,, 2019; Bucher and Glimcher,, 2022; Enke et
al., 2023a, ). It also explores asymmetries in reacting to good and bad news
in the observed data generating process, alongside preference-related biases
(Eil and Rao,, 2011; Zimmermann,, 2020; Möbius et al.,, 2022). Further, it
incorporates confirmation bias (Rabin and Schrag,, 1999; Charness and Dave,,
2017; Zhenxun, 2023b, ), sequence-related biases like the hot hand fallacy and
gambler’s fallacy (Rabin,, 2002; Rabin and Vayanos,, 2010), and confidence
biases (Serra-Garcia and Gneezy,, 2021; Huffman et al.,, 2022; Enke et al.,
2023b, ) that affect the variance of the agent’s posterior distribution, all
within the same framework.

The model is tested with help of a novel laboratory experiment. Participants
solved a series of tasks where they had to guess the percentage of red balls
in a selected urn from a pool of 99 urns, each containing varying proportions
of red and blue balls. Participants received information signals via ball
draws from the selected urn, forming and reporting their belief beta
distributions twice. Then they reported their updated beliefs after observing
another sequence of independently drawn signals from the same urn.

The results of the study support the idea that the inclusion of a more
complete set of biases mitigates the artificial presence of some biases in
more reduced models. At the population level, both overinference and base rate
neglect significantly affect updating behavior, when these are the only biases
that the model accounts for. However, when the model encompasses a broader
spectrum of biases, the impact of overinference vanishes, leaving base rate
neglect as the sole remaining persistent bias.

Nonetheless, if people exhibit different biases, findings at the aggregate
level might not be able to capture the importance of each bias, as many of
them will be averaged out across individuals. In other words, due to the
presence of individual heterogeneity333The idea of individual heterogeneity
being a key determinant of biased belief updating has been recently discussed
by Khaw et al., (2021) and Alós-Ferrer and Garagnani, (2023); supporting the
idea that while average reports might look almost Bayesian or noisy at the
population level, an individual analysis reveals that there are different but
systematic deviations among individuals., the experiment arguably offers even
more insightful findings when applied at the individual level. First, when
only a reduced amount of biases is accounted for, the model is unable to
identify biased inference in some subjects who would otherwise exhibit a clear
bias. In fact, when the full array of biases is factored in, the model shows
that all individuals exhibit at least one specific bias; and more importantly,
all tested biases are present to a certain extent. Finally, despite the
presence of individual heterogeneity, motivated-belief biases (optimism and
pessimism) and sequence-related biases (gambler’s fallacy and hot hand
fallacy) appear to be the main drivers of biased inference. On the other end,
the presence of confirmatory biases (confirmation and disconfirmation bias) is
scarce.

This paper does not only contribute to the literature of belief updating as a
compass to gauge the intensity of various belief biases, but also aims to be a
toolkit for researchers who identify belief-updating biases in settings where
these could be potentially confounded. In this direction, there are three ways
in which this paper contributes to the existing literature.

First, there has been quite some recent work trying to trace links between
different updating biases. For example, Heger and Papageorge, (2018) and
Gneezy et al., (2023) study how wishful thinking (optimism) can affect
overconfidence; while Charness and Dave, (2017) and Zhenxun, 2023b ; Zhenxun,
2023a  try to separate behaviour stemming from motivated beliefs and
unmotivated confirmation bias. In the absence of an overarching model that
encompasses a wide array of biases, findings can be confounded and results
about such links could therefore be biased. For example, bias A may appear to
be linked to bias B, only because a third bias C is missing.

Second, many researchers search for belief-based explanations of behavioural
phenomena, when conflicting biases come into play. For example, political
polarization has been separately explained from the standpoint of
overconfidence (Ortoleva and Snowberg,, 2015), and confirmation bias (Del
Vicario et al.,, 2017). In the financial literature, there is some debate as
to whether the disposition effect could be caused by discrepant biases, such
as motivated beliefs (Heinke et al.,, 2023), the gambler’s fallacy (Jiao,,
2017) or general underinference (Pitkäjärvi,, 2022). More examples range from
linking confirmation bias to several stylised facts in financial markets
(Pouget et al.,, 2017); or confidence biases to poor investment performance
(Ahmad and Shah,, 2020) and biased memory (Huffman et al.,, 2022). As
previously mentioned, utilizing an approach that is able to tell conflicting
biases apart has the advantage to better pinpoint what biases are precisely
underpinning certain behavioural regularities; or even let us know if some
biases are simply an artifact of using a less complete model.

Third, the problem of conflicting biases also arises in the field of
behavioural interventions. Boosting is a psychological technique aimed at de-
biasing individuals when they suffer some kind of cognitive bias. One of the
challenges mentioned in this area of research is the fact that these
interventions normally target one bias at a time, when indeed multiple biases
could be at stake (Kahneman et al.,, 2021). Once again, having a method to
distinguish between different conflicting biases could come in handy in order
to focus the intervention on those biases which end up being more common, or
drive inference the most.

The paper continues as follows: Section 2 explains the theoretical framework,
the updating setting and progressively introduces different biases into the
model. Section 3 deals with the experimental design and presents the belief-
measuring tool. In section 4, I bring the experimental data and the theory
together and compare two regression models, which differ in the amount of
biases they incorporate. Section 5 discusses the results of the experiment,
and section 6 concludes with some final remarks.

##  2 A model of multiple belief biases

###  2.1 Theoretical framework

Let an agent observe a signal $S=\\{s_{1},...,s_{n}\\}$ consisting of the
realization of $n$ independent and identically distributed Bernoulli trials.
Let $p$ denote the probability of success of each trial ($s_{i}=1$) and $1-p$
denote the probability of each failure ($s_{i}=0$). For such data generating
process (DGP), the likelihood function is the probability mass function of a
binomial distribution with parameters $(n,p)$:

$L(p|s_{1}...s_{n})={n\choose k}p^{k}(1-p)^{n-k}$ (1)

where $k={\sum_{i=1}^{n}s_{i}}$ and $(n-k)$ are the number of successes and
failures in the DGP respectively.

Let $\Omega=(0,1)$ be the set of possible values that $p$ may take444The
state-space set $\Omega$ can have two different interpretations. One could
either consider an agent who is not uncertain about her own beliefs, or one
who is indeed uncertain about her own beliefs (à la cognitive uncertainty
(Enke and Graeber,, 2023)). In the former case, the set $\Omega$ would specify
the state-space of an agent who forms beliefs over every possible realisation
of the objective parameter $p\in(0,1)$. In the latter case, the agent would
form beliefs over a binary state-space (whether the signal $s_{i}$ takes value
$1$ or $0$), and the set $\Omega$ would, in this case, represent the
subjective state-space ($p$), which describes the set of possible states where
the agent expresses uncertainty about her own beliefs. Both alternatives are
contemplated in this model., and let a prior belief $\pi(p)$ be beta
distributed with prior parameters $(a_{0},b_{0})$. Namely,

$\pi(p|a_{0},b_{0})=\frac{1}{B(a_{0},b_{0})}p^{a_{0}-1}(1-p)^{b_{0}-1}$ (2)

where $a_{0}$, $b_{0}$ $>0$, and $B(.)$ is the beta function.

Given a prior $\pi(p)$ and a likelihood $L(p|s_{1}...s_{n})$ the agent forms a
posterior. As the beta distribution is a conjugate prior of the binomial
distribution, a Bayesian agent updates her beliefs such that her posterior
distribution of $p$, $\pi(p|a_{n},b_{n})$, is also beta distributed with
parameters $a_{n},b_{n}$. This means that555See section A in the appendix for
a proof; as well as a proof to derive eq.(4) and (5), which suffice to derive
all other equations of posterior parameters in section 2.:

$a_{n}=k+a_{0}\qquad b_{n}=n-k+b_{0}$ (3)

In order to incorporate updating biases, let a non-Bayesian agent use a
distorted likelihood and prior when she updates666These distortions are often
seen (Benjamin,, 2019) as part of an “as-if model.” This means that the model
does not take the stand that biased agents actually follow Bayes’ Theorem with
different likelihood and prior functions, but instead, that these distortions
imply equivalent behaviour to agents interpreting too little, or too much,
from information signals (or prior beliefs) when they update.. These
distortions can be expressed as exponential deviations of the likelihood and
prior. Namely, $\tilde{L}(p|s_{1}...s_{n})=(L(p|s_{1}...s_{n}))^{\gamma}$
would represent the distorted likelihood and
$\tilde{\pi}(p)=(\pi(p))^{\delta}$ would represent a distorted prior of the
non-Bayesian agent ($\gamma,\delta>0$). The parameters $\gamma\neq 1$ and
$\delta\neq 1$ indicate deviations from Bayesian updating due to distortions
of the likelihood and prior respectively. With these modified functions such a
non-Bayesian agent has a posterior beta distribution with parameters
$\tilde{a}_{n},\tilde{b}_{n}$ such that:

$\tilde{a}_{n}=\gamma k+\delta(a_{0}-1)+1$ (4)

$\tilde{b}_{n}=\gamma(n-k)+\delta(b_{0}-1)+1$ (5)

In particular $\gamma>1$ indicates “overinference” while $\gamma<1$ shows
“underinference”; that is, believing that the information signal $S$ is
more/less informative than it actually is. Similarly, parameter $\delta>1$
indicates “base rate overuse” and $\delta<1$, “base rate neglect”; which
implies that the prior is more/less informative than a Bayesian agent
perceives it to be. This bias structure conceptually resembles Grether-
regressions777That is, binary-state regressions where inference biases and
base-rate biases are identified., as the same type of biases can be
identified. The next section shows how equations (4) and (5) can be extended
to incorporate more biases.

###  2.2 Introducing multiple belief biases

This section shows how an extended set of biases can be introduced into the
model.

####  2.2.1 Asymmetries between successes and failures & Preference-related
biases

Equations (4) and (5) assume that deviations from the Bayesian agent in the
likelihood are symmetric for successes and failures. That is, the agent over
or under-reacts to “positive information” the same way she over or under-
reacts to “negative” information. For a non-Bayesian agent this need not be
the case. Consider instead that the agent weights successes and failures of
the data generating process differently. Then her likelihood function would be
$\tilde{L}(p|s_{1}...s_{n})={n\choose k}p^{\alpha k}(1-p)^{\beta(n-k)}$. In
turn, her posterior would be beta distributed with shape parameters
$\tilde{a}_{n},\tilde{b}_{n}$ such that:

$\tilde{a}_{n}=\alpha k+\delta(a_{0}-1)+1$ (6)

$\tilde{b}_{n}=\beta(n-k)+\delta(b_{0}-1)+1$ (7)

where $\alpha,\beta>0$. And $\alpha\neq\beta$ indicates asymmetric reactions
to successes and failures in the DGP. Equations (6) and (7) are especially
interesting if the agent has preferences over the state-space. Suppose this is
the case; and suppose further that preferences are expressed by a utility
function, which is continuous and monotonically increasing over $p$. Then,
every success $k$ would be informative of a higher realisation of $p\in(0,1)$,
i.e. a preferred state. Conversely, every failure $(n-k)$ is informative of a
lower realisation of $p$. Therefore, successes and failures, can be
interpreted as pieces of good and bad news respectively. This means that
$\alpha>1$ (overreacting to positive information) or $\beta<1$ (underreacting
to negative information) can be interpreted as optimism bias; while $\alpha<1$
or $\beta>1$ would indicate pessimism. Furthermore, $\alpha>\beta$ represents
the good news effect, while $\alpha<\beta$ implies there is bad news
effect.888Asymmetries between successes and failures can also be present at
the level of the prior. These distinctions are introduced in section 4, but
left out of the main theoretical discussion. This is without loss of
generality as the resulting equations are identical to equations (9) and (10)
except for having separate $\rho$ and $\delta$ parameters for successes and
failures. (see section 4.2)

####  2.2.2 Confirmation Bias

Confirmation bias is modeled as a positive correlation between overreaction to
information signal $S$, and how confirming the signal is. The degree of
confirmation of the signal is unrelated to the agent holding preferences999A
discussion about the distinction between motivated and unmotivated
confirmation bias can be found in Zhenxun, 2023a  over the state-space. In
particular, the degree of confirmation $c$ is expressed as the area, in the
density function of the prior, comprised between the expected value of the
prior $E(\pi(p))$ and the mean of the information signal $k/n$. Formally:

$c=\bigg{|}\int_{(k\pm\varepsilon)/n}^{E(\pi(p))}\pi(p|a_{0},b_{0})\,dp\bigg{|}$
(8)

Equation (8) specifies a relative measure of confirmation101010In equation
(8), the signal mean $k/n$ is replaced by $(k\pm\varepsilon)/n$. This is
because $p$ is not strictly defined at $p=0$ and $p=1$. Thus, one must take
$(k+\varepsilon)/n$ as the inferior limit if $k=0$; and $(k-\varepsilon)/n$ if
$k=n$. (see Figure 1 for an example). The higher the value of $c$, the less
confirming a signal will be. To incorporate over or under reaction to
confirmation a separate term $\rho c$ is added to the distorted likelihood
function. Equations (6) and (7) are modified as follows:

$\tilde{a}_{n}=\alpha k+\rho c+\delta(a_{0}-1)+1$ (9)

$\tilde{b}_{n}=\beta(n-k)+\rho c+\delta(b_{0}-1)+1$ (10)

where $\rho<0$ indicates confirmation bias (as $c$ becomes smaller the
perception of the number of successes or failures grows); and $\rho>0$
indicates disconfirmation bias.

Figure 1: Example of confirmation measure for an updating problem with prior
mean $E(\pi(p))=0.5$ and information signal of 8 successes and 2 failures
($k/n=8/10$)

####  2.2.3 Sequence-related biases and unrelated inference biases

Over(under)-reaction to information signals can also be driven by other
reasons than the presence of preferences over the state-space. We consider two
kind of alternative deviations: Sequence-related biases (i.e. hot hand fallacy
and gambler’s fallacy) and over(under)-inference to information signals when
preference over the state-space are not at stake.

Sequence-related biases: Consider a partition of the signal space $S$. Namely,
$S_{1}=\\{s_{1}...s_{m}\\}$ and $S_{2}=\\{s_{m+1}...s_{n}\\}$ where $n>m$.
Suppose all of the $s_{i}\in S_{2}$ are either successes (i.e.
$\sum_{n-(m+1)}^{n}s_{i}=n-m+1$, $s_{i}\in S$) or failures (i.e.
$\sum_{n-(m+1)}^{n}s_{i}=0$, $s_{i}\in S$). In this context, the hot-hand
fallacy is defined as inferring too much from information signals after
observing a the last $n-m+1$ consecutive successes (i.e. $\alpha>1$) or
failures (i.e. $\beta>1$). Conversely, gambler’s fallacy is defined as
inference against the information signal after observing a the last $n-m+1$
consecutive successes (i.e. $\alpha<0$) or failures (i.e. $\beta<0$).

Inference biases: Consider an almost equivalent state-space set,
$\Omega_{NP}=(0,1)$ over which the agent form beliefs but where, contrary to
$\Omega$, the agent does not hold any preference. So any element $\hat{p}$ of
the set $\Omega_{NP}$, will have the characteristic that $u(\hat{p})=0$.
Because this is the only difference between sets, only $\alpha$ and $\beta$
coefficients in equations (9) and (10) differ between forming beliefs over
$\Omega$ or $\Omega_{NP}$. Therefore, when the agent forms beliefs over
$\Omega_{NP}$, $\alpha$ and $\beta$ coefficients can be interpreted as
over(under) inference for successes and failures, independent of preferences
over the state-space.

The biases in this subsection are only separably identified from one another
when the agent faces multiple belief-elicitation decisions and observes
different data generating processes coming from both $\Omega$ and
$\Omega_{NP}$.

####  2.2.4 Under and overconfidence

Confidence biases (overconfidence and underconfidence) are those which are
strictly related to the variance of the agent’s posterior distribution in
relation to the Bayesian variance. Overconfidence implies that the agent’s
overall distribution of posterior beliefs is more dispersed than that of a
Bayesian agent. Conversely, underconfidence yields a posterior distribution
which is more dispersed over the values of $p$ than the distribution a
Bayesian agent would have. Formally:

$\tilde{Var_{n}}=\nu\times Var_{n}$ (11)

where $\tilde{Var_{n}}$ is the variance of the agent’s posterior beta
distribution, and $Var_{n}$ is the Bayesian variance. In equation (11),
$\nu>1$ indicates overconfidence, while $\nu<1$ indicates underconfidence.

##  3 Experimental Design

The experiment tests the theoretical framework of section 2 both at the
individual and population level. The experiment was conducted at the
Behavioural and Experimental Economics Laboratory (BEELab) at Maastricht
University. 88 participants were recruited and, each of them had to make 30
belief-elicitation tasks. For each task, participants report their belief beta
distributions twice, after observing two consecutive information signals. The
average payment per participant was 15.9 euros. The experiment was pre-
registered in October, 2023 at aspredicted.org. The full instructions of the
experiment can be found in section D of the appendix.

###  3.1 The belief-elicitation task

In each of the 30 tasks, participants observe a pool of 99 urns; each urn
containing 100 balls. Each one of these urns contains a different distribution
of red and blue balls. That is, Urn 1 has only one red ball and 99 blue, in
Urn 2 there are only two red balls and 98 blue. This continues until Urn 99
where 99 balls are red and one is blue. Then, one of these urns is selected at
random (i.e. selected from a uniform distribution), but the content of the urn
is not revealed to the participants. Participants’ task is to guess the
percentage of red balls in the selected urn (see Figure 2).

Figure 2: Urn selection

In order to make their guesses, participants receive information signals by
observing two sequences of balls, drawn with replacement. In the first
sequence of draws, either one, two or three balls are extracted at random from
the selected urn. After seeing this sequence participants are asked to report
their belief distribution 111111see section 3.2 for details about the
elicitation of beta distributions (Figure 3(a)) about the percentage of red
balls in the selected urn. This first report, is taken as a prior. Once this
prior distribution is elicited, a second sequence of draws is drawn from the
same selected urn and shown to the subjects. In this case, either three, five
or seven balls are extracted at random from the urn. After observing this
second information signal, participants are asked to report their (posterior)
belief distribution121212Both in the prior and posterior elicitations,
participants need to update from a given default beta distribution. Before
eliciting their prior, the default beta distribution they see is a uniform
distribution (as this is the exogenously implemented “prior of the prior”).
Accordingly, the default beta distribution they update before eliciting their
posterior is their own prior report. (Figure 3(b)). Once this is done, a new
urn from the pool of 99 urns is selected, with replacement, and this process
is repeated thirty times. All of the subjects had an identical selection of
urns and sequences of draws, but the order of the tasks was randomised between
participants. All of the belief-elicitation tasks were incentiviced with a
binarised scoring rule (more details in section 3.2 and appendix B)

To assess the role of motivated beliefs, fifteen of the thirty urns (placed at
a random spots of the experiment) had an extra payment attached to it. I
henceforth refer to these urns as dollar urns. In particular, when
participants reported beliefs about these urns they receive a payment, in
cents, equal to the (unknown) number of red balls in the selected urn. This
means that participants should have a preference over the content of dollar
urns.

###  3.2 Eliciting beta distributions

I use a novel way of eliciting belief distributions, which is especially
convenient if the elicited belief distribution follows a specific functional
form, as opposed to already existing methods (Crosetto and de Haan,, 2022;
Harrison et al.,, 2017).

In order to report their beliefs, participants were presented with a dynamic
graphical interface that allows them to select their preferred beta
distribution (See figures 3(a) and 3(b)). By moving two sliders, each
associated with a different question, participants were able to select a
specific beta distribution. Those questions were:

1\. What percentage of red balls do you expect the selected urn to have?

2\. What is your uncertainty level about this percentage?

The first slider in figure, associated with question (1.), allows the subjects
to manipulate the expected value of a beta distribution, while the second
slider, associated with question (2.), allows the subjects to manipulate the
standard deviation of such beta distribution 131313The probability density
function of any beta distribution can also be parameterised by its expected
value and variance.. Participants were shown a five-minute explanatory video
on how to interpret the graph they selected and how to manipulate it. Subjects
were specifically instructed to solve the task graphically, and they were also
shown that the scale of the graph updates dynamically in order for the plot to
be informative. Participants also had the option to enable the graph to update
on a fixed scale if they desired to do so. Additionally, they answered related
comprehension questions 141414In order to participate in the experiment
subjects must have answered 3 out of 5 comprehension questions correctly. If
any comprehension question was answered incorrectly, participants had a second
chance to answer every question correctly and could test the software before
starting to answer the relevant belief-elicitation tasks.

In order to avoid the possibility that participants reported bimodal beta
distributions, the maximum range of the second slider (standard deviation)
depends on the value of the first slider (expected value). The possibility of
reporting bimodal beliefs was excluded for a number of reasons: First, it is
challenging to interpret what a bimodal beta distribution means in the context
of this experiment, secondly introducing a bimodal distribution could impair
the understanding of the graphical interface by the subjects; and thirdly, a
bimodal distribution could never be the Bayesian answer to the task, so
reporting a Bayesian distribution is an option which is never disallowed to
the participants.

(a) Figure 3a: Example of prior belief elicitation after observing a first
sequence of draws from a random urn showing “red, red”. This example shows a
specific selection of a scaled beta distribution with an expected value of 56
and a standard deviation of 18.47

(b) Figure 3b: Example of posterior belief elicitation after observing a
second sequence of draws showing “blue, red, blue, red, blue”. The updated
beta distribution is selected in this example shows an expected value of 51
and a standard deviation of 7.44

To incentivise truthful reporting, a binarised scoring rule was implemented in
every task. Specifically, I follow a similar method to the one suggested by
Schlag et al., (2015), who propose different scoring rules to incentivise
every moment of a probability distribution (see appendix B to understand the
specific scoring rule). At the same time, I closely follow Danz et al., (2022)
in not explicitly disclosing the exact scoring rule that is implemented.
Instead, participants were told that they should always truthfully report
their guess for the percentage red of balls, and their uncertainty level in
the selected urn (i.e. the mean and standard deviation of the beta
distribution) so as to maximise their expected payoff.

##  4 Baseline and Complete model regressions

This section presents the main analyses derived directly from the theoretical
framework of section 2. In Section 4.1, the baseline model regression is
outlined, introducing only a limited set of biases. Following that, Section
4.2 details the complete model (equations (14),(15) and (16)), encompassing
all biases of the model. The decision to present these two models separately
aims to facilitate comparison, investigating whether significant biases in the
baseline model persist when accounting for multiple belief biases. Both models
are executed and compared at both population and individual levels. In Section
4.3, a method is introduced to compare the relative importance of biases
proven significant at the individual level.

###  4.1 Baseline model regressions

The baseline model, closely resembles equations (4) and (5). It tests for
under(over)-inference and base rate neglect(overuse):

$\tilde{a}_{n}-1=\gamma_{s}k+\delta_{s}(a_{0}-1)+\varepsilon_{a}$ (12)

$\tilde{b}_{n}-1=\gamma_{f}(n-k)+\delta_{f}(b_{0}-1)+\varepsilon_{b}$ (13)

In equations (12) and (13), successes and failures (variables $k$ and
$(n-k)$), represent the number of red and blue balls observed in the second
sequence of draws provided to participants. Variables $a_{0}$ and $b_{0}$ are
the parameters of the beta distribution elicited by subjects after the first
sequence of draws, while $\tilde{a_{n}}$ and $\tilde{b_{n}}$ are the
parameters of the beta distribution elicited by subjects after observing the
second sequence of draws. $\varepsilon_{a}$ and $\varepsilon_{b}$ are the
error terms.

The parameter interpretation of equations (12) and (13) is akin to equations
(9) and (10). Parameters $\gamma$ and $\delta$ similarly indicate under/over
inference and base rate neglect/overuse respectively. However, equations (12)
and (13) differ by acknowledging that these biases ($\gamma_{s}$,$\gamma_{f}$)
and ($\delta_{s}$,$\delta_{f}$) may vary between successes and failures (i.e.
realisations of red and blue balls). The presence of those biases is tested by
comparing whether the estimated $\gamma$ and $\delta$ parameters are
significantly different from their Bayesian values (i.e.
$\gamma_{s}=\gamma_{f}=1$ and $\delta_{s}=\delta_{f}=1$).

Note that equations (12) and (13) do not have an intercept. This has a very
straightforward theoretical justification. If all the independent variables in
those equations take value $0$ (i.e. $k=0,n=0,a_{0}=1,b_{0}=1$), both
$\tilde{a}_{n}$ and $\tilde{b}_{n}$ take a value of $1$. This means that when
one starts updating from a uniform prior distribution ($a_{0}=1,b_{0}=1$), and
observes no information signals whatsoever ($k=0,n=0$), one must not update,
i.e. remain at such uniform prior after updating
($\tilde{a}_{n}=1,\tilde{b}_{n}=1$). Having an intercept different than zero
would imply that agents update in the absence of any kind of information.

###  4.2 Complete model regressions

In order to test for the presence of all the biases described in section 2.2,
I run a slightly modified version of equations (9), (10) and (11). Namely:

$\tilde{a}_{n}-1=(\alpha_{0}+\alpha_{Pref}I_{Pref}+\alpha_{Seq}I_{Seq_{s}})k+\rho_{s}c+\delta_{s}(a_{0}-1)+\varepsilon_{a}$
(14)

$\tilde{b}_{n}-1=(\beta_{0}+\beta_{Pref}I_{Pref}+\beta_{Seq}I_{Seq_{f}})(n-k)+\rho_{f}c+\delta_{f}(b_{0}-1)+\varepsilon_{b}$
(15)

$\tilde{Var_{n}}=\eta+\nu\times Var_{n}+\varepsilon_{v}$ (16)

Variables in equations (14) and (15) are identical to the baseline model with
the only exception of variable $c$, the relative measure of confirmation, as
described in section 2.2.2. Equation (16) is almost identical to equation (11)
in the interpretation of its variables and the $\nu$ parameter indicating over
or underconfidence ($\nu>1$ vs $\nu<1$). The inclusion of an intercept $\eta$
and an error term $\varepsilon_{v}$ are the only differences151515Similarly to
the baseline model, equations (14) and (15) do not have an intercept for the
very same reason as outlined in section 4.1..

Parameters $\rho$ and $\delta$ in equations (14) and (15) indicate
confirmation/disconfirmation bias, and base rate neglect/overuse respectively.
As in the baseline case, equations (14) and (15) differ by acknowledging that
these parameters may differ between successes and failures ($\rho_{s}$ vs
$\rho_{f}$) and ($\delta_{s}$ vs $\delta_{f}$). However, the notable
difference between equations (14), (15) and (9),(10) lies in accommodating the
biases of section 2.2.3. This involves the inclusion of three dummy variables
– $IPref$, $ISeq_{s}$, and $ISeq_{f}$ – interacting with the number of
successes (in (14)) or failures (in (15)). $IPref$ equals 1 when the subject
faces a decision with preferences over the state-space (that is, when beliefs
about a dollar urn where reported) and 0 otherwise. $ISeq_{s}$ ($ISeq_{f}$)
equals 1 when the last three balls observed in the second sequence are red
(blue). By doing so, one can distinguish motivated beliefs (optimism and
pessimism), sequence-related biases (gambler’s fallacy and hot hand fallacy)
and unrelated over/under inference. The presence of those biases is tested by
comparing whether the estimated parameters are significantly different from
their Bayesian values. Table 1 summarizes the baseline and complete models,
and specifically shows the ranges of values of the different parameters, which
correspond to each bias.

Baseline Model | Complete Model
---|---
Eq (12) (13) | Eq (14) (15) (16)
|  $\gamma_{s}$ or $\gamma_{f}>1$ Overinference
---
$\gamma_{s}$ or $\gamma_{f}<1$ Underinference
$\delta_{s}$ or $\delta_{f}>1$ Base Rate Overuse
$\delta_{s}$ or $\delta_{f}<1$ Base Rate Neglect
|  $\alpha_{0}$ or $\beta_{0}>1$ Overinference
---
$\alpha_{0}$ or $\beta_{0}<1$ Underinference
$\alpha_{0}+\alpha_{Pref}>1$ or $\beta_{0}+\beta_{Pref}<1$ Optimism
$\alpha_{0}+\alpha_{Pref}<1$ or $\beta_{0}+\beta_{Pref}>1$ Pessimism
$\alpha_{Pref}>\beta_{Pref}$ Good News Effect
$\alpha_{Pref}<\beta_{Pref}$ Bad News Effect
$\alpha_{0}+\alpha_{Seq}>1$ or $\beta_{0}+\beta_{Seq}>1$ Hot Hand Fallacy
$\alpha_{0}+\alpha_{Seq}<0$ or $\beta_{0}+\beta_{Seq}<0$ Gambler’s Fallacy
$\rho_{s}$ or $\rho_{f}<0$ Confirmation Bias
$\rho_{s}$ or $\rho_{f}>0$ Disconfirmation Bias
$\delta_{s}$ or $\delta_{f}>1$ Base Rate Overuse
$\delta_{s}$ or $\delta_{f}<1$ Base Rate Neglect
$\nu>1$ Overconfidence
$\nu<1$ Underconfidence
Table 1: Summary of biases for Baseline and Complete Model

###  4.3 The effect of each individual bias

In order to facilitate the comprehension and enhance the comparability of
biases in Table 1 for the individual analysis, I construct a measure that is
informative of the relative importance of each bias. This measure identifies
the specific effect of each bias in the relative changes of mean and variance
for each subject. In order to achieve this, I analyze the distortions to the
Bayesian expected value and Bayesian variance caused by exhibiting a
particular kind of bias. The idea is to compare these Bayesian values with a
bias-specific measure of how the expected value and variance would look like
if the subject exhibited only the specific bias we are interested in. I call
these measures Bias-specific expected value and Bias-specific variance. The
Bayesian expected value and variance of the posterior beta distribution are
given by:

$\displaystyle E_{n}$ $\displaystyle\equiv
E(p|a_{n},b_{n})=\frac{a_{n}}{a_{n}+b_{n}}$ (17) $\displaystyle Var_{n}$
$\displaystyle\equiv
Var(p|a_{n},b_{n})=\frac{a_{n}b_{n}}{(a_{n}+b_{n})^{2}(a_{n}+b_{n}+1)}$ (18)

The bias-specific expected value $E_{Bias}$ and bias-specific variance
$Var_{Bias}$161616In the case of confidence biases, $Var_{Bias}$ is already
given by $\tilde{Var}_{n}$, and $E_{Bias}$ would be unaffected. are given by:

$\displaystyle E_{Bias}$ $\displaystyle=\frac{a_{bias}}{a_{bias}+b_{bias}}$
(19) $\displaystyle Var_{Bias}$
$\displaystyle=\frac{a_{bias}b_{bias}}{(a_{bias}+b_{bias})^{2}(a_{bias}+b_{bias}+1)}$
(20)

where $a_{bias}$ and $b_{bias}$ are the hypothetical parameters of a posterior
beta distribution if they were distorted by only one specific
bias171717Because we are looking at the effect of specific biases for each
subject, it is theoretically possible that the effect of the bias in isolation
is so strong that it makes $a_{bias}$ or $b_{bias}$ negative. This would make
$E_{Bias}$ and $Var_{Bias}$ uninterpretable. Therefore, it is assumed that the
maximum effect that a bias in isolation can have, is such that the associated
parameter $a_{bias}$ or $b_{bias}$ is equal to 0 + $\varepsilon$. It is worth
nothing that these cases were very rare, as it would require strongly updating
against the information signal.. For example, suppose we want to evaluate the
impact of overinference. Suppose a particular subject exhibits overinference
only in successes: That is $\alpha_{0}>1$. Then,
$a_{bias}=a_{Overinfernce}=\alpha_{0}k+a_{0}$ and $b_{bias}=b_{n}$. Equations
(19) and (20) for every other bias are calculated analogously.

To capture the relative importance of each bias in inference, I compare the
distance between the Bayesian and distorted expected value and variance.
Namely:

$\displaystyle\Delta E_{Bias}$ $\displaystyle=|E_{n}-E_{Bias}|$ (21)
$\displaystyle\Delta Var_{Bias}$ $\displaystyle=|Var_{n}-V_{Bias}|$ (22)

Equations (21) and (22) allow us to quantify the discrepancy between the
Bayesian expected value and variance, and the bias-specific measures. These
discrepancies serve as indicative measures of the influence of each bias on
the overall inference process. Subsequently, by assessing these differences
across subjects, we can assess the relative impact of each individual bias.

##  5 Results

###  5.1 Population-level analysis

Table (2) compares the baseline model (equations (12) and (13)) with the
complete model (equations (14) (15) (16)) at the population level. Columns (1)
and (2) of the table refer to the baseline model, while columns (3), (4) and
(5) correspond to the complete model. All regressions in table (2) are run
with clustered standard errors by participant. Importantly, the significance
of each coefficient does not necessarily show whether an estimate is
significantly different from 0, but whether it is significantly different from
the corresponding Bayesian value for a given coefficient (see Table 1).

Table 2: Baseline and Complete models at the population level. Significance is with respect to Bayesian values. Clustered standard errors | Dependent variable:
---|---
|  $a$ $posterior$ |  $b$ $post$ |  $a$ $post$ |  $b$ $post$ |  $Variance$ $post$
| (1) | (2) | (3) | (4) | (5)
$Successes$ |  37.339∗∗ |  | 43.678 |  |
| (11.66) |  | (36.34) |  |
$a$ $prior$ |  0.017∗∗∗ |  |  0.016∗∗∗ |  |
| (0.02) |  | (0.057) |  |
$Failures$ |  |  76.199∗∗ |  | 67.102 |
|  | (31.96) |  | (40.072) |
$b$ $prior$ |  |  $-$0.0002∗∗∗ |  |  $-$0.001∗∗∗ |
|  | (0.01) |  | (0.019) |
$Success:preference$ |  |  |  $-$30.889 |  |
|  |  | (27.86) |  |
$Success:Seq_{pos}$ |  |  | 14.726 |  |
|  |  | (32.75) |  |
$Failures:preference$ |  |  |  |  $-$27.602 |
|  |  |  | (71.12) |
$Failures:Seq_{neg}$ |  |  |  | 102.488 |
|  |  |  | (99.13) |
$Confirmation$ |  |  | 0.001 |  $-$0.096 |
|  |  | (0.001) | (0.09) |
$Bayesian$ $variance$ |  |  |  |  | 0.983
|  |  |  |  | (0.076)
$Constant$ |  |  |  |  |  0.002∗∗
|  |  |  |  | (0.001)
Observations | 2,640 | 2,640 | 2,636 | 2,636 | 2,640
R2 | 0.002 | 0.003 | 0.002 | 0.004 | 0.289
Adjusted R2 | 0.001 | 0.002 | 0.00000 | 0.002 | 0.288
Note: |  ∗p$<$0.1; ∗∗p$<$0.05; ∗∗∗p$<$0.01 from the Bayesian estimate

In the baseline model, there are two significant biases at the population
level: Overinference and base rate neglect. We see overinference both in
successes ($\hat{\gamma}_{s}>1$; $\hat{\gamma}_{s}=37.339$ ) and failures
($\hat{\gamma}_{f}>1$; $\hat{\gamma}_{f}=76.119$). This can be interpreted as
subjects, on average, interpreting each red ball they observe as 37 red balls;
and each blue ball they see as 76 blue balls, when they form their beliefs.
And we see base rate neglect (also in successes ($\hat{\delta}_{s}<1$;
$\hat{\delta}_{s}=0.017$) and failures $\hat{\delta}_{f}<1$;
$\hat{\delta}_{f}=-0.0002$)). This can be interpreted as subjects ignoring
their priors (number of red/blue balls they expected to see) when they form
their beliefs.

Despite the significance of overinference and base rate neglect at the
population level in the baseline model, the effect of overinference seems to
vanish when all of the biases of the model are controlled for (Columns (3),
(4) and (5)), and base rate neglect is the only remaining significant bias.
The complete model also minimizes the AIC and BIC as show in Table 3 (Appendix
C). This precisely shows that the lack of inclusion of a wider array of biases
within the same model, makes some biases look artificially
significant181818Additionally, comparing the adjusted $R^{2}$ column for the
individual baseline and complete models in Table 4 further emphasises this
point..

However, neither the baseline nor the complete model seem to be explanatory of
subject’s posterior belief formation at the population level. This can be seen
by the extremely low goodness of fit ($R^{2}$) of Columns (1) to (4) in Table
2. Yet, this is not the result of subjects being completely noisy, but rather
a consequence of individual heterogeneity. That is, subjects are distinctly
biased, but they exhibit widely different biases. These distinct biases
average out at the population level, but clearly present at the individual
level. Table 4 (Appendix C) compares the $R^{2}$ of all the equations at the
population with the mean of all $R^{2}$s in these same equations after running
the model at the individual level. As one can see in the first and second
column of the table, the goodness of fit from the population to the individual
model shows a substantial improvement191919With the only exception of the
variance equation (Eq. (16)).. This is also the case for both the AIC and BIC
criteria (columns (3) and (4) of table 3), which tremendously diminish when
compared to running the models at the population level, with the only
exception of the variance equation (equation (16)), which performs better at
the population level. This is evidence pointing to the fact that there is
considerable individual heterogeneity in the data, and therefore, the model
would benefit from an individual analysis.

###  5.2 Individual-level analysis

Figures 4(a) and 4(b) provide an overview of the biases detected at the
individual level within the context of the two models: the baseline and the
complete model. In Figure 4(a), the bar chart depicts the occurrences of
statistically significant biases at the 5% level within the baseline model.
The color-coded representation distinguishes biases: red means significance
solely for successes, blue for failures, and green for significance in both
successes and failures202020The column “against” classifies those individuals
who update against the information signals.. Notably, overinference and base
rate overuse emerge prominently. However, Figure 4(b), illustrating individual
biases in the complete model, shows a more complete picture. Here, a broader
array of biases is accounted for, with the hot hand effect emerging as the
most commonly exhibited bias among the subjects. Further comparing these
models yields intriguing insights. First, at the descriptive level, it becomes
evident that individual heterogeneity persists across both models. This
observation underscores the varying proportions with which biases manifest
among individuals: All tested biases were present among subjects to a certain
extent. That is, there is no bias which fully vanishes after incorporating the
complete set of biases. Notably, the prevalence of the hot hand fallacy
contrasts sharply with the sparsity of confirmatory biases or overconfidence
among subjects. But secondly, the complete model makes another important
revelation: every subject exhibits at least one form of bias when the full
spectrum of biases is considered.

While the “No Bias” column in Figure 4(a) shows that 15 individuals (17% of
the sample) could not be categorised, this column is empty in Figure 4(b).
This implies that the shift from the baseline to the complete model is able to
identify biased individuals, who would otherwise be categorised as either
Bayesian or “too noisy”212121Because the individual-level analysis tests many
null hypotheses (eleven per subject), one may wonder whether some of these
results are driven by a high rate of false positives. To address this concern,
section C.2 of the appendix replicates Figures 4(a) and 4(b) for a 1%
significance level..

(a) Figure 4a: Number of times a specific bias is found to be significant
($p<0.05$) in the baseline model at the individual level. Overinference and
base rate overuse are the most common biases. 15 individuals (17%) exhibit no
bias or are too noisy to be distinguished from a Bayesian agent.

(b) Figure 4b: Number of times a specific bias is found to be significant
($p<0.05$) in the complete model at the individual level. There is significant
individual heterogeneity. All subjects exhibit at least one type of bias. The
hot hand fallacy is found to be the most common bias.

Despite these analyses showing the prevalence of different biases at the
individual level, it is also paramount to also consider how much each of these
biases drives inference in more tangible and common units. Thus, I apply the
methodology outlined in section 4.3, to asses the relative importance of each
bias.

Figures 5(a) and 5(b) provide an examination of biases within the complete
model, focusing on the expected value deviations from the Bayesian framework
for each specific bias. While both figures provide insights into how important
these deviations are, they take different approaches. Figure 5(a) presents the
expected value deviations for each bias, irrespective of their frequency of
occurrence. On the other hand, Figure 5(b) adjusts for frequency by weighting
these deviations with the number of times a bias was found to be statistically
significant.

#### Effects on Expected Value and Variance

(a) Figure 5a: Average expected-value deviations from each individual bias.

(b) Figure 5b: Expected-value deviations adjusted by the amount of times a
bias was found to be significant.

In Figure 5(a), two biases, gambler’s fallacy and optimism, stand out with
prominent expected value deviations, emphasizing their substantial impact on
inference. In contrast, confirmation and disconfirmation bias exhibit less
pronounced deviations. Remarkably, even after correcting for frequency in
Figure 5(b), gambler’s fallacy and optimism retain their prominence,
suggesting their enduring influence on biased inference.

Beyond these observations, Figure 5(a) and 5(b) also reveal interesting
patterns. The biases exerting the most substantial influence on expected value
deviations cluster into two distinct categories: Motivated beliefs (optimism
and pessimism), and biases associated with updating against the information
signal, including gambler’s fallacy. Additionally, there is a notable
observation regarding the hot hand fallacy: While identified as the most
common bias in Figure 4(b), its impact on expected value deviations appears
relatively modest in Figure 5(a). However, when it comes to its effect on
variance, the hot hand fallacy demonstrates a notably strong influence, as can
be seen in Figures 6(a) and 6(b).

Figure 6(a) and 6(b) are analogous to 5(a) and 5(b), but represent variance
deviations with respect to the Bayesian framework instead. While the relative
contribution of each bias appears to be quite homogeneous (Figure 6(a)), once
we correct for the frequency such biases (figure 6(b)), the hot hand fallacy
shows its prominence, while optimism and pessimism come second and third
respectively. This underlines the importance of motivated-belief biases in
overall inference: Both in expected-value and variance deviations.

(a) Figure 6a: Average variance deviations from each individual bias.

(b) Figure 6b: Variance deviations adjusted by the amount of times a bias was
found to be significant.

Putting it all together, this subsection emphasises the importance of
incorporating a more complete spectrum of biases into account. When doing so,
all biases are present at the individual level and there is no single subject
who does not exhibit some kind of bias. It also showcases a key interplay
between bias frequency and inference. While, the hot hand fallacy is the most
common bias, it is mainly so through its effect on variance. That is,
overreaction to a consecutive set of signals in the same direction does not
seem to distort the perception of how likely the outcome is as much as
distorting how certain individuals are. However, optimism and pessimism seem
to be driving inference in both dimensions. Overall, in spite of the presence
of individual heterogeneity, two categories of biases (sequence-related biases
and motivated-belief biases) clearly stand out above the rest.

##  6 Concluding remarks

In conclusion, this paper addresses the challenges in identifying and testing
multiple belief-updating biases simultaneously. This is important because in
the absence of such a model, many of these biases could be potentially
confounded. The theoretical model introduces a novel approach by considering
belief distributions over a continuous state space, allowing for a complete
identification of multiple conflicting belief biases. The corresponding
laboratory experiment applies this model, using a novel technique to elicit
belief distributions.

The population-level analysis shows that, despite the prevalence of
overinference and base rate neglect in the baseline model, only base rate
neglect remains significant in the complete model, emphasizing the importance
of accounting for a broader spectrum of biases. Individual-level analysis
uncovers significant heterogeneity, with every participant exhibiting at least
one identifiable bias. Further exploration into biases’ impact on expected
value and variance exposes motivated-belief biases (optimism and pessimism)
and sequence-related biases (gambler’s and hot hand fallacies) as key drivers
of biased inference. Thus, capturing a wide array of biases provides a much
more subtle understanding of belief updating dynamics. Overall, addressing
confoundedness in belief-updating biases is key to researchers interested in
understanding links between different biases, or delving deeper into the
causes of various behavioural phenomena (political polarization, investing
performance, disposition effect…) when conflicting updating biases are at
stake.

I would like to conclude by pointing to some related future research avenues.
While this paper takes detailed care in separating behaviour stemming from
different biases, it is important to note that it approaches biases as
systematic departures from rational behaviour. That is, it does not dig into
the cognitive mechanisms or channels by which a certain bias might be
exhibited. In this direction Bordalo et al., 2023a , look at how some biases
become more prominent when salient features of the information become more
relevant. At the same time, Bordalo et al., 2023b  also emphasise the
importance of memory and recall to explain well-known cognitive biases.
Further exploring more of these cognitive mechanisms in a setting where biases
can be de-confounded, could deliver a complete picture of what updating biases
matter the most, and pinpoint the primitive traits that underlie biased
behaviour.

## References

  * Abdellaoui et al., (2021) Abdellaoui, M., Hill, B., Kemel, E., and Maafi, H. (2021).  Ambiguity attitudes and learning.  Working Paper.
  * Ahmad and Shah, (2020) Ahmad, M. and Shah, S. Z. A. (2020).  Overconfidence heuristic-driven bias in investment decision-making and performance: mediating effects of risk perception and moderating effects of financial literacy.  Journal of Economic and Administrative Sciences, 38(1):60–90.
  * Alós-Ferrer and Garagnani, (2023) Alós-Ferrer, C. and Garagnani, M. (2023).  Part-time bayesians: Incentives and behavioral heterogeneity in belief updating.  Management Science.
  * Augenblick et al., (2021) Augenblick, N., Lazarus, E., and Thaler, M. (2021).  Overinference from weak signals and underinference from strong signals.  arXiv preprint arXiv:2109.09871.
  * Ba et al., (2022) Ba, C., Bohren, J. A., and Imas, A. (2022).  Over-and underreaction to information.  Available at SSRN.
  * Benjamin et al., (2019) Benjamin, D., Bodoh-Creed, A., and Rabin, M. (2019).  Base-rate neglect: Foundations and implications.  Working Paper, 1:62.
  * Benjamin, (2019) Benjamin, D. J. (2019).  Errors in probabilistic reasoning and judgment biases.  Handbook of Behavioral Economics: Applications and Foundations 1, 2:69–186.
  * (8) Bordalo, P., Conlon, J. J., Gennaioli, N., Kwon, S. Y., and Shleifer, A. (2023a).  How people use statistics.  Technical report, National Bureau of Economic Research.
  * (9) Bordalo, P., Conlon, J. J., Gennaioli, N., Kwon, S. Y., and Shleifer, A. (2023b).  Memory and probability.  The Quarterly Journal of Economics, 138(1):265–311.
  * Bucher and Glimcher, (2022) Bucher, S. and Glimcher, P. (2022).  Dynamics of base-rate neglect: Disregarding prior probabilities while accumulating evidence.  Available at SSRN 4068498.
  * Charness and Dave, (2017) Charness, G. and Dave, C. (2017).  Confirmation bias with motivated beliefs.  Games and Economic Behavior, 104:1–23.
  * Crosetto and de Haan, (2022) Crosetto, P. and de Haan, T. (2022).  Comparing input interfaces to elicit belief distributions.  working paper or preprint.
  * Danz et al., (2022) Danz, D., Vesterlund, L., and Wilson, A. J. (2022).  Belief elicitation and behavioral incentive compatibility.  American Economic Review, 112(9):2851–2883.
  * Del Vicario et al., (2017) Del Vicario, M., Scala, A., Caldarelli, G., Stanley, H. E., and Quattrociocchi, W. (2017).  Modeling confirmation bias and polarization.  Scientific reports, 7(1):40391.
  * Edwards, (1968) Edwards, W. (1968).  Conservatism in human information processing.  Formal representation of human judgment.
  * Eil and Rao, (2011) Eil, D. and Rao, J. M. (2011).  The good news-bad news effect: asymmetric processing of objective information about yourself.  American Economic Journal: Microeconomics, 3(2):114–138.
  * (17) Enke, B., Gneezy, U., Hall, B., Martin, D., Nelidov, V., Offerman, T., and Van De Ven, J. (2023a).  Cognitive biases: Mistakes or missing stakes?  Review of Economics and Statistics, 105(4):818–832.
  * Enke and Graeber, (2023) Enke, B. and Graeber, T. (2023).  Cognitive uncertainty.  The Quarterly Journal of Economics, 138(4):2021–2067.
  * (19) Enke, B., Graeber, T., and Oprea, R. (2023b).  Confidence, self-selection, and bias in the aggregate.  American Economic Review, 113(7):1933–1966.
  * Gneezy et al., (2023) Gneezy, U., Hoffman, M., Lane, M. A., List, J. A., Livingston, J. A., and Seiler, M. J. (2023).  Can wishful thinking explain evidence for overconfidence? an experiment on belief updating.  Oxford Economic Papers, 75(1):35–54.
  * Grether, (1980) Grether, D. M. (1980).  Bayes rule as a descriptive model: The representativeness heuristic.  The Quarterly journal of economics, 95(3):537–557.
  * Harrison et al., (2017) Harrison, G. W., Martínez-Correa, J., Swarthout, J. T., and Ulm, E. R. (2017).  Scoring rules for subjective probability distributions.  Journal of Economic Behavior & Organization, 134:430–448.
  * Heger and Papageorge, (2018) Heger, S. A. and Papageorge, N. W. (2018).  We should totally open a restaurant: How optimism and overconfidence affect beliefs.  Journal of Economic Psychology, 67:177–190.
  * Heinke et al., (2023) Heinke, S., Trutmann, K., and Rieskamp, J. (2023).  Belief updating and investment decisions: The impact of good or bad news varies with prior returns.  Available at SSRN 3935798.
  * Huffman et al., (2022) Huffman, D., Raymond, C., and Shvets, J. (2022).  Persistent overconfidence and biased memory: Evidence from managers.  American Economic Review, 112(10):3141–3175.
  * Jiao, (2017) Jiao, P. (2017).  Belief in mean reversion and the disposition effect: An experimental test.  Journal of Behavioral Finance, 18(1):29–44.
  * Jiao et al., (2020) Jiao, P., Veiga, A., and Walther, A. (2020).  Social media, news media and the stock market.  Journal of Economic Behavior & Organization, 176:63–90.
  * Kahneman et al., (2021) Kahneman, D., Sibony, O., and Sunstein, C. R. (2021).  Noise: a flaw in human judgment.  Hachette UK.
  * Khaw et al., (2021) Khaw, M. W., Stevens, L., and Woodford, M. (2021).  Individual differences in the perception of probability.  PLoS computational biology, 17(4):e1008871.
  * Möbius et al., (2022) Möbius, M. M., Niederle, M., Niehaus, P., and Rosenblat, T. S. (2022).  Managing self-confidence: Theory and experimental evidence.  Management Science, 68(11):7793–7817.
  * Ortoleva and Snowberg, (2015) Ortoleva, P. and Snowberg, E. (2015).  Overconfidence in political behavior.  American Economic Review, 105(2):504–535.
  * Pitkäjärvi, (2022) Pitkäjärvi, A. (2022).  Slow belief updating and the disposition effect.  Available at SSRN 4231331.
  * Pouget et al., (2017) Pouget, S., Sauvagnat, J., and Villeneuve, S. (2017).  A mind is a terrible thing to change: confirmatory bias in financial markets.  The Review of Financial Studies, 30(6):2066–2109.
  * Rabin, (2002) Rabin, M. (2002).  Inference by believers in the law of small numbers.  The Quarterly Journal of Economics, 117(3):775–816.
  * Rabin and Schrag, (1999) Rabin, M. and Schrag, J. L. (1999).  First impressions matter: A model of confirmatory bias.  The quarterly journal of economics, 114(1):37–82.
  * Rabin and Vayanos, (2010) Rabin, M. and Vayanos, D. (2010).  The gambler’s and hot-hand fallacies: Theory and applications.  The Review of Economic Studies, 77(2):730–778.
  * Schlag et al., (2015) Schlag, K. H., Tremewan, J., and Van der Weele, J. J. (2015).  A penny for your thoughts: A survey of methods for eliciting beliefs.  Experimental Economics, 18:457–490.
  * Serra-Garcia and Gneezy, (2021) Serra-Garcia, M. and Gneezy, U. (2021).  Mistakes, overconfidence, and the effect of sharing on detecting lies.  American Economic Review, 111(10):3160–3183.
  * Tversky and Kahneman, (1974) Tversky, A. and Kahneman, D. (1974).  Judgment under uncertainty: Heuristics and biases: Biases in judgments reveal some heuristics of thinking under uncertainty.  science, 185(4157):1124–1131.
  * (40) Zhenxun, L. (2023a).  Dual elicitation of motivated and unmotivated confirmation bias.  Working Paper.
  * (41) Zhenxun, L. (2023b).  A model of confirmation bias.  Working Paper.
  * Zimmermann, (2020) Zimmermann, F. (2020).  The dynamics of motivated beliefs.  American Economic Review, 110(2):337–363.

Appendix to Belief Bias Identification

##  Appendix A Conjugate analyses of the binomial distribution

This section shows the derivation of equations of posterior parameters in
section 2. I present two cases: The Bayesian case (eq. (3)), and introducing
the biases of the baseline model (eq.(4) and (5)). The rest of the proofs for
the remaining biases are analogous to these cases. Once the distorted
likelihood and prior are given, the proof follows the exact same steps.

###  A.1 The Bayesian case

######  Statement 1.

Given likelihood equation (1) and prior equation (2) of section 2.1., a
Bayesian agent updates her beliefs such that her posterior distribution of
$p$: $\pi(p|a_{n},a_{n})$ is beta distributed with parameters $a_{n},b_{n}$
such that:

$a_{n}=k+a_{0}\qquad b_{n}=n-k+b_{0}$

###### Proof.

Let us apply Bayes’ Theorem given likelihood eq.(1) and prior eq.(2). This
yields:

$\displaystyle\pi(p|s_{1}...s_{n},a_{0},b_{0})$
$\displaystyle=\frac{L(p|s_{1}...s_{n})\pi(p|a_{0},b_{0})}{\int_{p=0}^{1}\big{(}L(p|s_{1}...s_{n})\pi(p|a_{0},b_{0})\big{)}dp}$
$\displaystyle=\frac{{n\choose
k}p^{k+a_{0}-1}(1-p)^{n-k+b_{0}-1}/B(a_{0},b_{0})}{\int_{p=0}^{1}\Big{(}{n\choose
k}p^{k+a_{0}-1}(1-p)^{n-k+b_{0}-1}/B(a_{0},b_{0})\Big{)}dp}$
$\displaystyle=\frac{p^{k+a_{0}-1}(1-p)^{n-k+b_{0}-1}}{B(a_{0}+k,b_{0}+n-k)}$

Which is itself the probability density function of a beta distribution with
parameters ($a_{0}+k,b_{0}+n-k$). Therefore, the posterior Bayesian
distribution is beta distributed with parameters ($a_{n},b_{n}$) as defined in
Statement 1.

∎

###  A.2 Biases of the baseline model: Inference and base-rate biases

######  Statement 2.

Given likelihood $\tilde{L}(p|s_{1}...s_{n})=(L(p|s_{1}...s_{n}))^{\gamma}$
and prior $\tilde{\pi}(p)=(\pi(p))^{\delta}$ of section 2.1., where
$\gamma,\delta$ indicate deviations from Bayesian updating, a non-Bayesian
agent follows a posterior beta distribution with parameters
$\tilde{a}_{n},\tilde{b}_{n}$ such that:

$\tilde{a}_{n}=\gamma k+\delta(a_{0}-1)+1$

$\tilde{b}_{n}=\gamma(n-k)+\delta(b_{0}-1)+1$

###### Proof.

Applying Bayes theorem yields:

$\displaystyle\tilde{\pi}(p|s_{1}...s_{n},a_{0},b_{0})$
$\displaystyle=\frac{(L(p|s_{1}...s_{n}))^{\gamma}(\pi(p))^{\delta}}{\int_{p=0}^{1}\big{(}(L(p|s_{1}...s_{n}))^{\gamma}(\pi(p))^{\delta}\big{)}dp}$
$\displaystyle=\frac{\big{[}{n\choose k}\big{]}^{\gamma}p^{\gamma
k}(1-p)^{\gamma(n-k)}\big{(}1/B(a_{0},b_{0})\big{)}^{\delta}p^{\delta(a_{0}-1}(1-p)^{\delta(b_{0}-1)}}{\int_{p=0}^{1}\Big{(}\big{[}{n\choose
k}\big{]}^{\gamma}p^{\gamma
k}(1-p)^{\gamma(n-k)}\big{(}1/B(a_{0},b_{0})\big{)}^{\delta}p^{\delta(a_{0}-1}(1-p)^{\delta(b_{0}-1)}\Big{)}dp}$
$\displaystyle=\frac{p^{\gamma
k+\delta(a_{0}-1)}(1-p)^{\gamma(n-k)+\delta(b_{0}-1)}}{\int_{p=0}^{1}\Big{(}p^{\gamma
k+\delta(a_{0}-1)}(1-p)^{\gamma(n-k)+\delta(b_{0}-1)}\Big{)}dp}$
$\displaystyle=\frac{p^{\gamma
k+\delta(a_{0}-1)}(1-p)^{\gamma(n-k)+\delta(b_{0}-1)}}{B\Big{(}\gamma
k+\delta(a_{0}-1)+1,\gamma(n-k)+\delta(b_{0}-1)+1\Big{)}}$

This is the probability density function of a beta distribution with
parameters $\Big{(}\gamma
k+\delta(a_{0}-1)+1,\gamma(n-k)+\delta(b_{0}-1)+1\Big{)}$. Therefore, the
posterior distribution of an agent that exhibits inference bias $\gamma$, and
base-rate bias ($\delta$) is beta distributed with parameters
($\tilde{a_{n}},\tilde{b_{n}}$) as defined in Statement 2. ∎

##  Appendix B Incentivising mean and variance of beta distributions

The scoring rules follow Schlag et al., (2015) with a slight modification. In
particular, random realisations of the Bayesian posterior distributions are
taken as the random draws.

###  B.1 Incentivising the mean

Let $\tilde{m}$ be the reported mean of the agent’s posterior beta
distribution $\pi(p|\tilde{a}_{n},\tilde{b}_{n})$, and let $d$ be a random
draw of the Bayesian posterior beta distribution $\pi(p|a_{n},b_{n})$. Then
the Quadratic Scoring Rule is given by
$g_{QSR}(\tilde{m},d)=-(\tilde{m}-d)^{2}$. Let $A,B$ be the boundaries of the
state-space $\Omega$, and $M$ be any arbitrary amount of money222222In the
experiment $A=1$, $B=99$ and $M=25/3$ cents for each report (that is a total
maximum of 10 euros). Then, the randomised quadratic scoring rule is given by
the following lottery:

$\tilde{g}_{QSR}(\tilde{m},d)=l\Bigg{(}M,0;1+\frac{g_{QSR}(\tilde{m},d)}{(B-A)^{2}}\Bigg{)}$

###  B.2 Incentivising the variance

Let $\tilde{v}$ be the the reported variance of the agent’s posterior beta
distribution. In order to elicit the variance consider two random draws of the
agent’s posterior beta distribution $\pi(p|\tilde{a}_{n},\tilde{b}_{n})$.
Namely, $d_{1}$ and $d_{2}$. Then the variance scoring rule is given by
$g_{v}(\tilde{v},d_{1},d_{2})=-\Big{(}\tilde{v}-\frac{1}{2}(d_{1}-d_{2})^{2}\Big{)}^{2}$.
Applying randomisation, the randomised variance scoring rule yields:

$\tilde{g}_{v}(\tilde{v},d_{1},d_{2})=l\Bigg{(}M,0;\frac{g_{v}+\frac{1}{4}(B-A)^{4}}{\frac{1}{4}(B-A)^{4}}\Bigg{)}$

##  Appendix C Extra tables and figures

###  C.1 Information criteria and $R^{2}$ comparisons

Model | AIC(pop.) | BIC(pop.) | AIC(ind.) | BIC(ind.)
---|---|---|---|---
Baseline model |  |  |  |
Eq. (12) | 42454.31 | 42466.07 | 270.41 | 274.61
Eq. (13) | 43636.76 | 43648.52 | 265.84 | 270.05
Complete model |  |  |  |
Eq. (14) | 42398.72 | 42428.1 | 267.99 | 276.39
Eq. (15) | 43577.64 | 43607.03 | 254.45 | 262.84
Eq. (16) | -22181.66 | -22169.9 | -187.21 | -183.01
Table 3: Information Criteria Comparison at population and individual level Model |  $R^{2}$ Population | Mean $R^{2}$ Ind. | Mean adj.$R^{2}$ Ind. |
---|---|---|---|---
Baseline model |  |  |  |
Eq. (12) | 0.002 | 0.564 | 0.533 |
Eq. (13) | 0.003 | 0.533 | 0.501 |
Complete model |  |  |  |
Eq. (14) | 0.002 | 0.761 | 0.713 |
Eq. (15) | 0.003 | 0.812 | 0.774 |
Eq. (16) | 0.289 | 0.139 | 0.108 |
Table 4: $R^{2}$ Comparison for Baseline and Complete model at population vs
individual level

###  C.2 Individual bias frequency at a 1% significance level

As the individual-level analysis in section 5.2 tests a considerable number of
null hypotheses (eleven per subject), one may wonder whether some of these
results are driven by a high rate of false positives. To address this concern,
this section replicates Figures 4(a) and 4(b) for a 1% significance level,
lowering the rate of potential false positives.

(a) Figure 7a: Number of times a specific bias is found to be significant
($p<0.01$) in the baseline model at the individual level.

(b) Figure 7b: Number of times a specific bias is found to be significant
($p<0.01$) in the complete model at the individual level.

As one can see from figures 7(a) and 7(b), the three main results from section
5.2 still hold. Namely, i) all biases are exhibited to a certain extent in the
complete model, ii) transitioning from the baseline to the complete model
shows that all subjects exhibit some kind of bias (empty “No Bias” column),
and iii) the hot hand effect stands out as the most-frequently exhibited bias.

In fact, the second result seems to be stronger in this condition, with the
vast majority of subjects appearing to be “too noisy” in the baseline model,
but showing some type of bias in the complete model. However, the frequency of
significant motivated-belief biases (optimism/pessimism) does diminish under
$p<0.01$.

##  Appendix D Experiment instructions

In this study, you will be asked to complete 30 guessing tasks. For each
guessing task you have to make 2 related guesses. At the beginning of each
guessing task, there is always a pool of 99 URNS, each containing 100 BALLS.
Some balls in the urns are red, and some are blue. Each one of these urns
contains a different percentage of red balls. For example, in Urn 1 there is
only one red ball and 99 blue (1% of the balls are red), in Urn 2 there are
only two red balls and 98 blue (2% of the balls are red). This continues until
Urn 99 where 99 balls are red and one is blue. (See picture below).

For each guessing task, out of these 99 urns, one of them (say Urn X) has been
selected at random. Each urn has the same chances of being selected from the
pool. That is, you do not know how many of the balls are red and how many of
them are blue in the selected urn. All combinations are possible. (See picture
below).

Your task is to guess what the percentage of red balls in the selected urn is.
In each one of the 30 guessing tasks, this process will be repeated with a new
urn.

Bear in mind: Whenever a new urn is selected, it is always drawn from the same
pool of 99 urns (WITH REPLACEMENT). This means that the same urn can be chosen
either once or multiple times. The urns have letters (or combinations of
letters) on top of them. This is just to highlight the fact that when a new
letter(or combination of letters) is on top of the urn, it means that a new
urn has been drawn.

The guessing task

This section will explain how each one of the 30 guessing tasks works.
Initially, as explained above, the computer has randomly selected, with equal
probability, ONE out of the 99 urns. Remember that initially, you know nothing
about the content of this urn. Once a given urn is selected, you will be given
some information about the urn to help you make your guess. First, you will
see a sequence of balls which have been randomly drawn from the urn. Each ball
in the selected urn has the same chances of being drawn, and each of these
draws is done WITH REPLACEMENT. This means that after a ball has been drawn
and taken out of the urn, it is immediately replaced with one of the same
colour. The urn will always have the same 100 balls. In order to help you make
your guesses, two sequences of draws will be made from each urn.

  * •

1st sequence of draws: Either one, two or three balls from the urn are
selected at first. After this selection, you will have to make your first
guess. For your first guess you will answer two questions:

    1. 1.

What percentage of red balls do you expect the selected urn to have?

    2. 2.

What is your uncertainty level about this percentage?

Example: Guessing Task A

  * •

2nd sequence of draws: The second draw follows very similar rules to the first
one. In this case three, five or seven balls from THE SAME URN are selected
with replacement. After this selection, you will have to make your second
guess. Once again, for your second guess you will answer the same two
questions:

    1. 1.

What percentage of red balls do you expect the selected urn to have?

    2. 2.

What is your uncertainty level about this percentage?

In order to further help you with your guesses a dynamic graph of your choice
will be provided. Please watch the following video (next screen) to understand
how this works.

——————————————[Page Break]—————————————

Click here to see the explanatory video

——————————————[Page Break]—————————————

Your Payment

You can earn up to €29,70 in this experiment. In particular, your payment is
broken down as follows:

  * •

You will receive €5 for taking the time to complete this experiment.

  * •

You will receive up to €10 for your responses related to the guessing tasks.

How much of this amount (€10) you receive depends on the actual percentage of
red balls in the selected urn. You can get money for EVERY SINGLE ONE of your
guesses.

The payment rule we use, is optimized so that in order to maximize your
expected payoff, you should ALWAYS give your best estimate of the percentage
of red balls in the selected urn. In the same manner, the payment rule we use,
is also optimized so that in order to maximize your expected payoff, you
should ALWAYS give your best estimate of your uncertainty level.

  * •

You can receive up to €14,70 as extra payment.

This extra payment is divided across the 30 guessing tasks. Whether or not a
guessing task has an extra payment attached depends on whether a dollar urn
has come up. (See picture below. Urn A is in this example, a dollar urn). In
particular, a dollar urn will come up randomly in 15 of the 30 guessing tasks.

If a dollar urn comes up, you will receive as many cents as red balls the
selected dollar urn has. For example, if the selected dollar urn has 50 red
balls you will get 50 cents.

At the end of the experiment, you will be informed about the number of red
balls in each urn and your total payment. If you want to know more about the
details of the payment rule, you can let me know after the experiment or write
an email to<EMAIL_ADDRESS>
——————————————[Page Break]—————————————

Get Familiar with the Tools

Before you answer the comprehension questions you have the chance to get
familiar with the guesses, the payment and the sliders with a trial guessing
task.

——————————————[Page Break]—————————————