# Social learning via actions in bandit environments111I am thankful to Ian
Ball, Stephen Morris, and Kartik Vira for detailed comments and feedback. I
also thank Anne Carlstein and Nathaniel Hickok for insightful suggestions.

Aroon Narayanan

###### Abstract

I study a game of strategic exploration with private payoffs and public
actions in a Bayesian bandit setting. In particular, I look at cascade
equilibria, in which agents switch over time from the risky action to the
riskless action only when they become sufficiently pessimistic. I show that
these equilibria exist under some conditions and establish their salient
properties. Individual exploration in these equilibria can be more or less
than the single-agent level depending on whether the agents start out with a
common prior or not, but the most optimistic agent always underexplores. I
also show that allowing the agents to write enforceable ex-ante contracts will
lead to the most ex-ante optimistic agent to buy all payoff streams, providing
an explanation to the buying out of smaller start-ups by more established
firms.

Keywords: strategic exploration, bandits, social learning, heterogenous priors
JEL Classification: D83

##  1 Introduction

Many choices that economic agents face, be it a consumer deciding whether to
try a new restaurant or a firm deciding whether to invest in a new technology,
involve deciding between a risky and a safe option. The standard way to model
this dichotomy is in the form of a bandit problem, in which an agent faces the
choice of choosing between two arms to pull, representing the two actions.
Moreover, many of the environments that economic agents operate in are social
- they explore new options at the same time as other agents who are doing the
same. This is the setting I study, with the assumption that agents can see
each others actions but not each others payoffs. For example, when deciding
whether to try a new restaurant, a consumer may observe whether or not the
restaurant is busy i.e. whether people keep returning to it, but probably not
whether they enjoyed their meal (ignoring for this exercise the small subset
that posts reviews online).

In particular, my interest is in an intuitive type of equilibria, which I term
cascade equilibria. In these, agents will keep taking the risky action until
they become sufficiently pessimistic about its value, and then stop playing it
forever, unless they observe someone else taking the risky action longer than
they were supposed to. My main results revolve around establishing conditions
under which such equilibria exist, and characterizing their salient
properties.

The existence results bring out the crucial dilemma that agents face when
exploring in a social context. If an agent take the risky action today, she
can expect to get some immediate payoff and also some information about its
worth, but at the risk of bearing some bad payoff in case the risky arm is
actually detrimental. However, she could also take the safe action today and
wait for the actions of her co-explorers to give her some indication about the
right action to take. Cascade equilibria exist under conditions where the
former outweighs the latter, leading agents to choose to explore rather than
free-ride.

I also establish comparative statics for these equilibria. I allow for
heterogenous priors and show that the most optimistic agent is always the last
to stop exploring, but it need not always be the case that more optimistic
agents explore more than less optimistic agents. I then compare exploration in
cascade equilibria to the single-agent setting. If agents start out with a
common prior, each agent explores less than what she would have, had she been
exploring alone. With a heterogenous prior, however, agents may actually
explore more than their single-agent level, in order to prevent more
optimistic agents from becoming pessismistic and hence starting a cascade too
soon, because they value the information that they can get from the
exploration of others.

Finally, I show that allowing agents to write enforceable ex-ante contracts
will lead to the most optimistic agent buying all of the payoff streams and
then implementing her own efficient level of exploration. This phenomenon can
be interpreted as the more established, better informed firm buying out or
investing equity in smaller start-ups in the same space.

Background and related literature: The bandit problem, in particular the
single-agent multi-arm bandit, has a vast literature in computer science and
economics. In a Bayesian setting where one is comfortable assigning a prior to
the payoff from each arm, the solution was derived first by [Gittins, 1979].
The agent assigns an index to each arm every period, and pulls the arm with
the highest index. When faced with one risky action and one safe action, the
outcome under optimal play has a cutoff flavour - the agents chooses to take
the risky action until some predetermined cutoff.

In a multi-agent setting, a third component of strategic benefit comes into
play along with the immediate payoff and the option value of the risky action.
This was first studied by [Bolton and Harris, 1999], who showed that a free-
riding effect dominates in Markov equilibrium, leading agents to explore less
in the social setting than they would have done had they been exploring alone.
In their continuous-time model, both payoffs and actions are public
information, which overloads the information benefit of the social setting -
an agent can effectively depend on the exploration of others for her own
benefit, introducing a free-rider problem. Moreover, the unique Markov
equilibrium features complicated mixed strategies, rather than the simple
cutoff strategies that were optimal in the single agent setting. However,
[Hörner et al., 2021] shows that this is an artefact of the solution concept
rather than the problem itself, by showing that efficiency is in fact
attainable under some conditions in strongly symmetric equilibria and the best
equilibrium payoff is always achievable using cutoff strategies.

[Rosenberg et al., 2007] study a discrete time setting with two players and
private payoffs, imposing the restriction that playing the safe arm is
irreversible, and find that all equilibria are cutoff equilibria. [Heidhues et
al., 2015] show that although Nash implementing the efficient outcome with
common priors and observed payoffs is impossible even when the agents can
communicate with teach other, it can be implemented under some conditions in
sequential equilibrium when payoffs are unobserved and players can communicate
with each other. They are silent on the case without communication.

Another strand of literature that my work closely relates to is the herding
literature, started by [Banerjee, 1992] and [Bikhchandani et al., 1992]. The
idea of information cascades is a running theme in this literature, with
myopic agents ignoring their information when the information from predecessor
actions are too valuable. The closest paper from this literature is [Mossel et
al., 2015], who introduce forward-looking agents in a network setting and show
that full learning can happen in the limit in egalitarian networks.

##  2 Model

Each agent in a group of $n$ agents faces a choice between playing one of two
actions, labelled $r$ (risky) and $s$ (safe), at time periods $t=1,2...$. The
state of the world, $\theta\in\\{0,1\\}$, is drawn before start of play and is
common for all agents. The safe action gives a constant payoff of 0 always.
The risky action’s payoff each period is i.i.d. across players conditional on
$\theta$ \- it pays $X_{h}>0$ with probability $\pi$ and pays $X_{l}<0$
otherwise if $\theta=1$, and pays $X_{l}<0$ always if $\theta=0$. Assume
$E_{1}=\mathbb{E}[X|\theta=1]>0$. Note that
$E_{2}=\mathbb{E}[X|\theta=0]=X_{l}<0$. I informally say that an agent
“explores" when she plays the risky action.

Agent $i$ has prior $q_{i}$ about the state at time $t=1$. I denote $q_{i}(1)$
by $q_{i}$ for ease of notation i.e. $q_{i}$ is the probability that agent $i$
places on $\theta=1$. I assume that the priors of the agents are commonly
known. Wlog, I label the agents $1,...,n$ in decreasing order of $q_{i}$. The
belief of agent $i$ about the state at time $t$ after history $h_{t}$ is
written as $p_{i}(t,h_{t})$. Agents observe the actions of other agents, but
payoffs are private.

In this simple model, it is straightforward to calculate the cutoff belief at
which a single agent exploring alone would stop exploring, by simply equating
the benefit from exploring just one more period to the benefit from stopping
forever:

$p^{a}=\frac{(1-\delta)E_{0}}{(1-\delta)(E_{1}+E_{0})+\delta\pi E_{1}}$

Starting at any prior $p$, this gives a time $\tau^{a}(p)$ until which the
agent would explore. Similarly, given that the agent has $n$ risky actions to
take, she stops at a cutoff:

$p^{e}=\frac{(1-\delta)E_{0}}{(1-\delta)(E_{1}+E_{0})+\delta n\pi E_{1}}$

This again corresponds to a time $\tau^{e}(p)$ until which the agent would
explore. Note that $\tau^{e}(p)$ could be more or less than $\tau^{a}(p)$,
since in the $n$ action case the agent gets $n$ draws per period instead of
just $1$. But since $p^{e}<p^{a}$, the total number of draws that the agent
takes in the $n$ action case will be more than that she takes when can take
one action, so $n\tau^{e}(p)\geq\tau^{a}(p)$.

The focus of this paper will be on an intuitive class of equilibria, which I
call cascade equilibria. In such equilibria, on the equilibrium path, agents
start out exploring and if an agent draws a success at any point, she plays
the risky action forever. If however she draws only failures, then she will
continue exploring till some pre-determined time, after which she will stop,
unless she sees someone else explore past the pre-determined time that they
were supposed to stop.

######  Definition 1.

I call a weak perfect Bayesian equilibrium, $s$, a cascade equilibrium if $s$
induces the following “cascade" outcome on the equilibrium path: there are
cutoffs $(\tau_{i})_{i\in N}\geq 1$ such that

  1. 1.

agent $i$ plays the risky action until $t=\tau_{i}$

  2. 2.

conditional on all agents $j$ with cutoffs $\tau_{j}<\tau_{i}$ having already
switched to playing the safe action, agent $i$ switches to playing the safe
action at $t=\tau_{i}$ if and only if she has received $X_{l}$ every period
that she played the risky action

  3. 3.

if some agent $j$ does not switch at her designated cutoff $\tau_{j}$, all
agents play the risky action forever from $t=\tau_{i}+1$

Figure 1: Cascade equilibrium outcomes

Note that cascade equilibria are described by the outcomes that are observed
on path, remaining agnostic towards the equilibrium strategies that produce
these outcomes. The primary motivation for defining them this way is that our
interest lies in these outcomes themselves rather than the strategies employed
to reach these outcomes.

##  3 Equilibrium exploration

###  3.1 Existence of cascade equilibria

First, I show that cascade equilibria are guaranteed to exist under some
conditions. This is important because otherwise any results I prove using
these equilibrium dynamics that I conjecture could be vacuously true. Since
these are just sufficient conditions, I do not rule out that they may not
exist outside of these conditions, but a more general proof will be a focus of
future continuation of this work.

######  Proposition 1.

With a common prior, that is when $p_{i}=p_{j}$ for all $i,j$, the following
is a sufficient condition for the existence of cascade equilibria:

$\frac{E_{1}}{E_{0}}\geq\frac{1-\pi}{\delta+\pi-(1-\pi)(1+\frac{\delta\pi}{1-\delta})}$
(1)

###### Proof.

Consider the following strategy profile - for any history $h_{t}$, each agent
plays the risky action till her belief drops below $p_{a}$, after which she
switches to the safe action and plays the safe action as long as her belief
stays below $p_{a}$. If at any history her belief is $1$, she plays the risky
action forever. Drawing successes and failures lead to standard Bayesian
updating. If at any history an agent switches to the safe action from the
risky action, the other agents infer that the deviating agent drew only
failures when playing the risky action and update their beliefs about the good
state according to Bayes rule. If any agent plays the risky action at any
period where her beliefs can only either be $1$ or below $p_{a}$, other agents
infer that it must be $1$ and everyone plays the risky action forever after.
It is straightforward to check that these beliefs are consistent on path. As a
sidenote, note that this is a symmetric strategy profile, and on path everyone
has the same cutoff $\tau$.

Thus we are left to check sequential rationality of the strategies using the
one shot deviation principle. Note that given a profile of beliefs, the
equilibrium strategies require the same path of play, regardless of the
history that led to the profile of beliefs. So it is sufficient to check one
shot deviations at all possible profile of beliefs. Consider an agent’s
incentives at some belief $p_{i}>p_{a}$ to play the risky action as prescribed
by the strategy vs deviating for one period and playing the safe action. If
$p_{i}$ is such that the agent’s beliefs will fall below $p_{a}$ after
observing one more failure, then whether she deviates or not, she’ll play the
safe action from the next period if she draws no success, so she faces the
single agent incentives, and hence she will find it optimal to play the risky
action. For $p_{i}>p_{a}$ such that this is not the case, we have
$p_{i}\geq\frac{p_{a}}{p_{a}+(1-\pi)(1-p_{a})}$ by Bayes’ rule, and the
difference in payoffs from risky vs safe is bounded below by
$(1-\delta)p_{i}[E_{1}-\frac{(1-p_{i})}{p_{i}}E_{0}]-p_{i}\delta^{2}E_{1}$.
The first term is the minimum additional payoff from playing the risky action
this period. The second term in the maximum additional payoff from playing the
safe action this period, obtained when playing the safe action makes the other
agents reveal their draws via their actions, leading to the agent being
perfectly match the state two periods from now. This bound is non-negative due
to the inequality in the claim. When the agent’s belief $p_{i}<p_{a}$, if she
plays the risky action, everyone else plays the risky action forever after so
she gains no information externality, and hence she will find it better to
play the safe action. ∎

The interpretation of this sufficient condition in words relates to the trade
off that the agent faces in terms of the informational benefit of other
agents’ exploration versus the informational benefit of taking the risky
action. The left hand side quantifies the ratio of payoff benefits from the
arms, and the right hand side connects it to a threshold based on
informational considerations. The right hand side of the inequality,
signifying the threshold that the payoff benefits must surpass, is low when
$\pi$ is high and $\delta$ is low. In effect, a higher value for $\pi$
indicates that the risky action provides a signal of its worth more often,
while a lower value of $\delta$ means that the agents place lower value of the
informational benefit of other agent’s exploration (since it always comes in
the future).

A concern with showing existence only for common prior profiles would be that
such profiles are measure zero in the space of belief profiles. I show below
that the common prior assumption can be relaxed for this result, for priors
that are sufficiently close to some common prior. This is because generically,
the agents will not be indifferent between pulling the safe and risky arms
when they pull the risky arm for the last time, and will also not be
indifferent between the two arms when they pull the safe arm for the first
time. This slackness allows for a small variation in priors while keeping
incentives the same.

######  Proposition 2.

For any generic common prior belief profile $q$ and any set of parameters such
that Equation 1 is satisfied, there exists an open ball $B(q)$ around $q$ such
that for any $p\in B(q)$, there is a cascade equilibrium with prior $p$.

###### Proof.

Denote by $V_{i}(t,p_{i})$ the continuation value of agent $i$ at time $t$
with belief $p_{i}$. Denote by $p(t,q_{i})$ the belief of an agent at time $t$
given that she started out with a prior $q_{i}$ at time $t=0$, and then drew
only failures till time $t$. Note that the equilibrium in Proposition 1 was
symmetric, so on path agents will explore till some time $\tau$ and then all
together stop exploring if none of them drew a success.

The continuation value in the last period that they explore can be written as

$V(\tau,p(t,q_{i}))=p(t,q_{i})E_{1}+(1-p(t,q_{i}))E_{0}$

Since $p(t,q_{i})$ is derived by Bayes rule, it is continuous in $q_{i}$. And
hence so is $V(\tau,p(t,q_{i}))$. Now suppose the common prior profile $q$ is
such that agents are not indifferent between arms when stopping exploration
i.e. there exists $\rho$ such that the value from playing the risky arm for
the last time $V_{i}(\tau,p(\tau,q_{i}))>\rho$ and the value from playing the
risky arm the next time period $V_{i}(\tau+1,p(\tau+1,q_{i}))<-\rho$. Then by
continuity there is some $\epsilon>0$ such that for priors
$p_{i}\in(q_{i}-\epsilon,q_{i}+\epsilon)$, we have that
$V_{i}(\tau,p(\tau,p_{i}))\geq 0$ and $V_{i}(\tau+1,p(\tau+1,p_{i}))\leq 0$.
Thus agent $i$ will continue to find it optimal to explore till $\tau$ and
then stop. ∎

The proposition above establishes that the existence of cascade equilibria is
not restricted to a measure zero subset of the space of prior profiles, since
they exist in open balls around the set of common prior beliefs. Hence it is
not something that one can rule out generically. However, it does leave open
the question of whether one can expect them to exist for a significant
proportion of prior beliefs.

###  3.2 Properties of cascade equilibria

Cascade equilibria were motivated by the intuitive way that agents explore in
them, so the next logical step is to say more about the properties of agents’
exploration. Since their exploration is quantified by the cutoff $\tau_{i}$,
this is the relevant quantity that we must analyze.

A natural conjecture that one might have is that $\tau_{i}$ are ordered by
agent optimism i.e. agents that start out with a higher $q_{i}$ will have a
larger $\tau_{i}$. First I establish that this intuition holds for the most
optimistic agent (labelled agent $1$ wlog).

######  Proposition 3.

In any cascade equilibrium, $\tau_{1}\geq\tau_{j}$.

###### Proof.

Suppose $\tau_{1}<\tau_{j}$ for some agent $j\neq 1$ i.e. agent $1$ stops
exploring before agent $j$. Wlog let agent $j$ be the last to switch to the
safe action. Then, conditional on both of them getting only failures, we have
that at $t=\tau_{j}$, $p_{j}(t)<p_{1}(t)$ since agent $j$ started out with a
more pessimistic prior and has explored more. Moreover both agent $1$ and
agent $j$ face single agent incentives at $t=\tau_{j}$ since neither agent’s
action influences how the other will act in the future. But then, if it is
best for agent $j$ to play the risky action, it must be best for agent $1$ as
well. ∎

This need not be true in general i.e. it is possible that $\tau_{i}$ <
$\tau_{j}$ even though $p_{i}>p_{j}$. To see this, consider the following
example.

Example 1.  Suppose there are three agents, and the priors are such that the
first agent reaches the single-agent belief cutoff after observing $4$
failures, while the second and third reach it after $2$ draws, with the third
agent a bit more pessimistic than the second. Specify the following
strategies:

  * •

On the equilibrium path with no successes, agent $1$ plays the risky action
for $3$ time periods, agent $2$ for $1$ period, and agent $3$ for $2$ periods.

  * •

Off the equilibrium path, agents play the risky action until they reach their
single-agent cutoff belief. If they see an agent play the risky action when
they should’ve been playing the safe action, they infer that the state is good
with probability $1$. If they see an agent switching from the risky to safe
action, they infer that the agent only got failures so far.

I show in Appendix A.1 that this is an equilibrium for some parameter values.
Note that agent $3$ explores more than agent $2$ even though her prior is more
pessimistic. Both of them find it optimal to continue with this arrangement,
for their own reasons. Agent $2$ knows that if she plays the risky action when
she is supposed to switch, the others will assume that the state is good and
always plays risky forever, which will lead to her losing access to their
information. Agent $3$ knows that if she switches to the safe action, agent
$1$ will play the risky action only until $t=2$, which is bad for her since it
reduces her informational externality.

A corollary of Example 3 is that cascade equilibria need not be unique for a
given prior. Note that we could have switched the roles of agents $2$ and $3$,
and continued to have the same incentives for staying on the equilibrium path
as in the example. Agent $3$ would be willing to stop exploring early, since
she is more pessimistic than agent $2$, and given that agent $3$ stops early,
agent $2$ is willing to play the risky action for $2$ periods, since she is
more optimistic than agent $3$.

This analysis carries over for comparative statics across different priors. We
can compare $\tau_{1}$ across different prior profiles, but it is not possible
to compare $\tau_{i}$ in general. This is established in the proposition and
example that follow.

######  Proposition 4.

Suppose for some $p,p^{\prime}$, $p^{\prime}_{1}\geq p_{1}$. Then
$\tau_{1}(p^{\prime})\geq\tau_{1}(p)$.

###### Proof.

By Proposition 3, it must be the case that agent $1$ must be the last to stop
exploring for both priors (note that agent $1$ refers to the most optimistic
agent here so it is possible that this may not denote the same agent). Since
the last agent to stop exploring faces single-person incentives, agent $1$
stops exploring at $\tau_{a}(p_{1})$ with prior profile $p$ and at
$\tau_{a}(p^{\prime}_{1})$ with prior profile $p^{\prime}$. Given that
$p_{1}\geq p^{\prime}_{1}$, we must have that
$\tau_{a}(p_{1})\geq\tau_{a}(p^{\prime}_{1})$. Thus the result follows. ∎

Example 2.  It is not possible to compare the cutoffs of the other agents,
even if we fix the priors of all but one of these agents. To see this,
consider the setting of Example 3. In that example, agent $1$ explores $1$
period in equilibrium. However, as noted earlier, there is an equilibrium for
the same prior in which the agent $1$ explores for $2$ periods. In fact, with
a slightly smaller prior than in that example, agent $1$ would still explore
for $2$ periods in an equilibrium, which means that she explores more for a
smaller prior.

The intuition behind this is that the amount of exploration done by an agent
depends on both her own beliefs and what other agents do in equilibrium. If
the payoff that she anticipates from the actions of the other agents are
sufficiently high, she will be willing to stop exploration at an earlier
stage, even at a higher belief, compared to the case where the other agents
aren’t doing much socially beneficial exploration.

###  3.3 Comparison to single-agent setting

How does experimentation in the social setting compare to experimentation in
the single agent setting? The answer to this question is nuanced. On the one
hand, we can show that with a common prior, conditional on not having observed
a success, an agent explores less in the social setting than if she were
experimenting alone. To see this, define $\tau^{a}(p_{i})$ as the time at
which the agent switches to the risky action when experimenting alone,
conditional on no successes. Given the set of cascade equilibrium cutoffs
$T(p_{i},p_{-i})$ for an agent in the social setting, also define

$\bar{\tau}(p_{i},p_{-i})=\max\\{\tau\mid\tau\in T(p_{i},p_{-i})\\}$

######  Proposition 5.

If $p_{i}=p$ for all $i$, then $\tau^{a}(p)\geq\bar{\tau}(p)$.

###### Proof.

Suppose not. Then, there is some equilibrium of the social setting where an
agent $i$ explores for $\bar{\tau}(p)>\tau^{a}(p)$. Consider the time period
$t=\bar{\tau}(p)$, which is the last period that agent $i$ chooses the risky
action, conditional on no successes so far. By the definition of
$\bar{\tau}(p)$, either all other agents have already switched to the safe
action, or this will be the last period that they play the risky action,
conditional on no successes. So the play of other agents cannot condition on
what agent $i$ plays at $t=\bar{\tau}(p)$, so her continuation payoff for both
actions has the same information spillover benefits. Thus, she faces the
single-agent trade-offs, which means she cannot play the risky action at
$t=\bar{\tau}(p)$. ∎

With heterogenous beliefs, however, this is no longer necessary, as I show in
Example 6. There can be equilibria in which agents explore more than when they
explore alone. The intuition behind this is related to how agents anticipate
the effects of their actions on the exploration of other agents. When an agent
switches from the risky to the safe action, it immediately reveals the results
of all her past draws to all other agents. This can cause the other agents to
become pessimistic and stop exploring, even if they would’ve continued
exploring without the release of such information. Thus agents can
strategically explore a bit more than they would’ve in the single-agent
setting, in order to benefit from the information spillover from the others.
Note that for this story to be feasible, the other agents should be more
optimistic, which necessitates heterogenous beliefs.

Example 3.  Let $n=3$. The initial belief $p_{1}^{0}$ for the first agent is
such that three bad draws drop her belief below $p^{a}$, while for the other
two, it is such that five bad draws drop their beliefs below the single-agent
cutoff. Specify the following strategies:

  * •

If $t\leq 4$ and everyone has played the risky action so far, everyone plays
risky.

  * •

At $t=5$, agent $1$ switches to playing the safe action while agents $2$ and
$3$ play the risky action.

  * •

On and off path, agents $2$ and $3$ switch to the safe arm when their belief
drops below $p^{a}$. Off path, agent $1$ switches to playing the safe action
when her belief drops below $p^{a}$.

In Appendix A.2, I show that this strategy profile is indeed a PBE for some
values of the parameters. Note that in this equilibrium, agent $1$ explores
for $4$ periods, which is more than what she would have explored if she had
been exploring alone.

However the logic from Proposition 5 carries through for the most optimistic
agent, labelled agent $1$ wlog, if such an agent exists uniquely. This is
because in any cascade equilibrium, she must be the last agent to explore. But
then, she cannot be exploring more than her single agent levels.

######  Proposition 6.

In any cascade equilibrium, $\tau^{a}(p_{1})\geq\tau_{1}$.

###### Proof.

The proof for this carries through in the same way as the proof for
Proposition 5, since by Proposition 3, agent $1$ is the last to stop
exploring. ∎

###  3.4 Exploration with contracts

Suppose agents now have access to money, using which they can write
enforceable formal contracts ex ante222I assume away betting on the underlying
state, which can lead to agents offering and accepting unbounded bets.. I
assume that there is a contracting stage before exploration, in which agents
have access to as much financing as they expect to get in the best case on the
path of play. This can be interpreted as there being a financial institution
that can learn the prior of the agent by verifying the private information
that an agent has regarding the value of the project, and then providing the
agent with financing up to the expected value under the agent’s prior.

######  Proposition 7.

Suppose there exists a most optimistic agent $i$ i.e. $p_{i}>p_{j}$ for all
$j\neq i$. Then if a contract is written ex ante, the outcome with money will
be that agent $i$ owns all payoff streams and explores for $\tau^{e}(p_{i})$
periods.

###### Proof.

Consider an agent with prior $p$ about the state. For now, let each of the $n$
payoff streams be represented by a bandit. Her maximum payoff is when she can
utilize all bandits to learn about the state and maximize the total payoff
from all bandits - this is the efficient level of exploration. The efficient
cutoff belief at which the agent stops exploring, i.e. when the agent is
indifferent between pulling just one arm and stopping all exploration, is
given by

$p^{e}=\frac{(1-\delta)E_{0}}{(1-\delta)(E_{1}+E_{0})+\delta n\pi E_{1}}$

Given the prior $p$, this corresponds to a cutoff $\tau^{e}(p)$ at which the
agent stops all exploration. Note that $\tau^{e}(p)$ is increasing in $p$ as a
step function, i.e. it is constant in intervals and then jumps up by $1$ at
the right end point of the intervals.

Thus, the total payoff that the agent ex ante expects to get is given by:

$\displaystyle\sum_{t=1}^{\tau^{e}(p)}$ $\displaystyle
n\\{p(1-\delta)E_{1}-(1-p)(1-\delta)E_{0}\\}+np[1-(1-\pi)^{{n\tau^{e}(p)}}]\delta^{\tau^{e}(p)+1}E_{1}$
$\displaystyle=$ $\displaystyle
n(1-\delta^{\tau^{e}(p)+1})[pE_{1}-(1-p)E_{0}]+np[1-(1-\pi)^{{n\tau^{e}(p)}}]\delta^{\tau^{e}(p)+1}E_{1}$

As $p$ increases, whenever $\tau^{e}(p)$ is constant, this is clearly
increasing in $p$. At the cutoff where $\tau^{e}(p)$ jumps up, the agent with
that cutoff belief is indifferent between exploring at the two levels, so she
gets a higher payoff than those with a slightly smaller prior, since the
payoff is increasing whenever $\tau^{e}(p)$ is constant. Thus, it is clear
that the total payoff is increasing in $p$.

Since an agent with a higher prior can expect to earn a higher payoff and
hence utilize a higher ex ante credit line, she can offer higher side payments
to the other agents than any other agent can offer. The result then follows.

∎

This result predicts that the contracts that will be seen in practice will be
ones where the most optimistic agent acquires the payoff streams of the less
optimistic ones. Consider a setting where firms are developing new technology
socially. Their priors at the beginning of the exploration stage are informed
by their research ex ante, which leads to different signals of how lucrative
the technology happens to be. In cases where the most established firm can
research the technology the best ex ante and hence have the most optimistic
signal, the result above amounts to saying that the most established firm will
“buy out" smaller startups. Hence it provides a belief-centric explanation as
to why these buy outs are seen in practice at all. In many standard settings,
results like the [Milgrom and Stokey, 1982] no trade theorem and [Myerson and
Satterthwaite, 1983] impossibility theorem establish the impossibility of
trade under the common prior even when agents have private information,
precluding such buy outs from being rationalized.

##  4 Discussion

Existence of non-cascade equilibria: In this work, I focus on equilibria in
which behaviour is similar in spirit to the cutoff strategies that are
prevalent in earlier work. There will, however, be other interesting outcomes,
produced by strategies with complicated behaviour. For example, agents can
switch from the risky to the riskless action in continuous coordination,
releasing information about their own draws early on in return for information
for other agents’ draws, so that they can match their action to the state
sooner. These run the risk of making the agents pessimistic earlier than
necessary, reducing equilibrium exploration and hence reducing overall
welfare. Yet another class of equilibria can have mixing on the equilibrium
path. This class of equilibria will have the property that agents may switch
from playing the risky action to playing the safe action and then again switch
back, without having gained any additional information about the state or the
draws of others. Thus although interesting from a theoretical viewpoint, they
exhibit dynamics that unlikely to be observed in practice.

Welfare properties of cascade equilibria: Cascade equilibria were informally
motivated by the observation that they feature delayed information release,
but a result that formalizes the positive implications of this would need to
establish that cascade equilibria indeed have superior welfare or exploration
properties. An objective of future work could be to formally establish that
cascade equilibria indeed have superior welfare and exploration properties,
similar in spirit to the results in [Hörner et al., 2021].

## References

  * [Banerjee, 1992] Banerjee, A. V. (1992).  A simple model of herd behavior.  The quarterly journal of economics, 107(3):797–817.
  * [Bikhchandani et al., 1992] Bikhchandani, S., Hirshleifer, D., and Welch, I. (1992).  A theory of fads, fashion, custom, and cultural change as informational cascades.  Journal of political Economy, 100(5):992–1026.
  * [Bolton and Harris, 1999] Bolton, P. and Harris, C. (1999).  Strategic experimentation.  Econometrica, 67(2):349–374.
  * [Gittins, 1979] Gittins, J. C. (1979).  Bandit processes and dynamic allocation indices.  Journal of the Royal Statistical Society: Series B (Methodological), 41(2):148–164.
  * [Heidhues et al., 2015] Heidhues, P., Rady, S., and Strack, P. (2015).  Strategic experimentation with private payoffs.  Journal of Economic Theory, 159:531–551.
  * [Hörner et al., 2021] Hörner, J., Klein, N., and Rady, S. (2021).  Overcoming Free-Riding in Bandit Games.  The Review of Economic Studies.  rdab078.
  * [Milgrom and Stokey, 1982] Milgrom, P. and Stokey, N. (1982).  Information, trade and common knowledge.  Journal of economic theory, 26(1):17–27.
  * [Mossel et al., 2015] Mossel, E., Sly, A., and Tamuz, O. (2015).  Strategic learning and the topology of social networks.  Econometrica, 83(5):1755–1794.
  * [Myerson and Satterthwaite, 1983] Myerson, R. B. and Satterthwaite, M. A. (1983).  Efficient mechanisms for bilateral trading.  Journal of economic theory, 29(2):265–281.
  * [Rosenberg et al., 2007] Rosenberg, D., Solan, E., and Vieille, N. (2007).  Social learning in one-arm bandit problems.  Econometrica, 75(6):1591–1611.

##  Appendix A Examples

###  A.1 Reversed order of cutoffs

The only time period to check for deviations is time $t=2$. For agent $2$, the
differential benefit from playing the risky versus the safe action is given
by:

$p_{2}\pi E_{1}-(1-p_{2})(1-\delta)E_{0}-I_{2}$

where $I_{2}>0$ is the information benefit that the agent gets by staying on
the equilibrium path.

For agent $3$, the differential benefit from playing the risky versus the safe
action is given by:

$p_{3}\pi E_{1}-(1-p_{3})(1-\delta)E_{0}+I_{3}^{p}-I_{3}^{d}$

where $I_{3}^{p}=\delta^{3}p_{3}[1-(1-\pi)^{3}]E_{1}$ is the information
benefit that the agent gets by staying on the equilibrium path and
$I_{3}^{d}=\delta^{2}p_{3}[1-(1-\pi)^{2}]E_{1}$ is the information benefit
from deviating.

Note that given the other parameters, $\pi$ can be chosen such that
$I_{3}^{p}-I_{3}^{d}>0$. Then, there are priors close enough for the two
agents such that

$p_{3}\pi E_{1}-(1-p_{3})(1-\delta)E_{0}+I_{3}^{p}-I_{3}^{d}>0>p_{2}\pi
E_{1}-(1-p_{2})(1-\delta)E_{0}-I_{2}$

Thus agent $2$ will play the safe action and agent $3$ will play the risky
action.

###  A.2 Exploring more with heterogenous priors

We check for deviations at the key decision nodes for the players:

  1. 1.

Agent $1$ at $t=4$: the agent’s on path payoff is given by:

$p_{1}[(1-\delta)E_{1}+\delta\pi
E_{1}+\delta(1-\pi)(1-(1-\pi)^{11})\delta^{2}E_{1}]+(1-p_{1})[-(1-\delta)E_{0}]$

while if she deviates, she gets

$p_{1}[(1-(1-\pi)^{8})\delta^{2}E_{1}]$

The difference between these two is

$p_{1}[(1-\delta)E_{1}+\delta\pi
E_{1}]+(1-p_{1})[-(1-\delta)E_{0}]-p_{1}\delta^{2}E_{1}[(1-\pi)(1-(1-\pi)^{11})\delta-(1-(1-\pi)^{8})]$

Note that $p_{1}[(1-\delta)E_{1}+\delta\pi
E_{1}]+(1-p_{1})[-(1-\delta)E_{0}=0$ is solved by the single-agent cutoff
belief, so this term can be made arbitrarily small by choosing the initial
belief of agent $1$ appropriately, independent of $\delta$ and $\pi$. We can
also choose $\delta$ close enough to $1$ so that it is sufficient to check
that

$(1-\pi)(1-(1-\pi)^{11})-(1-(1-\pi)^{8})>0$

which is true when $\pi$ is small.

  2. 2.

Agents 2 and 3 at t=4: the on path payoff for agent 2 (and symmetrically,
agent 3) at t=4 is:

$p_{2}[(1-\delta^{2})E_{1}+\delta^{3}(1-(1-\pi)^{2})(1-\delta)E_{1}+\delta^{4}(1-(1-\pi)^{7})E_{1}]+(1-p_{2})[-(1-\delta^{2})E_{0}]$

while if she deviates, she gets

$p_{2}[(1-(1-\pi)^{4})\delta^{2}E_{1}]$

For $\pi$ small, the deviation payoff is small but the on path payoff is still
large since the two agents are two draws away from their single-agent cutoffs.

  3. 3.

Agents 2 and 3 at t=5: Both agents face their single-agent incentives at $t=5$
since they will get the information of the other agent at $t=6$ no matter what
they do, so they will both choose to play the risky action.