# Avoiding the bullies: The resilience of cooperation among unequals

Michael Foley
Network Science Institute
Northeastern University
Boston, MA
&Rory Smead
Department of Philosophy and Religion
Northeastern University
Boston, MA
&Patrick Forber
Department of Philosophy
Tufts University
Boston, MA
&Christoph Riedl
D’Amore-McKim School of Business
Northeastern University
Boston, MA
Other affiliations include Network Science Institute, Northeastern University,
Boston, MA; Khoury College of Computer Sciences, Northeastern University,
Boston, MA

###### Abstract

Can egalitarian norms or conventions survive the presence of dominant
individuals who are ensured of victory in conflicts? We investigate the
interaction of power asymmetry and partner choice in games of conflict over a
contested resource. Previous models of cooperation do not include both power
inequality and partner choice. Furthermore, models that do include power
inequalities assume a static game where a bully’s advantage does not change.
They have therefore not attempted to model complex and realistic properties of
social interaction. Here, we introduce three models to study the emergence and
resilience of cooperation among unequals when interaction is random, when
individuals can choose their partners, and where power asymmetries dynamically
depend on accumulated payoffs. We find that the ability to avoid bullies with
higher competitive ability afforded by partner choice mostly restores
cooperative conventions and that the competitive hierarchy never forms.
Partner choice counteracts the hyper dominance of bullies who are isolated in
the network and eliminates the need for others to coordinate in a coalition.
When competitive ability dynamically depends on cumulative payoffs, complex
cycles of coupled network-strategy-rank changes emerge. Effective
collaborators gain popularity (and thus power), adopt aggressive behavior, get
isolated, and ultimately lose power. Neither the network nor behavior converge
to a stable equilibrium. Despite the instability of power dynamics, the
cooperative convention in the population remains stable overall and long-term
inequality is completely eliminated. The interaction between partner choice
and dynamic power asymmetry is crucial for these results: without partner
choice, bullies cannot be isolated, and without dynamic power asymmetry,
bullies do not lose their power even when isolated. We analytically identify a
single critical point that marks a phase transition in all three iterations of
our models. This critical point is where the first individual breaks from the
convention and cycles start to emerge.

## Introduction

Individuals often differ in their ability to fight and win conflicts over
contested resources. Power asymmetries in conflicts can lead to the emergence
of hierarchies and dominant alphas. Such social structures present a serious
risk of destabilizing cooperative social interactions or norms. Dominant
individuals—bullies, in effect—undermine collaboration among other individuals
and are very dangerous to confront. Why work together to find food when the
stronger individual can simply take all of it?

Yet our human ancestors were nomadic, lived in groups (individuals did not
hold territories), and foraged via collaborative interactions [130, 84]. Human
cooperation and collaboration has only accelerated since, especially through
trade and information sharing. When resources are gained through social
interactions, cooperation is threatened by bullies who have no incentive to
cooperate. The presumed evolutionary solutions to this problem involve
coalition-building where individuals organize to collectively punish, exile,
or kill the destabilizing individual[18, 128, 145]. Coalition-building can be
challenging because it requires coordination and communication—tools that
require the stability of cooperation to emerge. We seek to investigate whether
and how bullies can be controlled or managed in this scenario without
presuming rich coordination and communication skills. We develop a model using
evolutionary game theory where payoffs are generated during social interaction
rather than by holding territories or individual foraging, and individuals can
choose their interaction partner. Our model identifies a pathway for dealing
with powerful individuals that does not require collective action: partner
choice allows individuals to isolate bullies and mitigate their damaging
effects without the need for explicit coordination, communication or
agreement. Furthermore, when power asymmetry dynamically depends on cumulative
payoffs, complex interdependent cycles emerge among strategies, network
connections, and ranks.

We use a generalized hawk-dove game to represent the strategic interaction
involved in a competition over a contested resource [127, 126, 52]. In this
game of conflict, players simultaneously choose either an aggressive hawk or
deferential dove strategy. Playing hawk against an opponent playing dove
secures the resource and yields the best payoff. However, playing hawk against
another hawk results in a costly conflict and the worst payoff. The dove
strategy is the safer option against hawk opponents: avoiding fights with the
hawks by relinquishing the resources and sharing the resource with other
_doves_ (see SI for technical details).

The usual (evolutionarily stable) solution to the generalized hawk-dove game
can be interpreted as an equilibrium in which individuals randomize between
hawk and dove strategies. The randomizing solution results in a significant
number of costly hawk-hawk conflicts. However, if individuals can use an
external cue (such as whether they are the territory owner or intruder) to
coordinate their actions, then more efficient new solutions emerge. These
correlated equilibria avoid conflict entirely, outperform the randomizing
strategy, are stable, and can easily invade a population [6, 58, 126, 92, 42].
Since these correlated equilibria identify patterns of behavior that are
“customary, expected, and self-enforcing” [148], they are identified as
paradigm examples of conventions in game theory [123, 125]. Two equally
effective conventions are possible: individuals may behave aggressively when
they are hosts and defer when they are visitors (a sort of ownership norm) or
vice versa (a host-guest hospitality norm). Whichever convention emerges first
takes over. Many studies have focused on the ownership norm (the so-called
“bourgeois” solution) since it appears more prevalent in nature[58, 126, 123,
87]. However, partner choice has the striking effect of favoring a host-guest
norm (also called the “paradoxical” convention) instead of the ownership norm
[42]. Although there is rich work in this area, there needs to be a formal
investigation into the interaction between power asymmetry and partner choice,
and instances when power asymmetry can dynamically depend on cumulative
payoffs.

Partner choice is an important and realistic mechanism. Social networks change
continuously as individuals form new ties and dissolve existing ones [77, 22,
118]. Individuals modify their social surroundings by choosing who to interact
with and how much time to allocate to each interaction [53, 4]. It is now
increasingly clear that the heterogeneous structures we observe in empirical
social networks are the result of an interplay between behavior and partner
choice [72, 9]. Partner choice gives rise to interaction networks that change
over time. We adopt the term dynamic network for our partner choice model;
others have used the term “temporal” or “evolving networks” [104].

Building on previous dynamics networks studies [122, 101, 42] and two recent
studies that investigate inequality in humans using public goods games [65,
64], we introduce power asymmetries into games of conflict played on dynamic
networks. Power takes the form of differences in competitive ability and has
the implication that different individuals may effectively be playing
different games. Formulations of similar asymmetric games in which individuals
receive different payoffs or face different games have been studied in the
context of static networks [85] and two-player interactions [57].

Using the hawk-dove framework, we model power asymmetries by assigning
individuals ranks and supposing that ranking individuals secure an additional
positive reward ($f$) in instances of hawk-hawk conflicts. Without loss of
generality, if $i$ outranks $j$ the payoffs are given by (1).

| hawkj | dovej
---|---|---
hawki |  (25mm $f,hh$ |  $hd,dh$ )20mm
dovei | $dh,hd$ | $dd,dd$
(1)

The value of this additional reward ($f$) for ranking individuals in conflicts
is a central variable in our study. It significantly affects whether the
correlated conventions are resilient in the presence of bullies. Partner
choice also has an important effect. We find that when individuals interact
randomly, the convention persists even under substantial power asymmetry, but
it eventually disappears completely and the population breaks into pure hawks
and doves. However, with partner choice, the convention remains stable under
any level of power asymmetry—aggressive bullies are isolated and the
convention is preserved in the population. Finally, if ranks themselves
dynamically depend on accumulated payoffs, we see the emergence of cycles in
which individuals move through the ranks, adapt their strategy, and change
network position.

There is much literature on the evolution of cooperation in randomly mixing
populations, structured populations, on graphs and networks, and on dynamic
networks (see [55, 103] for reviews). Our work provides three advances over
prior work. First, the focal game in the literature on dynamic networks has
predominantly been the well-known Prisoner’s Dilemma. Here we examine games of
conflict which naturally lend themselves to the study of inequality and
exploitation. Network structure has been shown to be more important in games
of conflict than the Prisoner’s Dilemma [21], which warrants additional
analysis of this game in connection with partner choice. Second, prior work on
games of conflict has mostly focused on randomly mixing populations or static
networks, thus eliminating the critical mechanism of partner choice. There are
notable exceptions that study games of conflict on dynamic networks [100, 44,
42], but in these models either individuals cannot use an external cue to
coordinate their actions (i.e., the models do not allow for correlated
equilibria as solutions) or they do not investigate power asymmetry. Previous
models also do not include both asymmetric power and partner choice [7, 121].
Third, models that do include power inequalities [127, 58, 52, 87] do not
consider networked interaction or partner choice, and they assume a static
game where a bully’s advantage does not change over time. We summarize the
most closely related prior work in Table S1.

Figure 1:  Game play and updating mechanism. In each round agents must (1)
select an interaction partner, (2) play a game of conflict, and (3) update
both their network weight and strategy weights based on the payoff $\pi$.

## Results

We explore our model (see Methods) analytically and using agent-based
simulations. Each round, individuals independently choose an opponent to play
in a game of conflict (Fig. 1). They choose a strategy to play in the
interaction (Fig. 1a top) and receive a payoff (Eq. 1). They can distinguish
whether they initiate the interaction (i.e., are the host) or not (i.e., are
the visitor). Learning occurs in our agent-based simulations through simple
yet psychologically realistic Roth-Erev reinforcement learning [35, 124, 101].
Agents update both who they visit as well as their strategies, distinguishing
between hosting and visiting according to the accumulated payoffs of each
strategy in their respective role (Fig. 1 bottom). As a result, the structure
of the social network coevolves along with the behavior of individuals. Our
key modeling parameter is the degree of power asymmetry $f$, which we vary
between $0$ (no power asymmetry, ranks play no role) and $1$ (large asymmetry,
ranking individual can always receive the maximum payoff). All main results
are robust with respect to population size and variation in learning speed
(see SI).

We consider three iterations of our model. First, power asymmetries in the
random interaction case (i.e., static, fully connected networks with uniform
tie weights). Second, power asymmetries with partner choice (i.e., dynamic
networks). Third, dynamic power asymmetries that evolve as a function of
cumulative payoffs with partner choice.

### Power asymmetries and random interaction

When individuals interact randomly, the introduction of power asymmetry has a
dramatic impact: we see the emergence of top-ranked individuals who no longer
follow the convention—bullies—and the eventual collapse of the correlated
convention for the entire population. Whether the correlated convention breaks
(and for which individuals) depends critically on the value of $f$, the payoff
to the ranking individual in hawk-hawk interactions. To understand why and how
this happens we examine power asymmetries with random interaction both
analytically (Fig. 2a and SI) and numerically through agent-based modeling
(Fig. 2b).

A simple analytic model provides some insight into how power asymmetry affects
correlated conventions. Below the critical value of $f<dh$, agents of all
ranks face, on average, a game of conflict ($f=0$ corresponds to the baseline
symmetric game of conflict). At the critical value of $f\geq dh$, hawk becomes
a dominant strategy for at least one of the ranking players, while the
outranked player’s best strategy depends on the chance an opponent plays hawk
(see SI for the full derivation). Critically, this transition does not depend
on population size or any other parameters of the model besides the $dh$
payoff.

As $f$ increases beyond the threshold, the number of ranking individuals for
whom the game is dominance-solvable increases. The convention becomes
unsustainable at high values of $f$. If the expected probability that an
opponent will play hawk ($P(hawk_{j})$) rises above a certain value, then
bottom-ranked individuals will prefer to play dove as both visitors and hosts.
That value depends on the payoffs of the game and the chance of outranking an
opponent (given by $R$; see SI for the full derivation):

$P(hawk_{j})>\frac{hd-dd}{(dh-dd)-(R\cdot f-hd)}.$ (2)

The expectation for opponent $P(hawk_{j})$ depends on the proportion of the
population for whom the game is dominance-solvable. Moving from random
interaction to our model with partner choice has a significant effect on this
expectation and on $R$.

With the insight from the analytic model in hand, we turn to the main analysis
of our work using agent-based simulations (Fig. 2b). These simulations bear
out the analytical results. For values below the critical value of $f<dh$, the
entire population settles on one of the two correlated conventions (with equal
proportions of simulations settling into either the ownership or host-guest
equilibrium). At the critical value of $f\geq dh$, the transition occurs where
the top-ranked individual stops adhering to the correlated convention. Bullies
start to adopt the aggressive strategy as both hosts and visitors, while
outranked individuals continue to settle on one of the correlated conventions.
Interestingly, if $f>dh$ but does not significantly exceed it, the bottom-
ranked individuals do not adopt pure-dove strategies even if outranked by
everyone else. The reason is that most mid-ranked individuals learn to play
dove when hosting in response to visits from ranking individuals. Once there
are a significant number of individuals who play dove as hosts, this allows
the bottom-ranked individuals to be successful playing hawk when they visit
despite always being out-ranked by their hosts. This also explains why we see
individuals adopt mostly pure strategies: since they can always coordinate on
a correlated convention there is no need to randomize between strategies. As
$f$ increases, the number of bullies (individuals who always play hawk both at
home and away) increases, reducing the number of individuals playing the
correlated convention. Simulations show that the convention breaks for
populations of size twenty near $f=0.7$. A two-class system emerges where
individuals ranked in the top half of the population settle on pure hawk,
while those in the bottom half settle on pure dove.

Figure 2:  In games of conflict with power asymmetry and random interaction,
individuals with different ranks effectively face different games. a,
Analytical analysis of our model reveals that, under random interaction,
individuals play different games, depending on their rank and the degree of
the power asymmetry $f$ (based on equations 16-17)). For $f<dh$, power
asymmetry has no effect and individuals of all ranks engage in a mean game of
conflict (the orange area). For $f\geq dh$, the game is dominant-solvable for
the top ranked individual(s). b, Numerical results from evolutionary
simulations show that conventions prevail when $f$ is below the critical value
$dh$; individuals of all ranks play correlated conventions (either all
ownership or host-guest; each appear with equal frequency in simulation seeds;
ownership shown). At $f\geq dh$ a transition occurs as ranking individuals
break from the convention and adopt pure hawk behavior both home and away. The
correlated convention is resilient among most individuals until high values of
$f$ make it unsustainable. A hierarchy forms in which the top-half of
individuals are pure hawks while the bottom-half are pure doves. Note how the
analytic result on the left matches the numeric result on the right. (Both
panels use $n=20$; $dh=0.4;dd=0.6$.)

### Power asymmetries and partner choice

Partner choice profoundly affects individual behavior in equilibrium. The
correlated convention never breaks on dynamic networks—even for the highest
degree of power asymmetry—and the cooperative convention remains stable for
most of the population (Fig. 3a). The key mechanism behind this result is that
partner choice allows individuals to avoid interacting with bullies. The same
phase transition as happened in the case of random interaction occurs at
$f\geq dh$ when a few top ranked individual(s) adopt aggressive strategies
while the correlated convention is preserved among the rest of the population.
However, contrary to the random interaction case, the second transition never
occurs. The correlated convention never breaks, even at very high values of
$f$. Partner choice restores the stability of the convention because
individuals learn to visit those they outrank and to play hawk when they do
so. As a result, a competitive hierarchy forms (Fig. 3c). The ranking
individual adopts aggressive strategies both at home and away, does not care
who to visit, but hosts no visitors in return. At lower ranks, individuals
target an increasingly smaller set of others who they outrank. They receive
visitors in proportion to the number of others who outrank them and tend to
play dove at home because they tend to be visited by individuals playing hawk.
Consequently, very few hawk-hawk conflicts occur despite the increased
incentive—the increased $f$ value—for ranking individuals. As in the random
interaction case, this allows even the lowest-ranking individual to play hawk
when visiting as there are always available dove hosts to visit. The presence
of a few aggressive bullies preserves the correlated convention for all other
individuals: avoiding the bullies changes how the rest interact.

Figure 3:  Partner choice restores the correlated convention and increases
cooperation in the presence of bullies. a, With partner choice and low
$f$-value, individuals of all ranks play correlated conventions (predominantly
host-guest as first shown in [42]). A transition occurs at $f\geq dh$,
identical to the random interaction case, where top ranked individual(s) break
from the convention and become pure hawks. The correlated convention is
preserved among outranked individuals. However, contrary to the random
interaction case, partner choice allows the convention to remain sustainable
among a majority of individuals even for high $f$ values. This is infeasible
under random interaction. b, The pattern in which some top ranked
individual(s) break from the convention is consistent across population sizes.
Notice how the shape of the curve in (b) matches the boundary where strategies
change in (a). Pure hawks are defined conservatively as agents with a
likelihood of at least $0.8$ for playing hawk both at home and away. c, The
network structure that emerges resembles a hierarchy. Ranking individuals play
pure hawk strategies but receive few or no visitors at all. Outranked
individuals adopt the correlated convention and attract many visitors. The
individual who is outranked by all others is visited by everyone. The graph
shows average in-weights across seeds grouped by rank. Node size is scaled by
incoming edge weight ($dh=0.4;dd=0.6;$ $f=0.6$). d, The highest degree of
network centralization (most hub-like) is reached at $f=0.2$. Nodes with
disproportionate (too many) connections stop emerging entirely at $f=0.8$
(color by mean network centralization). e, Network centralization over time,
averaged across seeds. Networks with heterogeneous node weights emerge for a
period of time. Network centralization of random Erdős–Rényi networks of the
same size and density are shown as reference (the dashed line shows the
median, the grey area is 95% CI).

We find the highest proportion of pure hawks for intermediate values of
$0.6<f<0.8$, while the number of pure hawks decreases for very high $f$
values. The reason for this is that as $f$ approaches 1, the alpha (top-
ranking) individual stops distinguishing among all other individuals. This
individual simply learns to always play hawk and all possible hosts are seen
as equivalent. At the same time, additional early hawk behavior from ranking
individuals may lead lower-ranked individuals to avoid visits from others.
Consequently, ranking individuals (with the exception of the alpha) have
higher proportions of hawk visitors compared to lower $f$ and therefore have a
greater incentive to play dove when hosting. Pure hawks reach a maximum of
around 20% of the population at $f=0.7$ with only around 15% of pure hawks at
$f=0.9$. This shows that excessive power asymmetry is not always detrimental.
It can in fact increase cooperation in a population by helping preserve the
correlated convention. Those results are robust across different population
sizes (Fig. 3b).

Introducing power asymmetry into a population affects which of the two
possible correlated conventions emerges—the ownership or host-guest norm. The
population always settles on the host-guest equilibrium rather than the
ownership equilibrium after the first phase transition. The reason is that
with partner choice, all but the lowest ranked individual can locate an even
lower-ranked individual to act aggressively toward. This means all visitors
will learn aggressive strategies and that the optimal response as a
(typically) outranked host with aggressive visitors is to play dove. At the
same time, the ranking individual adopts aggressive strategies and does not
discriminate among which other individuals to visit. This ensures that
everyone except the ranking individual receives at least some visits from a
ranking individual who cannot be beaten in conflicts, thereby incentivizing
individuals to play dove as hosts. The ownership solution would require that
all outranked individuals play dove when visiting, but that behavior is not
stable as even the second-lowest ranked individual can pursue the aggressive
strategy against the lowest-ranked individual and secure the $f$ payoff by
playing aggressively. As a result, only the host-guest equilibrium is stable.
Note that in the case of $f=0$—the baseline case in which ranks do not
matter—the correlated convention that emerges is also predominantly host-guest
rather than ownership. This is the main finding reported in [42].

We analyze the structure of the emerging interaction networks. Remember that
our networks are weighted, with tie weights updated through reinforcement
learning. Our analysis of network structure hence focuses on the analysis of
the distribution of in-weights (i.e., how many visitors an individual is
expected to receive), rather than the discrete number of connections (node
degree). We quantify network heterogeneity which we label Network
Centralization as the variance of in-degree network weights
$Var(\sum_{j}w_{j1},\sum_{j}w_{j2},...,\sum_{j}w_{jn})\forall j\in N\&j\neq i$
(see also SI). High values of network centralization indicate the presence of
“hubs”: nodes with disproportionately strong in-weights compared with the
average value. We show the distribution of the maximum network centralization
at any point during the evolution (Fig. 3d). The network structures that
emerge critically depend on $f$. For low $f$ values, we find large variation
in the types of networks that emerge, ranging from mostly homogeneous networks
to centralized hub-and-spoke networks (Fig. S2). When ranks play only a minor
role, individuals have no strong preferences for who to visit and a variety of
equilibrium solutions are possible as reported in [42]. In those cases,
networks range from almost completely homogeneous to fully star-like. For high
degrees of power asymmetry, individuals have clear preferences who to visit
and a hierarchical social network forms that is similar across all simulations
(Fig. 3c). Thus, the highest network centrality of any individual is reached
at $f=0.2$ and it is generally lower for higher values of $f$. For $f\geq
0.7$, centralized structures are virtually absent. Strong hierarchies form
with each individual visiting all lower-ranked individuals with equal
probability.

We show network centralization averaged across seeds over time along with a
random Erdős–Rényi networks of the same size and density as reference (Fig.
3e). The variance of the network weights at initialization is $0$ (all weights
are uniformly set to $\frac{L}{N-1}$; $L=19$). For comparison, the expected
variance of a Erdős–Rényi random network of size $N=20$ is $1.12$, while the
variance of a $N=20$ star network is $17.95$. Network centralization increases
over time with a significant spike in centralization at around 5,000 time
steps (Fig. 3e and Fig. S2). That is when we see outranked individuals turning
into highly connected hubs that attract many visitors. Hubs disappear once the
convention is established: there is no reason to visit the dove hub anymore as
most other individuals play dove as hosts as well. If $f<dh$, networks
converge to a nearly completely homogenous state and maintain more
heterogeneity when $f\geq dh$. For a period during the evolution (roughly
between time 2,000 and 50,000) the networks that emerge for low-to-middle $f$
values share qualitative and quantitative properties with those that we
observe in human social networks. There are a few nodes with disproportionate
(too many) connections, while most nodes have few connections [140, 10, 142].

Is the convention restored even when partner choice and strategies evolve on
different timescales? Previous studies show that the timescale of strategy and
partner choice dynamics can greatly alter the cooperation level [124, 48,
146]. Here however, we find that outcomes do not depend crucially on the ratio
between the partner choice and strategy updating timescales. Faster partner
choice dynamics increase the proportion of host-guest interactions relative to
the equal speed case and shorten the overall time needed for simulations to
converge (Fig. S4). Slower partner choice dynamics reduce the number of host-
guest interactions somewhat and lengthen the time needed to converge. However,
even with very slow partner choice updating, the convention continues to
emerge and the vast majority of interactions are host-guest (e.g., around 98%
in the $f=0.7$ case). The convention continues to survive even if network
learning speed is at $\nicefrac{{1}}{{100000}}$ th of partner choice speed
(Fig. S5). Thus, even very slow partner updating is sufficient to allow
populations to isolate bullies and thus preserve the correlated convention.

### Dynamic power asymmetries

So far, we have restricted our analysis to instances in which power
asymmetries—i.e., the ranks of individuals—are determined randomly and remain
fixed over time. However, in the real world, power does not remain fixed and
may depend on an individual’s accumulated wealth or resources [94, 41]. We
address this possibility by allowing agents’ ranks to change dynamically as a
function of total cumulative payoffs (see Methods). Individuals of different
rank not only face different games (as shown in Fig. 2a) but they do so
dynamically as their rank changes. Allowing rankings to be dynamic results in
striking cycles (Fig. 4), but the correlated convention remains resilient.

Figure 4:  Cycles of interacting rank and strategy changes. Allowing
individuals’ ranks to change based on cumulative payoffs (dynamic ranks) leads
to coupled cycles of aggressive-cooperative strategy, shifting network
position, and rising and falling rank.

Ranking individuals do well in aggressive interactions which they exploit—they
adopt increasingly aggressive strategies and become bullies. Doing so leads
others to learn to avoid them, which gives bullies less access to shared
resources from collaboration. As a result, their rank drops and the success of
the aggressive strategy fades. As their rank falls, they learn to adopt a more
cooperative strategy. They become part of a cooperative cluster that has
coordinated on the correlated convention. As these individuals become dove-
host hubs, thereby attracting many visitors, they accumulating high total
payoffs. They climb in the rankings as a result, and the cycle repeats. Rank,
network position, and strategy change in parallel such that adopting an
aggressive strategy is quickly followed by a sharp drop in visitors (becoming
a network spoke), which leads to a steady decrease in rank (Fig. 5a).

Do all individuals cycle through ranks or just some? We find that all
individuals cycle through ranks and spend a similar amount of time at each
rank (S3). Only the host strategy changes as individuals’ ranks change, while
the visiting strategy remains unaffected and is close to 100% hawk. This
illustrates the mechanism behind the temporal dynamics: the interaction
between partner choice and dynamic power asymmetry is crucial. Without partner
choice, bullies cannot be isolated, and without dynamic power asymmetry,
bullies do not lose their power even when isolated. So we only see complex
temporal dynamics emerge with the two mechanisms together—partner choice and
dynamic power. Since the payoffs are $\geq 0$, individuals never strictly
prefer to opt out of any interaction. This contributes to the rank cycling.
Individuals are able to accumulate positive payoffs as hubs playing dove
hosting many hawk visitors, thereby enabling their rise in the rankings. In
effect, having more interactions tends to increase an individual’s social
ranking.

The length of the cycles depends on the degree of power asymmetry (Fig. 5b).
Below the critical value of $f<dh$, no cycles emerge as the game remains a
mean game of conflict for all individuals. At the critical value $f\geq dh$, a
phase transition occurs and cycles first emerge. Consequently, the same phase
transition occurs in all three of our models (random mixing, partner choice,
and dynamic ranks) at precisely $f\geq dh$. The higher the asymmetry in
competitive ability, the shorter cycles become. When the benefits of winning
fights are higher, ranking individuals adopt aggressive strategies more
quickly. This in turn leads to isolation and a fall in the rankings to happen
more quickly. The result is shorter cycles. Beyond the effect of the degree of
power asymmetry, timescales for partner choice updating also affect cycle
length (Fig. S6). Cycles continue to emerge even with very slow partner
updating, but they may be long and can even exceed our simulation timescales.

Our model also suggests several welfare and wealth inequality insights. Total
population payoffs are lowest in random interaction case as several high-
ranking individuals become bullies and break from the correlated convention
(Fig. 5c). This introduces significant friction through hawk-hawk conflict
(31% of interactions), which reduces population welfare. The correlated
convention is mostly restored with partner choice, leading to high population
welfare. Individuals avoid visiting hawk-playing others who outrank them. Once
the network settles, those collisions can be entirely avoided and may only be
due to trembling. As a result, only a few hawk-hawk conflicts occur so that
population welfare is close to the correlated convention equilibrium. This is
different in the case of dynamic ranks. As individuals do not know each
others’ ranks before they visit, the welfare loss due to hawk-hawk conflict is
larger. As some individuals adopt the hawk-at-home strategy as a result of
their improved rank, others must then learn not to visit them, leading to some
hawk-hawk conflict in the meantime (around 11% of total interactions). This
drives down population welfare somewhat, but it remains higher than it is in
the case without partner choice.

Wealth inequality is highest for static networks in which high-ranking
individuals reap more rewards and further entrench their ranks, leading to a
“the rich get richer” phenomenon (Fig. 5d). Partner choice reduces inequality,
but does not eliminate it completely. In fact, partner choice reverses the
relationship between power and accumulated payoffs (Fig. S9). Now lower-ranked
individuals reap more rewards as they attract more visitors: they are
efficient cooperators who earn high rewards via frequent social interactions.
When ranks are dynamic, the population achieves perfect equality over time as
no individual remains a bully for long. Each individual cycles through ranks
to even out payoffs.

In summary, even with these dynamic cycles occurring, the host-guest behavior
of avoiding conflicts emerges and persists in the population. The broad
pattern of behavior (the host-guest norm) is stable despite the fact that no
individual remains static in rank, behavior, or social ties. As a result,
costly hawk-hawk conflicts are rare, occurring only during transitions.
Individuals learn to avoid others who become aggressive or to alter their
behavior in response.

Figure 5:  Cycles have equalizing force that reduce wealth inequality. a, Two
example cycles of coupled changes in network positions, strategies, and ranks
($dh=0.4;dd=0.6;f=0.4$). b, Below the critical value $f<dh$, ranks remain
static and no cycles emerge (the grey shaded area). At $f\geq dh$, a phase
transition occurs and coupled cycles emerge. The cycle length varies as a
function of $f$. Cycles are shorter when power asymmetry is higher. c, Static
networks lead to low total payoff due to inefficient hawk-hawk conflict while
partner choice stabilizes the convention and leads to high payoffs. d, Static
networks lead to high inequality as bullies earn higher payoffs. Partner
choice with static ranks reverses the pattern with low-ranking individuals
earning high payoffs as they attract more visitors. Partner choice and dynamic
ranks preserving perfect equality among individuals. ($dh=0.4;dd=0.6;f=0.6$).

## Discussion

Understanding the emergence and stability of cooperative rules is critical to
avoid the many paths that erode cooperation, including the tragedy of the
commons [60, 98, 97, 108, 129]. Our model provides further insight into the
evolution of cooperation among unequals. Cooperation in this context is
understood as adopting rules or conventions to avoid conflict and adhering to
them. In the hawk-dove game of conflict, correlating one’s strategy on host
and guest roles acts as a sort of egalitarian norm. Often such egalitarian
behavior collapses in the context of hierarchies where dominant alphas can
take what they want. Our study shows that partner choice can maintain the
egalitarian behavior in the presence of power asymmetries. Such conventions
would otherwise break down, as our random interaction model demonstrates. When
an individual’s hierarchical position depends on past competitive success and
is modeled by treating rank as a function of cumulative payoffs, the
egalitarian behavior tends to persist and we see dynamic cycles emerge.
Despite the instability of the power dynamics and the cycles of bullies, the
host-guest rule remains stable and resilient in most of the population. The
presence of bullies ensures some individuals will adopt aggressive strategies,
but the host-guest norm minimizes their impact since all individuals,
including those of low ranks, are aggressive when visiting. The norm of
deference to visitors prevents costly conflict throughout the population and,
in conjunction with partner choice, cannot be destabilized by bullies.
Interestingly, partner choice in the presence of power asymmetries allows only
for host-guest rules and not ownership rules. Thus, whether the host-guest
constellation in this case is properly called a “convention” depends on the
precise definition of convention, and whether conventions require viable
alternative behavioral rules [82, 148].

Developing our work along three models (starting from the random interaction
base case) is useful for comparative analysis. An important common feature of
the three models is that the $f\geq dh$ transition point is robust. It marks
the point at which the first ranking individual breaks from the convention in
all three models. In the dynamic power asymmetry case it also marks the point
at which cycles start to emerge: $f\geq dh$ is the only condition that needs
to be satisfied. The differences in results between the models reveals the
significance of dynamic networks, power asymmetries, and changing payoffs in
understanding social interactions. Traditional analyses often assume fixed
interaction structures and symmetric games with fixed payoffs. Our results
show that the interdependence of these elements can have a major impact on
dynamics. Thus, it is crucial to consider these interdependencies when
attempting to generalize lessons from game theoretic and network models to
real systems and to provide directions for future research.

An innovative feature of our modeling framework involves representing how a
game may change over time[144, 129]. We analyze how the ranks that determine
payoff asymmetry evolve as a function of cumulative payoffs. When individuals
interact in social dilemma games over multiple rounds, it is typically assumed
that any asymmetry that may exist between them is both exogenously determined
and independent of the outcome of previous interactions [91, 62, 78]. Here we
have introduced the idea that the source of asymmetry between individuals may
dynamically depend on past payoffs. This reflects a change in the source of
the ranking hierarchy from prior attributes to a more fluid source determined
by the accumulation of capital. The contrast between static and dynamic
ranking parallels the two main empirical hypotheses about the emergence of
dominance hierarchies[33, 23, 46, 81]. Payoffs in one round of our model
affect payoffs in the next, as is the case in many real-world situations.
Individuals in our model eventually learn (through trembling hands) that they
outrank most of the population and resort to bullying, which leads to the
cycles.

Our model uses relatively simple reinforcement learning mechanism. This
implies that managing cooperative rules in the presence of bullies may be
relatively easy to achieve as long as individuals are able to adjust their
network ties to others. It also presumes that bullies may not create general
obstacles that may change the strategic interactions, game payoffs, etc. When
individuals cannot avail themselves of dynamic network connections—when they
do not have the freedom to choose or change interaction partners—then bullies
become more problematic. Most work on the destabilizing effects of bullies
presumes this sort of environment [18, 128, 145]. Conversely, our results
suggest that when power asymmetries are minor and/or network learning is slow
relative to strategy learning (i.e., when individuals cannot very easily
change their social networks), the cycles that emerge under dynamic power
asymmetry are very long, to the point of effectively being absent. This could
explain why we see “rich-get-richer” phenomena in the real world despite
social networks (in principle) being dynamic.

Finally, the positive effect of partner choice has been recognized in
evolutionary contexts, with a focus on games like the Prisoner’s Dilemma [36,
44]. Our results reveal that this lesson generalizes to situations of conflict
and convention. Thus, our model contributes to the growing body of work that
explains network formation beyond preferential attachment and fitness models
[10, 72]. Learning to avoid conflict with aggressive bullies in dynamic
networks yields network structures that are at times qualitatively similar to
those we observe in human social networks. The majority of network formation
models do not involve strategic choice, but are a stochastic process of
forming ties [9, 72], active linking [99] or assortative interactions [16].
Models that do allow for strategic choice assume that the basis on which
choices are founded remains static. This results in stable (possibly unique)
equilibrium networks [17, 67, 19, 73]. None of these models captures realistic
dynamics such as more powerful individuals adopting more aggressive strategies
and others may choosing to avoid interacting with them. None of the existing
models can therefore explain the cycles we observe. Our work is part of a
family of models being developed now that can explain richer patterns of
network formation [8]. In particular, the realization that the maintenance of
network ties requires effort and is dependent on the positive utility derived
from those interactions.

## Methods

We use agent-based modeling which we analyze through computer simulation. We
adapt the modeling framework developed by [42] but extend it with power
asymmetries that are either static or dynamic. Except for diagrams that show
examples, all results are averaged across $200$ random seeds of the
simulation. We model $N$ agents engaged in pairwise games of conflict with
payoffs determined by the payoff matrix in Eq. 1. All agents are ranked
${1,2,...,N}$ according to their “fighting ability” with the ranking agent
winning $f$ in aggressive hawk-hawk interactions while the outranked agent
receives 0. That is, we implement a rank-order contest in which the ranking
agent receives the entire $f$ payoff, irrespective of how large the difference
in ranks is.

Agents interact through a dynamic network and simultaneously learn which
strategy to play and whom to interact with by updating both strategy and
network weights via Roth-Erev reinforcement [35]. The sequence of actions
taken every round is as follows. First, each agent chooses one other agent to
visit with a probability proportional to its outgoing network weights. Second,
the visitor and the host agents chooses an action of either cooperate (dove)
or defect (hawk) according to their visitor and host strategy weights,
respectively. Third, each agent receives a payoff according to the payoff
matrix. Fourth, all outgoing network weights and strategy weights are updated
simultaneously for all agents via Roth-Erev reinforcement. Note that all
agents are guaranteed exactly one interaction as visitor and between $0$ and
$N-1$ interactions as a host. Agents then interact repeatedly over 1 million
rounds.

This model has two important types of asymmetries. First, agents can play
different strategies depending on their role as host or visitor. Second, since
network ties are directed, who an agent decides to visit can be different from
who it is visited by. Together, these asymmetries allow for the possibility of
correlated conventions to emerge. During the update step, hosts only update
their strategy choices when hosting, whereas the visitors update both their
network connection and strategy choices when visiting. The rationale for this
particular asymmetry is that while individuals can control their behavioural
strategies when hosting or visiting, and who they visit, they cannot control
who decides to visit them.

#### Strategy and partner choice updating.

Each agent $i$ has two vectors $(w_{H},w_{D})$ and $(w_{h},w_{d})$ giving
$i$’s reinforcement weights for playing the hawk and dove strategy when
hosting and visiting, respectively. We use two modular dynamical components
commonly used in reinforcement learning [35] in our model: discounting of
learned weights ($\delta$) and random trembling (errors, $\epsilon$).
Trembling works identically for all actions (choosing a partner, choosing an
action as visitor, choosing a response as host). The probability of choosing
an action $s$ is proportional to the current relevant weights:

$Pr(s)=(1-\epsilon)\frac{w_{s}}{\sum_{s^{\prime}}w_{s^{\prime}}}+\epsilon\frac{1}{|S|}$
(3)

where $\epsilon$ is the error rate, $S$ is the relevant set of available
choices (i.e., visiting strategies, hosting strategies, or choice of player to
visit), $s\in S$ and $s^{\prime}\in S$.

All strategy weights are updated simultaneously for all agents at the end of
each round with the received payoffs $(\pi)$ and discounted by a factor
$\delta$ (set at 0.01 unless otherwise noted) according to

$w_{s}^{\prime}=(1-\delta)w_{s}+\pi_{s}$ (4)

where $w_{s}^{\prime}$ is the weights after updating and $s$ is the action
that was chosen; $\pi_{s}=0$ if action $s$ was not chosen. All visits and
strategy choices occur simultaneously within a round and all weights are
updated simultaneously after each round. Agents keep separate strategy weights
for hosting and visiting and this asymmetry is maintained during the updating
process such that the payoffs received for hosting do not affect strategy
weights for visiting or vice versa.

Each agent $i$ has a vector representing their directed weighted network ties
with other agents $(w_{i1},w_{i2},...,w_{in})$ where $w_{ij}$ represents the
weight that agent $i$ chooses to visit agent $j$; $(w_{ii}=0)$. We initialize
weights as $1$ for both strategy choices and network weights uniformly as
$\frac{L}{N-1}$. To standardize learning speeds between strategy updating and
network updating, we set $L=19$ which yields starting weights of $1$ for all
network ties in our standard population of $N=20$ ($w_{ij}=1$ for all $i,j$
and $i\neq j$). Keeping $L$ constant across populations of different sizes
allows us to keep the reinforcement learning at a similar speed relative to
total initial weights.

When selecting an interaction partner for a given round, the probability of
choosing agent $j$ is proportional to agent $i$’s outgoing network weights as

$Pr(j)=(1-\epsilon)\frac{w_{ij}}{\sum_{k}w_{ik}}+\epsilon\frac{1}{|N|}$ (5)

where $\epsilon$ is the error rate, $N$ is the set of agents, $j\in N$ and
$k\in N$.

After each round of interactions, the weights for all outgoing partner choice
links are updated by discounting the prior weight by a factor ($\delta$) and
adding the received payoff ($\pi$)

$w_{ij}^{\prime}=(1-\delta)w_{ij}+v\pi_{ij}$ (6)

where $w_{ij}^{\prime}$ is the link weight after updating and $\pi_{ij}$ is
the most recent payoff and $v$ is an adjustment factor for the timescale of
network learning. We investigate differences in relative learning speeds by
changing the network learning speed relative to the speed of strategy
learning. We modify the speed of network learning by multiplying the payoffs
received after each round by an adjustment factor $v$ before updating network
weights but leaving payoffs unmodified for strategy updating. We investigate
network updating both at speeds slower than strategy updating (e.g., $v=0.1$)
and faster than the strategy updating speed (e.g., $v=10$).

#### Rank updating.

We consider two versions of this game: static and dynamic. In the static case,
each agent is randomly assigned a rank between ${1,...,N}$ at the beginning
which remains the same throughout all interactions. Ranks are private
information but players learn the ranks of others through reinforcement
learning. The payoff for the ranking agent $i$ playing hawk against an
outranked agent $j$ also playing hawk is $f$, which we vary from $0$ (ranks
effectively do not matter at all) to 1.0 (ranks matter greatly), while the
outranked agent always receives a payoff of $0$.

In the dynamic ranks case, we update ranks based on cumulative payoffs every
$1,000$ rounds. Updating ranks every $1,000$ rounds avoids noise from tremble
and updating ranks more/less often does not substantively alter our our
results. In case of a hawk-hawk interaction among agents with identical
cumulative payoffs (identical cumulative payoffs are extremely rare given
discounting but can happen in early rounds) each agent receives a payoff of
$f/3$, which represents an equal chance of winning the resource but with some
cost of conflict. Agents who are successful in their interactions (achieve
high payoffs) and attract many visitors, accumulate more payoffs and are
better ranked.

## References

  * [1] J. M. Alexander.  The Structural Evolution of Morality.  Cambridge University Press, 2007.
  * [2] B. Allen, G. Lippner, Y.-T. Chen, B. Fotouhi, N. Momeni, S.-T. Yau, and M. A. Nowak.  Evolutionary dynamics on any population structure.  Nature, 544:227–230, 2017.
  * [3] Abdullah Almaatouq, Alejandro Noriega-Campero, Abdulrahman Alotaibi, P. M. Krafft, Mehdi Moussaid, and Alex Pentland.  Adaptive social networks promote the wisdom of crowds.  Proceedings of the National Academy of Sciences, 2020.
  * [4] Sinan Aral and Marshall Van Alstyne.  The diversity-bandwidth trade-off.  American Journal of Sociology, 117(1):90–171, 2011.
  * [5] Jasmina Arifovic, Joshua F. Boitnott, and John Duffy.  Learning correlated equilibria: An evolutionary approach.  Journal of Econmic Behavior & Organization, 2016.
  * [6] Robert J Aumann.  Subjectivity and correlation in randomized strategies.  Journal of Mathematical Economics, 1(1):67–96, 1974.
  * [7] Robert Axelrod and William D Hamilton.  The evolution of cooperation.  Science, 211(4489):1390–1396, 1981.
  * [8] Abhijit V Banerjee, Arun G Chandrasekhar, Esther Duflo, and Matthew O Jackson.  Changes in social network structure in response to exposure to formal credit markets.  Available at SSRN 3245656, 2018.
  * [9] Albert-László Barabási and Réka Albert.  Emergence of scaling in random networks.  Science, 286(5439):509–512, 1999.
  * [10] Albert-László Barabási and Réka Albert.  Emergence of scaling in random networks.  Science, 286(5439):509–512, 1999.
  * [11] Pranab Bardhan.  Irrigation and cooperation: An empirical analysis of 48 irrigation communities in south india.  Economic Development and cultural change, 48(4):847–865, 2000.
  * [12] Alain Barrat, Marc Barthelemy, Romualdo Pastor-Satorras, and Alessandro Vespignani.  The architecture of complex weighted networks.  Proceedings of the National Academy of Sciences, 101(11):3747–3752, 2004.
  * [13] Alain Barrat, Marc Barthélemy, and Alessandro Vespignani.  Weighted evolving networks: coupling topology and weight dynamics.  Physical review letters, 92(22):228701, 2004.
  * [14] Alan W Beggs.  On the convergence of reinforcement learning.  Journal of Economic Theory, 122(1):1–36, 2005.
  * [15] Alex Bell, Raj Chetty, Xavier Jaravel, Neviana Petkova, and John Van Reenen.  Who becomes an inventor in america? the importance of exposure to innovation.  The Quarterly Journal of Economics, 134(2):647–713, 2018.
  * [16] Theodore C Bergstrom.  The algebra of assortative encounters and the evolution of cooperation.  International Game Theory Review, 5(03):211–228, 2003.
  * [17] Francis Bloch, Garance Genicot, and Debraj Ray.  Informal insurance in social networks.  Journal of Economic Theory, 143(1):36–58, 2008.
  * [18] Christopher Boehm.  Hierarchy in the Forest: The Evolution of Egalitarian Behavior.  Harvard University Press, 2001.
  * [19] Vincent Boucher.  Structural homophily.  International Economic Review, 56(1):235–264, 2015.
  * [20] Robert Boyd, Peter J Richerson, Ruth Meinzen-Dick, Tine De Moor, Matthew O Jackson, Kristina M Gjerde, Harriet Harden-Davies, Brett M Frischmann, Michael J Madison, Katherine J Strandburg, et al.  Tragedy revisited.  Science, 362(6420):1236–1241, 2018.
  * [21] Yann Bramoullé.  Anti-coordination and social interactions.  Games and Economic Behavior, 58(1):30–49, 2007.
  * [22] Ronald S Burt and Jennifer Merluzzi.  Network oscillation.  Academy of Management Discoveries, 2(4):368–391, 2016.
  * [23] Ivan D Chase, Craig Tovey, Debra Spangler-Martin, and Michael Manfredonia.  Individual differences versus social dynamics in the formation of animal dominance hierarchies.  Proceedings of the National Academy of Sciences, 99(8):5744–5749, 2002.
  * [24] Todd L Cherry, Stephan Kroll, and Jason F Shogren.  The impact of endowment heterogeneity and origin on public good contributions: evidence from the lab.  Journal of Economic Behavior & Organization, 57(3):357–365, 2005\.
  * [25] Raj Chetty, Nathaniel Hendren, and Lawrence F Katz.  The effects of exposure to better neighborhoods on children: New evidence from the moving to opportunity experiment.  American Economic Review, 106(4):855–902, 2016.
  * [26] Aaron Clauset, Cosma Rohilla Shalizi, and Mark EJ Newman.  Power-law distributions in empirical data.  SIAM Review, 51(4):661–703, 2009.
  * [27] R M Dawes.  Social dilemmas.  Annual Review of Psychology, 31(1):169–193, 1980.
  * [28] Stephen J DeCanio and Anders Fremstad.  Game theory and climate diplomacy.  Ecological Economics, 85:177–187, 2013.
  * [29] Peter Sheridan Dodds and Duncan J Watts.  A generalized model of social and biological contagion.  Journal of Theoretical Biology, 232(4):587–604, 2005.
  * [30] Michael Doebeli and Christoph Hauert.  Models of cooperation based on the prisoner’s dilemma and the snowdrift game.  Ecology Letters, 8:748–766, 2005.
  * [31] W-B Du, X-B Cao, M-B Hu, and W-X Wang.  Asymmetric cost in snowdrift game on scale-free networks.  EPL (Europhysics Letters), 87(6):60004, 2009.
  * [32] John Duffy and Nick Feltovich.  Correlated equilibria, good and bad: An experimental study.  International Economic Review, 51(3):701–721, 2010.
  * [33] Lee Alan Dugatkin.  Winner and loser effects and the structure of dominance hierarchies.  Behavioral Ecology, 8(6):583–587, 1997.
  * [34] Yvonne Durham, Jack Hirshleifer, and Vernon L Smith.  Do the rich get richer and the poor poorer? experimental tests of a model of power.  The American Economic Review, 88(4):970–983, 1998.
  * [35] Ido Erev and Alvin E Roth.  Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria.  American Economic Review, pages 848–881, 1998.
  * [36] Ilan Eshel and Luigi Luca Cavalli-Sforza.  Assortment of encounters and evolution of cooperativeness.  Proceedings of the National Academy of Sciences, 79(4):1331–1335, 1982.
  * [37] Katrin Fehl, Daniel J. van der Post, and Dirk Semmann.  Co-evolution of behaviour and social network structure promotes human cooperation.  Ecology Letters, 6(11):546–551, 2011.
  * [38] Ernst Fehr and Urs Fischbacher.  Social norms and human cooperation.  Trends in cognitive sciences, 8(4):185–190, 2004.
  * [39] A. J. Figueredo and P. S. A. Wolf.  Assortative pairing and life history strategy – a cross-cultural study.  Human Nature, 20:317–330, 2009.
  * [40] Jessica C. Flack.  Multiple time-scales and the developmental dynamics of social systems.  Philosophical Transactions of the Royal Society B: Biological Sciences, 367(1597):1802–1810, 2012.
  * [41] Jessica C Flack, Michelle Girvan, Frans BM De Waal, and David C Krakauer.  Policing stabilizes construction of social niches in primates.  Nature, 439(7075):426, 2006.
  * [42] Michael Foley, Patrick Forber, Rory Smead, and Christoph Riedl.  Conflict and convention in dynamic networks.  Journal of the Royal Society Interface, 15(140):20170835, 2018.
  * [43] Dean P. Foster and Rakesh V. Vohara.  Calibrated learning and correlated equilibrium.  Games and economic behavior, 21(1-2):40–55, 1997.
  * [44] Tom Lenaerts Francisco C. Santos, Jorge M. Pacheco.  Cooperation prevails when individuals adjust their social ties.  PLoS Computational Biology, 2(10):e140, 2006.
  * [45] Robert H Frank and Philip J Cook.  The winner-take-all society: Why the few at the top get so much more than the rest of us.  Random House, 2010.
  * [46] Mathias Franz, Emily McLean, Jenny Tung, Jeanne Altmann, and Susan C Alberts.  Self-organizing dominance hierarchies in a wild primate population.  Proceedings of the Royal Society B: Biological Sciences, 282(1814):20151512, 2015.
  * [47] Feng Fu, Christoph Hauert, Martin A Nowak, and Long Wang.  Reputation-based partner choice promotes cooperation in social networks.  Physical Review E, 78(2):026117, 2008.
  * [48] Feng Fu, Te Wu, and Long Wang.  Partner switching stabilizes cooperation in coevolutionary prisoner’s dilemma.  Physical Review E, 79(3):036101, 2009.
  * [49] Drew Fudenberg and David K Levine.  The theory of learning in games, volume 2.  MIT Press, 1998.
  * [50] Drew Fudenberg and Jean Tirole.  Game theory, 1991.  Cambridge, Massachusetts, 393:12, 1991.
  * [51] S. Goyal and F. Vega-Redondo.  Network formation and social coordination.  Games and Economic Behavior, 50:178–207, 2005.
  * [52] Alan Grafen.  The logic of divisively asymmetric contests: respect for ownership and the desperado effect.  Animal Behaviour, 35(2):462–467, 1987.
  * [53] Mark S Granovetter.  The strength of weak ties.  American Journal of Sociology, 78(6):1360–1380, 1973.
  * [54] T. Gross and B. Blasius.  Adaptive coevolutionary networks: a review.  Journal of the Royal Society Interface, 5:259–271, 2008.
  * [55] Thilo Gross and Bernd Blasius.  Adaptive coevolutionary networks: a review.  Journal of the Royal Society Interface, 5(20):259–271, 2008.
  * [56] Francesco Guala and Luigi Mittone.  How history and convention create norms: an experimental study.  Journal of Economic Psychology, 31:749–756, 2010.
  * [57] Cameron L Hall, Mason A Porter, and Marian S Dawkins.  Dominance, sharing, and assessment in an iterated hawk–dove game.  Journal of Theoretical Biology, 493:110101, 2020.
  * [58] Peter Hammerstein.  The role of asymmetries in animal contests.  Animal Behaviour, 29(1):193–205, 1981.
  * [59] Z. Hao, A. AghaKouchak, N. Nakhjiri, and A Farahmand.  Global integrated drought monitoring and prediction system (GIDMaPS) data sets.  _figshare_ http://dx.doi.org/10.6084/m9.figshare.853801, 2014\.
  * [60] Garrett Hardin.  The tragedy of the commons.  Science, 162(3859):1243–1248, 1968.
  * [61] Sergiu Hart and Andreu Mas-Colell.  a simple adaptive procedure leading to corrlated equilibrium.  Econometrica, 68(5):1127–1150, 2000.
  * [62] Ch Hauert and Heinz Georg Schuster.  Effects of increasing the number of players and memory size in the iterated prisoner’s dilemma: a numerical approach.  Proceedings of the Royal Society of London. Series B: Biological Sciences, 264(1381):513–519, 1997.
  * [63] Christoph Hauert and Michael Doebeli.  Spatial structure often inhibits the evolution of cooperation in the snowdrift game.  Nature, 428(6983):643–646, 2004.
  * [64] Oliver P Hauser, Christian Hilbe, Krishnendu Chatterjee, and Martin A Nowak.  Social dilemmas among unequals.  Nature, 572(7770):524–527, 2019.
  * [65] Oliver P Hauser, Gordon T Kraft-Todd, David G Rand, Martin A Nowak, and Michael I Norton.  Invisible inequality leads to punishing the poor and rewarding the rich.  Behavioural Public Policy, pages 1–21, 2019.
  * [66] Shaun P Hargreaves Heap, Abhijit Ramalingam, and Brock V Stoddard.  Endowment inequality in public goods games: A re-examination.  Economics Letters, 146:4–7, 2016.
  * [67] P Jean-Jacques Herings, Ana Mauleon, and Vincent Vannetelbosch.  Farsightedly stable networks.  Games and Economic Behavior, 67(2):526–541, 2009.
  * [68] Richard J Herrnstein.  On the law of effect.  Journal of the Experimental Analysis of Behavior, 13(2):243–266, 1970.
  * [69] Christian Hilbe, Štěpán Šimsa, Krishnendu Chatterjee, and Martin A Nowak.  Evolution of cooperation in stochastic games.  Nature, 559(7713):246, 2018.
  * [70] Jack Hirshleifer.  The paradox of power.  Economics & Politics, 3(3):177–200, 1991.
  * [71] Simon Huttegger, Brian Skyrms, Pierre Tarres, and Elliott Wagner.  Some dynamics of signaling games.  Proceedings of the National Academy of Sciences, 111(suppl. 3):10873–10880, 2014.
  * [72] Matthew O Jackson.  Social and economic networks.  Princeton University Press, 2008.
  * [73] Matthew O Jackson, Tomas Rodriguez-Barraquer, and Xu Tan.  Social capital and social quilts: Network patterns of favor exchange.  American Economic Review, 102(5):1857–97, 2012.
  * [74] Matthew O Jackson and Alison Watts.  The evolution of social and economic networks.  Journal of Economic Theory, 106(2):265–295, 2002.
  * [75] Matthew O Jackson and Alison Watts.  On the formation of interaction networks in social coordination games.  Games and Economic Behavior, 41(2):265–291, 2002.
  * [76] Sham Kakade, Michael Kearns, John Langford, and Luis Ortiz.  Correlated equilibria in graphical games.  In Proceedings of the 4th ACM Conference on Electronic Commerce, pages 42–47, San Diego, California, USA, 2003. ACM.
  * [77] Martin Kilduff and Jung Won Lee.  The integration of people and networks.  Annual Review of Organizational Psychology and Organizational Behavior, 7, 2020.
  * [78] Timothy Killingback and Michael Doebeli.  The continuous prisoner’s dilemma and the evolution of cooperation through reciprocal altruism with variable investment.  American Naturalist, 160(4):421–438, 2002.
  * [79] H. Kokko, A. López-Spulcre, and L. J. Morrell.  From hawks and doves to self-consistent games of territorial behavior.  The American Naturalist, 167:901–912, 2006.
  * [80] J. R. Krebs and N. B. Davies.  An Introduction to Behavioral Ecology.  Blackwell, London, 1993.
  * [81] Kate L Laskowski, Max Wolf, and David Bierbach.  The making of winners (and losers): how early dominance interactions determine adult social structure in a clonal fish.  Proceedings of the Royal Society B: Biological Sciences, 283(1830):20160183, 2016.
  * [82] David Lewis.  Convention.  Harvard University Press, 1969.
  * [83] M. W. Macy and A. Flache.  Learning dynamics in social dilemmas.  Proceedings of the National Academy of Sciences, 99:7229–7236, 2002\.
  * [84] C. W. Marean.  An evolutionary anthropological perspective on modern human origins.  Annual Review of Anthropology, 98:533–556, 2001.
  * [85] Alex McAvoy and Christoph Hauert.  Asymmetric evolutionary games.  PLoS Computational Biology, 11(8):e1004349, 2015.
  * [86] M. Mesterton-Gibbons, T. Karabiyik, and T. N. Sherratt.  The iterated hawk-dove game revisited: the effect of ownership uncertainty on bourgeois as a pure convention.  Dynamic Games and Applications, 4:407–431, 2014.
  * [87] Mike Mesterton-Gibbons and Tom N Sherratt.  Bourgeois versus anti-bourgeois: a model of infinite regress.  Animal Behaviour, 89:171–183, 2014.
  * [88] Elena Miu, Ned Gulley, Kevin N Laland, and Luke Rendell.  Innovation and cumulative culture through tweaks and leaps in online programming contests.  Nature communications, 9(1):2321, 2018.
  * [89] Akihiro Nishi, Hirokazu Shirado, David G Rand, and Nicholas A Christakis.  Inequality and visibility of wealth in experimental social networks.  Nature, 526(7573):426, 2015.
  * [90] M. A. Nowak.  Five rules for the evolution of cooperation.  Science, 314:1560–1563, 2006.
  * [91] Martin Nowak and Karl Sigmund.  A strategy of win-stay, lose-shift that outperforms tit-for-tat in the prisoner’s dilemma game.  Nature, 364(6432):56, 1993.
  * [92] Martin A Nowak and Karl Sigmund.  Evolutionary dynamics of biological games.  Science, 303(5659):793–799, 2004.
  * [93] Cailin O’Connor.  The Origins of Unfairness: Social Categories and Cultural Evolution.  Oxford University Press, USA, 2019.
  * [94] F. J. Odling-Smee, K. N. Laland, and M. F. Feldman.  Niche Construction: The Neglected Process in Evolution.  Princeton University Press, 2003.
  * [95] Hisashi Ohtsuki, Christoph Hauert, Erez Lieberman, and Martin A Nowak.  A simple rule for the evolution of cooperation on graphs and social networks.  Nature, 441(7092):502–505, 2006.
  * [96] Elinor Ostrom.  Governing the commons: The evolution of institutions for collective action.  Cambridge University Press, 1990.
  * [97] Elinor Ostrom, Joanna Burger, Christopher B Field, Richard B Norgaard, and David Policansky.  Revisiting the commons: local lessons, global challenges.  Science, 284(5412):278–282, 1999.
  * [98] Elinor Ostrom, James Walker, and Roy Gardner.  Covenants with and without a sword: Self-governance is possible.  American Political Science Review, 86(2):404–417, 1992.
  * [99] Jorge M Pacheco, Arne Traulsen, and Martin A Nowak.  Active linking in evolutionary games.  Journal of theoretical biology, 243(3):437–443, 2006.
  * [100] Jorge M Pacheco, Arne Traulsen, and Martin A Nowak.  Coevolution of strategy and structure in complex networks with dynamical linking.  Physical Review Letters, 97(25):258103, 2006.
  * [101] Robin Pemantle and Brian Skyrms.  Network formation by reinforcement learning: the long and medium run.  Mathematical Social Sciences, 48:315–327, 2004.
  * [102] M. Perc and A. Szolnoki.  Coevolutionary games—a mini review.  Biosystems, 99:109–125, 2010.
  * [103] Matjaž Perc and Attila Szolnoki.  Coevolutionary games: A mini review.  BioSystems, 99(2):109–125, 2010.
  * [104] Nicola Perra, Bruno Gonçalves, Romualdo Pastor-Satorras, and Alessandro Vespignani.  Activity driven modeling of time varying networks.  Scientific Reports, 2:469, 2012.
  * [105] Thomas Piketty and Emmanuel Saez.  Inequality in the long run.  Science, 344(6186):838–843, 2014.
  * [106] F. L. Pinheiro, Jorge M Pacheco, and Francisco C Santos.  From local to global dilemmas in social networks.  PLoS One, 7:e32114, 2012.
  * [107] F. L. Pinheiro, Francisco C Santos, and Jorge M Pacheco.  Linking individual and collective behavior in adaptive social networks.  Physical Review Letters, 116:128702, 2016.
  * [108] William H. Press and Freeman J. Dyson.  Iterated prisoner’s dilemma contains strategies that dominate any evolutionary opponent.  Proceedings of the National Academy of Sciences, 109(26):10409–10413, 2012.
  * [109] David G Rand, Samuel Arbesman, and Nicholas A Christakis.  Dynamic social networks promote cooperation in experiments with humans.  Proceedings of the National Academy of Sciences, 108(48):19193–19198, 2011.
  * [110] David G Rand and Martin A Nowak.  Human cooperation.  Trends in cognitive sciences, 17(8):413–425, 2013.
  * [111] David G Rand, Martin A Nowak, James H Fowler, and Nicholas A Christakis.  Static network structure can stabilize human cooperation.  Proceedings of the National Academy of Sciences, 111(48):17093–17098, 2014.
  * [112] Bruno Ribeiro, Nicola Perra, and Andrea Baronchelli.  Quantifying the effect of temporal resolution on time-varying networks.  Scientific Reports, 3:3006, 2013.
  * [113] Alvin E Roth and Ido Erev.  Learning in extensive-form games: Experimental data and simple dynamic models in the intermediate term.  Games and Economic Behavior, 8(1):164–212, 1995.
  * [114] Larry Samuelson.  Evolutionary Games and Equilibrium Selection.  MIT Press, 1997.
  * [115] Larry Samuelson.  Evolution and game theory.  Journal of Economic Perspectives, 16(2):47–66, 2002.
  * [116] W. Sandholm.  Population games and evolutionary dynamics.  MIT Press, 2010.
  * [117] Francisco C Santos and Jorge M Pacheco.  Scale-free networks provide a unifying framework for the emergence of cooperation.  Physical Review Letters, 95(9):098104, 2005.
  * [118] Zuzana Sasovova, Ajay Mehra, Stephen P Borgatti, and Michaéla C Schippers.  Network churn: The effects of self-monitoring personality on brokerage dynamics.  Administrative Science Quarterly, 55(4):639–670, 2010.
  * [119] Marten Scheffer, Bas van Bavel, Ingrid A van de Leemput, and Egbert H van Nes.  Inequality in nature and society.  Proceedings of the National Academy of Sciences, 114(50):13154–13157, 2017.
  * [120] Anuj K Shah, Sendhil Mullainathan, and Eldar Shafir.  Some consequences of having too little.  Science, 338(6107):682–685, 2012.
  * [121] Karl Sigmund.  The calculus of selfishness.  Princeton University Press, 2010.
  * [122] Brian Skyrms.  The stag hunt and the evolution of social structure.  Cambridge University Press, 2004.
  * [123] Brian Skyrms.  Evolution of the social contract.  Cambridge University Press, 2014.
  * [124] Brian Skyrms and Robin Pemantle.  A dynamic model of social network formation.  Proceedings of the National Academy of Sciences, 97(16):9340–9346, 2000.
  * [125] R. Smead and P. Forber.  Convention and the origins of ownership.  Philosophy of Science, 87(5):doi:10.1086/710626, 2020.
  * [126] John Maynard Smith.  Evolution and the Theory of Games.  Cambridge University Press, 1982.
  * [127] John Maynard Smith and Geoffrey A Parker.  The logic of asymmetric contests.  Animal Behaviour, 24(1):159–175, 1976.
  * [128] Kim Sterelny.  The evolved apprentice.  MIT Press, 2012.
  * [129] Alexander J. Stewart and Joshua B. Plotkin.  Collapse of cooperation in evolving games.  Proceedings of the National Academy of Sciences, 111(49):17558–17563, 2014.
  * [130] M. C. Stiner.  Thirty years on the “broad spectrum revolution” and paleolithic demography.  Proceedings of the National Academy of Sciences, 98:6993–6996, 2001\.
  * [131] G. Szabó and I. Borsos.  Evolutionary potential games on lattices.  Physics Reports, 624:1–60, 2016.
  * [132] G. Szabó and G. Fáth.  Evolutionary games on graphs.  Physics Reports, 446:97–216, 2007.
  * [133] Lionel Trilling.  Sincerity and authenticity.  Harvard University Press, 2009.
  * [134] Robert L Trivers.  The evolution of reciprocal altruism.  The Quarterly review of biology, 46(1):35–57, 1971.
  * [135] Christian Tröster, Andrew Parker, Daan Van Knippenberg, and Ben Sahlmüller.  The coevolution of social networks and thoughts of quitting.  Academy of Management Journal, 62(1):22–43, 2019.
  * [136] Milena Tsvetkova and Vincent Buskens.  Coordination on egalitarian networks from asymmetric relations in a social game of chicken.  Advances in Complex Systems, 16(01):1350005, 2013.
  * [137] Sven Van Segbroeck, Francisco C Santos, Tom Lenaerts, and Jorge M Pacheco.  Reacting differently to adverse ties promotes cooperation in social networks.  Physical Review Letters, 102(5):058105, 2009.
  * [138] Sven Van Segbroeck, Francisco C Santos, Ann Nowé, Jorge M Pacheco, and Tom Lenaerts.  The evolution of prompt reaction to adverse ties.  BMC Evolutionary Biology, 8(1):287, 2008.
  * [139] P. Vanderschraaf.  Convention as correlated equilibrium.  Erkenntnis, 42:65–87, 1995.
  * [140] Ivan Voitalov, Pim van der Hoorn, Remco van der Hofstad, and Dmitri Krioukov.  Scale-free networks well done.  Physical Review Research, 1(3):033034, 2019.
  * [141] Wen-Xu Wang, Jie Ren, Guanrong Chen, and Bing-Hong Wang.  Memory-based snowdrift game on networks.  Physical Review E, 74(5):056113, 2006.
  * [142] Duncan J Watts and Peter Sheridan Dodds.  Influentials, networks, and public opinion formation.  Journal of Consumer Research, 34:441–458, 2007.
  * [143] Jörgen W Weibull.  Evolutionary Game Theory.  MIT Press, 1997.
  * [144] Lee Worden and Simon A Levin.  Evolutionary escape from the prisoner’s dilemma.  Journal of Theoretical Biology, 245(3):411–422, 2007.
  * [145] Richard Wrangham.  The Goodness Paradox: The Strange Relationship Between Virtue and Violence in Human Evolution.  Pantheon, 2019.
  * [146] B. Wu, D. Zhou, F. Fu, Q. Luo, and A. Wang, L.AND Traulsen.  Evolution of cooperation on stochastic dynamical networks.  PloS One, 5(6):e11187, 2010.
  * [147] Soon-Hyung Yook, Hawoong Jeong, A-L Barabási, and Yuhai Tu.  Weighted evolving networks.  Physical Review Letters, 86(25):5835, 2001.
  * [148] H. P. Young.  The evolution of conventions.  Econometrica, 61:57–84, 1993.
  * [149] L-X Zhong, D-F Zheng, B Zheng, Chen Xu, and PM Hui.  Networking effects on cooperation in evolutionary snowdrift game.  EPL (Europhysics Letters), 76(4):724, 2006.

## Supplementary Information

### Note on games and representation

The baseline representation for the strategic interactions we aim to
investigate is a game of conflict. By game of conflict we mean the generic set
of symmetric $2\times 2$ games where payoffs are strictly ordered such that
there are two strict asymmetric Nash and one symmetric mixed Nash equilibria
[143]. Given the payoff matrix below (eq 7), that payoff ordering condition is
$hd>dd>dh>hh$. Traditional models that use infinite populations with random-
mixing have a single stable point: the mixed Nash equilibrium. In the
literature most descriptions of hawk-dove, snowdrift, chicken, and anti-
coordination games count as generic games of conflict.

| hawkj | dovej
---|---|---
hawki |  (25mm $f,hh$ |  $hd,dh$ )20mm
dovei | $dh,hd$ | $dd,dd$
(7)

Games of conflict provide a natural and well-studied representation of
competitive interactions over resources [127, 126, 52]. These games, despite
being idealized, provide insight into many cooperative interactions that are
not adequately captured by, say, the prisoner’s dilemma [11]. Avoiding costly
conflicts is essential for cooperation, hence these games represent anti-
coordination scenarios such as congestion, pollution, and provision of public
goods [30, 21]; these and similar games have also been used to explore
division of labor [93] and situations with negative externalities [28].
Individuals that have no incentive to avoid conflict pose a substantial threat
to cooperative resolutions of these strategic interaction.

The novel feature of the current model involves introducing asymmetry in
conflict ability by assigning each individual a rank, where the ranking
individual dominates aggressive (hawk-hawk) interactions and secures the
contested resource. This is a departure from the baseline symmetric game of
conflict (see Table S1 for a review of related work). In the current model,
the ranking individual receives a fixed payoff $f$ whenever conflict occurs
and the outranked individual receives a payoff of $0$ in a winner-take-all
type of contest based on the relative standing between the two individuals.

### Related Work

We review closely related work in Table S1.

| Game | Coevolutiona | Asymmetryb | Weighted Networkc | Inequalityd
---|---|---|---|---|---
This Study | Conflict | Yes | Yes | Yes | Yes & Dynamic
Games of Conflict on Dynamic Networks |
[42] | Conflict | Yes | Yes | Yes | No
[100] | Conflict | Yes | No | No | No
[44] | Conflict | Yes | No | No | No
Games of Conflict on Static Networks |
[127] | Conflict | No | Yes | No | No
[87] | Conflict | No | Yes | No | No
[85] | Conflict | No | Yese | No | No
[136] | Conflict | No | Yese | No | No
[31] | Conflict | No | No | No | Yesf
[83] | Conflict | No | No | No | No
[30] | Conflict | No | No | No | No
[79] | Conflict | No | No | No | No
[141] | Conflict | No | No | No | No
[149] | Conflict | No | No | No | No
[21] | Conflict | No | No | No | No
Dynamic Networks but not Games of Conflict |
[124] | Stag Hunt | Yes | No | No | No
[75] | Coordination | Yes | No | No | No
[51] | Coordination | Yes | No | No | No
[47] | PD | Yes | No | No | No
[48] | PD | Yes | No | No | No
[138] | PD | Yes | No | No | No
[137] | PD | Yes | No | No | No
[146] | PD | Yes | No | No | No
[107] | PD | Yes | No | No | No
Inequality |
[65] | Public Goods | No | No | No | Yes
[64] | Public Goods | No | No | No | Yes

Table S1:  Summary of most closely related work.

Note. We include only the most closely related studies that address one of the
key aspects of our work: Games of conflict, dynamic networks, or inequality.
Two review papers of work on dynamic networks are Gross & Blasius[55] and Perc
& Szolnoki [103].
a This column focuses on whether the study uses networked interaction and
allows partner choice. That is, network ties are allowed to coevolve with
behavioral strategies.
b This column indicates whether or not the model comprises asymmetric
strategies for hosting vs. visiting behavior, thus allowing correlated
conventions as possible solution equilibria.
c This column indicates whether or not the model uses discrete network ties,
which requires an assumption about network density (or model network density
directly through a parameter).
d This column indicates if models includes inequality among agents such as
differences in fighting ability. Both studies in this category are human-
subject experiments.
e Their models do not distinguish asymmetry between hosting and visiting (and
thus precludes the correlated equilibria as a solution) but include
interesting asymmetry in the games faced by different agents, or allow actors
to deploy different actions against different partners, similar to ours.
f Inequality is based on network connectivity (which is static) and affects
the payoffs in cooperative dove-dove interactions, but not in hawk-hawk
conflict.

### Static game analysis

For the sake of simplicity, presume $hh=0$ and the payoff preference ordering
for a game of conflict: $hd>dd>dh>hh$. For group size $N$, each player has a
rank $i\in\\{1,2,\ldots,N\\}$. If $i<j$ then the hawk-hawk payoff to player
$i$ is $f$ (and the payoff to player $j$ is 0—the ranking individual wins the
contest). If $i>j$ then hawk-hawk payoff to player $i$ is $hh=0$ (and the
payoff to player $j$ is $f$—the outranked individual loses the contest).

Suppose a player $i$ is paired at random with another player $j$ without
knowledge of individual rankings. For any two players with ranks $i,j:i\neq j$
(i.e., there are no ties in rankings). Let $R$ be the probability player $i$
outranks partner $j$ where

$R=1-\frac{i-1}{N-1}.$ (8)

Then the expected payoffs for hawk and dove depend on the probability of
outranking an opponent and the probability an opponent plays hawk:

$E(hawk_{i})=P(hawk_{j})[R\cdot f+(1-R)\cdot hh]+P(dove_{j})\cdot hd,$ (9)

$E(dove_{i})=P(hawk_{j})\cdot dh+P(dove_{j})\cdot dd.$ (10)

Assuming $hh=0$ and $P(dove_{j})=1-P(hawk_{j})$:

$E(hawk_{i})=hd+P(hawk_{j})(R\cdot f-hd),$ (11)

$E(dove_{i})=dd+P(hawk_{j})(dh-dd).$ (12)

Player $i$ strictly prefers to play hawk iff $F(hawk_{i})>F(dove_{i})$ which
is true just in case:

$P(hawk_{j})<\frac{hd-dd}{(dh-dd)-(R\cdot f-hd)}.$ (13)

Or equivalently:

$R>\frac{dh-dd+hd}{f}-\frac{(hd-dd)}{P(hawk_{j})\cdot f}.$ (14)

In the situation where $E(hawk_{i})=E(dove_{i})$ (opponents play $hawk$ with a
probability equal to right hand side of condition 13) then player $i$ is
indifferent between playing $hawk$ or $dove$. Note that for the bottom ranked
individual ($R=0$) this condition amounts to the mixed Nash equilibrium for
the background game of conflict.

### Finding the dominant solvable breakpoint

Given the asymmetry created by differences in power or competitive ability,
the game may be dominant solvable for some top ranking individuals. We can
identify this region by plotting an indifference line as a function of the
probability of encountering a ranking or outranked individual. That function
is

$P(hawk_{j})=\frac{(hd-dd)}{(dh-dd)-(R\cdot f-hd)}$ (15)

This is the point at which, if your opponent should play $hawk$ with this
probability, you are indifferent between $(hawk,dove)$ presuming
$P(hawk_{j})\in[0,1]$. This point will not exist for all players. Depending on
the exact payoffs, only some players will have an indifference point. So, for
instance, top ranked individuals will have an “indifference point” outside the
unit interval, which simply means that $hawk$ is their dominant strategy. If
$0<P(h_{j})<1$ then there exists a point at which, should an opponent play
$hawk$ with higher probability, the player prefers $dove$; and should an
opponent play $hawk$ with lower probability, the player prefers $hawk$. In
effect, the player does not have a dominant strategy in the population. Note
that the lowest ranked individual in the population has an indifference point
equal to the mixed Nash equilibrium for the standard hawk-dove (where $f=hh$)
game since the probability that player ever plays the alternative game is
zero.

In the rank ordering there is a breakpoint ($i_{t}$) between ranking
individuals that have a dominant strategy ($hawk$) and outranked individuals
that have an indifference point below one ($P(hawk_{j})<1$). To solve where in
the ranking the breakpoint exists set $P(hawk_{j})=1$ and solve for $R$ (the
probability $i$ outranks $j$):

$R\cdot f=dh.$ (16)

Using equation 8 to substitute for $R$ we can solve for the rank transition
point $i_{t}$:

$i_{t}=(1-\frac{dh}{f})(N-1)+1.$ (17)

If a player’s rank is better than $i_{t}$ (i.e., the player’s rank value is
strictly less than the breakpoint $i_{t}$ or $i<i_{t}$) then they have a
strongly dominant strategy ($hawk$). In the situation where $i=i_{t}$ the
player prefers to play $hawk$ if their opponent plays $dove$ but is
indifferent between $hawk$ and $dove$ if their opponent plays $hawk$; in
effect, $hawk$ is a weakly dominant strategy for player $i$. Therefore one
should play $hawk$ when $i\leq i_{t}$. Otherwise (when $i>i_{t}$) the player
has an indifference point that identifies when they would prefer to play
$hawk$ or $dove$. This is represented in Fig 2A.

### Breakdown of conventions

In this strategic scenario bullies (individuals that can win any contest and
can break with conventions) emerge when $f\geq dh$. This transition point does
not depend on population size. The breakpoint conditions (equations 16-17)
identify the threshold where top-ranked individuals may stop following
correlated convention and emerge as bullies (individuals that can win any
contest and can ignore the convention). Since $R=1$ for the top-ranked
individual the condition for this transition is $f\geq dh$. This marks the
point where the top ranked individual faces a dominant solvable game. When
$hh<f<dh$ even top-ranked individuals prefer to adhere to the correlated
equilibrium convention for the hawk-dove game. In cases where $f<dh$ the rank
transition $i_{t}$ is negative and therefore no one in the population faces a
dominant solvable game. Once $f\geq dh$ then ranking individuals near the top
of the hierarchy will stop adhering to the convention, though when $f$ is
close to $dh$ most of the population continues to adhere to the convention.
This transition does not depend on population size.

Is there a point when the convention becomes impossible to sustain? When $f$
reaches a high value there looks to be a point where the convention starts to
breakdown across the population and simulations show that conventions tend to
go extinct. Analytically, there is a useful comparison to assess this
question: when does does the expected payoff of following the convention drop
below the expected payoff for pure doves? Consider the $host-guest$ convention
where player $i$ plays $dove$ as host and $hawk$ as visitor. When is it the
case that $E(dove_{i})>E(para_{i})$? To simplify the comparison, presume
players can encounter either pure strategies ($hawk,dove$) or $para$ (play
$dove$ at home and $hawk$ away).

The expected payoffs depend on rank, opponent strategy, and whether the player
is host or visitor. Note that both $dove$ and $para$ play the same strategies
as host. The key part of the comparison involves how they do when visiting.
Assuming individuals interact once as host and once as visitor with a randomly
selected individual in the population then whether $E(dove_{i})>E(para_{i})$
depends on a comparison of expected payoffs when visiting:

$\displaystyle P(para_{j})\cdot dd+P(dove_{j})\cdot dd+P(hawk_{j})\cdot dh>$
(18) $\displaystyle P(para_{j})\cdot hd+P(dove_{j})\cdot hd+P(hawk_{j})\cdot
R\cdot f.$

Given that $P(para_{j})+P(dove_{j})+P(hawk_{j})=1$ and
$P(para_{j})+P(dove_{j})=1-P(hawk_{j})$ this inequality simplifies to:

$P(hawk_{j})>\frac{hd-dd}{(dh-dd)-(R\cdot f-hd)}.$ (19)

Note that this is a version of the decision scenario described by condition 13
just reversed to consider when a player prefers to play $dove$ rather than
$hawk$. However, when $f$ is high several individuals will confront a dominant
solvable game where they always play $hawk$ and therefore $P(hawk_{j})=1$ when
encountering these individuals. If there are sufficient numbers of these pure
$hawk$ individuals, visitors will always prefer $dove$ to $para$ and the
convention becomes impossible to sustain. For instance, consider the case
where $N=20$ and the game payoffs are $hh=0,dh=0.4,dd=0.6,hd=1$. According to
the mixed Nash equilibrium for these payoffs in a standard game of conflict, a
player (strictly) prefers to play $dove$ when $P(hawk_{j})>0.5$. In a mixed
population of pure $hawk$ and $dove$, a player will prefer to be a $dove$ when
$hawk$ makes up more than half the population. For the $N=20$ case, half the
population has faces a dominant solvable confrontation (i.e., $i_{t}=10$), and
therefore play as pure $hawk$, when $f=0.76$. At this point, it does not
matter what the rest of the population is doing because the chances of
encountering a hawk (assuming random interactions) are sufficiently high that
a new member of the population prefers to play $dove$ regardless. The exact
threshold can be determined by assessing the expected $P(hawk_{j})$ of
possible opponents weighted by the probability of encountering different
individual strategy profiles. This will depend on the payoffs of the game, the
network structure, an individual’s ranking, and the population size. The
learning dynamics will also affect how populations tend to behave (i.e.,
whether conventions go extinct) close to the threshold.

### Static networks

As individuals learn their own rank through reinforcement learning, they move
through the space of game strategies (Fig S1). On static networks and high
$f$–cases in which the correlated convention breaks–out-ranked individuals
adopt defensive strategies playing dove both at home and away, while ranking
individuals adopt aggressive strategies playing hawk in both cases.

Fig. S1:  Evolution of strategies shown as trajectories over time. Individuals
of all ranks start at center at $t=0$ with 50:50 hawk:dove. Ranking
individuals learn hawk-hawk and move to the top-right. Outranked individuals
learn dove-dove and move to bottom-left ($dh=0.4;dd=0.6;f=0.9$).

### Dynamic networks

Expanding on the network evolution shown in the main text which show different
seeds aggregated by $f$, we show raw, unaggregated data in S2.

Fig. S2:  Network centralization over time for each seed, grouped by $f$
value. As reference, an ideal star network has Network Centralization
$=17.95$. Stars disappear for $f\geq 0.4$.

### Partner choice and dynamic ranks

As we have seen, cycles start to emerge at the threshold $f\geq dh$ but do all
individuals cycle through ranks or just some? To explore this question, we
analyze the percentage of time that each individual spends in each rank (Fig
S3). We find that each individuals cycle through ranks and spends close to an
equal amount of time in each rank (5% in our the case of our standard
population size of 20). However, we notice one interesting pattern. Occupying
the top rank and the bottom rank are somewhat correlated: only in the bottom
rank can an individual accumulate enough payoffs to make it all the way to the
top of they hierarchy. Simply being near the bottom is not enough to make it
all the way to top. So while some individuals oscillate between the extremes,
others spend most of their time oscillating only between middle ranks.

Fig. S3:  All individuals cycle through ranks and spend close to an equal
amount of time in each rank. Each line shows one individual from one example
seed of a population of 20 ($f=0.8)$. Individual with the highest standard
deviation of ranks highlighted in blue. Dashed line shows expected time in
each rank for population of (5% in the case of population of 20).

The ability of individuals to accumulate payoffs is crucial to the cycles
observed in the central model. If dove-playing individuals were unable to
accumulate payoffs, perhaps because losses were significantly costly or
negative, cycles would not emerge. To illustrate this we can introduce an
artificial modification to the model (artificial because Roth-Erev
reinforcement learning cannot accommodate negative payoffs without being
modified). We explore rank updating dynamics with a rule that mimics negative
$hh$ and $dh$ payoffs by subtracting a constant $c$ per interaction for
assessing rank changes only (the reinforcement of strategy and network weights
proceeded as usual with the Roth-Erev updating). When $c=dh$ this effectively
treats a $dh$ payoff as zero and $hh$ as negative. When $c>dh$, both $dh$ and
$hh$ are negative. When we adopt this rule for rank updating we do not see the
cycling behavior because individuals cannot accumulate payoffs from playing
dove as host to enable a rise in the ranks. Instead, we see noisy rank changes
across the length of simulation. This noise occurs because few individuals are
receiving positive per-interaction payoffs (i.e., $hd$ or $dd$) so rank
changes for many individuals become a random walk. These individuals are
unable to learn their effective ranks and so play more noisily towards highly
ranked individuals which introduces additional fluctuations in rank. However,
the correlated convention (playing hawk away and dove at home) still emerges
in these cases with one small difference. Some (top-ranked) individuals are
very noisy in their strategy choice as hosts, often oscillating between dove
and hawk host plays. Visitor strategy choice quickly converges on hawk for all
individuals.

### Timescale comparison

#### Partner choice and static power asymmetries.

First, we investigate the effect of different learning speed timescales in the
case of static ranks. We change the timescale of network updating relative to
strategy updating by adjusting the payoffs used to update network weights (see
Methods). Our results are robust to differences in timescales of network
learning and strategy learning (Fig S4). When power asymmetry is low, faster
network learning (relative to strategy learning) leads to higher proportion of
paradoxical interactions at the final time step, and slower network learning
leads to lower proportion of paradoxical interactions. However, paradoxical
interactions are still a significant proportion overall (about $90\%$). When
power asymmetry is high, network updating speed does not have a discernable
effect on the resulting proportion of paradoxical interactions.

Fig. S4:  Comparison of relative learning speeds. A, $f=0.2$. B, $f=0.7$.
(Both panels use $dh=0.4;dd=0.6$).

We also analyze the evolution of pure hawk nodes across different relative
learning speeds (Fig fig:supNetworkLearningSpeeds2). For the purpose of this
analysis, we defined nodes as “pure hawk” if they had least 0.8 hawk strategy
weight for both visitor and host (the exact specification does not change the
results substantively and is made only for illustration purposes). While the
convention breaks for random interaction (static network; black line $v=0.0$),
the convention continues to survive even when networks evolve very slowly, or
faster.

Fig. S5:  Even slow updating of partner choice allows the convention to
survive. Without network updating ($v=0.0$), the convention breaks but
survives with both faster and slower network learning speeds
($dh=0.4;dd=0.6;f=0.8$).

#### Partner choice and dynamic power asymmetries.

Next, we investigate the effect of varying discount rates in the dynamic ranks
case. Varying the discount for network learning, but not strategy learning,
changes how fast network ties can be revised after changes in strategy
relative to strategic learning. The lower the discount rate ($\delta$) the
longer it takes for network ties to be revised. We examine discount rates of
0.001 and 0.0001 in comparison to the default rate of 0.01, meaning network
learning speed is $\nicefrac{{1}}{{10}}$ th and $\nicefrac{{1}}{{100}}$ th of
the strategy learning speed in this respect. In both cases, the observed
cycles take much longer. (Fig S6). As cycle length increases, some cycles may
become longer than the 1 million time steps of our simulations. This is the
case in several individual simulations as noted in the figure caption.

Fig. S6:  Cycle length naturally depends on speed of network learning. In some
cases, cycle lengths exceed the 1 million time steps of our simulations. For
power asymmetry of $f=0.4$ this is the case in 2.5% of simulations at $1/10$
network speed and 25% of simulations at $1/100$ network speed; for power
asymmetry of $f=0.7$ 1% of simulations exceed the time limit at $1/100$
network speed ($dh=0.4;dd=0.6$).

### Inequality

In this section we provide detailed results of total payoffs and inequality
across the full range of $f$-values. We focus on the contrast between static
and dynamic networks, and between static and dynamic ranks. Across all
regimes, cumulative payoffs are highest in the case of $f$ below the critical
value of $dh$: in this regime, ranks effectively do not matter and thus there
are no bullies that cause “friction” compared to the correlated convention
(Fig S7). With $f>dh$ bullies start to disrupt the correlated convention and
cumulative payoffs dip. The cases with static networks are identical as
individuals do not change their rank if they cannot be isolated. In those two
cases where individuals cannot avoid powerful bullies and the correlated
convention breaks payoffs dip the most, reaching the lowest payoff at $f=0.5$.
After that, cumulative payoffs increase slightly as the increasingly large $f$
payoff earned by bullies starts to offset some of the friction caused by the
broken correlated convention.

Fig. S7:  Total payoffs by $f$.

Inequality is lowest in the case of dynamic networks with dynamic ranks as the
existence of cycles allow each individual to spend equal time at the top of
the hierarchy (Fig S8). Inequality is highest in the static network cases and
is highest in the case of more extreme asymmetry (higher $f$). In the case of
dynamic networks and static ranks, inequality increases with increasing
asymmetry, but then decreases with very high asymmetry as fewer top
individuals break away from the correlated convention to adopt aggressive pure
hawk strategies as shown in Fig 3A.

Fig. S8:  Gini by $f$.

How do payoffs differ across individuals of different ranks? We show results
using $f=0.6$ as a canonical case in which power asymmetry is large enough for
bullies to emerge, but not large enough for the convention to break completely
(Fig S9). On static networks, high-ranking individuals become bullies and earn
higher payoffs than individuals who are part of the cooperative convention
(who all earn the same payoffs). In static networks, individuals cannot be
isolated, so ranks never change even when they are allowed to change in
principle. With partner choice and static ranks, the pattern reverses: high-
ranking bullies get isolated while low-ranking cooperators attract more
visitors and thus earn higher payoffs. As there is some variation in the
network structures that emerge in this case, there is variation in how many
visitors low-ranked individuals attract and hence their payoffs can vary
substantially more than in all other cases. The overall pattern is clear: the
lowest ranked individual earns the highest payoffs, followed by the second-
lowest ranked and so on. Finally, on dynamic network with dynamic ranks, all
individuals move through ranks in cycles, spending about equal time in each
rank (as shown above in Fig S3). As a result, all individuals earn similar
payoffs, no matter their initial rank.

Fig. S9:  Total payoffs by initial rank contrasting all four combinations of
our model. ($f=0.6$; box-plot shows distribution of payoffs at each rank
across 200 seeds)

### Population Size

We find all our results are robust across different population sizes. For our
analysis we define an individual as a “pure hawk” if the likelihood of playing
hawk both at home and as visitor is greater than $0.8$. We make this
definition purely for convenience so that we can show proportions of agents by
“type”. The substantive result of the overall pattern of the curves shown in
Fig 3B does not critically depend on the likelihood threshold and looks
similar for higher (e.g., $0.9$) or lower thresholds (e.g., $0.6$). All
simulations with population size up to 100 are 200 seeds, population 200 is 50
seeds, and population 500 is 20 seeds.

The presence of cycles in the model with network learning and dynamic ranks
does not depend on population size. Fig S10 shows distribution of cycle
lengths across $f$ for different population sizes.

Fig. S10:  Cycle lengths are somewhat longer in larger populations. The
overall pattern of shorter cycles with higher power asymmetry remains robust.