Fairness and Deception in Human Interactions with Artificial Agents

Theodor Cimpeanu1 and Alexander J. Stewart1∗

1 School of Mathematics and Statistics, University of St Andrews, St Andrews,
KY16 9SS, United Kingdom
∗ E-mail<EMAIL_ADDRESS>
Online information ecosystems are now central to our everyday social
interactions. Of the many opportunities and challenges this presents, the
capacity for artificial agents to shape individual and collective human
decision-making in such environments is of particular importance. In order to
assess and manage the impact of artificial agents on human well-being, we must
consider not only the technical capabilities of such agents, but the impact
they have on human social dynamics at the individual and population level. We
approach this problem by modelling the potential for artificial agents to
“nudge” attitudes to fairness and cooperation in populations of human agents,
who update their behavior according to a process of social learning. We show
that the presence of artificial agents in a population playing the ultimatum
game generates highly divergent, multi-stable outcomes in the learning
dynamics of human agents’ behaviour. These outcomes correspond to universal
fairness (successful nudging), universal selfishness (failed nudging), and a
strategy of fairness towards artificial agents and selfishness towards other
human agents (unintended consequences of nudging). We then consider the
consequences of human agents shifting their behavior when they are aware that
they are interacting with an artificial agent. We show that under a wide range
of circumstances artificial agents can achieve optimal outcomes in their
interactions with human agents while avoiding deception. However we also find
that, in the donation game, deception tends to make nudging easier to achieve.

## Introduction

Online information ecosystems are increasingly shaped by artificial agents.
Recent advances have seen AI chatbots based on large language models (LLMs),
such as ChatGPT, progressing at a rate that provokes both excitement and alarm
[1, 5, 13, 46, 2, 39, 7] among researchers, policy-makers and commentators.
The ability of current AIs to converse, reason and inform is remarkable, and
will likely be transformative for many areas of human activity. Among the
questions this raises is how the presence of hyper-realistic artificial
interlocutors will remake online environments – particularly the deleterious
aspects of those environments such as widespread misinformation, anti-social
behavior and distorted decision-making [21, 30, 14, 40].

While there is ample scope for AI to become a tool for bad actors online [11,
27, 29], this is far from the whole story. The ability of AI to rapidly
synthesize disparate sources of information, and produce relevant insight on a
topic, may provide opportunities to improve the health of online information
ecosystems. Whether AI can reshape online spaces for the better depends on how
“nudges” by artificial agents impact the social dynamics in such environments.
This includes the attitudes of individual human agents to artificial ones –
are AI agents perceived as fair, impartial and honest? It also depends on the
population dynamics of the system i.e. how attitudes, ideas and behaviors
spread and how the introduction of artificial agents alters that process. To
address these questions we must consider not just the capabilities of
artificial agents, but the social dynamics of human agents interacting with a
mixture artificial agents and other humans.

Here we model the social dynamics of human interactions in a population
containing a minority of artificial agents. We focus on a game theoretic
setting in which human decision-making evolves via a process of social
learning. We focus on the ability of artificial agents to engender “fairness”
in a population of human agents who lack information about one another’s
intentions. We make general assumptions about the ability of artificial agents
to infer the preferences of individual human agents, and we study the
conditions under which such agents can “nudge” [36, 6, 8] a population of
human agents towards greater fairness and cooperation, first under the setting
of an ultimatum game [25] and then under the more complex setting of an
iterated donation game [41, 18, 42].

We show that the introduction of artificial agents into a population of
“human” agents engaging in a pairwise ultimatum game and evolving via social
learning, radically alters the equilibrium states of the system. When
artificial agents are assumed to nudge the behavior of human agents by
incentivising them to increase the fairness of their strategies, we observe
three outcomes. I) Successful nudging, in which the evolving population
converges on a level of fairness set by the artificial agents. II) Failed
nudging, in which the evolving population converges on the unfair equilibrium
found in populations where artificial agents are absent [25]. III) A novel
equilibrium of unintended consequences, in which the population evolves
fairness towards artificial agents and selfishness towards other human agents.
We characterize the conditions for these equilibria to exist, and the size of
their respective basins of attraction, as a function of the initial strategy
of the population of human agents, and the proportion of interactions that
occur between human and artificial agents. We then generalise our results for
the ultimatum game to describe the outcomes of an iterated donation game, and
characterise the findability of “nudge” strategies for artificial agents that
seek to encourage cooperation.

Key to any efforts aimed at using artificial agents to improve online
information ecosystems is the question of trust [20, 38, 12]. As AI chatbots
advance, we expect the quality of the information they provide to become
better [13], even as it becomes more difficult to distinguish bot from human
in an online setting. To address this, we consider how our results change if
human agents shift their behavior in response to a known artificial agent. If
an artificial agent’s ability to nudge the human agent’s behavior is reduced
when the human agent knows their interaction partner is artificial, there is a
tradeoff between the “honesty” of an artificial agent and its efficacy.
Nonetheless we show that, under a wide range of circumstances, artificial
agents can produce optimal outcomes while avoiding deception.

Our results show that the introduction of sophisticated artificial agents –
that have detailed information about the behavioral strategies of their humans
interaction partners – into an evolving population has profound consequences
for the behavior that emerges. While such agents can sometimes effectively
nudge a population towards a desired behavior, the feedback loops between the
artificial agents and the dynamics of social learning in the population can be
unpredictable, leading to new, unintended equilibria. Attempts at
interventions which seek to improve online environments through this sort of
nudging must therefore take account of these dynamics.

## Results

Modelling human agents. We study the evolution of fairness using the standard
game-theoretic framework of a two-player, one-shot ultimatum game [25]. Under
the ultimatum game one player is randomly assigned to the role of “proposer”
while the other adopts the role of “responder”. The proposer makes an offer to
split an endowment (assumed for simplicity to have total value 1), such that
the responder receivers a proportion $p$ of the endowment, while the proposer
retains the remaining $1-p$. The responder must decide either to accept the
proposer’s splitting of the endowment, or else reject it such that both
players receive nothing.

We assume that each player $i$ uses a strategy $\\{p_{i},q_{i}\\}$, where $p$
is the proportion of the endowment they offer when in the role of proposer,
and $q$ is the minimum offer they will accept in the role of responder. In a
population setting of $N_{h}$ human agents we assume that all possible
$N_{h}(N_{h}-1)/2$ pairwise games occur, resulting in an expected total payoff
to a player $i$ from interacting with $N_{h}-1$ other human agents of
$w_{i}=\sum_{j\neq i}(1-p_{i})(1-H(q_{j}-p_{i}))+p_{i}(1-H(q_{i}-p_{j}))$
where $H(x)$ is the Heaviside function.

We conceptualize “human” agents as those players engaging in the ultimatum
game, and updating their individual strategies $\\{p,q\\}$ via a process of
social learning [43] (Figure 1), in which a player $i$ imitates the strategy
of player $j$ with a probability $1/(1+\exp[\sigma(W_{i}-W_{j})])$, where
$\sigma$ characterizes the “strength of selection” i.e. the degree of noise in
the social learning process [44] and $W_{i}$ is the average payoff from all
interactions with human and artificial agents for player $i$. This imitation
process is coupled with a process of “innovation” in which players “innovate”
by adopting entirely novel strategies at rate $\mu$. Details of the social
learning dynamics are given in the Methods section.

Figure 1: Modelling nudges by artificial agents in a population of evolving
human agents. a) A population of only human agents playing the ultimatum game.
Pairs of players are randomly assigned to the role of proposer (purple) or
responder (green). The proposer suggests to split an endowment of total value
1, such that they keep a proportion $(1-p)$ of the endowment, and the
responder receives the remaining $p$. The responder accepts the proposal if it
is above a threshold $q$, otherwise both players receive 0. If players lack
prior information about the offers that will be accepted, the dynamics of
social learning converge on an unfair equilibrium $p=q=0$. b) When a minority
of artificial agents are present in the population (red outline), they can
seek to nudge the dynamics of social learning among the artificial agents,
e.g. by playing in a manner that incentivises greater fairness.

The ultimatum game has a “fair” outcome if both players split the endowment
equally, i.e. if $p=0.5$. The social dynamics of fairness in the ultimatum
game have been widely studied, and shown to often produce highly unfair
outcomes, with evolution towards $p=q=0$. However, this result can be reversed
if players have information about the proposals responders have accepted in
the past [25], in which case the system evolves towards fairer outcomes, more
consistent with laboratory observations of human behavior [15, 34].

In an online environment, where social interactions may be anonymous or
fleeting, the scope for fairness driven by past experience may be limited. And
so we study the dynamics of social learning in a population of human agents
who lack information about their interaction partners, and ask whether
artificial agents can nonetheless “nudge” the human agents towards fairness in
the ultimatum game.

Modelling artificial agents. In order to model artificial agents, we make some
generic assumptions about their capabilities. We assume that:

  * •

Artificial agents are designed with knowledge of human agent social learning
dynamics.

  * •

Artificial agents are able to infer the individual behavioral strategies of
human agents.

  * •

Artificial agents are part of the population (i.e. artificial agents seek to
nudge the behavior of human agents by engaging in the same kinds of social
interactions human agents engage in with one another).

In the context of the ultimatum game, we characterize artificial agent
strategies by three functions, $\\{f(p),g(s),G(s)\\}$ where $f(p)$ is the
probability that an artificial agent in the role of responder will accept a
proposal $p$. The function $g(s)$ describes the probability that an artificial
agent infers a threshold acceptance level $s$ for a given human agent,
assuming that the human _does not_ know that the artificial agent is non-
human. Similarly, $G(s)$ describes the probability that the artificial agent
infers a threshold acceptance level $s$ for a given human agent, assuming that
the human _does_ know that the artificial agent is non-human. We further
assume that if the human agent knows that the artificial agent is non-human,
they adjust their proposal level by a factor $(1+c_{p})$ and their acceptance
threshold by an amount $(1+c_{q})$ (where both $c_{p}$ and $c_{q}$ may be
either positive or negative). Finally we assume that artificial agents seek to
nudge human behavior by making proposals that human agents will accept.

For a given human agent $i$ interacting with an artificial agent of this type,
the expected payoffs for both players are

$\displaystyle
v^{h}_{i}=(1-\beta)\left[(1-p_{i})f(p_{i})+\int^{1}_{q_{i}}sg(s)ds\right]+$
$\displaystyle\beta\left[(1-p_{i}(1+c_{p}))f(p_{i}(1+c_{p}))+\int^{1}_{q_{i}(1+c_{q})}sG(s)ds\right]$
(1)

for the human agent

$\displaystyle
v_{i}^{a}=(1-\beta)\left[p_{i}f(p_{i})+\int^{1}_{q_{i}}(1-s)g(s)ds\right]+$
$\displaystyle\beta\left[p_{i}(1+c_{p})f(p_{i}(1+c_{p}))+\int^{1}_{q_{i}(1+c_{q})}(1-s)G(s)ds\right]$
(2)

for an artificial agent interacting with human agent $i$. The parameter
$\beta$ describes the “honesty” of the artificial agent, i.e. the probability
that the human agent knows the artificial agent is non-human. We assume that
there are $N_{a}$ artificial agents in the population, all of which use the
same strategy, meaning that the overall payoff $W_{i}$ to a human agent $i$ is
a weighted sum $W_{i}=(1-\gamma)w_{i}+\gamma v^{h}_{i}$, where
$\gamma=\frac{N_{a}}{N_{a}+N_{h}-1}$. Table 1 gives a full list of parameters
used in our model, along with their meaning.

Nudging towards fairness in the ultimatum game. We first explore the impact of
a minority of artificial agents on the social learning dynamics of human
agents engaged in a pairwise ultimatum game (Figure 1), assuming that the
human agents are unaware that artificial agents are present ($\beta=0$). In
general the impact of artificial agents on the outcome of social learning
among human agents will depend on the relative frequency of their
interactions, $\gamma$, as well as the strategies of the artificial agents.
This in turn is constrained by the technical capabilities of the artificial
agents, and their intended goal. We consider scenarios in which artificial
agents seek to generate a new equilibrium, $p=q=r^{*}$, such that $r^{*}>0$,
corresponding to a fairer outcome for the population than the equilibrium when
artificial agents are absent (i.e. $p=q=0$).

In order to successfully nudge human agents towards such fairer strategies,
artificial agents must incentivise them to increase both their proposal level
and acceptance threshold. In order to incentivise higher proposal levels $p$
from human agents, we assume that artificial agents accept proposals with a
probability $f(p)=1/(1+\exp[\theta(p_{c}-p)])$ where $p_{c}$ is the level at
which a proposal is accepted with probability $0.5$, and $h$ controls the
slope of the sigmoid function. This choice ensures that higher proposals from
human agents are always more likely to succeed. In order to incentivise higher
threshold acceptance levels from human agents we assume that artificial agents
attempt to infer the acceptance threshold of each human agent they interact
with, and make proposals at that level. This ensures that higher threshold
acceptance levels among human agents always lead to higher payoffs in
interactions with artifial agents.

Figure 2: The consequences of artificial agents for the dynamics of social
learning in the donation game. a-b) We calculated the equilibrium strategies
(solid lines) for Case I (successful nudging), Case II (failed nudging) and
Case III (unintended consequences) as described in the main text, as a
function of the slope $\theta$ of the acceptance probability function $f(p)$
for the artificial agent, assuming that human agents are unaware when
interacting with artificial agents (i.e. $\beta=0$). We see that successful
nudging only emerges when the $\theta$ becomes sufficiently large, while the
other two equilibria are always present. Analytic solutions (lines) are
compared to individual-based simulations (dots, see Supplementary Information)
for both a) the average equilibrium proposal level $p$ and b) the average
equilibrium acceptance threshold $q$ in the population of human agents. c) We
explored the basin of attraction for each of the three equilibria through
simulation. Results shown are for a $101\times 101$ grid of initial conditions
$\\{p,q\\}$ of the population of human agents, with both parameters varying
between 0 and 0.5 at regular intervals. We see that all three equilibria have
substantial basins of attraction, and vary with the initial conditions in a
complicated manner. Model parameters used in this figure are $\sigma=2000$,
$\gamma=0.1$, $p_{c}=0.25$ and $N_{h}=100$. Simulations for each intial
condition were replicated 20 times.

The efficacy of an artificial agent at nudging human agents’ behavior towards
fairness depends on their ability to correctly infer the human agents’
acceptance threshold. Under-estimating the threshold leads to an artificial
agent proposal being rejected, whereas over-estimates will lead to proposals
being accepted. We can therefore define the artificial agent quality as the
probability that it does not underestimate the acceptance threshold when
$\beta=0$ (i.e. when the human does not know the artificial agent is non-
human). This probability is given by $\alpha=1-\int^{q}_{0}g(s)ds$. Under this
model, the payoffs to a human agent $i$ from interacting with an artificial
agent is

$\displaystyle v_{i}^{h}=(1-p)f(p)+\alpha q$ (3)

and the payoff to the artificial agent is

$\displaystyle v_{i}^{a}=pf(p)+\alpha(1-q)$ (4)

where we have assumed any effects of over-estimating the human agent
acceptance threshold on artificial agent payoff are negligible. We show in the
Methods section that these choices of artificial agent strategy lead to
evolution towards fairer strategies when humans interact with only artificial
agents (i.e. $\gamma\to 1$). The question remains how artificial agents impact
the dynamics of social learning when $\gamma<1$. In order to study this
question, we analyse the dynamics of invasion in a monomorphic human
population, in which all human agents have the same strategy $\\{p,q\\}$. We
then consider the effect of an “innovation” which changes the strategy of a
single agent by a small amount, and ask whether such a novel strategy is
favored to spread via imitation.

We show (see Methods) that the dynamics of social learning in the system have
three equilibria as follows:

  * •

Case I: Successful nudging. The human agents evolve towards a fair equilibrium
$p=q=r^{*}$ where $r^{*}$ is the solution to

$(1-r^{*})\frac{df}{dp}\big{|}_{p=r^{*}}=f(r^{*})+\frac{1-\gamma}{\gamma}$

  * •

Case 2: Failed nudging. The human agents evolve towards a unfair equilibrium
$p=q=0$ (i.e. the same equilibrium as when artificial agents are absent.

  * •

Case 3: Unintended consequences. The human agents evolve towards an
equilibrium in which the acceptance threshold $q$ is maximized, while the
proposal level evolves towards a level $p^{*}$ which satisfies

$(1-p^{*})\frac{df}{dp}\big{|}_{p=p^{*}}=f(p^{*})$

.

We refer to Case 3 as “unintended consequences”, because the equilibrium is
such that human agents always reject proposals from other human agents, while
accepting fair proposals from artificial agents. Figure 2 shows a comparison
of analytic predictions to individual-based simulations (see Supplementary
Information), with the equilibrium strategies for human agents determined as a
function of the slope of the artificial agent acceptance function, $\theta$.
We see that successful nudging is only possible once $\theta$ reaches a
critical value, i.e. when the slope is steep enough. We also numerically
explore (Figure 2b) the basins of attraction of the three equilibria. We see
that the potential for artificial agents to generate unintended consequences
can be substantial. However we also find (see Supplementary Information) that
Case III can be completely avoided if the acceptance threshold, $p_{c}$, is
chosen appropriately, and in particular when $p_{c}=0.5$ only Case I and Case
II occur, with Case II corresponding to complete fairness i.e. $p=q=0.5$ for
all human agents.

Optimising artificial agent behavior. So far we have assumed that artificial
agents can nudge the dynamics of social learning among human agents in such a
way that their interactions are not detected by the human agents ($\beta=0$).
When this is not the case ($\beta>0$), it is reasonable to assume that a human
agent might treat their interaction with an artificial agent differently than
they would another human. As described previously, we assume that a human
agent changes their proposal level $p$ by a factor $(1+c_{p})$ when they are
aware they are interacting with an artificial agent (where $c_{p}$ can be
positive or negative, reflecting different qualitative attitudes to artificial
agents). Similarly we assume that a human agent changes their acceptance
threshold $q$ by a factor $(1+c_{q})$ when they are aware they are interacting
with an artificial agent.

We also assume that the ability of an artificial agent to estimate the
acceptance threshold of human agents changes when the human agent is aware
they are interacting with an artificial agent, i.e. $s(s)\neq G(s)$ in
general. And so we can define the relative change in accuracy of the
acceptance threshold estimate when a human agent knows they are interacting
with an artificial agent as
$\epsilon=1-\frac{\int^{q(1+c_{q})}_{0}G(s)ds}{\int^{q}_{0}g(s)ds}$. Under
this model the payoff to a human agent interacting with an artificial agent is

$\displaystyle v_{i}^{h}=(1-\beta)\left[(1-p_{i})f(p_{i})+\alpha
q_{i}\right]+$
$\displaystyle\beta\left[(1-p_{i}(1+c_{p}))f(p_{i}(1+c_{p}))+(\alpha+\epsilon(1-\alpha))q_{i}(1+c_{q})\right]$
(5)

and the payoff to the artificial agent is

$\displaystyle v_{i}^{a}=(1-\beta)\left[p_{i}f(p_{i})+\alpha(1-q_{i})\right]+$
$\displaystyle\beta\left[p_{i}(1+c_{p})f(p_{i}(1+c_{p}))+(\alpha+\epsilon(1-\alpha))(1-q_{i}(1+c_{q}))\right]$
(6)

Of particular interest is a scenario in which $\beta$ can be optimised, i.e.
we can interpret $\beta$ as the “honesty” of the artificial agent, and ask
what level of honesty optimises artificial agent payoff $v_{i}^{a}$, given a
human agent strategy $\\{p_{i},q_{i}\\}$. In general the optimum level of
honesty depends on $c_{p}$, $c_{q}$, $\alpha$ and $\epsilon$, i.e. on the
attitudes of human agents towards artificial agents, and on the accuracy of
the artificial agents at inferring human behavior (see Methods).

Figure 3 shows how the incentives for artificial agent honesty change as these
features of human attitudes and artificial agent capabilities change. In
particular we find that human agent attitudes do not reliably inform the
incentives for artificial agent honesty. In a wide range of cases, honesty is
preferable, provided human agents do not change their behavior too drastically
when they are aware they are interacting with artificial agents. However this
only holds if there is some accuracy gain from honesty, i.e. if $g(s)$ tends
to underestimate the human agent acceptance thresholds more than $G(s)$ (see
Methods). Such a scenario might arise if, for example an artifical agent were
able to gain better information about the human agent’s preferences by being
honest.

Figure 3: The impact of human agent attitudes on the honesty of artificial
agents. a) We determined the conditions for artificial agents to be honest
($\beta=1$) as a function of the percent change in human agent offer level
when artificial agents are known (i.e. $100\times c_{p}$) and the percent
change in the human agent rejection threshold when artificial agents are known
(i.e. $100\times c_{q}$), compared to when such agents are unknown. Results
shown are for high levels of human agent fairness, $p=0.4$ and $q=0.4$ [15,
34] and $\epsilon=0.1$, corresponding to an improvement in artificial agent
inferences about human agent acceptance threshold when they are known, along
with $\theta=10$ and $p_{c}=0.25$. We see four regions, represented by
different colors. When $c_{p}>0$ and $c_{q}>0$, corresponding to unambiguously
positive responses to known artificial agents, it is always beneficial for
artificial agents to be honest (red). However in other circumstances the
outcome depends on the precise values of $c_{p}$ and $c_{q}$, with artificial
agents either being incentivesd to be honest if their inferences are
inaccurate (dark blue), or only when their inferences are accurate (purple) or
else to never be honest (light blue). b) An equivalent plot with
$c_{p}=c_{q}=0.33$, in which we vary human agent strategy $\\{p,q\\}$, shows a
similar picture, i.e. the optimal honesty of artificial agents varies with the
strategy of the human agent.

More generally, there are four qualitatively different scenarios for
artificial agent honesty. First, the artificial agent is always honest
($\beta=1$). Second, the artificial agent is always deceptive ($\beta=0$).
Third, the artificial agent is honest if it is insufficiently accurate (i.e.
if $\alpha$ is below a threshold). Fourth, the artificial agent is honest if
it is sufficiently accurate (i.e. if $\alpha$ is above a threshold, see
Methods). Which of these scenarios hold depends on the strategy of the human
agents (Figure 3b) in addition to their attitudes towards artificial agents
(i.e. $c_{p}$ and $c_{q}$). This reflects a trade-off between the (assumed)
greater accuracy of threshold estimates when artificial agents are honest
($\epsilon>0$), and the change in human agent strategies when they are aware
they are interacting with artificial agents.

Emergence of fairness in the donation game. So far we have focused our
analysis on the ultimatum game, which is a widely studied game theoretic model
of fairness. However, in reality the types of social interactions it encodes –
a one-shot, asymmetrical interaction between a randomly assigned proposer and
responder – do not capture most real-world interactions. The pairwise donation
game [41], in which players decide whether to pay a cost $C$ to provide a
benefit $B$ to an interaction partner, captures a basic type of prosocial
interaction in which people can choose to provide cooperative help to one
another. The decision whether to provide such help is in general the product
of past experience, and over the long run players may assess the total of
their donation game interactions and decide whether the balance of costs paid
to benefits received is “fair”.

In this light, the iterated donation game can be seen as comprising the
microscopic, everyday interactions that make-up a macroscopic ultimatum game
[33], in which players engage with each other in a way that “proposes” a split
in the benefits and costs of cooperation and, after a time, they decide to
“respond” by continuing with the interaction or breaking it off entirely. In
order to model this, we calculate the equilibrium payoffs for the infinitely
iterated donation game (see Methods) and compare the proportion of the overall
payoff to both players, $S$, to that received by a focal human agent $i$,
$S_{i}$. The human agent’s acceptance threshold level $q_{i}$ then means that
they will only accept their donation game interaction partner if $S_{i}\geq
q_{i}S$ (see Methods). We further assume that, as in the ultimatum game,
$q\leq 0.5$, and that artificial agents seek to play such that they receive at
least half of the available payoff (i.e. artificial agents demand fairness).

Figure 4: The findability of strategies for artificial agents in the
infinitely iterated donation game. a) We consider the findability of memory-1
strategies that ensure a human agent receives an equilibrium payoff above an
ultimatum game acceptance threshold $q$ but below $50\%$ of the total payoff
generated (see Methods). The findability is the probability that a randomly
selected strategy generates the desired outcome. We numerically calculate the
findability of strategies as a function of $q$ and of the returns on
individual acts of cooperation $B/C$. b) For an acceptance threshold of
$q=0.4$ along with returns of $B/C=1.5$ (red dot) we look at the change in the
findability of strategies when artificial agents are known, as a function of
the change in the returns on cooperation and the acceptance threshold of the
human agent. We see that in most circumstances honesty reduces findability.
Here $\gamma=0.1$, $p_{c}=0.25$ and $\theta=10$

The key question we consider in this context is the ability of an artificial
agent to generate a donation game strategy that meets the human agent’s
acceptance threshold, and therefore incentivise cooperation. Figure 4 shows
the “findability” of memory-1 donation game strategies (i.e. strategies that
condition their play at round $t+1$ only on the outcome of round $t$) which
satisfy a given acceptance threshold for a human agent, as a function of the
returns on cooperation, $B/C$. We see that the findability of strategies
declines rapidly with acceptance threshold and with returns on cooperation,
such that when $B/C=1.5$ and the acceptance threshold is $q=0.4$ around
$0.01\%$ of strategies are able to satisfy the threshold. Finally we consider
the impact of artificial agent honesty on findability, assuming that when
human agents know they are interacting with an artificial agent they may
change their behavior such that the returns on cooperation and their
acceptance threshold also change (Figure 4b). In general we find that honesty
decreases the findability of successful artificial agent strategies for the
donation game, unless human agent attitudes are ‘positive” in the sense they
become more accepting of unfairness or generate higher returns from
cooperation.

## Discussion

Understanding how artificial agents shape human experience in online
information ecosystems is one of the most important issues raised by recent
advances in the ability of AI to converse, reason and inform. There is scope
for artificial agents to be employed for deleterious purposes [11, 27, 29] or
to promote the health of such ecosystems. One aspect of this question, which
is easily overlooked, is the impact of such agents on the population dynamics
of humans engaged in a process of individual or social learning within these
environments. The evolution of social behavior can often generate surprising
or counter-intuitive outcomes [23, 29]. We have shown, in the abstract game
theoretic context of an ultimatum game, how big the impact of such agents, and
how unpredictable the outcomes. Of course there is no particular reason that a
real artificial agent should behave exactly like those of our model. What we
show is that, when placed in the context of an evolving system, artificial
agents can easily generate multiple novel equilibria not present in their
absence. These equilibria include both the intended outcome of the artificial
agents, as well as unintended outcomes, that are not obvious either from the
dynamics of social learning absent artificial agents, or from the design of
the artificial agents themselves. We therefore argue that characterising the
unintended outcomes of artificial agents on the population dynamics of human
agents is a key challenge for regulating safe AI development [10, 16].

A second key feature of our model is the role of the attitudes of human agents
towards artificial ones. In particular, we focus on the impact of human agent
attitudes on the optimal behavior of artificial agents. Our analysis shows
that conditions for honesty, in which it is optimal for artificial agents to
“make themselves known” does not depend on the attitudes of human agents in a
straightforward way, and may change depending on the technical capabilities of
the artificial agents themselves – i.e. as artificial agents become more
advanced, their incentives to be honest can shift. Similarly, depending on the
strategy of the human agent, it may or may not be advantageous for the
artificial agent to be honest (Figure 3b). Of course, the attitudes of human
agents towards artificial ones will shift over time, depending on the behavior
of the artificial agents and in response to exogenous factors such as the
political or economic environment. And so the question of the long-term
evolutionary dynamics of human attitudes towards artificial agents must be
considered alongside the more short-term dynamics of everyday social
interactions studied here.

What our model also shows, in addition to the potential for unintended
consequences and deception when artificial agents seek to shift human social
dynamics, is that a minority of artificial agents _can_ in principle, be
successful at shifting those dynamics towards fairness and cooperation in
settings where it is not normally favored. How best to produce this outcome
depends in turn on human attitudes to artificial agents. Currently, on the
question of human attitudes to artificial agents, the picture is mixed. Some
empirical and theoretical work suggest that nudging by such agents may be most
effective when deception is involved [29, 3, 19, 8]. At the same time, some
work shows that human agents might be more amenable to cooperative
interactions with artificial agents, as shown by the relative lack of
physiological responses associated with rejection in such interactions [35,
45], supported as well by the public perception that lying is positive if done
for the greater good (in human-AI interactions) [8]. Our results can also be
interpreted in the context of recent work on role-play in artificial agent
interactions [37], which suggests that artificial agents should (at least
sometimes) not disclose their identity, and instead act out a human persona to
promote fairness in otherwise anonymous online interactions.

In some cases the behavior of artificial agents is human-like by design [31,
9], the most obvious examples being LLMs, which are often specifically trained
to mimic text written by humans [22, 32]. In such cases, the question of
“honesty” becomes most relevant, since the decision to reveal whether or not a
social interaction is with an artificial agent lies with that agent. The
question of honesty becomes even more acute in scenarios where artificial
agents can learn behaviors such as strategic deception, as seen for example in
LLMs learning behaviours such as lying in social deduction or text-based
adventure games in order to win [26, 28]. In contrast under our model, any
“deception” is always in service of generating fair outcomes in the dynamics
of social learning among human agents, and in that respect is quite unlike the
negative connotations often depicted when considering other examples of AI
deception [29]. Heuristics such as trust are often used as a cognitive
shortcut to reduce the complexity of interactions where the co-player’s
decisions are not transparent, and it has been suggested that humans might use
trust-based strategies more frequently in interactions with artificial agents
[17, 4]. All of this suggests that how human agents will respond to artificial
agents is likely to be highly context dependent. Whatever the attitudes of
humans to AI and other artificial agents, the key insight from our work is the
capacity for such agents to generate novel and unexpected outcomes in the
dynamics of social learning in human populations. Characterizing and assessing
the potential for such outcomes must be set alongside other key sources of
risk when considering the development of such agents.

## Methods

Here we analyse the dynamics of social learning of the ultimatum game and the
findability of strategies in the donation game, under the strategies described
in the main text.

Dynamics of social learning of human agents. In order to study the dynamics of
social learning among human agents playing the ultimatum game in a population
containing artificial agents, we adopt the assumptions of adaptive dynamics
[24], under which the population of human agents is assumed to be monomorphic,
with resident strategy $\\{p,q\\}$, and the ability of an invader, which
differs from the resident strategy by a small amount, to increase in the
population is assessed.

The payoff to a player $i$ using the resident strategy is given by

$W_{i}=(1-\gamma)H(p-q)+\gamma v_{i}^{h}$

where $v_{i}^{h}$ is given by Eq 4 and $H(x)$ is the Heaviside function. In
order to determine whether a resident strategy can be invaded, we calculate
the selection gradient for a player $i$ who adopts the invading strategy,
while the rest of the population continues to use the resident strategy.

The selection gradient for the proposal level $p$ when $p\neq q$ is

$\frac{\partial W_{i}}{\partial p_{i}}\Big{|}_{p_{i}=p}=\gamma\frac{\partial
v_{i}^{h}}{\partial p_{i}}\Big{|}_{p_{i}=p}$

where

$\gamma\frac{\partial v_{i}^{h}}{\partial
p_{i}}\Big{|}_{p_{i}=p}=-f(p)+(1-p)\frac{df}{dp}$

Similarly, the selection gradient for the proposal level $q$ when $p\neq q$ is

$\frac{\partial W_{i}}{\partial q_{i}}\Big{|}_{q_{i}=q}=\gamma\frac{\partial
v_{i}^{h}}{\partial p_{i}}\Big{|}_{q_{i}=q}$

where

$\gamma\frac{\partial v_{i}^{h}}{\partial q_{i}}\Big{|}_{p_{i}=p}=\alpha$

And so the acceptance level $q$ will always be selected to increase, whereas
the proposal level $p$ will reach equilibrium $p^{*}$ such that

$-f(p^{*})+(1-p^{*})\frac{df}{dp}\Big{|}_{p=p^{*}}$

which is the unintended consequences equilibrium given for Case III, in which
human agents only engage in fair interactions with artificial agents.

If the resident strategy is such that $p=q$ we must account for the impact of
small perturbations on $H(x)$. To determine the selection gradient in this
scenario we consider $W_{i}(p\pm\delta,q)-W_{i}(p,q)$ and
$W_{i}(p,q\pm\delta)-W_{i}(p,q)$ (where $|delta|\ll 1$ and $\delta>0$). This
leaves us (after neglecting terms $O(\delta^{2})$ and higher) with four cases:

$W_{i}(p+\delta,q)-W(p,q)=-(1-\gamma)\delta+\gamma\delta\frac{\partial
v_{i}^{h}}{\partial p}$

and

$W(p-\delta,q)-W_{i}(p,q)=-(1-\gamma)(1-p)-\gamma\delta\frac{\partial
v_{i}^{h}}{\partial p}$

and

$W_{i}(p,q+\delta)-W_{i}(p,q)=-(1-\gamma)p+\gamma\delta\frac{\partial
v_{i}^{h}}{\partial q}$

and

$W_{i}(p,q-\delta)-W_{i}(p,q)=-\gamma\delta\frac{\partial\pi_{h}}{\partial
q}.$

If we then take the limit $\delta\to 0$ we see that only invading strategies
that increase $p$ can spread, all other invaders are selected against. This
has two solutions. The first is when $p=q=r^{*}$ such that

$-(1-\gamma)=\gamma f(r^{*})-\gamma(1-r^{*})\frac{df}{dp}\Big{|}_{p=r^{*}}$

which is the solution given for Case II, successful nudging. Finally, if
$p=q=0$ then $W_{i}(p+\delta,q)-W(p,q)$ is negative provided

$(1-\gamma)<\gamma\frac{\partial v_{i}^{h}}{\partial p}\Big{|}_{p=0}$

which holds for the choices of $f(p)$ considered here. This means that
invaders with $p>0$ are selected against. This gives us the equilibrium for
Case I, i.e. nudging fails. Note that this implies that the only way to ensure
that the the unfair equilibrium is disrupted is to ensure that

$\gamma>\frac{1}{\frac{df}{dp}\big{|}_{p=0}-f(0)+1}$

i.e. when artificial agents become sufficiently common, they can be sure to
disrupt the equilibrium of the human agent dynamics of social learning.

Optimal behavior of artificial agents Next we consider the optimal behavior of
artificial agents when $\beta>0$. The payoffs for artificial agents under this
scenario are given by Eq. 6. To calculate the optimal value of $\beta$ we
simply calculate

$\displaystyle\frac{\partial
v_{i}^{a}}{\partial\beta}=-p_{i}f(p_{i})+p_{i}(1+c_{p})f(p_{i}(1+c_{p}))$
$\displaystyle-\alpha(1-q_{i})+(\alpha+\epsilon(1-\alpha))(1-q_{i}(1-c_{q}))$

Since this is independent of $\beta$, this implies that the only optimal
outcomes for the artificial agent are to either 1) always be honest
($\beta=1$, gradient is positive) or 2) always deceive ($\beta=0$, gradient is
negative) in an interaction between a given human agent and a given artificial
agent . We can express the conditions for honesty as follows. If
$q_{i}>\frac{\epsilon}{c_{q}(1-\epsilon)+\epsilon}$ then

$\alpha>\frac{p_{i}f(p_{i})-p_{i}(1+c_{p})f(p_{i}(1+c_{p}))-\epsilon(1-q_{i}(1+c_{q}))}{q_{i}c_{q}(1-\epsilon)-\epsilon(1-q_{i})}$

or else if $q_{i}<\frac{\epsilon}{c_{q}(1-\epsilon)+\epsilon}$ then

$\alpha<\frac{p_{i}f(p_{i})-p_{i}(1+c_{p})f(p_{i}(1+c_{p}))-\epsilon(1-q_{i}(1+c_{q}))}{q_{i}c_{q}(1-\epsilon)-\epsilon(1-q_{i})}$

i.e. depending on the threshold of of acceptance for the human agent $q_{i}$,
the attitude of the human agent to the artificial agent $c_{q}$ and the change
in accuracy of the artificial agent when human agent knows they are artificial
$\epsilon$, there is either a minimum or a maximum accuracy level $\alpha$
beyond which the artificial agent is incentivised to be honest. It is this
relationship that is used to produce the region plots in Figure 3.

The donation game Finally we consider an ultimatum game that arises as the
result of an infinitely iterated donation game [33]. We assume that the human
agent has a sense of fairness, wherein they are willing to participate in the
donation game provided they receive at least a proportion $q$ of the total
payoff generated by the interaction, i.e. $S_{h}=q(S_{h}+S_{a})$ where $S_{h}$
is the average payoff received by the human agent and $S_{a}$ is the payoff
received by the artificial agent at equilibrium in the infinitely iterated
donation game (see [33]). As above we assume that the artificial agent
attempts to infer the human agent’s acceptance threshold, and adopts a
donation game strategy that enforces the minimum the human will accept (we
will assume for now that the artificial agent is able to do this perfectly).
As shown in [33] if a player (in this case the artificial agent) in the
infinitely iterated donation game adopts a memory 1 strategy of the form

$\displaystyle p_{cc}$ $\displaystyle=$ $\displaystyle
1-\phi(1-\chi)(B-C-\kappa)$ $\displaystyle p_{cd}$ $\displaystyle=$
$\displaystyle 1-\phi(B+\chi C-(1-\chi)\kappa+\lambda)$ $\displaystyle p_{dc}$
$\displaystyle=$ $\displaystyle\phi(\chi B+C+(1-\chi)\kappa-\lambda)$
$\displaystyle p_{dd}$ $\displaystyle=$ $\displaystyle\phi(1-\chi)\kappa$ (7)

then the equilibrium payoffs of the two players follow the relationship

$S_{h}-\chi S_{a}-(1-\chi)\kappa+\lambda(v_{cd}+v_{dc})=0$

where $v_{ij}$ is the equilibrium rate at which the play $ij$ occurs in the
donation game (where the artificial agent’s play is listed first) and
$(\phi,\chi,\kappa,\lambda)$ define the artificial agent’s strategy as shown
above. Here $B$ and $C$ are the benefits and costs of the donation game.

In order to determine whether the artificial agent’s strategy generates
equilibrium payoffs that satisfy

$S_{h}\geq q(S_{h}+S_{a})$

first set $z=q/(1-q)$ so that we can write

$S_{h}\geq zS_{a}$

as our condition.

Let us also assume that as in the ultimatum game, we constrain $q\leq 0.5$,
meaning that the artificial agent seeks to ensure

$S_{a}>S_{h}$

Substituting again for $S_{h}$ we recover

$S_{a}(1-\chi)>(1-\chi)\kappa-\lambda(v_{cd}+v_{dc})$

Next note that payoffs are constrained by

$S_{a}+S_{h}>(v_{cd}+v_{dc})(B-C).$

Substituting again for $S_{h}$ this becomes

$S_{a}(1+\chi)+(1-\chi)\kappa-\lambda(v_{cd}+v_{dc})>(v_{cd}+v_{dc})(B-C)$

If we now consider whether a strategy for the human agent exists such that
$S_{h}>S_{a}$ we find (combining the above two inequalities) that such a
strategy can exist providing the artificial agent strategy satisfies

$2\kappa(1-\chi)>(1-\chi)(v_{cd}+v_{dc})(B-C)+2\lambda(v_{cd}+v_{dc})$

This can always be satisfied unless $\kappa=0$ and so we conclude that
artificial agents play strategies with $\kappa=0$.

Following a similar logic we can also look whether human agent strategies
exist such that

$S_{h}<zS_{a}$

Substituting for $S_{h}$ gives us

$(\chi-z)S_{a}-\lambda(v_{cd}+v_{dc})<0$

We must now consider the separate cases. If $\chi<z$ we have

$(z-\chi)S_{a}>-\lambda(v_{cd}+v_{dc})$

Note that the two player’s payoffs are constrained by

$S_{a}+S_{h}<2(B-C)-(v_{cd}+v_{dc})(B-C)$

i.e.

$S_{a}(1+\chi)-\lambda(v_{cd}+v_{dc})<2(B-C)-(v_{cd}+v_{dc})(B-C)$

Comparing these two conditions, we recover

$2(z-\chi)(B-C)-(v_{cd}+v_{dc})(B-C)(z-\chi)+\lambda(1+z)(v_{cd}+v_{dc})>0$

which can be satisfied in general and so artificial agents should not play
strategies with $\chi<z$. Next assume $\chi\geq z$ and note that the players
payoffs are constrained by

$S_{a}-S_{h}>-(v_{cd}+v_{dc})(B+C)$

meaning that

$S_{a}(1-\chi)+\lambda(v_{cd}+v_{dc})>-(v_{cd}+v_{dc})(B+C)$

we find that human agent strategies exist that violate our constraint if

$\lambda(1-z)>-(\chi-z)(B+C)$

This leaves us with the following constraints on the artificial agent
strategies:

$z\leq\chi\leq\min\left[\frac{2\lambda}{B-C}+1,\frac{z}{1-z}-\frac{\lambda}{B+C}\right]$

This can be used to numerically compute the findability of artificial agent
strategies as shown in Figure 4.

## Individual-based simulations

We performed simulations to explore the effect of artificially agents on the
dynamics of social learning among human agents in the ultimatum game.
Initially each of the $1-\gamma$ human agents in a monomorphic population of
size $N$ is assigned a random behavioural strategy, $\\{p,q,c_{p},c_{q}\\}$
($p,q\in[0,0.5]$ and $c_{p},c_{q}\in[-1,1]$). Agents’ interactions are
modelled using the ultimatum game, and calculated using the payoffs described
in the main text.

At each time step or generation, each agent plays the ultimatum game with
every other agent in the population. As artificial agents’ behaviours are
fixed (i.e. do not ”mutate”), we only compute the payoffs of human agents. The
total payoff for each agent is the sum of the payoffs in these encounters.

At the end of the generation, a random pair of human agents is selected, and
their strategies are updated using a stochastic rule that simulates social
learning dynamics. An agent $i$ with payoff $W_{i}$ chooses to copy the
strategy of a randomly selected neighbour agent $j$ with total payoff $W_{j}$,
using a pairwise comparison rule, with an imitation probability given by the
Fermi-Dirac function [43]:

$(1+e^{sigma*(f_{A}-f_{B})})^{-1},$

where $\sigma$ denotes the intensity of selection in the imitation process
[44]. With some probability $\mu$, this process is instead replaced with a
local mutation in one of the dimensions $\\{p,q,c_{p},c_{q}\\}$ (or solely in
$\\{p,q\\}$ if $c_{p}$ and $c_{q}$ are fixed for a specific simulation, or
when they would not be expressed in the payoffs, i.e. when AI never reveals
itself). For each behavioural strategy, a strategy is chosen from a matrix of
size $101$ ($p,q\in[0,0.5]$ and $c_{p},c_{q}\in[-1,1]$), where a local
mutation represents a step-wise transition to one of the neighbouring values
in one of the matrices. For instance, a human agent with strategy
$\\{0.25,0.25,0,0\\}$ could mutate to $\\{0.255,0.25,0,0\\}$ or
$\\{0.245,0.25,0,0\\}$ if $p$ is chosen as the parameter of reference, and
like-wise for $q$, $c_{p}$ or $c_{q}$, given a total of eight possible
strategies available in the case of mutation typically. Exceptions arise at
the edge cases, in which case mutations can only happen in one direction. In
other words the value matrices for each strategy do not wrap around (e.g.
$p=0$ cannot mutate to $p=-0.005$).

We simulate this evolutionary process until a stationary state. Furthermore,
the results for each combination of parameter values are obtained from
averaging 20 independent realisations. For region plots or heat map figures,
we initialise all human agents in one realisation with the strategy
$\\{p,q,c_{p},c_{q}\\}$ corresponding to the $x,y$ coordinate of the heat map.

## Basins of attraction

Here we explore the basins for attraction for Case I, II, and III outlined in
the main text, for the ultimatum game.

Figure S1. Replicate of Main text Figure 2b as a heat map, with colors
reflecting the proportion of replicates that end up at Case I (brown), Case II
(cream) or Case III (blue)

Nest we explored how the basin of attraction changes as we change $p_{c}$,
i.e. the threshold of the response function for the artificial agent, $f(p)$.
As we see in Figure S2, this results in the loss of the Case III equilibrium,
so that only successful nudging or failed nudging occur.

Figure S2. Replicate of Figure S1 with $p_{c}=0.5$

Finally we explored the impact of varying $\gamma$ and $\alpha$ on the basins
of attraction (Figure S3).

Figure S3. Replicate of Figure S1 with $\gamma$ and $\alpha$ varied as
indicated.

## References

  * [1] The malicious use of artificial intelligence: Forecasting, prevention, and mitigation, 8 2018.
  * [2] Statement on ai risk — cais, 5 2023.
  * [3] E. Adar, D. S. Tan, and J. Teevan.  Benevolent deception in human computer interaction.  pages 1863–1872. Association for Computing Machinery, 2013.
  * [4] P. Andras, L. Esterle, M. Guckert, T. A. Han, P. R. Lewis, K. Milanovic, T. Payne, C. Perret, J. Pitt, S. T. Powers, N. Urquhart, and S. Wells.  Trusting intelligent machines: Deepening trust within socio-technical systems.  IEEE Technology and Society Magazine, 37:76–83, 2018.
  * [5] H. Belfield.  Activism by the ai community: Analysing recent achievements and future prospects.  pages 15–21. Association for Computing Machinery, 2020.
  * [6] J. Borenstein and R. Arkin.  Robotic nudges: The ethics of engineering a more socially just human being.  Science and Engineering Ethics, 22:31–46, 2016.
  * [7] M. Brundage, S. Avin, J. Clark, H. Toner, P. Eckersley, B. Garfinkel, A. Dafoe, P. Scharre, T. Zeitzoff, B. Filar, H. Anderson, H. Roff, G. C. Allen, J. S. andCarrick Flynn, S. O. hEigeartaigh, S. Beard, H. Belfield, S. Farquhar, C. Lyle, R. Crootof, O. Evans, M. Page, J. Bryson, R. Yampolskiy, and D. Amodei.  The malicious use of artificial intelligence: Forecasting, prevention, and mitigation.  https://img1.wsimg.com/blobby/go/3d82daa4-97fe-4096-9c6b-376b92c619de/downloads/MaliciousUseofAI.pdf?ver=1553030594217, 2018\.  Accessed: 11/29/2023.
  * [8] T. Chakraborti and S. Kambhampati.  (when) can ai bots lie?  pages 53–59. Association for Computing Machinery, 2019.
  * [9] T. Cimpeanu, L. M. Pereira, and T. A. Han.  Co-evolution of social and non-social guilt in structured populations.  pages 2739–2741. International Foundation for Autonomous Agents and Multiagent Systems, 2023.
  * [10] T. Cimpeanu, F. C. Santos, L. M. Pereira, T. Lenaerts, and T. A. Han.  Artificial intelligence development races in heterogeneous settings.  Scientific Reports, 12:1723, 2022.
  * [11] E. Commission.  White paper on artificial intelligence – an european approach to excellence and trust, 2020.
  * [12] E. Glikson and A. W. Woolley.  Human trust in artificial intelligence: Review of empirical research.  Academy of Management Annals, 14:627–660, 2020.
  * [13] K. Grace, J. Salvatier, A. Dafoe, B. Zhang, and O. Evans.  Viewpoint: When will ai exceed human performance? evidence from ai experts.  J. Artif. Int. Res., 62:729–754, 5 2018.
  * [14] A. Guess, J. Nagler, and J. Tucker.  Less than you think: Prevalence and predictors of fake news dissemination on facebook.  Asian-Australasian Journal of Animal Sciences, 32, 2019.
  * [15] W. Güth, R. Schmittberger, and B. Schwarze.  An experimental analysis of ultimatum bargaining.  Journal of economic behavior & organization, 3:367–388, 1982.
  * [16] T. A. Han, L. M. Pereira, F. C. Santos, and T. Lenaerts.  To regulate or not: A social dynamics analysis of an idealised ai race.  Journal of Artificial Intelligence Research, 69:881–921, 2020.
  * [17] T. A. Han, C. Perret, and S. T. Powers.  When to (or not to) trust intelligent machines: Insights from an evolutionary game theory analysis of trust in repeated games.  Cognitive Systems Research, 68:111–124, 2021.
  * [18] C. Hilbe, M. A. Nowak, and K. Sigmund.  Evolution of extortion in iterated prisoner’s dilemma games.  Proceedings of the National Academy of Sciences of the United States of America, 110:6913–6918, 4 2013.
  * [19] A. Isaac and W. Bridewell.  White Lies on Silver Tongues: Why Robots Need to Deceive (and How), pages 157–172.  11 2017.
  * [20] A. Jacovi, A. Marasović, T. Miller, and Y. Goldberg.  Formalizing trust in artificial intelligence: Prerequisites, causes and goals of human trust in ai.  pages 624–635. Association for Computing Machinery, 2021.
  * [21] D. M. Lazer, M. A. Baum, Y. Benkler, A. J. Berinsky, K. M. Greenhill, F. Menczer, M. J. Metzger, B. Nyhan, G. Pennycook, D. Rothschild, M. Schudson, S. A. Sloman, C. R. Sunstein, E. A. Thorson, D. J. Watts, and J. L. Zittrain.  The science of fake news: Addressing fake news requires a multidisciplinary effort.  Science, 359:1094–1096, 3 2018.
  * [22] S. Lin, J. Hilton, and O. Evans.  Truthfulqa: Measuring how models mimic human falsehoods.  pages 3214–3252. Association for Computational Linguistics, 5 2022.
  * [23] L. McNally and A. L. Jackson.  Cooperation creates selection for tactical deception.  Proceedings of the Royal Society B: Biological Sciences, 280:20130699, 2013.
  * [24] C. Mullon, L. Keller, and L. Lehmann.  Evolutionary stability of jointly evolving traits in subdivided populations.  Am Nat, 188(2):175–95, Aug 2016.
  * [25] M. A. Nowak, K. M. Page, and K. Sigmund.  Fairness versus reason in the ultimatum game.  Science, 289:1773–1775, 2000.
  * [26] A. O’Gara.  Hoodwinked: Deception and cooperation in a text-based game for language models, 2023.
  * [27] C. O’neil.  Weapons of math destruction: How big data increases inequality and threatens democracy.  Crown, 2016.
  * [28] A. Pan, J. S. Chan, A. Zou, N. Li, S. Basart, T. Woodside, H. Zhang, S. Emmons, and D. Hendrycks.  Do the rewards justify the means? measuring trade-offs between rewards and ethical behavior in the machiavelli benchmark.  JMLR.org, 2023.
  * [29] P. S. Park, S. Goldstein, A. O’Gara, M. Chen, and D. Hendrycks.  Ai deception: A survey of examples, risks, and potential solutions, 2023\.
  * [30] G. Pennycook, J. McPhetres, Y. Zhang, J. G. Lu, and D. G. Rand.  Fighting covid-19 misinformation on social media: Experimental evidence for a scalable accuracy-nudge intervention.  Psychological science, 31:770–780, 7 2020.
  * [31] L. M. Pereira, T. Lenaerts, L. A. Martinez-Vaquero, and T. A. Han.  Social manifestation of guilt leads to stable cooperation in multi-agent systems.  pages 1422–1430, 2017.
  * [32] E. Perez, S. Ringer, K. Lukosiute, K. Nguyen, E. Chen, S. Heiner, C. Pettit, C. Olsson, S. Kundu, S. Kadavath, A. Jones, A. Chen, B. Mann, B. Israel, B. Seethor, C. McKinnon, C. Olah, D. Yan, D. Amodei, D. Amodei, D. Drain, D. Li, E. Tran-Johnson, G. Khundadze, J. Kernion, J. Landis, J. Kerr, J. Mueller, J. Hyun, J. Landau, K. Ndousse, L. Goldberg, L. Lovitt, M. Lucas, M. Sellitto, M. Zhang, N. Kingsland, N. Elhage, N. Joseph, N. Mercado, N. DasSarma, O. Rausch, R. Larson, S. McCandlish, S. Johnston, S. Kravec, S. E. Showk, T. Lanham, T. Telleen-Lawton, T. Brown, T. Henighan, T. Hume, Y. Bai, Z. Hatfield-Dodds, J. Clark, S. R. Bowman, A. Askell, R. Grosse, D. Hernandez, D. Ganguli, E. Hubinger, N. Schiefer, and J. Kaplan.  Discovering language model behaviors with model-written evaluations.  pages 13387–13434. Association for Computational Linguistics, 7 2023\.
  * [33] W. H. Press and F. J. Dyson.  Iterated prisoner’s dilemma contains strategies that dominate any evolutionary opponent.  Proc Natl Acad Sci U S A, 109(26):10409–13, Jun 2012.
  * [34] D. G. Rand, C. E. Tarnita, H. Ohtsuki, and M. A. Nowak.  Evolution of fairness in the one-shot anonymous ultimatum game.  Proceedings of the National Academy of Sciences, 110:2581–2586, 2013.
  * [35] A. G. Sanfey, J. K. Rilling, J. A. Aronson, L. E. Nystrom, and J. D. Cohen.  The neural basis of economic decision-making in the ultimatum game.  Science (New York, N.Y.), 300:1755–1758, 8 2003.
  * [36] F. P. F. C. F. P. Santos, J. M. Pacheco, A. Paiva, and F. P. F. C. F. P. Santos.  Evolution of collective fairness in hybrid populations of humans and agents.  volume 33, pages 6146–6153, 2019.
  * [37] M. Shanahan, K. McDonell, and L. Reynolds.  Role play with large language models.  Nature, 623:493–498, 2023.
  * [38] K. Siau and W. Wang.  Building trust in artificial intelligence, machine learning, and robotics.  Cutter Business Technology Journal, 31:47–53, 8 2018.
  * [39] Z. Stein-Perlman, B. Weinstein-Raun, and K. Grace.  2022 expert survey on progress in ai.  https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/, 2022\.  Accessed: 11/29/2023.
  * [40] A. J. Stewart, M. Mosleh, M. Diakonova, A. A. Arechar, D. G. Rand, and J. B. Plotkin.  Information gerrymandering and undemocratic decisions.  Nature 2019 573:7772, 573:117–121, 9 2019.
  * [41] A. J. Stewart and J. B. Plotkin.  From extortion to generosity, evolution in the iterated prisoner’s dilemma.  Proceedings of the National Academy of Sciences of the United States of America, 110:15348–15353, 10 2013.
  * [42] A. J. Stewart and J. B. Plotkin.  Collapse of cooperation in evolving games.  Proceedings of the National Academy of Sciences, 111(49):17558–17563, 2014.
  * [43] G. Szabo and C. Toke.  Evolutionary prisoner’s dilemma game on a square lattice.  Physical Review E, 58:69–73, 8 1998.
  * [44] G. Szabó and G. Fáth.  Evolutionary games on graphs.  Physics Reports, 446:97–216, 2007.
  * [45] M. van ’t Wout, R. S. Kahn, A. G. Sanfey, and A. Aleman.  Affective state and decision-making in the ultimatum game.  Experimental Brain Research, 169:564–568, 2006.
  * [46] B. Zhang, N. Dreksler, M. Anderljung, L. Kahn, C. Giattino, A. Dafoe, and M. C. Horowitz.  Forecasting ai progress: Evidence from a survey of machine learning researchers.  8 2022.