# Human-machine social systems

Milena Tsvetkovaa, Taha Yasserib,c, Niccolo Pescetellid, and Tobias Wernere

a Department of Methodology, London School of Economics and Political Science,
London, United Kingdom

b School of Sociology, University College Dublin, Dublin, Ireland

c Geary Institute for Public Policy, University College Dublin, Dublin,
Ireland

d Collective Intelligence Lab, New Jersey Institute of Technology, Newark, New
Jersey, USA

e Center for Humans and Machines, Max Planck Institute for Human Development,
Berlin, Germany

## Abstract

From fake accounts on social media and generative-AI bots such as ChatGPT to
high-frequency trading algorithms on financial markets and self-driving
vehicles on the streets, robots, bots, and algorithms are proliferating and
permeating our communication channels, social interactions, economic
transactions, and transportation arteries. Networks of multiple interdependent
and interacting humans and autonomous machines constitute complex adaptive
social systems where the collective outcomes cannot be simply deduced from
either human or machine behavior alone. Under this paradigm, we review recent
experimental, theoretical, and observational research from across a range of
disciplines – robotics, human-computer interaction, web science, complexity
science, computational social science, finance, economics, political science,
social psychology, and sociology. We identify general dynamics and patterns in
situations of competition, coordination, cooperation, contagion, and
collective decision-making, and contextualize them in four prominent existing
human-machine communities: high-frequency trading markets, the social media
platform formerly known as Twitter, the open-collaboration encyclopedia
Wikipedia, and the news aggregation and discussion community Reddit. We
conclude with suggestions for the research, design, and governance of human-
machine social systems, which are necessary to reduce misinformation, prevent
financial crashes, improve road safety, overcome labor market disruptions, and
enable a better human future.

## Introduction

At 2:32 pm on May 6, 2010, the Dow Jones Industrial Average plunged nearly
1000 points wiping out temporarily $1 trillion from the stock market, only to
recover about half an hour later. This flash crash has been largely attributed
to high-frequency traders responding to and reinforcing the consequences of a
single large sell order [1]. High-frequency traders are sophisticated
algorithms that conduct automated financial trading at high speeds and with
high turnover rates. In other words, the machines did it, and they did it
collectively.

Today, practically all daily human interactions are mediated by algorithms and
bots: web crawlers decide what information we find, ranking algorithms
determine what we consume, auction bots outbid us for purchases, ChatGPT
answers our questions, while social media bots fool us to believe dubious
facts and news. Modern society is a complex human-machine social system in
which machines are increasing in numbers, human interactions with machines are
becoming more frequent, but also, machine-machine interactions are starting to
have real consequences. With recent advances in generative AI models, the
existential threat of unexplainable and uncontrollable general AI has started
looming large again [2, 3, 4]. However, when they are numerous and
interdependent, even simple “dumb” artificial agents can produce unintended
and potentially undesirable outcomes. Imagine a “flash crash” but now in a
city packed with self-driving vehicles full of passengers! If we want to
prevent financial crashes, improve road safety, preserve market competition,
increase auction market efficiency, and reduce misinformation, it is no longer
sufficient to understand humans – we need to consider machines, understand how
humans and machines interact, and how the collective behavior of systems of
humans and machines can be predicted. We require a “new sociology of humans
and machines.”

The existing fields of human-computer interaction, social computing, computer-
supported cooperative work, and machine learning tend to approach machines as
media or interfaces, not as autonomous actors or agents. Artificial
intelligence does precisely this but focuses on cognition and decision-making,
with little attention devoted to how humans and machines interact in groups
and networks [5]. Several important and influential theories such as socio-
technical systems theory [6], actor-network theory [7, 8], cyber-physical
social systems theory [9, 10], and the concepts of social machines [11, 12]
and human-machine networks [13, 14] stem from the idea that humans and
machines form a single social system; yet, they still approach machines as a
single entity, “technology,” bracketing off their multiplicity, autonomy, and
heterogeneity, and leaving the network effects and emergent outcomes often out
of the discussion. In short, the existing theoretical frameworks mainly focus
on the function of one machine either in isolation or at best in interaction
with one or several individuals, leaving out the whole range of situations
when multiple humans and machines co-exist and interact at large scale.

This survey synthesizes research and ideas related to social systems composed
of multiple autonomous, yet interacting and interdependent humans and machines
(Fig. 1). We focus on bots, algorithms, and robots that act alongside humans
in tasks of competition, coordination, cooperation, communication, and
collective decision making. We approach these situations as complex adaptive
social systems where the collective outcomes cannot be necessarily deduced
from individual preferences and behavior alone [15]. Our approach directly
responds to a recent call for a systemic/ecological theoretical perspective
and behavioral empirical methods for studying machine behavior [16] and falls
along the “hybrid collective intelligence perspective,” which considers
humans, artificial agents, and their interactions at different levels of
analysis and design [17]. Similarly to the budding field of Social AI [18], we
operate at the intersection of complex systems, network science, and AI, but
we investigate systems with multiple independent AI agents interacting on par
with humans, rather than systems where human communication and interaction is
mediated by AI technology such as a recommendation algorithm.

Figure 1: In contrast to prior conceptualizations of human-machine aggregates,
human-machine social systems include _multiple autonomous algorithms, bots, or
robots that interact on par with humans_. We need to study human behavior,
machine behavior, and the human-human, human-machine, and machine-machine
interactions simultaneously to understand the collective outcomes in such
systems.

## Human-machine interactions

We use the term “machines” to refer to a wide range of computational
artifacts. These may be embodied in physical devices such as humanoid robots
or exist only in digital space such as bots and algorithms. They may
constitute anything from simple expert systems that use pre-defined if-else
rules to sophisticated generative deep-learning models that learn from data in
real time. We restrict our attention to _autonomous_ artificial agents that
participate in similar tasks as humans and thus operate in the same social
space. Hence, we exclude human-assisted bots and bot-assisted humans known as
“cyborgs” [19, 20], coordinated bot accounts known as botnets, swarm-bots, and
bot farms [21, 22, 23, 24], online bots that perform background
infrastructure-related tasks such as crawlers, indexers, and scrapers [20],
and smart electricity grids and traffic light control systems because, in
these cases, the machines are either non-independent or do not interact on par
with humans.

The machines we consider are similar to humans in that they exhibit goal-
oriented behavior shaped by information and subject to constraints, as well as
diversity in how exactly they do this. Within this high level of similarity,
however, the cognition and behavior of today’s bots and algorithms differ from
those of humans in several notable ways. Generally, machines’ behavior tends
to be predictable and persistent [25], with higher precision and faster
execution [26], better informed with access to global information [27], and
less adaptable and susceptible to influence [28, 29, 30]. In contrast, humans
tend to be limited to local information, satisfice, act with errors, learn and
adapt, succumb to social influence and peer pressure, yet also exhibit opinion
stubbornness and behavioral inertia; on occasions, they may also use
metacognition and revise their own perceptual and decision-making models.
Humans often exhibit cognitive biases due to limited information processing
capacity, bounded rationality, reliance on heuristics, vestiges of
evolutionary adaptation, and emotional motivations [31, 32], and algorithms
trained on data generated by humans tend to reproduce these biases [33, 34].
The ongoing quest for human-like general AI may erase the behavioral
differences, making it difficult to distinguish a human from a machine based
on speech, decisions, and actions. Nevertheless, market competition over
faster, smarter, and more precise algorithms is likely to exacerbate the
cognitive differences between humans and AI [35]. Hence, humans will remain
distinct from machines in the near future.

If humans are unaware that they are facing a machine, they will interact with
it as though they are interacting with a human. Thus, the underlying
mechanisms and assumptions behind interactions with covert artificial agents
do not differ from human-human interactions at the dyadic level. Nevertheless,
they may still produce different collective outcomes because machines act
differently.

Dyadic interactions when humans are aware of the machine’s identity merit more
attention. Research from the CASA (computers as social actors) paradigm in
psychology emphasizes that humans to a large extent treat and respond to
machines similarly to other humans: people reciprocate kind acts by computers
[36], treat them as politely as they treat humans [37], consider them as
competent, but also apply gender and racial stereotypes to them [38, 39];
people also humanize and empathize with machines, and experience distress when
observing the mistreatment of a robot [40, 41].

Nevertheless, there are visible neurophysiological differences in the brain
when humans interact with robots [42, 43], likely because humans do not
attribute agency and morals to them [44, 45]. AI is perceived to have lower
intentional capacity, lack self-interest, and be more unbiased than humans. As
a result, humans exhibit a narrower emotional spectrum with machines than with
humans, reacting with lower and flatter levels of social emotions such as
gratitude, anger, pride, and sense of fairness [46, 47, 48, 49], yet judging
machines more harshly when they commit mistakes, cause harm, or incur losses
[50, 51]. Further, hundreds of experimental studies concur that humans behave
more rationally and selfishly with machines, cooperating and sharing less and
demanding and exploiting more [52, 53, 54, 55]. People would design a machine
to be more cooperative than they are themselves [56] but act pro-socially
towards it only if it is more human-like in terms of physical appearance,
emotional adaptation, and social awareness [57], or if it benefits another
human [58]. Compared to a single person, small groups of people are even more
likely to exhibit competitive behavior and bullying toward robots due to
intergroup bias [59]. Despite this, humans are still susceptible to machine
influence when making decisions or solving problems [60]. Robots can cause
both informational and normative conformity in people [61, 62] and AI and
ChatGPT can corrupt humans’ moral judgment, helping them succumb to unethical
advice [63, 64]. Humans tend to trust algorithmic advice more than advice
coming from a human or a human crowd [65, 66] but may also avoid it if they
perceive a threat to their decision control or a lack of understanding and
cognitive compatibility [67, 68].

Figure 2: Collective outcomes in human-machine social systems differ from those in human-only systems because machines behave differently from humans, human-machine and machine-machine interactions differ from human-human interactions, but also the humans, the machines, and their interactions influence each other. Table 1: Types, examples, and collective outcomes of human-machine social systems. Situation |  Models |  General examples |  H-M examples |  H-M collective outcomes
---|---|---|---|---
Competition |  Zero-sum game Auction markets Buyer-seller markets Oligopoly market games |  Competitions Contests Auctions Markets |  Trading markets Pricing algorithms Online auctions with sniping algorithms Cheating in online games with bots |  \+ Increase efficiency by improving liquidity and price discovery – Increase volatility by causing price spikes and crashes – Increase in consumer prices from algorithmic collusion – Decrease human activity
Coordination |  Coordination game Stag Hunt game Battle of the Sexes Chicken game Graph Coloring game |  Conventions Technological standards Communication technology |  Traffic with autonomous vehicles Wikipedia |  \+ Improve coordination by introducing random behavior – Worsen coordination by failing to adapt
Cooperation |  Prisoner’s Dilemma Public Goods game Dictator game Ultimatum game Trust game |  Team collaboration Mutual aid |  Caring robots Wikipedia Reddit |  \+ Increase cooperation \+ Increase efficiency by handling large task volumes \+ Increased forecasting accuracy – Decrease efficiency by introducing new types of workload
Contagion |  Epidemiological models Threshold models of contagion |  Communicative disease Rumors Innovations |  Twitter Reddit |  – Increase spread of misinformation, opinion polarization, verbal conflict \+ Increase human activity and engagement
Collective decision making |  Vote aggregation Active learning |  Crowdsourcing Prediction markets Voting systems Cultural evolution |  Clinical diagnosis Citizen science Content moderation Hybrid forecasting |  \+ Increase innovation and accuracy by introducing diversity – Decrease human activity and engagement

## Collective outcomes

In complex adaptive social systems, individuals’ behavior and their
interactions affect the collective outcomes but the relationship can be
fundamentally different from a simple sum or average [69, 70, 71]. The
collective outcomes in human-machine social systems differ from those in
human-only systems because machines behave differently from humans, human-
machine (H-M) and machine-machine (M-M) interactions differ from human-human
(H-H) interactions, but also the humans, the machines, and their interactions
influence each other indirectly (Fig. 2). In what follows, we overview recent
findings on the collective dynamics and patterns in groups and networks of
humans and machines in the context of five different social interaction
situations: competition, coordination, cooperation, contagion, and collective
decision-making. We review theoretical and empirical studies based on agent-
based models, controlled experiments, online field interventions, and
observational analyses from robotics, human-computer interaction, web science,
complexity science, computational social science, finance, economics,
political science, social psychology, and sociology. We synthesize the common
patterns in the fragmented literature and identify gaps to fill and links to
build for future AI research, design, and governance.

### Competition

Competition occurs when multiple actors strive for a common goal that cannot
be shared, as is the case in contests, auctions, and product markets.
Algorithms have infiltrated all sorts of markets. These algorithms are
designed to benefit the owner, without regard for others and for the
efficiency and stability of the market; yet, they may still offer collective
benefits.

With more advanced data processing, learning, and optimization capabilities
than humans, algorithms are better able to discover arbitrage opportunities
and hence, eliminate mispricing and increase liquidity in markets. Thus, some
experimental studies show that algorithmic traders can increase market
efficiency [72], but possibly at the expense of human traders’ performance
[73]. Furthermore, with their presence, algorithmic traders make human traders
act more rationally and thus, reduce strategic uncertainty and confusion in
the market [74]. For instance, one study finds smaller price bubbles and
prices that are closer to the fundamental value when subjects expect some
algorithmic traders to participate in the market compared to markets where
they expect only human traders [75]. This study offers the strongest evidence
for the algorithms’ indirect effects on human behavior since it manipulated
only the participants’ beliefs; the markets were human-only in both cases.

Other research, however, finds that algorithms do not always improve
efficiency in trading markets. While perfectly optimizing arbitrage algorithms
eliminate mispricings, neither zero-intelligence algorithms that submit random
bids without profit maximization [76] nor profit-maximizing agents that update
their beliefs from trading history [77] can improve market quality. At the
same time, manipulator and spoofing algorithms that take specific actions to
mislead and influence other traders worsen market efficiency [78].

Jarrow and Protter [79] suggest that high-frequency traders can cause
mispricing and increase market volatility, to the disadvantage of regular
traders, because algorithms react fast and in similar ways to a common signal.
A field experiment in a small online cryptocurrency marketplace finds that
traders are more likely to buy after a bot buys, compared to when a bot
observes but does not intervene, producing a larger buying volume [80]. The
study, however, cannot clarify the extent to which bots or humans are
responsible for the herding effect. Another experiment suggests that reducing
trading speed improves profit dispersion and allocation between algorithmic
and human traders by leveling the playing field against fast algorithmic
actions [81].

While algorithms can reduce the rationality of professional traders, they can
alienate and drive away amateur ones. In online auction markets, freely
available sniping algorithms have greatly increased the prevalence of
effective last-minute bids [82]. Sniping works mainly because it exploits the
naivety of amateur online bidders, who tend to increase their bids
incrementally. However, human lack of rationality carries its own positive
effects on the market because squatting (placing a high early bid) deters new
entrants [83]. Correspondingly, field experiments reveal a negligibly small
[83] or non-existent [84] buyer gain to sniping. More consequentially, since
naive first-time bidders respond negatively to being outbid by snipers and are
less likely to return to another auction [85], algorithms appear to have a net
negative impact on the marketplace.

In addition to trading and auction markets, pricing algorithms have become
widespread in regular product markets [86]. Often, markets are not entirely
priced by algorithms but by a mix of humans and algorithms interacting in the
same environment [87]. This interaction can occur by pricing algorithms
providing recommendations to human pricing managers [88, 89] or between
competing firms, where some use algorithms and others do not [87, 90]. While
pricing algorithms can help firms scale their business and respond more
quickly to changes in demand, there are also anti-competition effects that can
arise from algorithmic pricing. Simulation experiments show that Q-learning
algorithms learn to set anti-competitive prices without communication [91, 92,
93, 94]. Recent experiments highlight that those algorithms are often more
collusive than humans in small markets [95] and foster collusion when
interacting with humans compared to fully human markets [96]. Observational
studies of gasoline markets [90] and e-commerce [97, 98] support this
experimental evidence. Furthermore, algorithms can weaken competition by
providing better demand predictions, thereby stabilizing cartels [99, 100,
101] or by asymmetries in pricing technologies and commitment [102, 103].

Overall, the general intuition is that markets with more actors should be more
efficient. Thus, one might expect markets populated by algorithmic
participants to demonstrate enhanced performance. However, in reality,
machines can behave unexpectedly and cause humans to behave unexpectedly.
Their beneficial effects are often in balance, crucially depending on the
machines’ prevalence, decision speed, and information quality [104], as well
as the humans’ experience and expectations.

### Coordination

The problem of coordination requires adopting a strategy identical to or, in
some cases, dissimilar from, other people’s strategies, as when deciding
whether to participate in collective action, agreeing on a convention such as
driving on the right-hand side of the road, adopting a specific communication
technology or technological standards such as the telefax or the pdf file
format, or avoiding a crowd or traffic congestion [105, 106]. In human-machine
systems, bots could be used to introduce more randomness and movement to steer
human groups toward better solutions.

In support of this idea, bots acting with small levels of random noise and
placed in central locations in a scale-free network decrease the time to
coordination, especially when the solutions are hard to find [107]. The bots
reduce unresolvable conflicts not only in their direct interactions but also
in indirect H-H interactions, even when the participants are aware that they
are interacting with machines. Another study, however, finds that hybrid
groups playing a cooperative group-formation game have lower performance than
human-only and bot-only groups [30]. The bots in this case probabilistically
choose actions based on empirical patterns of human behavior; thus, although
they exhibit human-like strategies, they process information less efficiently
and are less adaptive, reducing group performance in the end.

In sum, in situations where a group may get stuck on a suboptimal equilibrium,
non-human-like bots may be able to help by jittering the system with
randomness and unpredictability. Such simple bots may be more beneficial than
bots that superficially imitate human behavior without the ability to learn
and adapt.

### Cooperation

The problem of cooperation pertains to social dilemma situations where a
decision is collectively beneficial but individually costly and risky.
Although the economically rational decision in non-repeated anonymous
interactions is to free-ride and exploit others’ contributions, people’s
actual behavior tends to be informed by norms of reciprocity, fairness, and
honesty signaling. Thus, as a result of millennia of evolutionary adaptation,
people generally cooperate with each other. If people know they are
interacting with bots, however, they cooperate less [53, 54]. Yet, since
humans reciprocate to and imitate cooperative neighbors, introducing covert,
persistently cooperating bots could increase cooperation.

Computer simulations show that persistent prosocial bots favor the emergence
of fairness by eliciting generous offers in the Ultimatum game [108] and
increase the level of cooperation in a Prisoner’s Dilemma game with costly
punishment [109] or a “loner” option that avoids interaction [110]. The
effects appear stronger when humans are more prone to imitation and bots
occupy more central positions in networks with highly heterogeneous
connectivity [109]. A Public Goods game experiment in networks of 24
individuals confirms that just four under-cover cooperative bots can increase
cooperation, especially if the bots are well spread in the network,
interacting with a higher number of human players, rather than concentrated
with overlapping sets of partners [111]. The reason is that humans wait for
someone else to cooperate before they do but once they observe many
cooperators, they become more likely to exploit.

Yet, cooperative bots may sometimes fail to improve cooperation. For instance,
hybrid groups with identifiable bots do not perform better than human-only
groups playing a Public Goods dilemma with a collective risk leading to
uncertain individual returns [112]. When participants are informed of the
presence of artificial agents but without revealing who they are, there is a
small increase in the cooperation of the bots’ direct neighbors but no
significant boost in the overall network [113]. Similarly, multiple well-
dispersed covert bots, whether all-cooperating or reciprocating, fail to
improve cooperation [114], although a single overt network engineer bot who
suggests connecting cooperators and excluding defectors can successfully do
so.

In sum, covert, persistently cooperating bots (i.e., not very human-like) can
increase cooperation in the group but this crucially depends on the network of
interactions. Bots are successful if they either are strategically positioned
– well dispersed in regular and random networks or centrally located in
networks with skewed degree distributions – or have the power to strategically
engineer the network by offering opportunities to break links to defectors.

### Contagion

Contagion concerns the spread of information and behaviors in communication
networks: prominent examples include the spread of misinformation, slang,
fashion, emotions, and opinions [115, 116, 117]. In contrast to the strategic
interdependence under the competition, coordination, and cooperation
scenarios, the main mechanism here is social influence. Informational
influence is the tendency to accept information from others as accurate and
important when dealing with uncertainty, while normative influence is the
tendency to conform to the expectations of others to fit in a social group or
society [118, 119, 120]. In human-machine systems, bots can be remarkably
influential at the collective level, despite exerting limited direct influence
on individuals because, in networks, small effects can produce chain reactions
and trigger cascades [121, 122, 123].

This is how social media bots influence public opinion. In an agent-based
model of cultural dissemination, where bots with fixed beliefs can affect the
beliefs of humans, weak bots do not alienate their followers and their
followers’ friends and thus have their message spread farther than messages by
more pushy and assertive users [124]. In other words, network amplification
occurs through bots’ indirect influence precisely because their direct
influence on humans is weak, slow, and unobtrusive. In another model, bots
affect public opinion formation not necessarily by influencing people’s actual
opinion but by influencing their confidence to express it [29]. Essentially,
social media bots can amplify marginal voices by triggering the spiral of
silence amongst disagreeing humans. The bots are more influential when they
are more numerous and connected to central actors. Strategically placed zealot
bots can in fact bias voting outcomes and win elections [125].

Bots can also trigger emotional contagion in groups, even though they evoke
flatter emotional reactions from individual humans. Humanoid robots can
encourage and increase social interactions among older adults within care
facilities, between different generations, and for children with ASD [59]. In
small-team collaborative experiments, a robot’s verbal expressions of
vulnerability can show “ripple effects” and make the humans more likely to
exhibit vulnerability themselves by admitting mistakes, consoling team
members, and laughing together [126]. The vulnerable robots also make humans
more likely to engage in social conversations and to perceive the group more
positively [127]. The reported positive contagion effects, however, were
detected when comparing one machine to another [127, 128]. Overall, bots are
more effective than no bots to influence opinions, behavior, and emotions but
not necessarily more effective than humans. Yet, even when bots have a weak
direct influence on humans’ opinions, they can exert significant collective
influence via persistence, strategic placement, and sheer numbers.

### Collective decision-making

Collective decision-making involves groups making choices or solving problems,
using methods to combine individual opinions. It impacts social phenomena as
diverse as team collaboration, voting, scientific innovation, and cultural
evolution [129]. Originating with Galton’s work on estimation tasks [130], the
“wisdom of crowds” concept suggests that a crowd’s collective aggregated
estimate is often more accurate than any individual’s, or sometimes even
experts’ [131]. Crowds perform better when individual opinions are either
independent or diverse [132], while social interaction can hinder [133, 134,
135] or improve [136, 137] collective performance. In human-machine systems,
algorithms introduce diversity and can thus improve decision-making.

An analysis of professional Go players’ moves over 71 years suggests that
AlphaGo, the AI program Google DeepMind introduced in 2016, led human players
to novel strategies and improved their decision-making [138, 139, 140].
AlphaGo’s decisions, untethered by human bias, sparked human innovation in
this game. However, other research indicates that the positive influence of
machine-human social learning on problem-solving is limited. Brinkmann and
colleagues [141] introduce an algorithm in sequences of human participants
that solve a task by building on the previous participant’s solution.
Innovative algorithmic solutions benefit immediate followers but team accuracy
does not have lasting effects as humans are more likely to replicate human
solutions than algorithmic ones. Similarly, in a team prediction task, an
algorithm maintaining group diversity by promoting minority opinions improves
individual accuracy but the effects dissipate for team accuracy [142].

The question of how and when to combine human and algorithmic decision-making
constitutes an active area of research known as hybrid intelligence [143, 17].
It includes research on active and interactive learning and human-in-the-loop
algorithms [144]. An area of application is clinical decision-making, where
combining clinician and algorithmic judgments can improve cancer diagnoses
[145, 146]. Another application is citizen science, where platforms like
Zooniverse combine crowd-based classifications with machine classification.
This hybrid approach found supernovae candidates among Pan-STARRS images more
effectively than humans or machines alone [147]. However, employing AI can
damage citizen scientists’ retention [148, 149], suggesting a trade-off
between efficiency and volunteer engagement. Ultimately, without volunteers,
AI’s performance would diminish.

The emerging field of hybrid intelligence suggests that algorithms introduce
novel solutions but these may be too unfamiliar for humans to adopt.
Nevertheless, machine diversity and competition might inspire alternative
forms of human creativity and innovation. Developing methods to effectively
combine human and machine solutions could further improve collective
intelligence.

## Prominent examples of human-machine communities

Besides the interaction situation, the institutional, organizational,
technological, and cultural aspects of the social context shape the he
machines’ capabilities and behavior, inform the humans’ expectations and
attitudes towards the machines, and constrain the human-machine and machine-
machine interactions. Different human-machine communities function differently
and face different problems.

### Algorithmic and high-frequency trading markets

Markets with algorithmic traders arguably constitute the oldest, largest, and
best-studied human-machine communities we have today. Many different kinds of
algorithmic traders exist. The most prominent kind, high-frequency trading
algorithms, constitute automated scripts that rely on high-speed large-volume
transactions to exploit mispricings or market signals before they disappear or
are incorporated into the price. The phenomenon started in the mid-90s and has
since spread to dominate foreign equities, foreign exchange, commodities,
futures, and stock markets globally [104]. Given the highly competitive nature
of professional trading, the exact algorithms are proprietary but can be
roughly subdivided into market-making approaches and opportunistic trading
[150].

High-frequency trading algorithms process large amounts of trade history data
and current news to make decisions and are thus considered the better
“informed” traders [151]. Some of the algorithms appear to anticipate the
market and their trades consistently predict future order flow by human
traders [152]. However, since most algorithms react similarly to the same
public information, they exhibit less diverse trading strategies and more
correlated actions among themselves compared to humans [153]. Thus, although
their behavior generally improves market efficiency, it can also trigger
behavioral cascades and instability.

High-frequency trading algorithms generally act as market makers, increasing
trading opportunities, reducing transaction costs, connecting buyers and
sellers across venues, and submitting significant volumes of price quotes
[154, 104]. They facilitate price efficiency by trading in the direction of
permanent price changes but opposite temporary price errors [151]. This
regularly acts as a stabilizing force, reducing short-term volatility [153,
155, 150]. On a longer time scale, however, algorithms may decease market
quality by increasing volatility [156] and uncertainty [26], and by reducing
trading strategy diversity [157]. For example, an extensive analysis of the
2010 flash crash concludes that, although the algorithms did not cause it,
they exacerbated it by amplifying the volatility [1]. This has led to recent
efforts to regulate the speed of trading in markets, for example, by
processing trades in batches at slower intervals to diminish the advantage
that high-frequency trading algorithms have, thereby stabilizing markets
[158].

Overall, the empirical research on the market effects from algorithmic and
high-frequency trading is inconclusive, often because researchers do not
observe trader type and estimate the prevalence and influence of algorithms
indirectly. Much of the research, nevertheless, emphasizes generally positive
effects on market efficiency due to machines’ higher information processing
and optimization capabilities and humans’ anticipation of these. The improved
states, however, are occasionally perturbed by micro-bursts of price spikes
and crashes due to machines’ herding behavior and humans’ inability to
intervene at ultra-fast timescales. Paradoxically, in such extreme events,
humans are expected to be the rational actors with global information and
perspective, while the algorithms are considered impulsive and susceptible to
influence.

### Twitter

Until its rebranding as X in 2022 and its data access restrictions, the micro-
blogging platform Twitter was a valuable tool for human-machine research.
Social bots are the most prevalent and studied automated accounts on Twitter.
They are designed to mimic humans to boost followers, disseminate information,
and promote products. Bots and bot detection methods have co-evolved,
resulting in increasingly more sophisticated imitation or detection strategies
[159, 160, 161, 162, 28, 163, 164], but detection is inherently limited due to
the overlap between covert autonomous bots, managed user accounts, hacked
accounts, cyborgs, sock-puppets, and coordinated botnets [21, 165, 19, 166].
Estimates suggest that 9-15% of Twitter users are bots [167, 164], with bot
activity typically increasing around controversial political events [168].

Twitter social bots engage less in social interactions than humans but produce
more content [169]. In active sessions, humans increase social interactions
(retweets, replies, and mentions), but decrease new content production,
showing a pattern not seen in bots. This is likely because of humans’ social
instincts and cognitive limitations, leading to fatigue [169]. Overall, bots
are less connected, having fewer human friends and followers compared to
humans [164]. Interactions are predominantly human-human (76%), with bot-bot
(2%), bot-human (19%), and human-bot (3%) interactions being less common
[168]. Bots mainly retweet – a passive strategy to indicate support and gain
followers – but are less successful in attracting followers than humans [164].

Despite their lesser social behavior, Twitter bots significantly influence
communication and contagion. An early study suggests that bots can affect
human interaction networks by encouraging followings and conversations [170].
They play an important role in misinformation dissemination, notably in
relation to political events [171, 172, 173, 174, 175, 176, 177], COVID-19
[178, 179], and stock market investment [180]. Bots amplify low-credibility
content early on and target influential humans through replies and mentions
[175]. Interested parties can trigger deep information cascades by enhancing
bot visibility and influence with their large numbers [181]. Interestingly,
while bots equally link to true and false news from low-credibility sites,
people do not discriminate between humans and bots when they retweet low-
credibility content and are more likely to spread false news. Hence, it is
humans who are responsible for the spread of false news [182].

Bots significantly contribute to negative sentiment and conflict escalation.
Acting from the periphery, they target central human users to exert indirect
influence. Bots amplify existing negative sentiment and selectively promote
inflammatory content, often targeting only one faction in conflicts [168].
Their success stems from exploiting human tendencies to connect with similar
others and engage with messages that reinforce their beliefs [183].
Consequently, they increase ideological polarization and negatively affect
democratic discourse on social media, as seen in events like the 2016 US
presidential election [184], the 2016 UK Brexit Referendum [183], and the 2017
Catalan independence referendum [168].

In sum, Twitter’s covert social bots, which impersonate humans, are considered
harmful, prompting the platform to cull them [185, 186]. Despite their
rudimentary social behavior and weak network integration, these bots
significantly influence political communication, public information and
opinion, elections, and markets. Their strength lies in indirect action: they
skew the platform’s recommendation system, thus biasing popular content [142].
Additionally, they exploit human behavioral weaknesses like attention seeking,
confirmation bias, moral outrage, and ideological homophily. Bots’ influence
is thus primarily indirect, planting the seeds of manipulation and relying on
human tendencies to fertilize them.

### Wikipedia

Wikipedia, the largest and most popular free-content online encyclopedia,
hosts an ecology of bots that generate articles, fix errors on pages, link to
other sites and databases, tag articles in categories, identify vandals,
notify users, and so on [187, 188, 189]. These bots are open-source, approved,
registered, and tagged [190, 187]. They are not sophisticated: most use basic
regular expressions or straightforward heuristics and only some incorporate
machine learning techniques. They are significantly less numerous than human
editors but complete a disproportionately large volume of all edits [191, 25,
189].

Compared to human-human interactions, bot-bot interactions are more reciprocal
and balanced but do not exhibit status effects [25]. While bots are more
likely to be involved in back-and-forth reverts with each other over long
periods, these accidental encounters are rarely a sign of direct opinion
conflict but part of routine productive maintenance work or a reflection of
conflicts existing between their human owners [192]. Human editors interact
mainly with policing bots, mostly by criticizing the legitimacy of the norms
they enforce, rather than the sanctions themselves, suggesting that editors
perceive bots as extensions of their human owners rather than independent
agents [193].

Bots have been invaluably beneficial to the maintenance and operation of
Wikipedia. The diversity of the bot ecology guarantees the system’s
resilience. For instance, during the random outage of the anti-vandalism
ClueBot NG, the website eventually caught up, albeit twice as slowly as usual,
thanks to the heterogeneity of the quality control network, comprising
instantaneous fully automated robots, rapid tool-assisted humans (cyborgs),
humans editing via web browsers, and idiosyncratic batch scripts [194].
Wikipedia demonstrates that successful bot governance and regulation does not
have to come at the expense of independent development and diversity. The
simplicity, independence, and heterogeneity of the machines facilitate the
system’s success and resilience overall but may also introduce unexpected
complexities and uncertainties in communication at smaller scales [195].

### Reddit

Reddit is a popular news aggregation, content rating, and discussion website
founded in 2005. Bots on Reddit provide internal moderation and communication,
augment functionality (e.g., for mobile users), or post content similarly to
human users, ranging from comic and playful posts by evident automated
accounts such as haiku_robot and ObamaRobot, to trolling and provocative
comments by undercover social bots [196]. Similarly to Wikipedia, Reddit has
developed norms and protocols for deploying bots [197], but similarly to
Twitter, it has no effective service limitations to prevent covert and
malicious automated accounts. Nevertheless, Reddit differs from Twitter in
several crucial ways that mitigate problems from unstemmed bot activity –
content on the site is posted within communities, heavily moderated,
up-/downvoted, and extensively discussed. Due to these structural differences,
political misinformation, polarization, and conflict are less pronounced on
the platform.

Reddit offers evidence for collaboration and contagion between humans and
bots. Content moderators have widely adopted Automod – a bot that is flexible
to updates, adaptable to community rules, and interpretable. The bot aids with
menial tasks but does not necessarily decrease workloads as it requires
continuous updates in response to changes in user behavior and language and
involves high volumes of correspondence with incorrectly banned users [198].
From the other side, regular users engage with evident entertainment bots and
their direct replies imitate the sentiment and the words of the bot posts
[199]. Thus, emotional contagion and lexical entrainment can occur between
humans and bots, even when humans are aware of the simple automated script
behind the bot.

## Discussion

The algorithms in currently existing human-machine social systems are still
relatively simple. Few of them use sophisticated machine learning or AI
approaches, and even then, these inform narrow and well-defined behavior and
actions. With the notable exception of malicious social bots and customer-
service chatbots, most machines do not attempt to be human-like. In fact, most
machines are either super-human, capable of processing large quantities of
data, acting at ultra-fast speeds, and tirelessly persisting with tedious
jobs, or candidly non-human in the sense of resisting peer influence, not
reciprocating, and acting completely at random. There is also a clear
distinction between covert and overt bots: covert bots are more problematic
than bots that are encouraged or forced to be explicit about their identity
and follow well-defined norms and regulations. Overall, machines tend to have
both positive and negative effects on the system, and these can often be
unintuitive.

The effects of machines in human-machine social systems crucially depend on
the algorithms they follow, their position in the interaction network, their
numbers, the specifics of the interaction situation, institutional
regulations, technological affordances, the organizational setting, and
emergent norms and culture. Machines change outcomes because of their
different behavior, because humans interact differently with them, but also
because of their indirect effects – the presence of machines causes humans to
behave and interact with each other differently.

Machines can be beneficial when they act directly or steer humans in ways that
compensate for human weaknesses. For instance, intentionally noisy bots can
shake up sub-optimal outcomes and improve coordination, while persistently
cooperative bots can stem eye-for-an-eye cascades and maintain cooperation; a
machine placed in a central role as an arbitrageur can facilitate price
discovery and improve market quality, while network-engineering bots can
facilitate the assortment of cooperators and the exclusion of defectors,
boosting collective welfare. With access to global information, higher
processing power, and instantaneous execution, machines can quickly respond to
external events such as vandalism, political crises, or natural disasters,
ensuring the system’s robustness and resilience and improving operational
efficiency. Depending on the situation, machines can offer super-human
persistence, behavioral diversity, or decision-making speed that complement
humans’ behavior and adjust it towards better outcomes.

What helps in one context, however, can hinder in another. Machines’
unintuitive decisions and ultrafast speeds may confuse humans. Thus, humans
may not be able to learn from and adopt machines’ superior solutions that
could advance knowledge and aid technological progress. In other situations,
humans may be unable to prevent or correct promptly machines’ awry behavior,
resulting in episodic instabilities and flash failures. Machines are less
responsive than humans and cannot always react to changes in the interaction
situation or human behavior, impeding system adaptation and evolution.
Machines can also be designed to exploit human weaknesses and trigger behavior
and information cascades that exacerbate opinion polarization, emotional
contagion, ideological segregation, and conflict. Machines’ non-human
optimization logic, execution speed, and behavioral rigidity can clash with
humans and exacerbate behavior and interactions toward undesirable outcomes.

The review outlines several directions for research, recommendations for AI
design, and considerations for social policy.

### Implications for research

Existing research tends to be biased towards engineering and optimization,
lacking a deeper understanding and explanation from a social science
perspective. We require a new relational sociology of humans and machines that
embeds social psychology mechanisms such as outgroup bias, authority bias, and
personification into the micro-macro paradigm of complex systems to study _in
silico_ , _in vitro_ , and _in vivo_ the collective dynamics and patterns that
systems of different composition, structure, and interaction rules generate.
The time for this new sociology is particularly ripe, before AI becomes more
sophisticated and sophisticated AI starts proliferating. Complex generative AI
exhibits emergent behavior that itself requires explanation and understanding
[200, 201] and such unpredictable agents would considerably complicate the
task of understanding and predicting the dynamics and patterns of human-
machine social systems. We need to get going and get ahead now.

We have identified several productive avenues for research. First, researchers
would benefit from an agent-based modeling framework that outlines important
aspects of distinction for human and bot agents: utility function,
optimization ability, access to information, learning, innovation/creativity,
proneness to error, etc. The framework could borrow concepts and approaches
from agent-based models of other two-agent systems such as predator–prey,
principal–agent, and common pool resource models. Second, we would benefit
from more controlled experiments that explicitly compare human-machine
networks to human-only and machine-only networks and known bots to covert or
unknown bots. Experiments should manipulate participants’ perceptions of the
algorithms in terms of technical specifications, emotional capability, gender
and minority bias, and similar. Field interventions in online communities with
endemic bot populations present a third promising research direction. There
are already several compelling examples that demonstrate the promise of the
methodology: social bots that engage with human users to become popular and
influential [202, 203, 204, 205], trading bots that influence traders’
behavior to manipulate prices in cryptocurrency markets [206], political bots
that expose their followers to opposing political bias to decrease
polarization and radicalization [207], and random “drifters” to measure
platform bias [208]. Fourth, in addition to high-frequency trading markets,
Wikipedia, Reddit, and Twitter, we need observational research of more human-
machine communities and contexts as, for instance, traffic systems that
include human-driven and driverless vehicles, online multiplayer games
composed of human players, non-player characters, and cheating code, and
dating markets with AI-driven chatbots [209].

Finally, research with robots, bots, algorithms, and AI entails new ethical
problems that demand careful elaboration and mitigation. For instance, covert
bots should be used as confederates in field experiments only in communities
where bots are already present, ensuring their actions are not unusual and
they do not cause direct harm [80]. As any digital research, research with
bots should minimize interventions: researchers should consider replacing
experiments with less invasive methods, refining the treatments to minimize
risks, and reducing the number of participants [210]. Even then, bots may be
banned by site administrators because users get upset and voice privacy
concerns [202]. Do people perceive certain bots as inherently deceptive? Could
knowledge of the bot owner and algorithm mitigate this perception?

### Implications for AI design

The conceptualization of human-machine social systems has concrete
applications to AI design: since H-H, H-M, and M-M interactions differ,
machines should be explicitly and differentially designed for these.
Importantly, the algorithms should be trained separately for each type of
interaction. This will help avoid problematic cases such as when bargaining
algorithms trained on H-H interactions hurt market outcomes because humans
interact differently with them [52] and driverless vehicles trained on human
driving get stuck in traffic when co-appearing in large numbers [211].

Relatedly, AI design should consider a hierarchy of behavioral rules and
conventions to guide H-M and M-M interactions in the context of H-H
interactions. Isaac Asimov’s famous Three Laws of Robotics [212] regulate M-H
interactions and own preservation only: a robot should 1) not harm humans, 2)
obey humans, and 3) protect its own existence, where higher order rules take
precedence. We argue here that AI designers should think about M-M
interactions along similar lines, adapting the rules to specific contexts and
carefully considering their potential implications and unintended consequences
[213].

AI design should also take into consideration cultural context. How people
judge machines depends on their age, environment, personality traits, etc.,
and changes from place to place [214, 48]. Machines themselves have culture
because they reflect their owners’ culture but also act and interact in
settings with specific organizational principles and community norms [25]. For
instance, when it comes to self-driving cars and assistant bots, AI designers
should account for local driving culture and attitudes towards domestic
assistants when training the algorithms to interact with humans of the same or
different background, as well as with machines trained on the same or
different culture.

### Implications for policy

Our framework and survey emphasize the need for a system-focused approach to
AI policy and ethics: policymakers should approach AI not as a single
existential threat but as a multiplicity of machines and algorithms. Machines
appear to be more beneficial when they are superhuman or simply “alien” and
when they are diverse. Similarity in information sources, interaction speeds,
optimization algorithms, and objective functions can cause extreme
catastrophic events, such as flash crashes in markets. Thus, while AI
designers may chase optimization and superintelligence, policymakers should be
more concerned with the diversity of human-machine ecologies. This spotlights
a new set of questions: Should all machines be equal? Should we allow status
hierarchies and priority orderings among machines? Should we allow these
differences to align with, and thus exacerbate existing socio-economic
inequalities?

Policymakers should also prepare for the co-evolution of machines and humans,
which will inadvertently change the nature of existing institutions. Machines
can cause humans to withdraw interaction: for instance, outsourcing care to
robots can reduce caregivers’ empathy [215]. Intelligent machines are changing
the transmission and creation of human culture by altering social learning
dynamics and generating new game strategies, scientific discoveries, and art
forms [216]. Humans will need to adapt to autonomous machines just as much as
autonomous machines need to learn from and adapt to humans. Understanding the
complex interactions between humans and machines and eventually, their co-
evolution over time, will be essential to manage and govern future human-
machine society.

## Conclusion

This survey aimed to gather, link, and synthesize a relatively disparate and
disconnected literature from human-computer interaction, robotics, web
science, financial economics, and computational social science under a common
theoretical paradigm: human-machine social systems. We attempted to identify
common dynamics and patterns that emerge from the interactions of humans and
autonomous machines regardless of the specific context, as well as
peculiarities and unique problems that concrete techno-organizational and
socio-cultural environments generate. Our utmost ambition is to stimulate
cumulative empirically driven and mechanism-focused sociological research in
the emerging, fast-evolving field of human-AI science. At stake are new and
urgent social challenges such as online misinformation, market flash crashes,
cybersecurity, labor market resilience, and road safety. With increasing
social connectivity and accelerating developments in AI, understanding the
social systems of humans and autonomous machines is a challenging undertaking,
but one that is crucially important for a better human future.

## Review methodology

We compile an integrative narrative review [217, 218, 219] that summarizes and
synthesizes research on human-machine social systems combining different
methodologies. We employ an expansive search strategy, whereby we start from
several relevant search terms and key publications and then employ a backward
search of referenced articles and a forward search of citing articles,
repeating iteratively as we discover new key concepts and areas of research.
For the database search, we queried Google Scholar with the following
combination of keywords: robots + group, bots + network, bots + twitter, bots
+ reddit, bots + wikipedia, algorithmic trading, algorithmic pricing, and
sniping. Google Scholar is a suitable database because it includes publicly
accessible working papers, not just published articles. This helps cover the
most recent work in the actively growing field of research we review; it also
guards against publication bias.

The abstracts of the results from the direct searches and the cited and citing
articles were manually screened for inclusion. The inclusion criteria required
that the study involves at least one bot interacting on par with humans in
groups of three or more. The articles were grouped into themes and topics and
within each topic, we repeated the search procedure whenever we discovered
more specific and niche terminology. Since the field is emergent and the
conceptualization and terminology are not established yet, the review is not
necessarily exhaustive. Further, we aim to balance our overview across the
different areas of research, and since these emerged in different periods and
have developed to differing extents, the exhaustiveness of the reviewed
literature varies by research area too.

## Acknowledgments

M.T. acknowledges the generous support of the Santa Fe Institute during the
period when the research was conducted. The funders had no role in study
design, data collection and analysis, decision to publish, or preparation of
the manuscript.

## References

  * [1] Kirilenko, A., Kyle, A. S., Samadi, M. & Tuzun, T.  The flash crash: High-frequency trading in an electronic market.  _The Journal of Finance_ 72, 967–998 (2017).
  * [2] Metz, C.  The new chat bots could change the world. Can you trust them?  _The New York Times_ (2022).
  * [3] Milmo, D. & Stacey, K.  ‘It’s not clear we can control it’: What they said at the Bletchley Park AI summit.  _The Guardian_ (2023).
  * [4] Lipton, E.  As A.I.-Controlled Killer Drones Become Reality, Nations Debate Limits.  _The New York Times_ (2023).
  * [5] Nilsson, N. J.  _The Quest for Artificial Intelligence_ (Cambridge University Press, 2009).
  * [6] Emery, F.  Characteristics of socio-technical systems.  In _Characteristics of Socio-Technical Systems_ , 157–186 (University of Pennsylvania Press, 2016).
  * [7] Latour, B.  _Reassembling the Social: An Introduction to Actor-Network-Theory_ (OUP Oxford, 2007).
  * [8] Law, J.  Notes on the theory of the actor-network: Ordering, strategy, and heterogeneity.  _Systems practice_ 5, 379–393 (1992).
  * [9] Sheth, A., Anantharam, P. & Henson, C.  Physical-cyber-social computing: An early 21st century approach.  _IEEE Intelligent Systems_ 28, 78–82 (2013).
  * [10] Wang, F.-Y.  The emergence of intelligent enterprises: From CPS to CPSS.  _IEEE Intelligent Systems_ 25, 85–88 (2010).
  * [11] Buregio, V., Meira, S. & Rosa, N.  Social machines: A unified paradigm to describe social web-oriented systems.  In _Proceedings of the 22nd International Conference on World Wide Web_ , WWW ’13 Companion, 885–890 (Association for Computing Machinery, New York, NY, USA, 2013).
  * [12] Shadbolt, N. R. _et al._ Towards a classification framework for social machines.  In _Proceedings of the 22nd International Conference on World Wide Web_ , WWW ’13 Companion, 905–912 (Association for Computing Machinery, New York, NY, USA, 2013).
  * [13] Eide, A. W. _et al._ Human-machine networks: Towards a typology and profiling framework.  In Kurosu, M. (ed.) _Human-Computer Interaction. Theory, Design, Development and Practice_ , no. 9731 in Lecture Notes in Computer Science, 11–22 (Springer International Publishing, 2016).
  * [14] Tsvetkova, M. _et al._ Understanding human-machine networks: A cross-disciplinary survey.  _ACM Computing Surveys (CSUR)_ (2017).
  * [15] Miller, J. H. & Page, S.  _Complex Adaptive Systems: An Introduction to Computational Models of Social Life_ (Princeton University Press, 2009).
  * [16] Rahwan, I. _et al._ Machine behaviour.  _Nature_ 568, 477–486 (2019).
  * [17] Peeters, M. M. M. _et al._ Hybrid collective intelligence in a human–AI society.  _AI & SOCIETY_ 36, 217–238 (2021).
  * [18] Pedreschi, D. _et al._ Social AI and the challenges of the human-AI ecosystem (2023).  2306.13723.
  * [19] Chu, Z., Gianvecchio, S., Wang, H. & Jajodia, S.  Who is tweeting on Twitter: Human, bot, or cyborg?  In _Proceedings of the 26th Annual Computer Security Applications Conference_ , ACSAC ’10, 21–30 (Association for Computing Machinery, New York, NY, USA, 2010).
  * [20] Gorwa, R. & Guilbeault, D.  Unpacking the social media bot: A typology to guide research and policy.  _Policy & Internet_ 12, 225–248 (2020).
  * [21] Abokhodair, N., Yoo, D. & McDonald, D. W.  Dissecting a social botnet: Growth, content and influence in Twitter.  In _Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing_, CSCW ’15, 839–851 (Association for Computing Machinery, New York, NY, USA, 2015).
  * [22] Boshmaf, Y., Muslukhov, I., Beznosov, K. & Ripeanu, M.  The socialbot network: When bots socialize for fame and money.  In _Proceedings of the 27th Annual Computer Security Applications Conference_ , ACSAC ’11, 93–102 (Association for Computing Machinery, New York, NY, USA, 2011).
  * [23] Mondada, F. _et al._ The cooperation of swarm-bots: Physical interactions in collective robotics.  _IEEE Robotics & Automation Magazine_ 12, 21–28 (2005).
  * [24] Silva, S. S. C., Silva, R. M. P., Pinto, R. C. G. & Salles, R. M.  Botnets: A survey.  _Computer Networks_ 57, 378–403 (2013).
  * [25] Tsvetkova, M., García-Gavilanes, R., Floridi, L. & Yasseri, T.  Even good bots fight: The case of Wikipedia.  _PLoS ONE_ 12, e0171774 (2017).
  * [26] Hilbert, M. & Darmon, D.  How complexity and uncertainty grew with algorithmic trading.  _Entropy_ 22, E499 (2020).
  * [27] Koren, Y., Rendle, S. & Bell, R.  Advances in collaborative filtering.  In Ricci, F., Rokach, L. & Shapira, B. (eds.) _Recommender Systems Handbook_ , 91–142 (Springer US, New York, NY, 2022).
  * [28] Ferrara, E., Varol, O., Davis, C., Menczer, F. & Flammini, A.  The rise of social bots.  _Communications of the ACM_ 59, 96–104 (2016).
  * [29] Ross, B. _et al._ Are social bots a real threat? An agent-based model of the spiral of silence to analyse the impact of manipulative actors in social networks.  _European Journal of Information Systems_ 28, 394–412 (2019).
  * [30] Takko, T., Bhattacharya, K., Monsivais, D. & Kaski, K.  Human-agent coordination in a group formation game.  _Scientific Reports_ 11, 10744 (2021).
  * [31] Gilovich, T.  _How We Know What Isn’t So_ (Simon and Schuster, 2008).
  * [32] Kahneman, D.  _Thinking, Fast and Slow: Daniel Kahneman_ (Penguin, London, 2012), 1st edition edn.
  * [33] Kordzadeh, N. & Ghasemaghaei, M.  Algorithmic bias: Review, synthesis, and future research directions.  _European Journal of Information Systems_ 31, 388–409 (2022).
  * [34] O’Neil, C.  _Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy_ (Crown Publishing Group, New York, NY, 2016).
  * [35] Tegmark, M.  _Life 3.0: Being Human in the Age of Artificial Intelligence_ (Allen Lane, London, 2017), 1st edition edn.
  * [36] Fogg, BJ. & Nass, C.  How users reciprocate to computers: An experiment that demonstrates behavior change.  In _CHI ’97 Extended Abstracts on Human Factors in Computing Systems_ , CHI EA ’97, 331–332 (Association for Computing Machinery, New York, NY, USA, 1997).
  * [37] Nass, C., Steuer, J. & Tauber, E. R.  Computers are social actors.  In _Proceedings of the SIGCHI Conference on Human Factors in Computing Systems_ , 72–78 (1994).
  * [38] Nass, C. & Moon, Y.  Machines and mindlessness: Social responses to computers.  _Journal of Social Issues_ 56, 81–103 (2000).
  * [39] Siegel, M., Breazeal, C. & Norton, M. I.  Persuasive robotics: The influence of robot gender on human behavior.  In _2009 IEEE/RSJ International Conference on Intelligent Robots and Systems_ , 2563–2568 (2009).
  * [40] Rosenthal-von der Pütten, A. M., Krämer, N. C., Hoffmann, L., Sobieraj, S. & Eimler, S. C.  An experimental study on emotional reactions towards a robot.  _International Journal of Social Robotics_ 5, 17–34 (2013).
  * [41] Slater, M. _et al._ A virtual reprise of the Stanley Milgram obedience experiments.  _PLOS ONE_ 1, e39 (2006).
  * [42] Krach, S. _et al._ Can machines think? Interaction and perspective taking with robots investigated via fMRI.  _PLOS ONE_ 3, e2597 (2008).
  * [43] McCabe, K., Houser, D., Ryan, L., Smith, V. & Trouard, T.  A functional imaging study of cooperation in two-person reciprocal exchange.  _Proceedings of the National Academy of Sciences_ 98, 11832–11835 (2001).
  * [44] Gray, H. M., Gray, K. & Wegner, D. M.  Dimensions of mind perception.  _Science_ 315, 619–619 (2007).
  * [45] Zhang, J., Conway, J. & Hidalgo, C. A.  Why do people judge humans differently from machines? The role of agency and experience (2022).  2210.10081.
  * [46] Adam, M. T. P., Teubner, T. & Gimpel, H.  No rage against the machine: How computer agents mitigate human emotional processes in electronic negotiations.  _Group Decision and Negotiation_ 27, 543–571 (2018).
  * [47] Chugunova, M. & Sele, D.  We and It: An interdisciplinary review of the experimental evidence on how humans interact with machines.  _Journal of Behavioral and Experimental Economics_ 99, 101897 (2022).
  * [48] Hidalgo, C. A., Orghian, D., Canals, J. A., Almeida, F. D. & Martin, N.  _How Humans Judge Machines_ (MIT Press, 2021).
  * [49] Schniter, E., Shields, T. W. & Sznycer, D.  Trust in humans and robots: Economically similar but emotionally different.  _Journal of Economic Psychology_ 78, 102253 (2020).
  * [50] Dietvorst, B. J., Simmons, J. P. & Massey, C.  Algorithm aversion: People erroneously avoid algorithms after seeing them err.  _Journal of experimental psychology General_ 144, 114–126 (2015).
  * [51] Candrian, C. & Scherer, A.  Rise of the machines: Delegating decisions to autonomous AI.  _Computers in Human Behavior_ 134, 107308 (2022).
  * [52] Erlei, A., Das, R., Meub, L., Anand, A. & Gadiraju, U.  For what it’s worth: Humans overwrite their economic self-interest to avoid bargaining with AI systems.  In _Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems_ , CHI ’22, 1–18 (Association for Computing Machinery, New York, NY, USA, 2022).
  * [53] Ishowo-Oloko, F. _et al._ Behavioural evidence for a transparency–efficiency tradeoff in human–machine cooperation.  _Nature Machine Intelligence_ 1, 517–521 (2019).
  * [54] Karpus, J., Krüger, A., Verba, J. T., Bahrami, B. & Deroy, O.  Algorithm exploitation: Humans are keen to exploit benevolent AI.  _iScience_ 24, 102679 (2021).
  * [55] March, C.  Strategic interactions between humans and artificial intelligence: Lessons from experiments with computer players.  _Journal of Economic Psychology_ 87, 102426 (2021).
  * [56] de Melo, C. M., Marsella, S. & Gratch, J.  Human cooperation when acting through autonomous machines.  _Proceedings of the National Academy of Sciences_ 116, 3482–3487 (2019).
  * [57] Oliveira, R., Arriaga, P., Santos, F. P., Mascarenhas, S. & Paiva, A.  Towards prosocial design: A scoping review of the use of robots and virtual agents to trigger prosocial behaviour.  _Computers in Human Behavior_ 114, 106547 (2021).
  * [58] Hayes, B., Ullman, D., Alexander, E., Bank, C. & Scassellati, B.  People help robots who help others, not robots who help themselves.  In _The 23rd IEEE International Symposium on Robot and Human Interactive Communication_ , 255–260 (2014).
  * [59] Sebo, S., Stoll, B., Scassellati, B. & Jung, M. F.  Robots in groups and teams: A literature review.  _Proceedings of the ACM on Human-Computer Interaction_ 4, 176:1–176:36 (2020).
  * [60] Köbis, N., Bonnefon, J.-F. & Rahwan, I.  Bad machines corrupt good morals.  _Nature Human Behaviour_ 5, 679–685 (2021).
  * [61] Salomons, N., van der Linden, M., Sebo, S. S. & Scassellati, B.  Humans conform to robots: Disambiguating trust, truth, and conformity.  In _2018 13th ACM/IEEE International Conference on Human-Robot Interaction (HRI)_ , 187–195 (2018).
  * [62] Salomons, N., Sebo, S. S., Qin, M. & Scassellati, B.  A minority of one against a majority of robots: Robots cause normative and informational conformity.  _ACM Transactions on Human-Robot Interaction (THRI)_ 10, 1–22 (2021).
  * [63] Leib, M., Köbis, N. C., Rilke, R. M., Hagens, M. & Irlenbusch, B.  The corruptive force of AI-generated advice (2021).  2102.07536.
  * [64] Krügel, S., Ostermaier, A. & Uhl, M.  ChatGPT’s inconsistent moral advice influences users’ judgment.  _Scientific Reports_ 13, 4569 (2023).
  * [65] Bogert, E., Schecter, A. & Watson, R. T.  Humans rely more on algorithms than social influence as a task becomes more difficult.  _Scientific Reports_ 11, 8028 (2021).
  * [66] Logg, J. M., Minson, J. A. & Moore, D. A.  Algorithm appreciation: People prefer algorithmic to human judgment.  _Organizational Behavior and Human Decision Processes_ 151, 90–103 (2019).
  * [67] Burton, J. W., Stein, M.-K. & Jensen, T. B.  A systematic review of algorithm aversion in augmented decision making.  _Journal of Behavioral Decision Making_ 33, 220–239 (2020).
  * [68] Mahmud, H., Islam, A. K. M. N., Ahmed, S. I. & Smolander, K.  What influences algorithmic decision-making? A systematic literature review on algorithm aversion.  _Technological Forecasting and Social Change_ 175, 121390 (2022).
  * [69] Axelrod, R.  _The Evolution of Cooperation_ (Basic Books, New York, 1984).
  * [70] Schelling, T. C.  Dynamic models of segregation.  _The Journal of Mathematical Sociology_ 1, 143–186 (1971).
  * [71] Granovetter, M.  Threshold models of collective behavior.  _American Journal of Sociology_ 83, 1420–1443 (1978).
  * [72] Grossklags, J. & Schmidt, C.  Software agents and market (in) efficiency: A human trader experiment.  _IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews)_ 36, 56–67 (2006).
  * [73] Angerer, M., Neugebauer, T. & Shachat, J.  Arbitrage bots in experimental asset markets.  _Journal of Economic Behavior & Organization_ 206, 262–278 (2023).
  * [74] Akiyama, E., Hanaki, N. & Ishikawa, R.  It is not just confusion! Strategic uncertainty in an experimental asset market.  _The Economic Journal_ 127, F563–F580 (2017).
  * [75] Farjam, M. & Kirchkamp, O.  Bubbles in hybrid markets: How expectations about algorithmic trading affect human trading.  _Journal of Economic Behavior & Organization_ 146, 248–269 (2018).
  * [76] Gode, D. K. & Sunder, S.  Allocative efficiency of markets with zero-intelligence traders: Market as a partial substitute for individual rationality.  _Journal of Political Economy_ 101, 119–137 (1993).
  * [77] Gjerstad, S.  The competitive market paradox.  _Journal of Economic Dynamics and Control_ 31, 1753–1780 (2007).
  * [78] Bao, T., Nekrasova, E., Neugebauer, T. & Riyanto, Y. E.  _Algorithmic Trading in Experimental Markets with Human Traders: A Literature Survey_ (Edward Elgar Publishing, 2022).
  * [79] Jarrow, R. A. & Protter, P.  A dysfunctional role of high frequency trading in electronic markets.  _International Journal of Theoretical and Applied Finance_ 15, 1250022 (2012).
  * [80] Krafft, P. M., Macy, M. & Pentland, A. S.  Bots as virtual confederates: Design and ethics.  In _Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing_ , CSCW ’17, 183–190 (Association for Computing Machinery, New York, NY, USA, 2017).
  * [81] Cartlidge, J., De Luca, M., Szostek, C. & Cliff, D.  Too fast too furious: Faster financial-market trading agents can give less efficient markets.  _ICAART-2012: Proceedings of the Fourth International Conference on Agents and Artificial Intelligence, Vol. 2 (Agents)_ 126–135 (2012).
  * [82] Roth, A. E. & Ockenfels, A.  Last-minute bidding and the rules for ending second-price auctions: Evidence from eBay and Amazon auctions on the Internet.  _American Economic Review_ 92, 1093–1103 (2002).
  * [83] Ely, J. C. & Hossain, T.  Sniping and squatting in auction markets.  _American Economic Journal: Microeconomics_ 1, 68–94 (2009).
  * [84] Gray, S. & Reiley, D. H.  Measuring the benefits to sniping on eBay: Evidence from a field experiment.  _Journal of Economics and Management_ 9, 137–152 (2013).
  * [85] Backus, M., Blake, T., Masterov, D. V. & Tadelis, S.  Is sniping a problem for online auction markets?  In _Proceedings of the 24th International Conference on World Wide Web_ , WWW ’15, 88–96 (International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 2015).
  * [86] Aparicio, D. & Misra, K.  _Artificial intelligence and pricing_ , vol. 20 (Emerald Publishing Limited, 2023).
  * [87] Chen, L., Mislove, A. & Wilson, C.  An empirical analysis of algorithmic pricing on amazon marketplace.  In _Proceedings of the 25th international conference on World Wide Web_ , 1339–1349 (2016).
  * [88] Garcia, D., Tolvanen, J. & Wagner, A. K.  Strategic responses to algorithmic recommendations: Evidence from hotel pricing.  Tech. Rep. (2023).  CESifo Working Paper.
  * [89] Hunold, M. & Werner, T.  Algorithmic price recommendations and collusion: Experimental evidence.  _Available at SSRN_ (2023).
  * [90] Assad, S., Clark, R., Ershov, D. & Xu, L.  Algorithmic pricing and competition: Empirical evidence from the german retail gasoline market (2023).  Forthcoming.
  * [91] Calvano, E., Calzolari, G., Denicolò, V. & Pastorello, S.  Artificial intelligence, algorithmic pricing, and collusion.  _American Economic Review_ 110, 3267–3297 (2020).
  * [92] Calvano, E., Calzolari, G., Denicolò, V., Harrington, J. E. & Pastorello, S.  Protecting consumers from collusive prices due to AI.  _Science_ 370, 1040–1042 (2020).
  * [93] Klein, T.  Autonomous algorithmic collusion: Q-learning under sequential pricing.  _The RAND Journal of Economics_ 52, 538–558 (2021).
  * [94] Johnson, J. P., Rhodes, A. & Wildenbeest, M.  Platform design when sellers use pricing algorithms.  _Econometrica_ 91, 1841–1879 (2023).
  * [95] Werner, T.  Algorithmic and human collusion (2022).
  * [96] Normann, H.-T. & Sternberg, M.  Human-algorithm interaction: Algorithmic pricing in hybrid laboratory markets.  _European Economic Review_ 152, 104347 (2023).
  * [97] Musolff, L.  Algorithmic pricing facilitates tacit collusion: Evidence from e-commerce.  In _Proceedings of the 23rd ACM Conference on Economics and Computation_ , 32–33 (2022).
  * [98] Wieting, M. & Sapi, G.  Algorithms in the marketplace: An empirical analysis of automated pricing in e-commerce.  _Available at SSRN 3945137_ (2021).
  * [99] Miklós-Thal, J. & Tucker, C.  Collusion by algorithm: Does better demand prediction facilitate coordination between sellers?  _Management Science_ 65, 1552–1561 (2019).
  * [100] O’Connor, J. & Wilson, N. E.  Reduced demand uncertainty and the sustainability of collusion: How AI could affect competition.  _Information Economics and Policy_ 54, 100882 (2021).
  * [101] Martin, S. & Rasch, A.  Demand forecasting, signal precision, and collusion with hidden actions.  _International Journal of Industrial Organization_ 92, 103036 (2024).
  * [102] Brown, Z. Y. & MacKay, A.  Competition in pricing algorithms (2021).  28860.
  * [103] Leisten, M.  Algorithmic competition, with humans.  Tech. Rep. (2022).  Working Paper.
  * [104] Menkveld, A. J.  The economics of high-frequency trading: Taking stock.  _Annual Review of Financial Economics_ 8, 1–24 (2016).
  * [105] Ullmann-Margalit, E.  _The Emergence of Norms_ (OUP Oxford, 2015).
  * [106] Young, H. P.  The evolution of conventions.  _Econometrica_ 61, 57–84 (1993).
  * [107] Shirado, H. & Christakis, N. A.  Locally noisy autonomous agents improve global human coordination in network experiments.  _Nature_ 545, 370–374 (2017).
  * [108] Santos, F. P., Pacheco, J. M., Paiva, A. & Santos, F. C.  Evolution of collective fairness in hybrid populations of humans and agents.  _Proceedings of the AAAI Conference on Artificial Intelligence_ 33, 6146–6153 (2019).
  * [109] Shen, C., He, Z., Shi, L., Wang, Z. & Tanimoto, J.  Simple bots breed social punishment in humans (2022).  2211.13943.
  * [110] Sharma, G., Guo, H., Shen, C. & Tanimoto, J.  Small bots, big impact: Solving the conundrum of cooperation in optional Prisoner’s Dilemma game through simple strategies.  _Journal of The Royal Society Interface_ 20, 20230301 (2023).  2305.15818.
  * [111] Suri, S. & Watts, D. J.  Cooperation and contagion in web-based, networked public goods experiments.  _PLOS ONE_ 6, e16836 (2011).
  * [112] Fernández Domingos, E. _et al._ Delegation to artificial agents fosters prosocial behaviors in the collective risk dilemma.  _Scientific Reports_ 12, 8492 (2022).
  * [113] Kirchkamp, O. & Nagel, R.  Naive learning and cooperation in network experiments.  _Games and Economic Behavior_ 58, 269–292 (2007).
  * [114] Shirado, H. & Christakis, N. A.  Network engineering using autonomous agents increases cooperation in human groups.  _iScience_ 23, 101438 (2020).
  * [115] Centola, D.  _How Behavior Spreads: The Science of Complex Contagions_ (Princeton University Press, Princeton; Oxford, 2018).
  * [116] Christakis, N. A. & Fowler, J. H.  Social contagion theory: Examining dynamic social networks and human behavior.  _Statistics in Medicine_ 32, 556–577 (2013).
  * [117] Rogers, E. M.  _Diffusion of Innovations_ (Simon and Schuster, 2003), 5th edition edn.
  * [118] Cialdini, R. B.  _Influence: Science and Practice_ (Allyn & Bacon, Boston, MA, 2008).
  * [119] Deutsch, M. & Gerard, H. B.  A study of normative and informational social influences upon individual judgment.  _The Journal of Abnormal and Social Psychology_ 51, 629–636 (1955).
  * [120] Turner, J. C.  _Social Influence_.  Social Influence (Thomson Brooks/Cole Publishing Co, Belmont, CA, US, 1991).
  * [121] Fowler, J. H. & Christakis, N. A.  Cooperative behavior cascades in human social networks.  _Proceedings of the National Academy of Sciences_ 107, 5334–5338 (2010).
  * [122] Leskovec, J., Adamic, L. A. & Huberman, B. A.  The dynamics of viral marketing.  _ACM Transactions on the Web_ 1, 5–es (2007).
  * [123] Watts, D. J.  A simple model of global cascades on random networks.  _Proceedings of the National Academy of Sciences_ 99, 5766–5771 (2002).
  * [124] Keijzer, M. A. & Mäs, M.  The strength of weak bots.  _Online Social Networks and Media_ 21, 100106 (2021).
  * [125] Stewart, A. J. _et al._ Information gerrymandering and undemocratic decisions.  _Nature_ 573, 117–121 (2019).
  * [126] Strohkorb Sebo, S., Traeger, M., Jung, M. & Scassellati, B.  The ripple effects of vulnerability: The effects of a robot’s vulnerable behavior on trust in human-robot teams.  In _Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction_ , HRI ’18, 178–186 (Association for Computing Machinery, New York, NY, USA, 2018).
  * [127] Traeger, M. L., Strohkorb Sebo, S., Jung, M., Scassellati, B. & Christakis, N. A.  Vulnerable robots positively shape human conversational dynamics in a human–robot team.  _Proceedings of the National Academy of Sciences_ 117, 6370–6375 (2020).
  * [128] Zhang, A. W., Lin, T.-H., Zhao, X. & Sebo, S.  Ice-breaking technology: Robots and computers can foster meaningful connections between strangers through in-person conversations.  In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_ , CHI ’23, 1–14 (Association for Computing Machinery, New York, NY, USA, 2023).
  * [129] Bang, D. & Frith, C. D.  Making better decisions in groups.  _Royal Society Open Science_ 4, 170193 (2017).
  * [130] Galton, F.  Vox Populi.  _Nature_ 75, 450–451 (1907).
  * [131] Surowiecki, J.  _The Wisdom of Crowds_ (Anchor, New York, NY, 2005), reprint edition edn.
  * [132] Page, S.  _The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies_ (Princeton University Press, 2008).
  * [133] Frey, V. & van de Rijt, A.  Social influence undermines the wisdom of the crowd in sequential decision making.  _Management Science_ 67, 4273–4286 (2021).
  * [134] Lorenz, J., Rauhut, H., Schweitzer, F. & Helbing, D.  How social influence can undermine the wisdom of crowd effect.  _Proceedings of the National Academy of Sciences_ 108, 9020–9025 (2011).
  * [135] Muchnik, L., Aral, S. & Taylor, S. J.  Social influence bias: A randomized experiment.  _Science_ 341, 647–651 (2013).
  * [136] Becker, J., Brackbill, D. & Centola, D.  Network dynamics of social influence in the wisdom of crowds.  _Proceedings of the National Academy of Sciences_ 114, E5070–E5076 (2017).
  * [137] Navajas, J., Niella, T., Garbulsky, G., Bahrami, B. & Sigman, M.  Aggregated knowledge from a small number of debates outperforms the wisdom of large crowds.  _Nature Human Behaviour_ 2, 126–132 (2018).
  * [138] Choi, S., Kang, H., Kim, N. & Kim, J.  How does AI improve human decision-making? Evidence from the AI-powered Go program (2023).
  * [139] Shin, M., Kim, J. & Kim, M.  Human learning from Artificial Intelligence: Evidence from human Go players’ decisions after AlphaGo.  _Proceedings of the Annual Meeting of the Cognitive Science Society_ 43 (2021).
  * [140] Shin, M., Kim, J., van Opheusden, B. & Griffiths, T. L.  Superhuman artificial intelligence can improve human decision-making by increasing novelty.  _Proceedings of the National Academy of Sciences_ 120, e2214840120 (2023).
  * [141] Brinkmann, L. _et al._ Hybrid social learning in human-algorithm cultural transmission.  _Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences_ 380, 20200426 (2022).
  * [142] Pescetelli, N., Barkoczi, D. & Cebrian, M.  Bots influence opinion dynamics without direct human-bot interaction: The mediating role of recommender systems.  _Applied Network Science_ 7, 1–19 (2022).
  * [143] Dellermann, D., Ebel, P., Söllner, M. & Leimeister, J. M.  Hybrid intelligence.  _Business & Information Systems Engineering_ 61, 637–643 (2019).
  * [144] Wiethof, C. & Bittner, E.  Hybrid intelligence-combining the human in the loop with the computer in the loop: A systematic literature review.  In _Forty-Second International Conference on Information Systems, Austin_ , 1–17 (2021).
  * [145] Hekler, A. _et al._ Superior skin cancer classification by the combination of human and artificial intelligence.  _European Journal of Cancer_ 120, 114–121 (2019).
  * [146] Tschandl, P. _et al._ Human–computer collaboration for skin cancer recognition.  _Nature Medicine_ 26, 1229–1234 (2020).
  * [147] Wright, D. E. _et al._ A transient search using combined human and machine classifications.  _Monthly Notices of the Royal Astronomical Society_ 472, 1315–1323 (2017).
  * [148] Bowyer, A., Maidel, V., Lintott, C., Swanson, A. & Miller, G.  This image intentionally left blank: Mundane images increase citizen science participation.  In _2015 Conference on Human Computation & Crowdsourcing. Presented at the Conference on Human Computation & Crowdsourcing, San Diego, California, United States_, vol. 460 (2015).
  * [149] Trouille, L., Lintott, C. J. & Fortson, L. F.  Citizen science frontiers: Efficiency, engagement, and serendipitous discovery with human–machine systems.  _Proceedings of the National Academy of Sciences_ 116, 1902–1909 (2019).
  * [150] Hagströmer, B. & Nordén, L.  The diversity of high-frequency traders.  _Journal of Financial Markets_ 16, 741–770 (2013).
  * [151] Brogaard, J., Hendershott, T. & Riordan, R.  High-frequency trading and price discovery.  _The Review of Financial Studies_ 27, 2267–2306 (2014).
  * [152] Hirschey, N.  Do high-frequency traders anticipate buying and selling pressure?  _Management Science_ 67, 3321–3345 (2021).
  * [153] Chaboud, A. P., Chiquoine, B., Hjalmarsson, E. & Vega, C.  Rise of the machines: Algorithmic trading in the foreign exchange market.  _The Journal of Finance_ 69, 2045–2084 (2014).
  * [154] Hendershott, T., Jones, C. M. & Menkveld, A. J.  Does algorithmic trading improve liquidity?  _The Journal of Finance_ 66, 1–33 (2011).
  * [155] Hasbrouck, J. & Saar, G.  Low-latency trading.  _Journal of Financial Markets_ 16, 646–679 (2013).
  * [156] Boehmer, E., Fong, K. & Wu, J.  International evidence on algorithmic trading.  In _AFA 2013 San Diego Meetings Paper_ (2012).
  * [157] Johnson, N. _et al._ Abrupt rise of new machine ecology beyond human response time.  _Scientific Reports_ 3, 2627 (2013).
  * [158] Budish, E., Cramton, P. & Shim, J.  The high-frequency trading arms race: Frequent batch auctions as a market design response.  _The Quarterly Journal of Economics_ 130, 1547–1621 (2015).
  * [159] Beskow, D. M. & Carley, K. M.  Bot conversations are different: Leveraging network metrics for bot detection in Twitter.  In _2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM)_ , 825–832 (2018).
  * [160] Cresci, S.  A decade of social bot detection.  _Communications of the ACM_ 63, 72–83 (2020).
  * [161] Davis, C. A., Varol, O., Ferrara, E., Flammini, A. & Menczer, F.  BotOrNot: A system to evaluate social bots.  In _Proceedings of the 25th International Conference Companion on World Wide Web_ , WWW ’16 Companion, 273–274 (International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 2016).
  * [162] Duh, A., Slak Rupnik, M. & Korošak, D.  Collective behavior of social bots is encoded in their temporal Twitter activity.  _Big Data_ 6, 113–123 (2018).
  * [163] Orabi, M., Mouheb, D., Al Aghbari, Z. & Kamel, I.  Detection of bots in social media: A systematic review.  _Information Processing & Management_ 57, 102250 (2020).
  * [164] Varol, O., Ferrara, E., Davis, C., Menczer, F. & Flammini, A.  Online human-bot interactions: Detection, estimation, and characterization.  _Proceedings of the International AAAI Conference on Web and Social Media_ 11, 280–289 (2017).
  * [165] Bastos, M. T. & Mercea, D.  The Brexit botnet and user-generated hyperpartisan news.  _Social Science Computer Review_ 37, 38–54 (2019).
  * [166] Grimme, C., Preuss, M., Adam, L. & Trautmann, H.  Social bots: Human-like by means of human control?  _Big Data_ 5, 279–293 (2017).
  * [167] Chu, Z., Gianvecchio, S., Wang, H. & Jajodia, S.  Detecting automation of Twitter accounts: Are you a human, bot, or cyborg?  _IEEE Transactions on Dependable and Secure Computing_ 9, 811–824 (2012).
  * [168] Stella, M., Ferrara, E. & De Domenico, M.  Bots increase exposure to negative and inflammatory content in online social systems.  _Proceedings of the National Academy of Sciences_ 115, 12435–12440 (2018).
  * [169] Pozzana, I. & Ferrara, E.  Measuring bot and human behavioral dynamics.  _Frontiers in Physics_ 8 (2020).
  * [170] Hwang, T., Pearce, I. & Nanis, M.  Socialbots: Voices from the fronts.  _Interactions_ 19, 38–45 (2012).
  * [171] Ferrara, E.  Disinformation and social bot operations in the run up to the 2017 French presidential election.  _First Monday_ (2017).
  * [172] Forelle, M., Howard, P., Monroy-Hernández, A. & Savage, S.  Political bots and the manipulation of public opinion in Venezuela (2015).  1507.07109.
  * [173] Howard, P., Woolley, S. & Calo, R.  Algorithms, bots, and political communication in the US 2016 election: The challenge of automated political communication for election law and administration.  _Journal of Information Technology and Politics_ 15, 81–93 (2018).
  * [174] Howard, P. N. & Kollanyi, B.  Bots, #strongerIn, and #brexit: Computational propaganda during the UK-EU referendum (2016).  1606.06356.
  * [175] Shao, C. _et al._ The spread of low-credibility content by social bots.  _Nature Communications_ 9, 4787 (2018).
  * [176] Suárez-Serrato, P., Roberts, M. E., Davis, C. & Menczer, F.  On the influence of social bots in online protests.  In Spiro, E. & Ahn, Y.-Y. (eds.) _Social Informatics_ , Lecture Notes in Computer Science, 269–278 (Springer International Publishing, Cham, 2016).
  * [177] Yan, H. Y., Yang, K.-C., Shanahan, J. & Menczer, F.  Exposure to social bots amplifies perceptual biases and regulation propensity.  _Scientific Reports_ 13, 20707 (2023).
  * [178] Himelein-Wachowiak, M. _et al._ Bots and misinformation spread on social media: Implications for COVID-19.  _Journal of Medical Internet Research_ 23, e26933 (2021).
  * [179] Yang, K.-C., Torres-Lugo, C. & Menczer, F.  Prevalence of low-credibility information on Twitter during the COVID-19 outbreak (2020).  2004.14484.
  * [180] Fan, R., Talavera, O. & Tran, V.  Social media bots and stock markets.  _European Financial Management_ 26, 753–777 (2020).
  * [181] Stella, M., Cristoforetti, M. & Domenico, M. D.  Influence of augmented humans in online interactions during voting events.  _PLOS ONE_ 14, e0214210 (2019).
  * [182] Vosoughi, S., Roy, D. & Aral, S.  The spread of true and false news online.  _Science_ 359, 1146–1151 (2018).
  * [183] Gorodnichenko, Y., Pham, T. & Talavera, O.  Social media, sentiment and public opinions: Evidence from #Brexit and #USElection.  _European Economic Review_ 136, 103772 (2021).
  * [184] Bessi, A. & Ferrara, E.  Social bots distort the 2016 US presidential election online discussion (2016).
  * [185] BBC News.  Twitter ’shuts down millions of fake accounts’.  _BBC News_ (2018).
  * [186] Dang, S. & Paul, K.  Twitter says it removes over 1 million spam accounts each day.  _Reuters_ (2022).
  * [187] Halfaker, A. & Riedl, J.  Bots and cyborgs: Wikipedia’s immune system.  _Computer_ 45, 79–82 (2012).
  * [188] Niederer, S. & van Dijck, J.  Wisdom of the crowd or technicity of content? Wikipedia as a sociotechnical system.  _New Media & Society_ 12, 1368–1387 (2010).
  * [189] Zheng, L. N., Albano, C. M., Vora, N. M., Mai, F. & Nickerson, J. V.  The roles bots play in Wikipedia.  _Proceedings of the ACM on Human-Computer Interaction_ 3, 215:1–215:20 (2019).
  * [190] Geiger, R. S.  The lives of bots.  In Lovink, G. & Tkacz, N. (eds.) _Critical Point of View: A Wikipedia Reader_ , 78–93 (Institute of Network Cultures, Amsterdam, 2011).
  * [191] Steiner, T.  Bots vs. Wikipedians, anons vs. logged-ins (redux): A global study of edit activity on Wikipedia and Wikidata.  In _Proceedings of The International Symposium on Open Collaboration_ , OpenSym ’14, 1–7 (Association for Computing Machinery, New York, NY, USA, 2014).
  * [192] Geiger, R. S. & Halfaker, A.  Operationalizing conflict and cooperation between automated software agents in Wikipedia: A replication and expansion of ’Even Good Bots Fight’.  _Proceedings of the ACM on Human-Computer Interaction_ 1, 49:1–49:33 (2017).
  * [193] Clément, M. & Guitton, M. J.  Interacting with bots online: Users’ reactions to actions of automated programs in Wikipedia.  _Computers in Human Behavior_ 50, 66–75 (2015).
  * [194] Geiger, R. S. & Halfaker, A.  When the levee breaks: Without bots, what happens to Wikipedia’s quality control processes?  In _Proceedings of the 9th International Symposium on Open Collaboration_ , WikiSym ’13, 1–6 (Association for Computing Machinery, New York, NY, USA, 2013).
  * [195] Hilbert, M. & Darmon, D.  Large-scale communication is more complex and unpredictable with automated bots.  _Journal of Communication_ 70, 670–692 (2020).
  * [196] Massanari, A. L.  Contested play: The culture and politics of Reddit bots.  In _Socialbots and Their Friends_ (Routledge, 2016).
  * [197] Hurtado, S., Ray, P. & Marculescu, R.  Bot detection in Reddit political discussion.  In _Proceedings of the Fourth International Workshop on Social Sensing_ , SocialSense’19, 30–35 (Association for Computing Machinery, New York, NY, USA, 2019).
  * [198] Jhaver, S., Birman, I., Gilbert, E. & Bruckman, A.  Human-machine collaboration for content regulation: The case of Reddit automoderator.  _ACM Transactions on Computer-Human Interaction_ 26, 31:1–31:35 (2019).
  * [199] Ma, M.-C. & Lalor, J. P.  An empirical analysis of human-bot interaction on Reddit.  In _Proceedings of the Sixth Workshop on Noisy User-generated Text (W-NUT 2020)_ , 101–106 (Association for Computational Linguistics, Online, 2020).
  * [200] Ray, P. P.  ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope.  _Internet of Things and Cyber-Physical Systems_ 3, 121–154 (2023).
  * [201] Webb, T., Holyoak, K. J. & Lu, H.  Emergent analogical reasoning in large language models.  _Nature Human Behaviour_ 7, 1526–1541 (2023).
  * [202] Aiello, L. M., Deplano, M., Schifanella, R. & Ruffo, G.  People are strange when you’re a stranger: Impact and influence of bots on social networks.  _Proceedings of the International AAAI Conference on Web and Social Media_ 6, 10–17 (2012).
  * [203] Freitas, C., Benevenuto, F., Ghosh, S. & Veloso, A.  Reverse engineering socialbot infiltration strategies in Twitter.  In _Proceedings of the 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2015_ , ASONAM ’15, 25–32 (Association for Computing Machinery, New York, NY, USA, 2015).
  * [204] Messias, J., Schmidt, L., Oliveira, R. A. R. & de Souza, F. B.  You followed my bot! Transforming robots into influential users in Twitter.  _First Monday_ 18, 1–14 (2013).
  * [205] Savage, S., Monroy-Hernandez, A. & Höllerer, T.  Botivist: Calling volunteers to action using online bots.  In _Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing_, CSCW ’16, 813–822 (Association for Computing Machinery, New York, NY, USA, 2016).
  * [206] Krafft, P. M., Della Penna, N. & Pentland, A. S.  An experimental study of cryptocurrency market dynamics.  In _Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems_ , CHI ’18, 1–13 (Association for Computing Machinery, New York, NY, USA, 2018).
  * [207] Bail, C. A. _et al._ Exposure to opposing views on social media can increase political polarization.  _Proceedings of the National Academy of Sciences_ 115, 9216–9221 (2018).
  * [208] Chen, C., Li, G., Fan, L. & Qin, J.  The impact of automated investment on peer-to-peer lending: Investment behavior and platform efficiency.  _Journal of Global Information Management (JGIM)_ 29, 1–22 (2021).
  * [209] Lorenz, T.  Welcome to the age of automated dating.  _Washington Post_ (2023).
  * [210] Salganik, M. J.  _Bit by Bit: Social Research in the Digital Age_ (Princeton University Press, 2019).
  * [211] Chang, D.  Texas drivers are furious as 20 Cruise self-driving cars cause jam.  https://www.dailymail.co.uk/news/article-12550179/Texas-drivers-furious-20-Cruises-gridlock-Austin.html (2023).
  * [212] Asimov, I.  _I, Robot_ , vol. 1 (Spectra, [1950] 2004).
  * [213] Graham, T. & Ackland, R.  Do socialbots dream of popping the filter bubble? The role of socialbots in promoting deliberative democracy in social media.  In Gehl, R. W. & Bakardjieva, M. (eds.) _Socialbots and Their Friends: Digital Media and the Automation of Sociality_ , 187–206 (Routledge, United States of America, 2017).
  * [214] Awad, E. _et al._ The Moral Machine experiment.  _Nature_ 563, 59–64 (2018).
  * [215] Kenway, E.  ‘Care bots’: A dream for carers or a dangerous fantasy?  _The Observer_ (2023).
  * [216] Brinkmann, L. _et al._ Machine culture.  _Nature Human Behaviour_ 7, 1855–1868 (2023).
  * [217] Paré, G. & Kitsiou, S.  Chapter 9: Methods for literature reviews.  In Lau, F. & Kuziemsky, C. (eds.) _Handbook of eHealth Evaluation: An Evidence-based Approach_ , 157–178 (University of Victoria, Victoria, British Columbia, 2017).
  * [218] Sylvester, A., Tate, M. & Johnstone, D.  Beyond synthesis: Re-presenting heterogeneous research literature.  _Behaviour & Information Technology_ 32, 1199–1215 (2013).
  * [219] Whittemore, R. & Knafl, K.  The integrative review: Updated methodology.  _Journal of Advanced Nursing_ 52, 546–553 (2005).