# Competition and Collaboration in Crowdsourcing Communities: What happens
when peers evaluate each other?

Forthcoming in Organization Science, 2024

Christoph Riedla∗, Tom Gradb, Christopher Lettlc
aD’Amore-McKim School of Business, Northeastern University, Boston, MA
bDepartment of Strategy and Innovation, Copenhagen Business School, 2000
Frederiksberg, Denmark
vDepartment of Strategy and Innovation, Vienna University of Economics and
Business, 1020 Vienna, Austria

∗Corresponding author<EMAIL_ADDRESS>

###### Abstract

Crowdsourcing has evolved as an organizational approach to distributed problem
solving and innovation. As contests are embedded in online communities and
evaluation rights are assigned to the crowd, community members face a tension:
they find themselves exposed to both competitive motives to win the contest
prize and collaborative participation motives in the community. The
competitive motive suggests they may evaluate rivals strategically according
to their self-interest, the collaborative motive suggests they may evaluate
their peers truthfully according to mutual interest. Using field data from
Threadless on 38 million peer evaluations of more than 150,000 submissions
across 75,000 individuals over10 years and two natural experiments to rule out
alternative explanations, we answer the question of how community members
resolve this tension. We show that as their skill level increases, they become
increasingly competitive and shift from using self-promotion to sabotaging
their closest competitors. However, we also find signs of collaborative
behavior when high-skilled members show leniency toward those community
members who do not directly threaten their chance of winning. We explain how
the individual-level use of strategic evaluations translates into important
organizational-level outcomes by affecting the community structure through
individuals’ long-term participation. While low-skill targets of sabotage are
less likely to participate in future contests, high-skill targets are more
likely. This suggests a feedback loop between competitive evaluation behavior
and future participation. These findings have important implications for the
literature on crowdsourcing design, and the evolution and sustainability of
crowdsourcing communities.

Keywords: crowdsourcing, online communities, collaboration and competition,
self-promotion, sabotage

##  1 Introduction

Crowdsourcing as an organizational approach to distributed problem solving and
innovation (Afuah and Tucci, 2023) has come a long way. Originally conceived
as an approach through which firms solicit contributions from independent
individuals the past decade has seen a significant evolution in crowdsourcing
design. Two aspects stand out. First, crowdsourcing contests are now often
embedded in online communities (e.g., Jeppesen and Lakhani, 2010; Riedl and
Seidel, 2018; Grad et al., 2023; Zaggl et al., 2023; Riedl et al., 2024).
Second, many of these crowdsourcing communities have adopted open
participation architectures that assign evaluation rights to the crowd
(Majchrzak and Malhotra, 2013; Dahlander et al., 2019; Blohm et al., 2016).

When community members are asked to evaluate their rivals in contest-based
communities, they face a tension as they find themselves exposed to both
competitive motives to win the contest prize and collaborative participation
motives in the community (Deutsch, 1949; Fiske, 1992). Community members may
evaluate their rivals fairly according to the collaborative ideal of the
community (Fiske, 1992, calls this equality matching) or they may act in self-
interest and evaluate them strategically in order to maximize their own
chances of winning the contest (market pricing). That is, the incentives
designed to encourage effort during idea generation (i.e., the contest prize)
introduce a competitive element in the otherwise collaborative idea-evaluation
process. This may thus tempt idea generators to violate collaborative
community norms and evaluate ideas in strategic self-interest. Embedding
contests in online communities appears to have created a mismatch between
organizational context and incentives (Gallus et al., 2022): the
organizational context is collaborative yet the incentives are competitive.
Such incongruent incentive schemes can backfire with unintended consequences
and lower participation. It is not clear how individuals evaluate their peers
as they face this tension between competitive and collaborative motives; and
in case they act in self-interest and evaluate strategically: who do they
target and with what consequences?

Collaborative aspects of mutual interest, such as helping behavior,
reciprocity, and a desire for fairness (Wasko and Faraj, 2000; Gebauer et al.,
2013; Franke and Shah, 2003; Bauer et al., 2016) are key characteristics of
communities which bring individuals with shared interests in member or social
welfare together (Faraj and Johnson, 2011; Majchrzak and Malhotra, 2013).
Shared values and norms play a crucial role in sustaining good community
citizenship and mutually interested collaborative behavior (Chiu et al., 2015;
Ivaturi and Chua, 2019). Individuals may consequently choose to evaluate
fairly and truthfully along the meritocratic ideal. On the other hand, when
contests are embedded in such communities, the self-interested motivation to
win the contest prize is in conflict with the mutually interested
participation motivation of the community. The economics literature on
strategic behavior in contests (see Konrad, 2009, for a summary) provides
precise predictions for the self-interested evaluation behavior we may expect
to see in crowdsourcing contests. It predicts that competitors engage in
sabotage—defined as the effort of one individual that reduces the performance
of another—and self-promotion—a form of unproductive effort that makes one’s
own contributions appear better without increasing the quality of the
contributions itself (Lazear and Rosen, 1981; Magee and Galinsky, 2008;
Konrad, 2009).111In terms of rating behavior this corresponds to rate oneself
up (self-promotion) and to rate competitors down (sabotage). There are other
forms of strategic behavior such as strategic entry decisions (Liu et al.,
2014), cheap talk (Archak, 2010), and reciprocal rating (Hutter et al., 2011).
However, it is not at all clear if these predictions of self-interested
behavior from contest theory bear out given the community setting. Indeed,
research has shown that community members sometimes act collaboratively even
when they are direct competitors (Franke and Shah, 2003; Harhoff et al., 2003)
and that they are sometimes overly positive in their peer evaluations (Aadland
et al., 2019; Klapper et al., 2023). Investigating sabotage and self-promotion
together is crucial. While sabotage decreases the quality signal,
(self-)promotion increases it and the two may cancel each other out. Sabotage
also comes with an externality: beyond the saboteur it also increases the
chances of winning for all other competitors (Konrad, 2009). As a result, the
presence of one form of strategic behavior affects the effectiveness of the
other. Community members will therefore consider the two forms of strategic
behavior jointly as they attempt to resolve the tension between competition
and cooperation.

Past work has repeatedly identified such conflicting competitive and
collaborative motivations underlying behavior both in online communities and
social situations more broadly (Deutsch, 1949; Lewis, 2000). Self-interested
behavior refers to actions of an individual or entity being conducted for the
sole purpose of achieving personal benefits (Smith, 2010; Cropanzano et al.,
2005). In contrast, collaborative behavior refers to actions of individuals or
entities being conducted in pursuit of a common goal and thus for the purpose
of achieving mutual benefits (Deutsch, 1949; Fjeldstad et al., 2012). Examples
of collaborative behavior include knowledge sharing and contributing to joint
problem solving, providing fair peer evaluations, encouraging new members, or
the maintenance of shared resources (Li et al., 2007; Chambers and Baker,
2020; Chiu et al., 2015). These self-interested (competitive) and mutual-
interested (collaborative) motivations are often simultaneously present and
they interact with each other in complex ways (e.g., Adler and Chen, 2011;
Franke and Shah, 2003; Bauer et al., 2016; Bullinger et al., 2010; Hutter et
al., 2011; Nambisan and Baron, 2010; Roberts et al., 2006; Majchrzak and
Malhotra, 2013; Chambers and Baker, 2020).

However, existing studies do not offer precise predictions of how individuals
resolve tensions between competitive and collaborative participation motives
when asked to evaluate peers in crowdsourcing contests in which they are
competing. Recent research on crowdsourcing communities has started to
acknowledge the existence of strategic behavior (e.g., Hofstetter et al.,
2018a; Archak and Sundararajan, 2009; Liu et al., 2014; Hutter et al., 2011;
Chen et al., 2020; Deodhar et al., 2022). Notably, Klapper et al. (2023)
highlight that peer evaluations, when transparent, offer the possibility for
strategic behavior as they can provide individuals with a platform to shape
their own reputation. However, their work is focussed on non-competitive
environments with transparent evaluations which is only the case in some peer
evaluation settings.

Therefore we do not fully understand how and why strategic behaviors arise,
its dynamics, nor how it affects the structure of the crowdsourcing community
via long-term participation (cf. Balietti and Riedl, 2021) when evaluations
are anonymous and competitive. Thus, we complement existing research by
offering insights for these manifestations of peer evaluations. The covert
nature combined with competition is likely to spur an additional strategic
behavior beyond the self-promotion found in prior studies (Edelman and Larkin,
2015; Klapper et al., 2023), namely sabotage of competitors. Thus, we provide
a more comprehensive view on strategic behaviors in crowdsourcing communities
that differentiates between self-promotion and sabotage and investigates their
complementary effect.

Our paper addresses two research questions: (1) How do participants in
crowdsourcing communities resolve the tension between the competitive and
collaborative participation motive when asked to evaluate their peers? (2) How
does the way participants resolve the tension change the composition of the
community when some individuals are more motivated by the competitive aspects
of the contest? A core moderator for self-interested behavior suggested by
contest theory is skill222Skill in terms of generating high-quality contest
entries not in terms of acting strategically, although the two may be
correlated. because it affects the chance to win the contest and thus affects
the gain an individual can expect from making strategic peer evaluations
(Boudreau et al., 2016). This suggests that the strength of competitive and
collaborative motives may depend on skill and that the tension is greatest for
highly skilled participants who are most likely to win the contest, while it
is less acute for those of lower skill who know they are unlikely to win.

We investigate these questions using longitudinal panel data from a leading
crowdsourcing community (Threadless), analyzing more than 38 million peer
evaluations of 150,000 ideas by 75,000 individuals over 10 years. Using self-
interested behavior predicted by contest theory as a framework, we look at
self-promotion, sabotage, and skill as a moderator to identify both the
culprits as well as the targets of strategic behavior. We then draw on the
rich online community literature to explain collaborative behavior that is
currently not well explained by contest theory. We also explain how the
individual-level use of strategic evaluations translates into important
organizational-level outcomes by affecting the community structure through
individuals’ long-term participation.

We have two main findings. First, we find behavior that is both consistent
with self interest and mutual interest in a rich and nuanced way. Consistent
with self-interested behavior predicted by contest theory, we find lower-
skilled individuals do not sabotage but self-promote. As their skill level
increases, they increasingly adopt more competitive strategies and shift from
using self-promotion to sabotaging their closest competitors (other high-skill
individuals). On the other hand, we also find signs of collaborative behavior
when they show leniency toward those community members who do not threaten
their chance of winning. Using insights from two natural experiments—an
evaluation rule change and a change in incentives—we rule out alternative
explanations and establish that the observed behavior is indeed strategically
motivated. Second, we find that the future participation of the targets of
sabotage is potentially affected, depending on their skill level. Low-skill
targets of sabotage are less likely to participate in future contests, while
high-skill targets are more likely. This suggests a feedback loop between
competitive evaluation behavior and future participation. Individuals
increasingly act strategically as their skill level increases, and they find
the resulting fierce competition so engaging that they increase their future
participation even though it makes them the targets of sabotage.

Our paper makes three key contributions. First, we extend prior research on
the competitive and collaborative nature in crowdsourcing (Hutter et al.,
2011; Franke and Shah, 2003; Bauer et al., 2016; Nambisan and Baron, 2010).
Our theorizing explains why community members sometimes wear the competitive
hat rather than the collaborative one when they evaluate their peers.
Specifically, we theorize that leniency is a crucial collaborative element
that allows individuals to better justify self-interested strategic
evaluations through a form of moral licensing (Blanken et al., 2015). Second,
we challenge the assumption in much community research that the tension
between competitive and collaborative participation motives is a tension
across individuals due to stable attributes (some people are competitive while
others are collaborative; Lakhani and Wolf, 2003; Reuben et al., 2015; Erat
and Gneezy, 2012; Belenzon and Schankerman, 2015) by showing that the tension
is instead context specific and can be described coherently based on the
competitiveness of the situation (which depends on the skill of the
individuals involved). Third, our work contributes to our understanding of
important organizational-level outcomes by showing how ostensible negative
strategic behavior can have positive long-term effects. We theorize how the
self-reinforcing dynamic of strategic behavior affects the social structure of
communities (Kim et al., 2018; Faraj and Johnson, 2011; Huang et al., 2014;
Piezunka and Dahlander, 2019; Hofstetter et al., 2018b).

##  2 Contest-Theoretic Intuition

How would evaluation behavior look like in a world without community in which
evaluators act only according to self interest? In this section, we develop a
simple theoretical framework to formalize predictions of self-interested
evaluation behavior for heterogeneous participants in crowdsourcing contests
with an open participation architecture. Like much formal modeling work, our
model is a simplified version of what we might expect going on in real
contests but which can help build credible behavioral foundations about
mechanisms that may explain observed behavior (Knudsen et al., 2019).

###  2.1 Background

Sabotage and self-promotion are conceptually similar in that they affect the
relative ranking within contests: since the contest winner is determined based
on the relative rank-order of contestants, any action that increases the
likelihood to win for one contestant by necessity implies that the likelihood
of other contestants is reduced. The economics literature points to one
important difference between self-promotion and sabotage with regard to their
effect on the relative ranking (Konrad, 2009). Self-promotion affects the
likelihood of the culprit to win relative to everyone else (the relative
likelihood to win among all others remains the same) while sabotage has an
important externality. The action of the culprit affects the relative
likelihood of the target and all other contestants to win: everyone’s
likelihood to win is affected, not only the likelihood of the culprit
committing the sabotage. As we will show in our economic model, this
distinction has important implications on the cost-benefit calculation to
determine which form of strategic behavior to employ. In the Appendix (Section
A) we provide a brief overview of the economics literature on strategic
behavior. There we show that while sabotage is well understood theoretically,
empirical insights are scarce (especially with regard to the effect of skill)
and that so far prior work has not modeled sabotage and self-promotion
together. It is thus unclear how the two affect each other.

###  2.2 Basic Model Setup

To keep the modeling tractable and the exposition simple, we build on previous
models of sabotage in a single contest (so call one-shot contest), actors that
are differentiated by skill, and a single winner prize (see, e.g., Harbring
and Irlenbusch, 2011; Konrad, 2000). We term the participants in the contest
“agents”. The contest consists of three types of agents: high type agents, low
type agents, and neutral outsiders. The contest proceeds as follows. First,
agents will make contest submissions (every agent makes exactly one
submission). Second, the value of each submission is determined through peer
evaluation. Third, the submission with the highest value (determined through
peer evaluation) wins the single contest prize ($M$).

First we look at contest submissions. Low and high types produce a contest
submission (i.e., an idea) of low ($b_{l}$) and high quality ($b_{h}$),
respectively. Without loss of generality, we normalize the evaluation scale to
range from 0 and 1. Furthermore, by construction the quality of a low
submission is lower than that of a high submission ($0<b_{l}<b_{h}<1$).
Neutral outsiders are agents who participate only in the evaluation phase of
the contest but do not enter their own submission into the contest. As they
are not competing for the winner prize, they have no incentive to act
strategically and evaluate sincerely, thus they help us establish the “true
quality” of a submission.

Second, to determine the value of a submission, we make the simplifying
assumption that every agent (each outsider, low, and high type) evaluates
every submission. Since every submission thus receives the same amount of
evaluations, we can determine the value of a submission by summing up all the
evaluations each submission receives. While evaluations can be sincere (i.e.,
$b_{l}$ or $b_{h}$, respectively), an agent can also sabotage any other agent,
evaluating their submission with $0$ or promote them by evaluating their
submission with $1$. As an example, in the case of sincere evaluation by
everyone (no promotion and no sabotage), the value of a high type would be
$b_{h}$ times the number of high types ($h$), low types ($l$), and outsiders
($n$; i.e., $v_{h}=b_{h}(n+l+h)$). The only other assumption required for our
model is that there are fewer high types than low types, and even more
outsiders than low types (i.e., $n>l>h$), which should be realistic in most
cases.

Third, in order to determine how to act during peer evaluation (i.e., evaluate
sincerely, sabotage, or promote) agents consider the cost and benefits of
their actions. The benefit for an agent derives from that agents likelihood of
winning the contest multiplied by the prize of the contest ($M$)—that is, the
incentive of the idea generation phase of the contest. To calculate an agent’s
likelihood of winning, we rely on established contest literature and for
simplicity model this as a Tullock contest (Tullock, 1980). In a Tullock
contest, the probability of winning is proportional to the value of an agent’s
submission (i.e., the sum of the evaluations the agent receives on her
submission) in relation to the total contest output (i.e., aggregate
evaluations of all contest submissions). As far as the costs are concerned, we
assume that while evaluating sincerely is free of costs, sabotaging and
promoting incurs costs ($c_{s}$ and $c_{p}$, respectively). Costs associated
with promotion and sabotage might arise from the costs of identifying suitable
targets (Harbring et al., 2007; Münster, 2007), the moral costs associated
with lying (Abeler et al., 2019; Gneezy et al., 2018), or violating social
norms (Elster, 1989). In our empirical setting, evaluating is anonymous but
this is not an assumption of our model. If evaluation is anonymous, this
simply means that the costs associated with promotion and sabotage may be
relatively small as there are no reputation costs associated with it (but
moral costs may still be exist).

From this setup, we can already derive four important insights. Insight 1. The
damage done by sabotage to a high type (evaluating $0$ instead of $b_{h}$) is
larger than the damage done to a low type (evaluating $0$ instead of $b_{l}$
because $b_{l}<b_{h}$ by construction). Since the cost of a single act of
sabotage is the same whether a high type or low type are targeted. This
suggests that targeting high types is more attractive. Insight 2. Following a
parallel argument, the benefit gained by promoting a low type is larger than
the benefit gained by a high-type (i.e., $1-b_{l}>1-b_{h}$). Insight 3.
Promoting anyone other than oneself is not rational: one would incur the cost
of promoting, but has no benefit (in fact, one reduces one’s own chance of
winning by promoting others). With this setup, it is immediately clear that
sabotaging oneself and promoting any agent other than oneself are not helpful
in winning the contest prize and no rational agent would engage in such
behavior. In the following we thus speak of only “self-promotion” and simplify
the notation to consider at most one promotion act per contest entry. Together
with the previous insight, this suggests that self-promotion will be more
attractive to low types. Insight 4. All neutral outsiders evaluate others
sincerely as they have nothing to gain from strategic behavior (since they did
not submit to the contest their chance of winning the prize is zero) yet would
incur cost of evaluating strategically.

Regarding the last point, note that neutral outsiders are not necessarily
agents who never compete—they simply do not compete in the current contest. We
will use this feature to identify strategic behavior in our empirical analysis
where agents do not enter every contest: they evaluate as neutral outsiders in
some weeks and evaluate as competitors with stakes from idea generation—and
incentives to evaluate strategically—in the contest in other weeks.

###  2.3 Self-Promotion

Self-promotion increases an agent’s chance of winning. Since the expected gain
from self-promotion is higher for a low type than a high type, there is a cost
boundary at which all low type agents decide to self-promote, but none of the
high type agents self-promote. If the cost of self-promotion are low enough,
all high types will of course self-promote in addition to all low types who
will continue to self-promote. Even low types will not self-promote if the
cost exceeds their expected gain (see Appendix B for a formal proof).

###  2.4 Sabotage

The decision to sabotage—and who to sabotage—is more complicated. If an agent
sabotages another agent, she decreases that agent’s output and thus decreases
total contest output, which increases her probability of winning the contest
(in the utility function the numerator of the agent’s output remains the same
while the total contest output in the denominator decreases). However, this
decrease in the denominator also benefits all other agents and thus sabotage
has—contrary to self-promotion—an important negative externality (Konrad,
2000). In the Appendix we first show a proof that the marginal gain from
sabotaging one more agent of a given type is increasing in the number of
agents of that type that are already being sabotaged. This means that an agent
will either sabotage all agents of a given type, or none of that type. Second,
we show that high types have more to gain from sabotaging other agents of a
given type (all low types or all high types), than low types have to gain from
sabotaging those the same agents (see proof in appendix). This result follows
from the externality associated with sabotage: the negative externality of
sabotage that improves everyone else’s chance of winning has a relatively
larger impact for low types than high types. Hence, the relative gain from
sabotaging other agents is larger for high types than low types. Third, we
show that both high and low types have more to gain from sabotaging high types
than low types, which follows from the fact that the damage done by sabotage
to a high type is higher than the damage done to a low type.

A key question now is whether low types or high types find it more attractive
to sabotage low types, or low types sabotage high types. That is, at what cost
of sabotage will one group start to sabotage the other group (while also
considering everyone else’s behavior). In the appendix, we go through the
exercise of calculating all bounds to establish the precise order according to
which groups of agents will decide to sabotage other groups of agents. The
bounds reveal that low types have more to gain from sabotaging high types,
than high types have to gain from sabotaging low types. The intuition behind
this result rests on the fact that damage done by sabotage to high types is
higher than the damage done to low types.

###  2.5 Predictions for Empirical Analysis

Our model allows us to make several predictions of the self-interested
behavior of crowdsourcing participants. The first is that self-promotion is
prevalent: most agents, including low type agents, self-promote. The second is
that high types are the most likely to sabotage. The third is that high types
target each other with their sabotage. In addition to guiding our empirical
investigation—in particular with regard to the crucial role of skill—these
predictions also serve as a baseline expectation for self-interested strategic
behavior which we can use to explore contrasting collaborative behavior.

##  3 Empirical Context

Our empirical setting is Threadless, a prototypical crowdsourcing community
(Majchrzak and Malhotra, 2016). Threadless is a crowdsourcing and e-commerce
platform that hosts weekly T-shirt design contests (Nickell, 2010). Since
2001, the site has developed into a leading crowdsourcing platform, pioneering
a community-based business model. The company involves its community of 1.5
million members in nearly all aspects of the innovation process and does not
employ any in-house designers (Lakhani and Kanji, 2008). The platform draws on
the creative talent of designers from across the world and has a distinct
focus on building and nurturing an online community of creative individuals
(Riedl and Seidel, 2018).

The tension between competition and collaboration is apparent in our study
context, the Threadless T-shirt design community. Threadless describes itself
as being an “inspiring design community” (Nickell, 2010, title page). Like in
many other crowdsourcing communities (Majchrzak and Malhotra, 2013, 2016),
Threadless users freely reveal their work by posting draft designs, ask for
and provide feedback on designs in the forum, inspire each other with their
work, read interviews, and learn from each other (Nickell, 2010; Riedl and
Seidel, 2018). According to the founder, Threadless is about “real community,
friendships, and working on fun, cool projects together” (Nickell, 2010,
inside cover flap). Community members value participation as an experience in
itself with constructive and developmental spirit (Brabham, 2010), making the
community ethos front and center. The community has also developed strong
social norms supportive of collaborative behavior (Bauer et al., 2016).
Threadless as a company is seen as a member in the community in which it
participates rather than being its owner (Nickell, 2010, p. 45). The designs
printed on T-shirts and sold on the community’s e-commerce page are chosen by
the community and most of the proceeds from sales are distributed to the
community (Nickell, 2010). And yet, winning a design contest is coveted. The
name of the designer of a winning design is printed on each T-shirt thus
giving credit to the individual. Winners are inducted into an elite alumni
club, are invited to events, appear in interviews on the community’s blog, and
are profiled in books and teaching cases (Lakhani and Kanji, 2008; Nickell,
2010). Other communities even post public leader boards (Grad et al., 2023).
Cash prizes for winning a contest have increased over the years (Nickell,
2010). Designers have also been honored in additional Designer of the Year and
Most Printed Design awards (Nickell, 2010). Ultimately, being a successful
contest winner can be a launch pad for design careers beyond Threadless
(Nickell, 2010).

Contests are divided into two distinct and temporally separated stages: (1)
entering a contest by submitting a design, and (2) evaluating the submitted
designs. To enter a contest, Threadless provides a designer kit and template,
and submissions from any standard software program or digitized drawings are
accepted, thus creating low barriers to entry. The submission section of the
site is static and identical every week. Through a simple form, Threadless
enables designers to upload their design, provide a thumbnail, title, and
short description. The submission site provides no information as to how many
designs have been submitted, or by who, or what their quality might be. At
this stage, submissions undergo an editorial review by Threadless staff before
they go up for a seven-day rating period.333Threadless lists several reasons
for declining a submission at this stage: designs using copyrighted material,
duplication of prior work, inappropriate content, technical errors with the
image file such as low resolution, text only designs, designs that use too
many colors, are too large or cannot be printed for other reasons. The rating
period is seven days with the exception of the change in rules about dropping
design earlier which we exploit in the natural experiment on self-promotion in
Section 5.2.

Threadless posts designs publicly on their website for peer rating, using a
scale from 0 to 5 stars. The “score submissions” page shows a grid of all
submissions in the running with thumbnails, title (possibly trimmed to fit the
grid), and username. To navigate designs, participants can toggle between
“currently in the running” and “archive”, the design they submitted
themselves, and a single “filter by keyword” search. Clicking on the thumbnail
or title leads to the scoring page of a specific design which shows the full
title and description, a thread of comments, and the days remaining of the
scoring period. The page also shows a counter of how many others have already
scored the design, but no average rating is shown to avoid social influence
and herding. The average rating is shown once the scoring period is over and
the submission has entered the “archive”, but individual ratings are never
revealed. That is, ratings are completely anonymous to the community (but not
to us as the researchers).

Among the submissions that received the highest average rating, Threadless
typically selects three to six designs per week as contest winners. We show
the rating percentile of contest winners in the Appendix (Figure A.III). On
average, contest winners scored in the 95th percentile of all designs
submitted that week. The median rank of printed designs is in the 98th
percentile. The designs of contest winners are then printed and subsequently
sold through the Threadless e-commerce site. Designers receive a cash prize
and additional store credit if their design was selected for printing.
Threadless does not publish a global ranking with respect to designers’ past
design evaluations as other crowdsourcing platforms do (e.g., Topcoder).
Threadless generated $30 million in revenue in 2012 and provided over $775,000
in prizes over the observation period. We focus on the regular weekly contests
from 2001 to 2011, excluding special and themed contests that Threadless also
hosts occasionally from our analysis . The data was provided to us by
Threadless directly (i.e., it was not scraped from the website) under an NDA.
The authors declare that they have no relevant financial or non-financial
competing interests to report.

###  3.1 Identification Strategy

To empirically identify strategic contest behavior, we leverage the fact that
the competition-collaboration tension is most acute for designers who
participate in the evaluation when also being active submitters, and less
acute when they only participate in the evaluation process and are not
themselves competing: a designer may evaluate the submissions of others
without having her own design in the running in one week, and then enter her
own design in the contest and evaluate in the next week.444See Section C.2 for
more details on the empirical patterns. We model the effects of participating
in a contest on evaluation behavior using a difference-in-difference approach.
That is, we model within individual differences between evaluations cast on
competitors versus non-competitors (thus controlling for unobserved individual
differences such as, e.g., being a harsh critic in general) and within
submission differences between evaluations cast by those who submitted to the
contest and those that did not (thus controlling for submission quality).
Evaluations cast by an individual in a contest that the individual did not
compete in serve as control for those contests in which they did. At the same
time, evaluations of the same submission from individuals who are not
themselves competing in the contest serve as controls for individuals who are
competing in the contest. In terms of our model, a contestant who did not
submit a contest entry is a neutral outsider, and a (high/low type) contestant
otherwise.

Note that “neutral outsiders” are other individuals who submit to contests in
general, just not to the one of the current week. Therefore, outsiders are
still designers who are in a good position to judge the quality of submissions
and not consumers who never submit. Why are outsiders participating in the
rating? These outsiders appear to be predominantly motivated to participate in
the rating process in order to engage in the community and learn vicariously
from the work of others (Riedl and Seidel, 2018). We make the assumption that
these outsiders are “neutral” in the sense that they have no direct incentive
to evaluate strategically since they do not have anything to gain from doing
so while facing moral costs of lying if they evaluated strategically rather
than truthfully. This assumption seems particularly plausible given the
community focus of Threadless (Lakhani and Kanji, 2008). This assumption not
withstanding, it is possible that rivalries among a small set of competitors
may cause them to evalute strategically even when not competing (Kilduff et
al., 2010). We explore this in the Appendix D.2.555Note that fake accounts are
not of great concern to our analysis. While fake accounts may exist on
Threadless in principle, our analysis includes only ratings cast by
designers—users who also make design submission. To qualify as “designer” a
user needs to make a design submission that passes Threadless’ basic muster of
being a valid submission (a real t-shirt design on the Threadless template,
cannot be blank, cannot be an obvious copy of other work, etc.), and have
their design submission actually put up for voting.

We model sabotage as the change in probability that individual $i$ submits a
0-star rating—the lowest possible rating—on submission $j$ when having
submitted to the same contest relative to not having submitted to the same
contest. Conversely, we model self-promotion as the change in probability of
rating one’s own submission with 5-stars—the highest possible rating—when
rating one’s own submission versus rating someone else 5-stars.666By using the
two extreme outcomes, 0-stars and 5-stars respectively, we use a conservative
measure of sabotage and self-promotion respectively, because (a) we only
include the likely report of maximal lies (Gneezy et al., 2018), and (b) we
make sure not to conflate the two behaviors. An alternative specification for
sabotage would be, e.g., to use the probability that a design gets assigned a
0 or 1-star rating. However, this would not just include sabotage (all designs
that are of higher quality) but also self-promotion (all 0-star designs that
get assigned a 1-star rating) and therefore conflate the two behaviors. A
similar argument can be made for self-promotion. The two key explanatory
variables are (a) a dummy variable indicating whether an individual is a
participant in the contest (i.e., whether $i$ has submitted to the same
contest $c$ and is in the running for the prize); and (b) a dummy variable
indicating if the individual is rating his or her own submission (this is only
possible if the other dummy variable is also 1). That is, we estimate the
following equations:

$\displaystyle\text{0-Star Rating}_{ij}$
$\displaystyle=\beta_{11}\text{Submitted to same
contest}_{ij}+\beta_{12}\text{Rate own
submission}_{ij}+\alpha_{i}+\alpha_{s}+\epsilon_{ij}$ (1)
$\displaystyle\text{5-Star Rating}_{ij}$
$\displaystyle=\beta_{21}\text{Submitted to same
contest}_{ij}+\beta_{22}\text{Rate own
submission}_{ij}+\alpha_{i}+\alpha_{s}+\epsilon_{ij}$ (2)

where 0-Star Ratingij is an indicator that is $1$ if the rating submitted by
individual $i$ on the submission $j$ is a 0-star rating, 5-Star Ratingij is an
indicator that is $1$ if the rating submitted by individual $i$ on the
submission $j$ is a 5-star rating, $\alpha_{i}$ are individual-level and
$\alpha_{s}$ submission-level fixed effects, and $\epsilon_{ij}$ are error
terms.

We use this approach to separately identify the two types of strategic
behavior outlined in the theoretical model above:

  * •

Self-promotion: If a 5-star vote is assigned to one’s own submission, this
indicates self-promotion ($\beta_{22}$ in Eq. 1). We provide robustness tests
using a natural experiment to address concerns that this would not reflect
strategic behavior but rather the result of overconfidence or an increased
preference fit in Section 5.2

  * •

Sabotage: Sabotage is indicated if 0-star votes are assigned more liberally
when competing in the same contest versus being an outsider who has no stakes
in the contest (controlling for how likely the submission is to receive
0-stars overall; $\beta_{11}$ in Eq. 1). We provide results from a second
natural experiment to provide evidence that low ratings are strategic rather
than a result of endogenous entry into contests in Section 5.1

We estimate our models as linear probability models for ease of interpretation
(Angrist, 2001; Greene, 2012). A discussion of the usefulness of this approach
can be found in Angrist (2001) and Moffitt (2001). Furthermore,
notwithstanding the limitations of this approach, efficient estimation
approaches for OLS exist for large datasets that support demeaning of multiple
fixed effects. We estimate linear probability models using the lfe package for
R (Gaure, 2013), which supports demeaning of multiple fixed effects. This
algorithmic approach is mandatory as our dataset is extremely large with more
than 38 million observations, 74,525 individual fixed effects, and 154,086
submission fixed effects (or 511 contest fixed effects).

To investigate how sabotage and self-promotion vary across skill levels we
compute skill as the time lagged average submission quality.777We use the
quality of the best previous submission as a robustness check which is
consistent with our results, see Section C.4 in the Appendix. We compute skill
for both the individual casting the rating (we refer to this as the source)
and the individual who made the contest entry being evaluated (we refer to
this as the target). We then estimate versions of the models in Equation 2 and
1 in which we include source and target status and their interactions with the
Submitted to same contest and Rate own submission dummy variables. Since
target skill is time-invariant at the submission level, the main coefficient
drops out.

###  3.2 Data

Our data consists of an unbalanced cross-panel of 74,525 individuals
participating in 511 contests, with over 38 million ratings on 154,086 contest
entries (see Table 1). The skill distribution is right skewed (Appendix figure
A.IIa) and thus, our empirical setting matches our model, which assumes fewer
high types than low types. The average contest size is 407 submissions
(Appendix figure A.IIb shows the entire distribution).We show descriptive
statistics and correlations of main variables in Table 2.

| Rate own Submission
---|---
Submitted to same contest | No | Yes
No | 30,246,070 (13.58%) |
Yes | 7,772,772 (10.73%) | 114,914 (74.58%)
Table 1: Summary of rating behavior. Parentheses show probability of rating conditional on submitting. | Mean | SD | Min | Max | (1) | (2) | (3) | (4) | (5) | (6)
---|---|---|---|---|---|---|---|---|---|---
Rating (1) | 1.93 | 1.53 | 0.00 | 5.00 |  |  |  |  |  |
Submitted to same contest (2) | 0.21 | 0.41 | 0.00 | 1.00 | 0.03 |  |  |  |  |
Rate own submission (3) | 0.00 | 0.05 | 0.00 | 1.00 | 0.11 | 0.11 |  |  |  |
Contest size (4) | 406.73 | 131.88 | 2.00 | 848.00 | 0.07 | 0.11 | 0.00 |  |  |
Average score in contest (5) | 1.93 | 0.23 | 1.26 | 2.76 | 0.15 | 0.04 | 0.00 | 0.50 |  |
Source skill (6) | 2.26 | 0.61 | 0.00 | 5.00 | 0.02 | 0.05 | 0.00 | 0.17 | 0.29 |
Target skill (7) | 2.28 | 0.57 | 0.00 | 5.00 | 0.27 | -0.01 | -0.01 | 0.13 | 0.29 | 0.08
Table 2: Descriptive statistics and correlations of main variables. All
correlations are significant at $p<0.001$.

##  4 Results

We start with baseline estimates of a difference-in-difference model estimated
(Table 3).888We cluster standard errors on the submission level to account for
possible dependence of ratings of the same submissions and in the same
contest. One may alternatively be concerned about capturing auto-correlation
at the rater level. We examine the robustness of our specification in A.II.
The clustered standard errors are virtually identical to robust standard
errors. Furthermore, $p$-values are generally much smaller than $0.001$ (i.e.,
$1e-9$) so that our substantial conclusions are not affected by the level of
clustering of standard errors. With a continuous measure of ratings as
dependent variable (Model 1), we find no signs of sabotage but rather slight
leniency toward ones’ competitors ($\beta=0.024$; $p<0.001$) and strong signs
for self-promotion ($\beta=2.820$; $p<0.001$). Moving to a linear probability
model (Model 2) we find also find overall leniency: a slightly lower
likelihood to rate 0-stars when an individual submitted to the same contest
($\beta=-0.008$; $p<0.001$). That is, community members do not appear to act
strategically against their competitors but instead show leniency. Note that
our model with individual and submission fixed effects controls for the
possibility that some contests include more low-quality submissions that
deserve 0-stars. Next, we estimate a linear probability model on casting a
5-star rating. We find a significant level of self-promotion (Model 3:
$\beta=0.855$; $p<0.001$). The effect of self-promotion is extremely strong.
Seventy-five percent of individuals rate their own submissions and the
overwhelming majority of them does so with the highest possible rating: 97% of
self-votes are 5-stars. Given that the average rating for non-self-ratings is
only 1.80 stars, this indicates that contestants are self-promoting with
dramatically inflated ratings.

| OLS | Linear Probability
---|---|---
Dependent Variable: | Rating | Sabotage | Self-Promotion
|  | 0-Star Rating | 5-Star Rating
| (1) | (2) | (3)
Submitted to same contest: Yes | $0.024^{***}$ | $-0.008^{***}$ | $-0.002^{***}$
| $(0.001)$ | $(0.000)$ | $(0.000)$
Rate own submission: Yes | $2.820^{***}$ | $-0.195^{***}$ | $0.855^{***}$
| $(0.003)$ | $(0.001)$ | $(0.001)$
Individual | Fixed | Fixed | Fixed
Submission | Fixed | Fixed | Fixed
Adj. R2 | $0.382$ | $0.372$ | $0.210$
Num. obs. | 38,102,880
${}^{***}p<0.001$, ${}^{**}p<0.01$, ${}^{*}p<0.05$
Table 3: Estimates for strategic behavior in contest. Standard errors are in
parentheses, clustered at the submission level.

Next, we investigate if there is heterogeneity in the use of strategic
behavior among those of different skill. We estimate a variation of Equation 2
and include measures of idea generation skill for both the source and the
target of the rating, an interaction between source and target skill, and
interaction terms between the Submitted to same contest dummy and source and
target skill, respectively (Table 4).999This analysis relies on a reduced
sample as we need to exclude (a) all ratings that an individual casts before
making their first submission (because for those individuals we cannot compute
idea generation skill), and (b) all ratings on every contestant’s first
submission (again, because idea generation skill is still unknown for those
individuals). We check whether the use of subsamples drives our findings in
Section C.4 in the Appendix but find no evidence of this. We find significant
heterogeneity in sabotaging behavior. We find that higher-skill individuals
are more likely to be the source of sabotage (Model 1: $\beta=0.012$;
$p<0.001$) and are more likely to be the target of sabotage ($\beta=0.019$;
$p<0.001$). This pattern mirrors the predictions of our theoretical model.
Turning to self-promotion, we estimate a similar model using the Rate own
submission dummy variable but include only the interaction with source skill,
since source and target are the same in the case where a contestant is voting
on themselves. We find that higher-skill individuals are significantly less
likely to self-promote (Model 2: $\beta=-0.055$; $p<0.001$) compared to lower-
skill contestants. This matches our model prediction that low-skill agents are
more likely to engage in self-promotion.

| Linear Probability
---|---
Dependent Variable: | Sabotage | Self-Promotion
| 0-Star Rating | 5-Star Rating
| (1) | (2)
Submitted to same contest: Yes | $-0.071^{***}$ | $0.001^{***}$
| $(0.001)$ | $(0.000)$
Rate own submission: Yes | $-0.194^{***}$ | $0.957^{***}$
| $(0.001)$ | $(0.004)$
Source skill | $-0.028^{***}$ | $-0.007^{***}$
| $(0.001)$ | $(0.001)$
Source skill $\times$ Target skill | $-0.005^{***}$ | $0.008^{***}$
| $(0.000)$ | $(0.000)$
Submitted to same contest: Yes |  |
$\times$ Source skill | $0.012^{***}$ |
| $(0.000)$ |
$\times$ Target skill | $0.019^{***}$ |
| $(0.000)$ |
Rate own submission: Yes |  |
$\times$ Source skill |  | $-0.055^{***}$
|  | $(0.002)$
Individual | Fixed | Fixed
Submission | Fixed | Fixed
Adj. R2 | $0.391$ | $0.243$
Num. obs. | 18,787,584
${}^{***}p<0.001$; ${}^{**}p<0.01$; ${}^{*}p<0.05$
Table 4: Estimates for strategic behavior with skill heterogeneity. Standard
errors are in parentheses, clustered at the submission level.

To better interpret the heterogeneity across skill levels, we compute
predicted values for the expected change in the likelihood to rate 0-stars
when competing versus not competing (Figure 1). The heat map in Panel A shows
that high-skill contestants target other high-skill contestants with an up to
6% increased likelihood of assigning a 0-star rating when competing compared
to their baseline when not competing. The null-line marking the start of
sabotaging behavior is not perfectly symmetric: high-skill contestants are
sabotaged by contestants of all other skill levels. This matches our
theoretical predictions that low types sabotaging high types is the second
most lucrative form of sabotage (after high types sabotaging high types). We
similarly compute the relative change in probability of self-promotion (since
source and target skill are the same in the case of self-rating, this analysis
is effectively the diagonal of the heatmap in Panel A). Panel B in Figure 1
shows that the likelihood that contestants engage in self-promotion decreases
with their own skill. This finding is in line with the predictions from our
theoretical model that self-promotion is less effective among high
types.101010We perform three important robustness tests. First, we repeat the
same analysis using the quality of the best design an individual submitted in
the past (instead of the average of all past submissions) and find almost
identical coefficients (Appendix Table A.III and Figure A.IV). Second, we
explore alternative measures of skill: tenure, experience, and being a past
winner (Appendix Table A.IV). We find no significant effect of tenure. We find
mixed results for experience . We find a lower likelihood to sabotage for
those with more evaluation experience but higher likelihood to sabotage for
those with more submission experience. This is consistent with the
interpretation that competitive motivation is the driver behind strategic
behavior (i.e., those who are competitively motivated and make many
submissions, sabotage more while those who may be more socially motivated and
submit many evaluations). We find that past winners act significantly more
strategically. They are both much more likely to sabotage, and other past
winners are the targets. Being a past winner could be seen as socially
validated skill. Third, we explore whether the reported effect of strategic
behavior could be explained by dyadic rivalries instead (Kilduff et al.
(2010); Section D.2). We find that sabotage happens outside, and on top of,
any pre-existing dyadic rivalries. Even within the same dyad, we still find a
difference in individuals’ rating behavior between competing for the same
prize vs. not competing which indicates strategic behavior. In summary, dyadic
rivalry patterns are not sufficient to explain strategically motivated
sabotage.

Figure 1: Strategic behavior by competitors of heterogeneous skill levels.

Note: For these figures the mean submission quality is used as measure of
skill. Panel A (Sabotage). Relative change in probability of rating 0-stars
when competing compared to not competing across skill levels. Outer rugs show
distribution of data. There are 4,512 (1,091) observations for source (target)
skill $\geq 3.5$. Panel B (Self-promotion). Relative change in probability of
rating 5-stars when rating own submission compared to submissions by others of
same skill (error band is 95% confidence interval).

As further evidence for the use leniency as mutual-interested compensation for
self-interested strategic evaluation, the Appendix (Table A.V) shows
regressions on the contest-level using standard deviation of the ratings that
individuals cast as dependent variable. Those regressions show that when
individuals have submitted to the same contest, they make better use of the
full rating spectrum and submit ratings with a higher standard deviation. The
effect is amplified by skill so that higher-skilled individuals submit ratings
with an even higher standard deviation. This suggests that leniency is a
phenomenon within a single contest: individuals accumulate credits on an
invisible scorecard by promoting some community members (giving them higher
ratings than they deserve) and then spend these credits on self-interested
strategic evaluations (giving them lower ratings than they deserve).

##  5 Mechanisms of Strategic Behavior

Why do some community members behave strategically? Is it really driven by
incentives of the idea generation phase spilling over to idea evaluation or
are there other alternative explanations? Exploring our data further, we show
that  (a) sabotage is strategically motivated as a result of idea generation
incentives spilling over to the idea evaluation,  (b) self-promotion is
strategically motivated and not just a result of overconfidence,  (c) those
who act strategically mostly use sabotage and self-promotion together rather
than substituting one with the other, and finally  (d) we briefly explore the
effect that strategic behavior has on the selection of contest winners.

###  5.1 Sabotage is a result of idea generation incentives

If peer evaluation is strategically motivated by idea generation incentives
spilling over to idea evaluation, we would expect an increase in the
probability to assign 0-stars in contests with higher incentives (or more
precisely, the prize spread between the contest winner and the contest
loser(s); e.g., Lazear, 1989; Harbring and Irlenbusch, 2011). Conversely, if
idea evaluation is not strategically motivated, we would expect no change or
possibly even a decrease in the probability to rate 0-stars after the
incentive change because submission quality can be expected to increase as a
result of increased effort due to the incentive effect (Jeppesen and Lakhani,
2010). We leverage a platform change as a natural experiment—a change in prize
money awarded to contest winners. The prize for winning the weekly contest
doubled from $500 to $1,000 in 2005 (see Figure A.V in the Appendix). This
doubling of the contest prize was announced on the company’s blog on the day
it took effect and therefore, community members had no prior knowledge of it
and were not able to withhold submissions or otherwise alter their behavior in
anticipation of it.111111Note that this natural experiment also helps us
address another potential alternative explanation. An alternative explanation
for the observed negative rating behavior could be seen in the endogenous
decision to enter contests (see, e.g., Bockstedt et al., 2016). Contestants
may submit entries to contests where they expect that competition is weak and,
therefore, deserve lower ratings. Following the argument of endogenous entry
we would expect either no change in 0-star ratings as contestants should not
be able to time their contest entry around the incentive change.

We use a difference-in-difference design to identify the causal effects of the
incentive changes. We use a 6-months time window before and after the
incentive change. We estimate the same linear probability model for 0-Star
ratings with individual- and submission-level fixed effects as before (Table
5). We find a significant 2.2% increase in the probability to rate 0-stars
when competing after the incentive change (Model 1: $\beta=0.022$; $p<0.001$).
The effect is stronger (2.3%) in the first quarter after the incentive change
and slightly weaker (2.0%) in the second quarter (Model 2:
$\beta=0.023,\beta=0.020$; both $p<0.001$). Furthermore, we find that
submission quality (as determined by ratings of neutral outsiders who have no
incentive to rate strategically) is significantly higher after the incentive
change ($\mu_{before}=1.18$, $\mu_{after}=1.39$; t-test $p<0.001$). This
indicates that the incentive change, as intended, indeed increases the average
quality in the idea generation phase but also increases sabotage in the idea
evaluation phase. A placebo test in which we selected a fake date four weeks
before the actual incentive change provides further evidence for the causal
impact of the incentive change on the rating behavior (Model 3:
$\beta=-0.009$; $p<0.001$) as it rather indicates a negative time trend.

| Linear Probability
---|---
Dependent Variable: |  Sabotage: 0-Star Rating
| Base | Persistence | Placebo Test
| (1) | (2) | (3)
Submitted to same contest: Yes | $-0.037^{***}$ | $-0.037^{***}$ | $-0.035^{***}$
| $(0.000)$ | $(0.000)$ | $(0.000)$
Rate own submission: Yes | $-0.179^{***}$ | $-0.179^{***}$ | $-0.179^{***}$
| $(0.000)$ | $(0.000)$ | $(0.000)$
Submitted to same contest: Yes |  |  |
$\times$ After | $0.022^{***}$ |  | $0.028^{***}$
| $(0.000)$ |  | $(0.000)$
$\times$ 1st quarter after |  | $0.023^{***}$ |
|  | $(0.000)$ |
$\times$ 2nd quarter after |  | $0.020^{***}$ |
|  | $(0.000)$ |
$\times$ Fake after |  |  | $-0.009^{***}$
|  |  | $(0.000)$
Individual | Fixed | Fixed | Fixed
Submission | Fixed | Fixed | Fixed
Adj. R2 | $0.390$ | $0.390$ | $0.390$
Num. obs. | 825,504
${}^{***}p<0.001$, ${}^{**}p<0.01$, ${}^{*}p<0.05$
Table 5: Natural experiments of sabotage behavior after an incentive changes
from $500 to $1,000 prize money using $\pm 6$ months an observation windows.
Standard errors in parentheses, clustered at the submission level.

###  5.2 Self-promotion is strategically motivated and not only overconfidence

Why do community members evaluate their own submissions so highly? High
evaluations for one’s own submission could reflect overconfidence of
contestants in their own abilities (Benabou and Tirole, 2002; Camerer and
Lovallo, 1999) or an increased preference fit of one’s own creative work
compared to the work of others (e.g., Berg, 2016; Franke et al., 2010). To
explore the mechanism behind the high self-evaluation, we leverage another
natural experiment around a change in the scoring mechanism.

Before the change, every contest entry was posted on the website for the full
7-day rating period. After the change in 2005, contest entries with a rating
below 1.5 stars were eliminated from the rating process before the 7-day
rating period was complete (see Figure A.VI in the
Appendix).121212Specifically, contest entries that had received ratings from
100 different people but the average of those rating was less than 1.5 stars
were eliminated. Note that the average rating still was not visible during
rating process but only to the platform owner. This rule change was introduced
to help the platform owner to focus on a smaller set of contest entries by
removing low quality submissions from the contests earlier. The change was
announced and immediately implemented. If rating one’s own submission is not
intended strategically but only reflect biased perception and overconfidence,
we should not see any change in rating behavior around this rule change. If,
however, rating on one’s own submission is intended strategically, such an
evaluation should happen earlier to maximize the chance to progress past the
100 vote / 1.5 stars cutoff.

We analyze the sequence of evaluations for each contest entry and test when in
the sequence the self-promoting evaluation was cast (Figure 2). By focusing on
the relative position in the evaluation sequence we control for any behavior
change that might occur for community members evaluating the same contest
entry. We focus our analysis on the 20-week period around the evaluation rule
change ($\pm 10$ weeks before/after the rule change). We find that before the
rule change, self-evaluation happens roughly in the middle of the sequence
(43rd percentile). That is, self-evaluations were cast roughly on a random day
during the evaluation period. After the rule change, self-evaluations are cast
significantly earlier, falling in the 25th percentile of the evaluation
sequence. Our analysis of this natural experiment suggests that evaluating
one’s own submission has a strategic intention (make it past the 1.5 stars
cutoff) rather than simply a reflection of overconfidence or preference for
one’s own submission.

Figure 2: Analysis of Rating Sequence.

Note: Self-rating happens significantly earlier after the rule change.

###  5.3 Sabotage and self-promotion are mostly used together

Do individuals use sabotage and self-promotion together to maximize the impact
of their strategic behavior or do they substitute one for the other, maybe to
atone for moral cost? As noted before, investigating this question empirically
is difficult as we can identify sabotage only probabilistically through the
difference-in-difference approach. However, we can get some meaningful
leverage to identify possible substitution by using self-promotion (or rather
the lack thereof) as our starting point. Remember that self-promotion is very
common: 86% of individuals who had a chance to self-promote (i.e., submitted
to a contest and then rated at least some of their competitors) chose to do so
and 97% of those self-ratings are 5-stars. Conversely, only 14% of individuals
who had a chance to self-promote did not do so.

Using the same modeling approach with individual- and submission-level fixed
effects as in the main analysis, we find (Table A.VI) that the likelihood to
rate the competition with 0-stars if one has also self-promoted in the same
contest is significantly lower than for those who did not (Model 1,
$\beta=-0.002;p<0.01$; note that having self-promoted in a contest implies
also having submitted to that contest). This suggests that some individuals do
in fact sabotage somewhat less when they have self-promoted. Note, however,
that this coefficient estimate captures the effect of marginal individuals:
individuals who only sometimes self-promote but not always (as otherwise the
individual fixed effect captures this time-invariant behavior). As such, the
estimate may mask the more specific effect among that group of more strategic
individuals who always self-promote and for which the model cannot pick up
variation in behavior. To check this possibility, we estimate the same model,
but without individual-level fixed effects (Model 2). Here we find a large
positive effect: those who have self-promoted also have a much higher
likelihood to sabotage ($\beta=0.031;p<0.001$). So while the small group of
marginal raters may indeed commit less sabotage when they also self-promoted,
this is not the case for large majority of individuals who tend to use
sabotage and self-promotion together to maximize their likelihood of winning
the contest.

###  5.4 Consequences of strategic behavior: Effect on selection of contest
winners

Does strategic behavior affect idea selection? To quantify the impact that
strategic evaluations have on idea selection—compared to which ideas would be
selected if all evaluations were truthful and followed the meritocratic ideal
of the community—we perform two supplementary analyses. The first assesses the
effect of self-promotion, the second of sabotage (see Appendix 6). For self-
promotion, keeping everything else constant, we find that the winner changes
in 1.4% of contests (7 out of 511) and in 5.5% (28 of 511) there is a change
in at least one of the top three ranks. For sabotage, we find that the winner
changes in 12% of the contests and in 48% there is a change among the top
three. The effect is especially pronounced in close contests where the contest
winner would change in 25% of cases, while 65% would see a change in the top
three.

##  6 Consequences of Sabotage on Long-Term Participation

| Hazard: Participate in Next Round
---|---
| Sabotage | Leniency
| Skill | Skill | Skill | Skill
| Continuous | Quintiles | Continuous | Quintiles
| (1) | (2) | (3) | (4)
Sabotage received | $-0.017^{\dagger}$ | $-0.017$ |  |
| $(0.010)$ | $(0.019)$ |  |
Sabotage received |  |  |  |
$\times$ Skill | $0.066^{***}$ |  |  |
| $(0.009)$ |  |  |
$\times$ Skill Q1 |  | $-0.158^{***}$ |  |
|  | $(0.027)$ |  |
$\times$ Skill Q2 |  | $-0.076^{**}$ |  |
|  | $(0.027)$ |  |
$\times$ Skill Q4 |  | $0.075^{**}$ |  |
|  | $(0.026)$ |  |
$\times$ Skill Q5 |  | $0.119^{***}$ |  |
|  | $(0.026)$ |  |
Leniency received |  |  | $-0.110^{***}$ | $-0.044$
|  |  | $(0.018)$ | $(0.028)$
Leniency received |  |  |  |
$\times$ Skill |  |  | $-0.123^{***}$ |
|  |  | $(0.011)$ |
$\times$ Skill Q1 |  |  |  | $0.166^{***}$
|  |  |  | $(0.028)$
$\times$ Skill Q2 |  |  |  | $0.117^{***}$
|  |  |  | $(0.029)$
$\times$ Skill Q4 |  |  |  | $-0.208^{***}$
|  |  |  | $(0.034)$
$\times$ Skill Q5 |  |  |  | $-0.525^{***}$
|  |  |  | $(0.046)$
Controls |  |  |  |
Skill | $0.165^{***}$ |  | $0.170^{***}$ |
| $(0.013)$ |  | $(0.013)$ |
Skill Q1 |  | $-0.341^{***}$ |  | $-0.443^{***}$
|  | $(0.035)$ |  | $(0.035)$
Skill Q2 |  | $-0.084^{*}$ |  | $-0.114^{***}$
|  | $(0.032)$ |  | $(0.033)$
Skill Q4 |  | $0.122^{***}$ |  | $0.096^{**}$
|  | $(0.035)$ |  | $(0.035)$
Skill Q5 |  | $0.225^{***}$ |  | $0.036$
|  | $(0.044)$ |  | $(0.043)$
Submission quality | $0.976^{***}$ | $0.982^{***}$ | $0.821^{***}$ | $0.797^{***}$
| $(0.027)$ | $(0.028)$ | $(0.034)$ | $(0.036)$
Submission popularity | $0.067^{***}$ | $0.066^{***}$ | $0.071^{***}$ | $0.070^{***}$
| $(0.008)$ | $(0.008)$ | $(0.008)$ | $(0.008)$
Competition size | $-0.000$ | $-0.000$ | $-0.000^{\dagger}$ | $-0.000^{*}$
| $(0.000)$ | $(0.000)$ | $(0.000)$ | $(0.000)$
Average contest rating | $0.953^{***}$ | $0.933^{***}$ | $0.845^{***}$ | $0.779^{***}$
| $(0.053)$ | $(0.053)$ | $(0.056)$ | $(0.056)$
AIC | $912233.777$ | $911871.273$ | $922869.780$ | $921779.500$
R2 | $0.317$ | $0.321$ | $0.327$ | $0.337$
Num. obs. | $72162$ | $72162$ | $73018$ | $73018$
Num. events | $48052$ | $48052$ | $48639$ | $48639$
PH test | $0.000$ | $0.000$ | $0.000$ | $0.000$
${}^{***}p<0.001$; ${}^{**}p<0.01$; ${}^{*}p<0.05$; ${}^{\dagger}p<0.1$ |
Table 6: Analysis of long-term participation. Multiple event hazard model of
participation in the next week.

This section investigates a key organizational-level outcome resulting from
the emergence of strategic behavior: how does being the target of sabotage and
leniency affect long-term participation and hence community structure?
Retaining existing members is critical to sustaining the communities in which
contests are embedded (Faraj et al., 2011; Ransbotham and Kane, 2011). While
existing research has investigated the motivation to join communities (Lakhani
and Wolf, 2003), much less is known about individuals’ progression and long-
term participation in communities (Smirnova et al., 2022). There is a rich
body of research on fair processes which would suggest that the targets of
unfair behavior (such as sabotage), or those who experience random shocks to
their performance evaluation, adjust their future effort and engagement (e.g.,
Gomez-Mejia and Balkin, 1992; Balietti and Riedl, 2021). Several studies that
investigate sabotage in lab experiments find that being the target of sabotage
increases the likelihood of dropout (Balietti and Riedl, 2021; Charness et
al., 2014; Chen, 2003). Notice that while we consider self-promotion of equal
importance, this section focuses on the amount of sabotage and leniency
received simply because the amount of self-promotion received is virtually
identical across all individuals.

The analysis proceeds as follows. We analyze the likelihood to participate in
next week’s contest, based on the amount of sabotage and leniency an
individual experienced on their current week’s submission. For this analysis,
it is important to recall that sabotage (leniency) can be identified only
probabilistically. For example, a low-quality submission may deserve a 0-star
rating even if it comes from a competitor. Conversely, certain individuals may
simply be harsh critics who rate most submissions with 0-stars without any
intention to sabotage them. To measure the amount of sabotage (leniency) that
a submission received, we rely on our micro-level model of sabotage developed
above.

For each individual, we estimate the counterfactual likelihood to cast a
0-stars evaluation of neutral outsider (using Model 1 from Table 4). We then
compute the “residual” between the evaluation that was actually cast and a
counterfactual without strategic behavior. That is, our micro-level model of
sabotage allows us to estimate the degree to which any of the 0-star ratings
are likely due to sabotage (accounting for submission fixed effects, evaluator
fixed effects, and time varying covariates like source and target skill). We
also compute the amount of leniency a submission received as “residual”
between a non-0-stars rating that should have been a 0-star rating. We then
aggregate the evaluation-level residual to the submission level by taking the
average across all ratings. This then captures the level of sabotage
(leniency) that a submission received and gives us a panel dataset in which we
have an estimate for the degree of sabotage (leniency) received by an
individual for each contest. We then estimate a multiple event hazard model
(Therneau and Grambsch, 2000) to predict the likelihood of long-term
participation.131313We estimate an interval Cox proportional hazard model for
multiple-events, controlling for individual hazard rate of participation,
exploiting within-individual variation in received sabotage; robust standard
errors clustered at the individual level. We include the key observables that
a competitor observes after a contest: the rating their submission received,
the number of ratings received, the level of competition (number of
competitors and average quality of all submissions in the contest), as well as
our focal variable of interest, the amount of sabotage received.

We find strong positive main effects of skill (Table 6 Model 1:
$\beta=0.165;p<0.001$): higher-skilled individuals are more likely to
participate in future contests. There is a small marginally significant main
effect of sabotage ($\beta=-0.017;p<0.10$). We find a significant positive
interaction between the level of sabotage received and skill
($\beta=0.066;p<0.001$). That is, high-skilled community members who received
one standard deviation more sabotage, are about 6.6% more likely to
participate in the next contest compared to those who received only the
average amount of sabotage. To better interpret the effect across skill
levels, we convert skill to quintiles, using the middle quintile as our
reference category (Model 2). Looking at the interaction between the level of
sabotage received and skill, we find that individuals in first and second
quintile react negatively to being targets of sabotage: their likelihood of
participation in the next round drop significantly by 16% and 8%, respectively
($\beta=-0.158;p<0.001$ and $\beta=-0.076;p=0.003$). For high-skilled
competitors, we find the opposite: they react positively to sabotage and their
likelihood to participate increases. There is an 8% increase for individuals
in the 4th skill quintile ($\beta=0.075;p=0.003$) and a 12% increase
($\beta=0.119;p<0.001$) for individuals in the highest skill quintile in the
expected hazard to participate in the next round for a one unit change in
received sabotage (one standard deviation).141414We can also interpret the
coefficients of our control variables. Having done well in a contest (high
rating, many ratings) makes future participation significantly more likely.
Competition has mixed effects (positive effect of high average rating across
all contest submissions, no effect of competition size).

The pattern is reversed for the amount of leniency that a submission receives.
The lowest skilled individuals increase their future participation by 17%
(Model 4: $\beta=0.166;p<0.001$) when they receive an additional one standard
deviation of leniency. The highest-skilled individuals on the other hand
decrease their likelihood of future participation. For high-skilled
individuals, we need to caution against placing too much weight on the
coefficient for leniency: it is very rare that high-skilled individuals would
submit a design of such low quality that it deserves a 0-star rating and hence
there is virtually no room to show leniency toward high-skilled individuals.

##  7 Discussion

Crowdsourcing has evolved as an organizational approach to distributed problem
solving and innovation. As contests are embedded in online communities and
evaluation rights are assigned to the crowd, community members face a tension
between competitive and collaborative participation motives. This tension
tempts idea generators to violate community norms and evaluate their peers
strategically. Using large-scale digital trace data from Threadless, a
prototypical crowdsourcing community (Majchrzak and Malhotra, 2016), we answer
the questions of how community members balance the competitive and
collaborative motives as they evaluate their peers, and how the individual-
level decisions they make change the structure of the community. We show that
as their skill level increases, the competitive motive increasingly wins out:
community members shift from using self-promotion to sabotaging their closest
competitors (other high-skill individuals). However, community members also
act in the collaborative spirit of the community and show leniency toward
those who do not directly threaten their own chance of winning.

In addition to these immediate short-term effects, we also find surprising
long-term effects of the fierce competition that ensues. While low-skill
targets of sabotage are less likely to participate in future contests, high-
skill targets are more likely. Furthermore, low-skilled beneficiaries of
leniency are encouraged and are more likely to participate in the future. This
suggests a feedback loop between competitive evaluation behavior and future
participation.

###  7.1 Theoretical Implications

Our findings have three important implications for the literature on the
interplay between competitive and collaborative motives in crowdsourcing
design, as well as the evolution and sustainability of crowdsourcing
communities.

First, we extend prior research on the nature of competitive and collaborative
behavior in crowdsourcing. The tension between competition and collaboration
has been identified by several crowdsourcing researchers (Adler and Chen,
2011; Franke and Shah, 2003; Bullinger et al., 2010; Hutter et al., 2011;
Majchrzak and Malhotra, 2013; Boudreau and Lakhani, 2015) and features
prominently in the broader organizations literature (Deutsch, 1949; Tsai,
2002; Lado et al., 1997; Gallus et al., 2022). This work has shown that the
level of knowledge sharing and mutual support decreases in communities when
members are also competing (Franke and Shah, 2003; Harhoff et al., 2003). We
complement this research with insights into evaluation behavior. We show that
the competitive incentives designed to encourage effort during idea generation
spill over into the collaborative community, creating a mismatch between the
collaborative organizational context and competitive incentives (Gallus et
al., 2022). However, the resulting strategic behavior is not rampant and out
of control as contest theory might predict. This suggests that embedding
contests in communities drives up the cost of amoral strategic behavior even
in a community without reputation costs (because peer ratings are anonymous)
and low search costs (because sabotaging everyone would only cost a few mouse
clicks). The tension between competitive and collaborative participation
motives is felt most acutely by high-skilled individuals who are most likely
to win (and thus have the most to gain). Hence, they are the most willing to
accept the cost of violating community norms.

However, community members not only sabotage, but they also show leniency. We
theorize that showing leniency is a crucial collaborative element that allows
individuals to better justify violating community norms with competitive
evaluations through a form of moral licensing (Blanken et al., 2015). Moral
licensing describes behavior in which people who behave in a moral way can
also display behaviors that are immoral, unethical, or otherwise problematic
(e.g., Merritt et al., 2010). Specifically, we theorize that individuals
accumulate credits on an invisible scorecard by promoting some low-skilled
community members (giving them higher ratings than they deserve) and then
spend these credits on self-interested strategic evaluations (giving some
rivals lower ratings and self-promoting their own work). Often, the more
individuals are invested in the community, the stronger their perceived need
to atone for the violation of community norms may be (Ashforth and Johnson,
2001). From the perspective of contest theory, leniency is unexpected
collaborative behavior because promoting anyone other than oneself is counter
to the self-interest of winning the contest. However, this leniency is not
simply a collaborative act within the community. It has its own strategic
nature as it is targeted toward those who least threaten one’s own chance of
winning. Recognizing that showing leniency toward low-skilled community
members is strategically motivated has far-reaching implications for our
understanding of interactions in communities (e.g., Wasko and Faraj, 2005;
Faraj and Johnson, 2011; Ren et al., 2012). Future models of behavior in
online communities need to consider not only interactions themselves but also
how they are motivated. Thus, leniency appears to be a crucial collaborative
element that allows individuals to better justify self-interested strategic
evaluations. It alleviates the tension between the mismatched collaborative
organization structure and the competitive incentive (Gallus et al., 2022).

Second, our dyad-level approach offers complementary insights to previous
studies that examined participation motivations at the individual level (e.g.,
Hippel and Krogh, 2003; Wasko and Faraj, 2005). Prior work focuses on
competitive and collaborative participation motives as individual-level traits
(some individuals are competitive while others are collaborative; Lakhani and
Wolf, 2003; Reuben et al., 2015; Erat and Gneezy, 2012; Belenzon and
Schankerman, 2015) to explain why community members behave in certain ways. By
contrast, we explain how the contextual factor of the competitive situation
explains behavior. The same individual in the same contest may act
competitively toward one individual yet show leniency toward another. Our
theory and empirical results imply that within individual changes in skill and
contextual factors (whether the individual is itself competing for the prize
vs. a neutral outsider, and who the target of the behavior is) make some
interactions more (or less) competitive and explain the observed microlevel
behavior. When community members are not themselves competing, they evaluate
their peers fairly with equal treatment of everyone (i.e., according to
equality matching; Fiske, 1992). When they are competing, the evaluations
depend on the competitiveness of the situation. The evaluation of those who do
not threaten a community member’s own chance of winning fall into a different
category of relational behavior. Those evaluations are not governed by the
competitive participation motivation (i.e., they are not based on a rational
cost-benefit calculation of market pricing; Fiske, 1992) and are instead
evaluated under a collaborative scheme of kindness and selfless generosity
(i.e., under communal sharing; Fiske, 1992). As a result, skill (and social
confirmation of skill from past success) is a key factor that shapes the
strength and saliency of the self-interested motivation because it affects the
gains individuals can expect from such behavior and can thus explain whether
individuals wear a competitive or collaborative hat (Ashforth and Johnson,
2001). Differentiating behavior based on the target allows individuals to be
competitive and cooperative at the same time, thus alleviating the tension
(cf. Waldman et al., 2019). Individuals may have joined the community out of
intrinsic motivation but once they have high skill (and have experienced
success), the competitive motive takes over. This contributes to recent
research which has started to acknowledge that there are changes and growth
over individual “careers” in communities as they learn and improve their skill
(Riedl and Seidel, 2018; Soda et al., 2021; Smirnova et al., 2022; Dahlander
and O’Mahony, 2011).

Our dyad-level investigation also adds to the nascent crowdsourcing literature
that has started to acknowledge the existence of strategic behaviors
(Hofstetter et al., 2018a; Archak and Sundararajan, 2009; Liu et al., 2014;
Hutter et al., 2011; Chen et al., 2020; Klapper et al., 2023; Deodhar et al.,
2022) by providing well-identified empirical evidence for both sabotage and
self-promotion in a prominent crowdsourcing platform. In particular, our
findings complement prior research on peer evaluations by highlighting the
importance of competition and covertness of evaluations: While Klapper et al.
(2023) find that Wikipedia editors target negative evaluations at community
members who are unlikely to retaliate, we show that when in direct
competition—as it is the case in many crowdsourcing settings—negative
evaluations are targeted towards the most prolific competitors. Furthermore,
positive evaluations are more likely to be granted to those lower-skilled
competitors who matter little for the overall outcome of the contest. Thus the
dynamic reverses compared to the findings of Klapper et al. (2023). This has
important implications for crowdsourcing community organizers as it
underscores the important roles of competition and transparency when designing
the interactions of community members. While non-transparent peer evaluations
limit the possibility to use peer evaluations as a means to portray oneself,
it does reduce the costs to engage in sabotage behavior towards competitors.
For settings where competition is prevalent it also highlights the usefulness
of game-theoretic considerations to assess heterogeneous evaluation patterns.

On a more general note, we add important field evidence to the existing
economics literature on sabotage that has mostly relied on lab experiments
(Lazear, 1989; Charness et al., 2014; Harbring and Irlenbusch, 2011; Konrad,
2000) and found ambiguous evidence for the association between sabotage and
skill (Charness et al., 2014; Harbring et al., 2007; Chambers and Baker,
2020).

Third, our work contributes to our understanding of the evolution and
sustainability of online communities by showing how individual-level strategic
behavior affects the organizational-level social structure of communities (Kim
et al., 2018; Faraj and Johnson, 2011; Huang et al., 2014; Piezunka and
Dahlander, 2019; Hofstetter et al., 2018b). We document a self-reinforcing
dynamic in which increased member skill not only leads to more strategic
behavior but also increased future participation. This complements past
studies that have investigated the fluidity of communities focused on aspects
of self-selection and open boundaries—i.e., dynamics across individuals—with
insights of dynamics within individuals (Faraj et al., 2011; Felin et al.,
2017). Contrary to the expectation that unfair behavior would reduce
engagement (e.g., Gomez-Mejia and Balkin, 1992; Faullant et al., 2017; Franke
et al., 2013; Balietti and Riedl, 2021), strategic behavior appears to
facilitate future participation because it encourages low-skilled individuals
with leniency and engages high-skilled community members in challenging
competition through sabotage. Strategic behavior appears to play an important
organizational role in stabilizing the core of a community by engaging members
in intense competition.

Why does being a victim of sabotage make highly skilled community members more
likely to participate in future contests? We theorize that the fierce
competition among the high-skilled adds to their intrinsic motivation and
promotion focus. Competition can be thrilling and exciting, even when outcomes
are negative (Franken and Brown, 1995). Intrinsic motivation and corresponding
theories may provide an explanation. First, according to self-determination
theory (Ryan and Deci, 2000) skilled individuals may experience a sense of
competence and autonomy that can enhance their motivation. Having been a
victim of sabotage could serve as a confirmation of being considered a true
competitor, which could increase motivation and thus the likelihood of future
participation. Furthermore, the fierce competition among the high-skilled adds
to the status incentive of the competition and may be perceived as a challenge
to overcome, rather than a threat (To et al., 2020). Second, the challenge of
competition is a trigger of a flow state (Csikszentmihalyi, 1990). Higher-
skilled individuals have a wider range in which flow can occur when the
challenge-level rises compared to their lower-skilled counterparts. Being a
victim of sabotage may thus increase the perceived challenge: while low-
skilled individuals perceive sabotage as a threat, high-skilled individuals
perceive it as a challenge (promotion focus To et al., 2020), making their
participation in future contests more likely.

Prior work has focused on collaborative behavior and explained why members in
crowdsourcing act reciprocally toward others who have helped them in the past
(Safadi et al., 2021; Perry-Smith, 2006; Jeppesen and Lakhani, 2010; Dahlander
et al., 2016). By contrast, we explain why apparent reciprocity in terms of
social network structure may also result from competitive behavior in which
high-skilled members sabotage other high-skilled community members. We thus
provide an additional explanation for the phenomenon of a consolidated
community core of high-skill individuals that is a key characteristic of many
crowdsourcing communities (Safadi et al., 2021; Perry-Smith, 2006; Jeppesen
and Lakhani, 2010; Dahlander et al., 2016). Our contest theory model and
empirical results imply that behaviors (and resulting network structures)
which appear to be reciprocal and collaborative can in fact be deeply
competitive. Together, this shows why strategic behavior must be incorporated
in our theoretical understanding of crowdsourcing communities that have often
been studied using structural social network approaches. The understanding of
the implications of social structure in communities could therefore benefit
from a more systematic integration of, and attention to, how actors’
behavioral motivation guides their behavior.

With the proliferation of social network studies focused on static structural
aspects (Safadi et al., 2021; Perry-Smith, 2006; Jeppesen and Lakhani, 2010;
Dahlander et al., 2016), the feedback loops between dynamic interactions and
their effect on structure suggest important questions for future research
(e.g., Soda et al., 2021; Fulker et al., 2021; Foley et al., 2021). Our study
is a first step in that direction. It provides an alternative explanation for
why communities often have a core-periphery structure. Future work needs to
consider not only the structure of community interaction itself but also the
valence of interaction (e.g., are comments helpful or hurtful? Are ratings
positive or negative?) and how the interactions are motivated. This may reveal
more nuanced network structures in terms “negative” (competitive) ties
(Labianca and Brass, 2006), adding to our understanding of social networks
from structural analysis which has focused on ties based on social
relationships (e.g., friendship), communication, and information flow
(Wasserman and Faust, 1994; Borgatti et al., 2009).

###  7.2 Practical Implications

Our findings also have important practical implications. Counter to common
sense, platform design that allows strategic behavior may not necessarily be
bad. From an organizational perspective, strategic behavior is usually seen as
destructive and thus undesirable. According to this view, organizations are
well advised to limit its impact by designing incentives (and structures) that
reduce strategic behavior. Our study reveals that this view is not always
valid. The fierce competition among high-skilled community members which makes
them targets of sabotage increases their future participation. It also leads
to the encouragement of less skilled members through leniency. As a result,
the competitive strategic behavior that ostensibly runs counter to the notion
of a collaborative community does in fact have positive long-term effects on
the community. Therefore, our study challenges conventional wisdom on
strategic behavior: instead of eliminating strategic behavior entirely, the
organizers of crowdsourcing communities may encourage it within controlled
boundaries (e.g., anonymous voting).

###  7.3 Limitations, Generalizability, and Future Research

This paper is not without limitations. First, while Threadless is a
prototypical crowdsourcing community and therefore highly suitable for
generalization, there may still be some differences to other settings. For
example, Threadless does not publish an overall ranking which may limit
competitive motivation compared to other settings (e.g., Topcoder publishes
public rankings which can spur rivalry; see Grad et al., 2023). Future
research could therefore investigate the patterns of strategic behaviors in
settings that differ from ours. The contest theory model can serve as a
starting point for that research. The findings may apply to other contexts
beyond crowdsourcing. As the theoretical reasoning and empirical setting of
this study is concerned with individuals anonymously evaluating their rivals
in an otherwise collaborative environment, the study’s results may generalize
to contexts with similar characteristics such as peer evaluation in teams,
academic publishing, grant awarding, or political races. Second, this paper
focuses on two specific forms of strategic behavior on the individual level.
The study of other forms of strategic behaviors that emerge between two or
more participants over time, such as interpersonal rivalry and reciprocity,
are exciting avenues for future research. Finally, our work opens new fields
to investigate career dynamics in communities and the connection between
individual behaviors and organizational-level outcomes like overall community
structures.

## Acknowledgements

We gratefully acknowledge helpful comments by Jennifer Brown, Blair Davey,
Imke Reimers, Ulrich Berger, Ben Greiner as well as participants at the NBER
Summer Institute 2019, and participants of the Digital Innovation Workshop
2019 at Boston College. This research was funded in part by the National
Science Foundation [Grant IIS-1514283] and benefitted from computing
infrastructure available at the Network Science Institute funded by the US
Office of Naval Research [N00014-17-1-2542].

## References

  * Aadland et al. (2019) Aadland, E., Cattani, G., and Ferriani, S. (2019).  Friends, gifts, and cliques: social proximity and recognition in peer-based tournament rituals.  Academy of Management Journal, 62(3):883–917.
  * Abeler et al. (2019) Abeler, J., Nosenzo, D., and Raymond, C. (2019).  Preferences for truth-telling.  Econometrica, 87(4):1115–1153.
  * Adler and Chen (2011) Adler, P. S. and Chen, C. X. (2011).  Combining creativity and control: Understanding individual motivation in large-scale collaborative creativity.  Accounting, Organizations and Society, 36(2):63–85.
  * Afuah and Tucci (2023) Afuah, A. and Tucci, C. L. (2023).  Reflections on the 2022 amr decade award: crowdsourcing as a solution to distant search.  Academy of Management Review, 48(4):597–610.
  * Angrist (2001) Angrist, J. D. (2001).  Estimation of Limited Dependent Variable Models With Dummy Endogenous Regressors.  Journal of Business & Economic Statistics, 19(1):2–28.
  * Archak (2010) Archak, N. (2010).  Money, glory and cheap talk: analyzing strategic behavior of contestants in simultaneous crowdsourcing contests on TopCoder.com.  In Proceedings of the 19th International Conference on World Wide Web, pages 21–30.
  * Archak and Sundararajan (2009) Archak, N. and Sundararajan, A. (2009).  Optimal design of crowdsourcing contests.  In Proceedings of International Conference on Information Systems, page 200. Association for Information Systems.
  * Ashforth and Johnson (2001) Ashforth, B. E. and Johnson, S. A. (2001).  Which hat to wear.  In Social Identity Processes in Organizational Contexts, pages 31–48. Psychology Press, New York, NY.
  * Balafoutas et al. (2012) Balafoutas, L., Lindner, F., and Sutter, M. (2012).  Sabotage in Tournaments: Evidence from a Natural Experiment.  Kyklos, 65(4):425–441.
  * Balietti and Riedl (2021) Balietti, S. and Riedl, C. (2021).  Incentives, competition, and inequality in markets for creative production.  Research Policy, 50(4):104212.
  * Bauer et al. (2016) Bauer, J., Franke, N., and Tuertscher, P. (2016).  Intellectual Property Norms in Online Communities : How User-Organized Intellectual Property Regulation Supports Innovation.  Information Systems Research, 27(4):724–750.
  * Belenzon and Schankerman (2015) Belenzon, S. and Schankerman, M. (2015).  Motivation and sorting of human capital in open innovation.  Strategic Management Journal, 36(6):795–820.
  * Benabou and Tirole (2002) Benabou, R. and Tirole, J. (2002).  Self-Confidence and Personal Motivation.  Quarterly Journal of Economics, 117(3):871–915.
  * Berg (2016) Berg, J. M. (2016).  Balancing on the creative highwire: Forecasting the success of novel ideas in organizations.  Administrative Science Quarterly, 61(3):433–468.
  * Blanken et al. (2015) Blanken, I., Van De Ven, N., and Zeelenberg, M. (2015).  A meta-analytic review of moral licensing.  Personality and Social Psychology Bulletin, 41(4):540–558.
  * Blohm et al. (2016) Blohm, I., Riedl, C., Füller, J., and Leimeister, J. M. (2016).  Rate or trade? identifying winning ideas in open idea sourcing.  Information Systems Research, 27(1):27–48.
  * Bockstedt et al. (2016) Bockstedt, J., Druehl, C., and Mishra, A. (2016).  Heterogeneous Submission Behavior and its Implications for Success in Innovation Contests with Public Submissions.  Production and Operations Management, 25(7):1157–1176.
  * Borgatti et al. (2009) Borgatti, S. P., Mehra, A., Brass, D. J., and Labianca, G. (2009).  Network analysis in the social sciences.  Science, 323(5916):892–895.
  * Boudreau and Lakhani (2015) Boudreau, K. J. and Lakhani, K. R. (2015).  “Open” disclosure of innovations, incentives and follow-on reuse: Theory on processes of cumulative innovation and a field experiment in computational biology.  Research Policy, 44(1):4–19.
  * Boudreau et al. (2016) Boudreau, K. J., Lakhani, K. R., and Menietti, M. (2016).  Performance responses to competition across skill levels in rank-order tournaments: field evidence and implications for tournament design.  RAND Journal of Economics, 47(1):140–165.
  * Brabham (2010) Brabham, D. C. (2010).  Moving the crowd at threadless.  Information, Communication & Society, 13(8):1122–1145.
  * Bullinger et al. (2010) Bullinger, A. C., Neyer, A.-K., Rass, M., and Moeslein, K. M. (2010).  Community-based innovation contests: Where competition meets cooperation.  Creativity and Innovation Management, 19(3):290–303.
  * Camerer and Lovallo (1999) Camerer, C. and Lovallo, D. (1999).  Overconfidence and Excess Entry: An Experimental Approach.  American Economic Review, 89(1):306–318.
  * Carpenter et al. (2010) Carpenter, J., Matthews, P. H., and Schirm, J. (2010).  Tournaments and office politics: Evidence from a real effort experiment.  American Economic Review, 100(1):504–17.
  * Chambers and Baker (2020) Chambers, C. R. and Baker, W. E. (2020).  Robust systems of cooperation in the presence of rankings: how displaying prosocial contributions can offset the disruptive effects of performance rankings.  Organization Science, 31(2):287–307.
  * Charness et al. (2014) Charness, G., Masclet, D., and Villeval, M. C. (2014).  The Dark Side of Competition for Status.  Management Science, 60(1):38–55.
  * Chen (2003) Chen, K.-P. (2003).  Sabotage in promotion tournaments.  Journal of Law, Economics, and Organization, 19(1):119–140.
  * Chen et al. (2020) Chen, P., Sun, H., Fang, Y., and Liu, X. (2020).  Conan: A framework for detecting and handling collusion in crowdsourcing.  Information Sciences, 515(April):44–63.
  * Chiu et al. (2015) Chiu, C.-M., Huang, H.-Y., Cheng, H.-L., and Sun, P.-C. (2015).  Understanding online community citizenship behaviors through social support and social identity.  International Journal of Information Management, 35(4):504–519.
  * Cropanzano et al. (2005) Cropanzano, R., Goldman, B., and Folger, R. (2005).  Self-interest: Defining and understanding a human motive.  Journal of Organizational Behavior: The International Journal of Industrial, Occupational and Organizational Psychology and Behavior, 26(8):985–991.
  * Csikszentmihalyi (1990) Csikszentmihalyi, M. (1990).  Flow: The Psychology of Optimal Experience.  Harper & Row, New York, NY.
  * Dahlander et al. (2019) Dahlander, L., Jeppesen, L. B., and Piezunka, H. (2019).  How organizations manage crowds: Define, broadcast, attract, and select.  In Sydow, J. and Berends, H., editors, Managing Inter-organizational Collaborations: Process Views (Research in the Sociology of Organizations, Vol. 64), pages 239–270. Emerald, Leeds, UK.
  * Dahlander and O’Mahony (2011) Dahlander, L. and O’Mahony, S. (2011).  Progressing to the center: Coordinating project work.  Organization Science, 22(4):961–979.
  * Dahlander et al. (2016) Dahlander, L., O’Mahony, S., and Gann, D. M. (2016).  One foot in, one foot out: how does individuals’ external search breadth affect innovation outcomes?  Strategic Management Journal, 37(2):280–302.
  * Dato and Nieken (2014) Dato, S. and Nieken, P. (2014).  Gender differences in competition and sabotage.  Journal of Economic Behavior & Organization, 100(April):64–80.
  * Deodhar et al. (2022) Deodhar, S. J., Babar, Y., and Burtch, G. (2022).  The influence of status on evaluations: Evidence from online coding contests.  MIS Quarterly, 46(4):2085–2110.
  * Deutsch (1949) Deutsch, M. (1949).  A theory of co-operation and competition.  Human Relations, 2(2):129–152.
  * Deutscher et al. (2013) Deutscher, C., Frick, B., Gürtler, O., and Prinz, J. (2013).  Sabotage in Tournaments with Heterogeneous Contestants: Empirical Evidence from the Soccer Pitch.  Scandinavian Journal of Economics, 115(4):1138–1157.
  * Dixit (1987) Dixit, A. (1987).  Strategic Behavior in Contests.  American Economic Review, 77(5):891– 898.
  * Drago and Garvey (1998) Drago, R. and Garvey, G. T. (1998).  Incentives for Helping on the Job: Theory and Evidence.  Journal of Labor Economics, 16(1):1–25.
  * Edelman and Larkin (2015) Edelman, B. and Larkin, I. (2015).  Social comparisons and deception across workplace hierarchies: Field and experimental evidence.  Organization Science, 26(1):78–98.
  * Elster (1989) Elster, J. (1989).  Social norms and economic theory.  Journal of Economic Perspectives, 3(4):99–117.
  * Erat and Gneezy (2012) Erat, S. and Gneezy, U. (2012).  White lies.  Management Science, 58(4):723–733.
  * Faraj et al. (2011) Faraj, S., Jarvenpaa, S. L., and Majchrzak, A. (2011).  Knowledge collaboration in online communities.  Organization Science, 22(5):1224–1239.
  * Faraj and Johnson (2011) Faraj, S. and Johnson, S. L. (2011).  Network exchange patterns in online communities.  Organization Science, 22(6):1464–1480.
  * Faullant et al. (2017) Faullant, R., Füller, J., and Hutter, K. (2017).  Fair play: Perceived fairness in crowdsourcing competitions and the customer relationship-related consequences.  Management Decision, 55(9):1924–1941.
  * Felin et al. (2017) Felin, T., Lakhani, K. R., and Tushman, M. L. (2017).  Firms, crowds, and innovation.  Strategic Organization, 15(2):119–140.
  * Fiske (1992) Fiske, A. P. (1992).  The four elementary forms of sociality: framework for a unified theory of social relations.  Psychological Review, 99(4):689–723.
  * Fjeldstad et al. (2012) Fjeldstad, Ø. D., Snow, C. C., Miles, R. E., and Lettl, C. (2012).  The architecture of collaboration.  Strategic Management Journal, 33(6):734–750.
  * Foley et al. (2021) Foley, M., Smead, R., Forber, P., and Riedl, C. (2021).  Avoiding the bullies: The resilience of cooperation among unequals.  PLoS Computational Biology, 17(4):e1008847.
  * Franke et al. (2013) Franke, N., Keinz, P., and Klausberger, K. (2013).  “Does This Sound Like a Fair Deal?”: Antecedents and Consequences of Fairness Expectations in the Individual’s Decision to Participate in Firm Innovation.  Organization Science, 24(5):1495–1516.
  * Franke et al. (2010) Franke, N., Schreier, M., and Kaiser, U. (2010).  The “I Designed It Myself” Effect in Mass Customization.  Management Science, 56(1):125–140.
  * Franke and Shah (2003) Franke, N. and Shah, S. (2003).  How communities support innovative activities: an exploration of assistance and sharing among end-users.  Research Policy, 32(1):157–178.
  * Franken and Brown (1995) Franken, R. E. and Brown, D. J. (1995).  Why do people like competition? the motivation for winning, putting forth effort, improving one’s performance, performing well, being instrumental, and expressing forceful/aggressive behavior.  Personality and Individual Differences, 19(2):175–184.
  * Fulker et al. (2021) Fulker, Z., Forber, P., Smead, R., and Riedl, C. (2021).  Spite is contagious in dynamic networks.  Nature Communications, 12(1):1–9.
  * Gallus et al. (2022) Gallus, J., Reiff, J., Kamenica, E., and Fiske, A. P. (2022).  Relational incentives theory.  Psychological Review, 129(3):586–602.
  * Garicano et al. (2005) Garicano, L., Palacios-Huerta, I., and Prendergast, C. (2005).  Favoritism under social pressure.  Review of Economics and Statistics, 87(2):208–216.
  * Gaure (2013) Gaure, S. (2013).  lfe: Linear group fixed effects.  The R Journal, 5(2):104–116.
  * Gebauer et al. (2013) Gebauer, J., Füller, J., and Pezzei, R. (2013).  The dark and the bright side of co-creation: Triggers of member behavior in online innovation communities.  Journal of Business Research, 66(9):1516–1527.
  * Gneezy et al. (2018) Gneezy, U., Kajackaite, A., and Sobel, J. (2018).  Lying aversion and the size of the lie.  American Economic Review, 108(2):419–453.
  * Gomez-Mejia and Balkin (1992) Gomez-Mejia, L. R. and Balkin, D. B. (1992).  Determinants of faculty pay: An agency theory perspective.  Academy of Management Journal, 35(5):921–955.
  * Grad et al. (2023) Grad, T., Riedl, C., and Kilduff, G. J. (2023).  When rivalry backfires: How individual skill and risk of status loss moderate the effects of rivalry on performance.  Working Paper.
  * Greene (2012) Greene, W. W. H. . (2012).  Econometric analysis.  Pearson, New York, NY, 6th edition.
  * Haas and Park (2010) Haas, M. R. and Park, S. (2010).  To share or not to share? Professional norms, reference groups, and information withholding among life scientists.  Organization Science, 21(4):873–891.
  * Harbring and Irlenbusch (2011) Harbring, C. and Irlenbusch, B. (2011).  Sabotage in tournaments: Evidence from a laboratory experiment.  Management Science, 57(4):611–627.
  * Harbring et al. (2007) Harbring, C., Irlenbusch, B., Kräkel, M., and Selten, R. (2007).  Sabotage in corporate contests–an experimental analysis.  International Journal of the Economics of Business, 14(3):367–392.
  * Harhoff et al. (2003) Harhoff, D., Henkel, J., and Von Hippel, E. (2003).  Profiting from voluntary information spillovers: how users benefit by freely revealing their innovations.  Research Policy, 32(10):1753–1769.
  * Hippel and Krogh (2003) Hippel, E. v. and Krogh, G. v. (2003).  Open source software and the “private-collective” innovation model: Issues for organization science.  Organization Science, 14(2):209–223.
  * Hofstetter et al. (2018a) Hofstetter, R., Aryobsei, S., and Herrmann, A. (2018a).  Should you really produce what consumers like online? empirical evidence for reciprocal voting in open innovation contests.  Journal of Product Innovation Management, 35(2):209–229.
  * Hofstetter et al. (2018b) Hofstetter, R., Zhang, J. Z., and Herrmann, A. (2018b).  Successive open innovation contests and incentives: winner-take-all or multiple prizes?  Journal of Product Innovation Management, 35(4):492–517.
  * Huang et al. (2014) Huang, Y., Vir Singh, P., and Srinivasan, K. (2014).  Crowdsourcing new product ideas under consumer learning.  Management Science, 60(9):2138–2159.
  * Hutter et al. (2011) Hutter, K., Hautz, J., Füller, J., Mueller, J., and Matzler, K. (2011).  Communitition: The tension between competition and collaboration in community-based design contests.  Creativity and Innovation Management, 20(1):3–21.
  * Ivaturi and Chua (2019) Ivaturi, K. and Chua, C. (2019).  Framing norms in online communities.  Information & Management, 56(1):15–27.
  * Jeppesen and Lakhani (2010) Jeppesen, L. B. and Lakhani, K. R. (2010).  Marginality and problem-solving effectiveness in broadcast search.  Organization Science, 21(5):1016–1033.
  * Keum and See (2017) Keum, D. D. and See, K. E. (2017).  The influence of hierarchy on idea generation and selection in the innovation process.  Organization Science, 28(4):653–669.
  * Kilduff et al. (2010) Kilduff, G. J., Elfenbein, H. A., and Staw, B. M. (2010).  The psychology of rivalry: A relationally dependent analysis of competition.  Academy of Management Journal, 53(5):943–969.
  * Kilduff et al. (2016) Kilduff, G. J., Galinsky, A. D., Gallo, E., and Reade, J. J. (2016).  Whatever it takes to win: Rivalry increases unethical behavior.  Academy of Management Journal, 59(5):1508–1534.
  * Kim et al. (2018) Kim, Y., Jarvenpaa, S. L., and Gu, B. (2018).  External bridging and internal bonding: Unlocking the generative resources of member time and attention spent in online communities.  MIS Quarterly, 42(1):265–283.
  * Klapper et al. (2023) Klapper, H., Piezunka, H., and Dahlander, L. (2023).  Peer evaluations: Evaluating and being evaluated.  Organization Science, (in press).
  * Knudsen et al. (2019) Knudsen, T., A. Levinthal, D., and Puranam, P. (2019).  Editorial: A model is a model.  Strategy Science, 4(1):1–3.
  * Konrad (2000) Konrad, K. (2000).  Sabotage in rent-seeking contests.  Journal of Law, Economics, and Organization, 16(1):155–165.
  * Konrad (2009) Konrad, K. A. (2009).  Strategy and Dynamics in Contests.  Oxford University Press, Oxford, UK.
  * Labianca and Brass (2006) Labianca, G. and Brass, D. J. (2006).  Exploring the social ledger: negative relationships and negative asymmetry in social networks in organizations.  Academy of Management Review, 31(3):596–614.
  * Lado et al. (1997) Lado, A. A., Boyd, N. G., and Hanlon, S. C. (1997).  Competition, cooperation, and the search for economic rents: A syncretic model.  Academy of Management Review, 22(1):110–141.
  * Lakhani and Kanji (2008) Lakhani, K. R. and Kanji, Z. (2008).  Threadless: The Business of Community.  Harvard Business School Multimedia/Video Case.
  * Lakhani and Wolf (2003) Lakhani, K. R. and Wolf, R. G. (2003).  Why hackers do what they do: Understanding motivation and effort in free/open source software projects.  Technical Report September 2003, Open Source Software Projects.
  * Lazear (1989) Lazear, E. P. (1989).  Pay Equality and Industrial Politics.  Journal of Political Economy, 97(3):561.
  * Lazear and Rosen (1981) Lazear, E. P. and Rosen, S. (1981).  Rank-order tournaments as optimum labor contracts.  Journal of Political Economy, 89(5):841–864.
  * Lerchenmueller et al. (2019) Lerchenmueller, M. J., Sorenson, O., and Jena, A. B. (2019).  Gender differences in how scientists present the importance of their research: observational study.  BMJ, 367.
  * Lewis (2000) Lewis, M. W. (2000).  Exploring paradox: Toward a more comprehensive guide.  Academy of Management Review, 25(4):760–776.
  * Li (2017) Li, D. (2017).  Expertise versus bias in evaluation: Evidence from the NIH.  American Economic Journal: Applied Economics, 9(2):60–92.
  * Li et al. (2007) Li, H., Bingham, J. B., and Umphress, E. E. (2007).  Fairness from the top: Perceived procedural justice and collaborative problem solving in new product development.  Organization Science, 18(2):200–216.
  * Liu et al. (2014) Liu, T. X., Yang, J., Adamic, L. A., and Chen, Y. (2014).  Crowdsourcing with all-pay auctions: A field experiment on taskcn.  Management Science, 60(8):2020–2037.
  * Magee and Galinsky (2008) Magee, J. C. and Galinsky, A. D. (2008).  Social hierarchy: The self-reinforcing nature of power and status.  Academy of Management Annals, 2(1):351–398.
  * Majchrzak and Malhotra (2013) Majchrzak, A. and Malhotra, A. (2013).  Towards an information systems perspective and research agenda on crowdsourcing for innovation.  Journal of Strategic Information Systems, 22(4):257–268.
  * Majchrzak and Malhotra (2016) Majchrzak, A. and Malhotra, A. (2016).  Effect of knowledge-sharing trajectories on innovative outcomes in temporary online crowds.  Information Systems Research, 27(4):685–703.
  * Merritt et al. (2010) Merritt, A. C., Effron, D. A., and Monin, B. (2010).  Moral self-licensing: When being good frees us to be bad.  Social and Personality Psychology Compass, 4(5):344–357.
  * Milgrom and Roberts (1988) Milgrom, P. and Roberts, J. (1988).  An Economic Approach to Influence Activities in Organizations.  American Journal of Sociology, 94:154–179.
  * Moffitt (2001) Moffitt, R. (2001).  Policy interventions, low-level equilibria, and social interactions.  In Durlauf, S. and Young, P., editors, Social Dynamics. MIT Press, Cambridge, MA.
  * Münster (2007) Münster, J. (2007).  Selection tournaments, sabotage, and participation.  Journal of Economics and Management Strategy, 16(4):943–970.
  * Nambisan and Baron (2010) Nambisan, S. and Baron, R. A. (2010).  Different roles, different strokes: Organizing virtual customer environments to promote two types of customer contributions.  Organization Science, 21(2):554–572.
  * Nickell (2010) Nickell, J. (2010).  Threadless: Ten Years of T-shirts from the World’s Most Inspiring Online Design Community.  Abrams Image, New York, NY.
  * Perry-Smith (2006) Perry-Smith, J. E. (2006).  Social yet creative: The role of social relationships in facilitating individual creativity.  Academy of Management Journal, 49(1):85–101.
  * Piezunka and Dahlander (2019) Piezunka, H. and Dahlander, L. (2019).  Idea rejected, tie formed: Organizations’ feedback on crowdsourced ideas.  Academy of Management Journal, 62(2):503–530.
  * Ransbotham and Kane (2011) Ransbotham, S. and Kane, G. C. (2011).  Membership turnover and collaboration success in online communities: Explaining rises and falls from grace in wikipedia.  MIS Quarterly, 35,(3):613–627.
  * Reitzig and Sorenson (2013) Reitzig, M. and Sorenson, O. (2013).  Biases in the selection stage of bottom-up strategy formulation.  Strategic Management Journal, 34(7):782–799.
  * Ren et al. (2012) Ren, Y., Harper, F. M., Drenner, S., Terveen, L., Kiesler, S., Riedl, J., and Kraut, R. E. (2012).  Building member attachment in online communities: Applying theories of group identity and interpersonal bonds.  MIS Quarterly, 36(3):841–864.
  * Reuben et al. (2015) Reuben, E., Sapienza, P., and Zingales, L. (2015).  Taste for competition and the gender gap among young business professionals.  Technical Report 21695, National Bureau of Economic Research.
  * Riedl et al. (2024) Riedl, C., Hutter, K., Füller, J., and Tellis, G. (2024).  Enhancing crowdsourcing contest outcomes: Choice and market context in incentive design.  Journal of Management Information Systems, (in press).
  * Riedl and Seidel (2018) Riedl, C. and Seidel, V. P. (2018).  Learning from mixed signals in online innovation communities.  Organization Science, 29(6):1010–1032.
  * Roberts et al. (2006) Roberts, J. A., Hann, I.-H., and Slaughter, S. A. (2006).  Understanding the motivations, participation, and performance of open source software developers: A longitudinal study of the apache projects.  Management Science, 52(7):984–999.
  * Ryan and Deci (2000) Ryan, R. M. and Deci, E. L. (2000).  Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being.  American Psychologist, 55(1):68.
  * Safadi et al. (2021) Safadi, H., Johnson, S. L., and Faraj, S. (2021).  Who contributes Knowledge? Core-periphery tension in online innovation communities.  Organization Science, 32(3):752–775.
  * Scheiner et al. (2018) Scheiner, C. W., Baccarella, C. V., Bessant, J., and Voigt, K.-I. (2018).  Participation motives, moral disengagement, and unethical behaviour in idea competitions.  International Journal of Innovation Management, 22(06):1850043.
  * Schotter and Weigelt (1992) Schotter, A. and Weigelt, K. (1992).  Asymmetric tournaments, equal opportunity laws, and affirmative action: Some experimental results.  Quarterly Journal of Economics, 107(2):511–539.
  * Smirnova et al. (2022) Smirnova, I., Reitzig, M., and Sorenson, O. (2022).  Building status in an online community.  Organization Science, 33(6):2519–2540.
  * Smith (2010) Smith, A. (1759 (2010)).  The theory of moral sentiments.  Penguin, London, UK.
  * Soda et al. (2021) Soda, G., Mannucci, P. V., and Burt, R. S. (2021).  Networks, creativity, and time: Staying creative through brokerage and network rejuvenation.  Academy of Management Journal, 64(4):1164–1190.
  * Therneau and Grambsch (2000) Therneau, T. M. and Grambsch, P. M. (2000).  Modeling Survival Data: Extending the Cox Model.  Springer, New York, NY.
  * To et al. (2020) To, C., Kilduff, G. J., and Rosikiewicz, B. L. (2020).  When interpersonal competition helps and when it harms: An integration via challenge and threat.  Academy of Management Annals, 14(2):908–934.
  * Tsai (2002) Tsai, W. (2002).  Social structure of “coopetition” within a multiunit organization: Coordination, competition, and intraorganizational knowledge sharing.  Organization Science, 13(2):179–190.
  * Tullock (1980) Tullock, G. (1980).  Efficient rent seeking.  In Buchanan, J., Tollison, R., and Tullock, G., editors, Toward a theory of the rent-seeking society. Texas A&M University Press, College Station, TX.
  * Waldman et al. (2019) Waldman, D. A., Putnam, L. L., Miron-Spektor, E., and Siegel, D. (2019).  The role of paradox theory in decision making and management research.  Organizational Behavior and Human Decision Processes, 155(November):1–6.
  * Wasko and Faraj (2000) Wasko, M. M. and Faraj, S. (2000).  “it is what one does”: Why people participate and help others in electronic communities of practice.  Journal of Strategic Information Systems, 9(2-3):155–173.
  * Wasko and Faraj (2005) Wasko, M. M. and Faraj, S. (2005).  Why should i share? examining social capital and knowledge contribution in electronic networks of practice.  MIS Quarterly, 29(1):35–57.
  * Wasserman and Faust (1994) Wasserman, S. and Faust, K. (1994).  Social network analysis: Methods and applications.  Cambridge University Press, New York, NY.
  * Zaggl et al. (2023) Zaggl, M. A., Malhotra, A., Alexy, O., and Majchrzak, A. (2023).  Governing crowdsourcing for unconstrained innovation problems.  Strategic Management Journal, in press.

## Appendix

##  Appendix A A Brief Review of the Economics Literature on Strategic
Behavior

The threat of sabotage in which actors engage in non-productive effort to
reduce the performance of another actor has been at the forefront of economic
literature and noted in the earliest work on rank-order tournaments Lazear
(1989); Dixit (1987). However, past empirical work on sabotage is scarce.
Notable exceptions are the work on withholding helping behavior (Drago and
Garvey, 1998; Haas and Park, 2010), which can be considered a weaker form of
sabotage. The dearth of empirical insight into sabotage is (at least in part)
attributable to the challenge of investigating sabotage empirically: actors
try to hide their opportunistic behavior due to its immoral connotation so
that “company data on sabotage is generally not available for research” (Dato
and Nieken, 2014, p. 65).

Self-promotion is somewhat better understood empirically (Li, 2017; Keum and
See, 2017; Reitzig and Sorenson, 2013; Edelman and Larkin, 2015;
Lerchenmueller et al., 2019), but it is unclear how it interacts with sabotage
and when individuals would choose one form of strategic behavior over the
other (i.e., whether the two are complements or substitutes). Thus, there are
limited empirical insights into strategic behavior beyond theoretical work
(Lazear, 1989; Milgrom and Roberts, 1988; Münster, 2007), laboratory studies
(Harbring and Irlenbusch, 2011; Charness et al., 2014), or sports (Balafoutas
et al., 2012; Deutscher et al., 2013; Garicano et al., 2005).151515Of course,
there are many other forms of strategic behavior besides sabotage and self-
promotion in crowdsourcing and contest theory more generally. For example, Liu
et al. (2014) look at strategic entry decisions, Archak (2010) look at the use
of “cheap talk” to deter others from entering, Hutter et al. (2011) look at
instances in which individuals do not reciprocate cooperative efforts, and
Hofstetter et al. (2018a) look at socially motivated reciprocal rating (as
opposed to strategically motivated rating when actors are anonymous), Chen et
al. (2020) looks at collusion in micro-tasking where actors copy each others’
solutions, and Scheiner et al. (2018) study unethical behavior using survey
responses but not actual behavior.

Prior work from the economics literature has suggested that strategic behavior
is contingent on the skill of contestants (e.g., Schotter and Weigelt, 1992;
Dixit, 1987). However, existing empirical evidence is ambiguous. Some studies
find that low skill is associated with engagement in strategic behavior
(Carpenter et al., 2010; Charness et al., 2014) while others find evidence for
higher skill (Harbring and Irlenbusch, 2011). Yet others find no association
between skill and strategic behavior at all (Dato and Nieken, 2014). Hence,
the effect (or lack of it) remains unclear.

##  Appendix B Theoretical Framework

###  B.1 Model Setup in More Formalized Terms

We build on previous models of one-shot contest with sabotage (see, e.g.,
Harbring and Irlenbusch, 2011; Konrad, 2000), skill heterogeneous agents, and
a single winner prize. The contest consists of two types of agents: a set $H$
of $h$ high-skill agents, and a set $L$ of $l$ low-skill agents. We refer to
these as high and low types, respectively. Furthermore, the contest involves a
set $N$ of $n$ outsiders. We assume that $h<l<n$, which should be satisfied in
most realistic contest settings. Low and high types produce a contest
submission (i.e. idea) of low quality $b_{l}$ and high quality $b_{h}$,
respectively, which they enter into the contest. Outsiders do not enter the
contest and thus remain neutral. The submissions of all agents are rated by
each outsider and by each high and low type agent on a bounded quality scale
in $\mathbb{R}_{\geq 0}$ with $[r_{min},r_{max}]$. Without loss of generality,
we normalize the rating scale to the unit interval so that $r_{min}=0$ and
$r_{max}=1$. We assume that the quality bid of low type agents’ submissions is
$b_{l}$, whereas the quality bid of high type agents’ submissions is $b_{h}$
with $0<b_{l}<b_{h}<1$.

While ratings can be sincere (i.e., $b_{l}$ or $b_{h}$, respectively), an
agent can also sabotage any other agent, rating their submission at
$r_{min}=0$; or promote them by rating their submission at $r_{max}=1$. Let
$\Delta s_{h}=b_{h}$ be the damage done by sabotaging a high type and $\Delta
s_{l}=b_{l}$ be the damage done by sabotaging a low type. Since $b_{l}<b_{h}$,
it follows that the damage inflicted on a high type through sabotage is larger
than on a low type (i.e., $\Delta s_{l}<\Delta s_{h}$). Further, let $\Delta
sp_{h}=1-b_{h}$ denote the benefit gained from promoting a high type and
$\Delta sp_{l}=1-b_{l}$ denote the benefit gained from promoting a low type.
Since $b_{l}<b_{h}$ it follows that low types have more to gain from promotion
than high types (i.e., $\Delta sp_{l}>\Delta sp_{h}$).

We assume that while rating sincerely is free of costs, sabotaging another
agent costs $c_{s}$ and promoting any agent (including oneself) costs $c_{p}$.
Costs associated with promotion and sabotage might arise from the costs of
identifying suitable targets (Harbring et al., 2007; Münster, 2007), the moral
costs associated with lying (Abeler et al., 2019; Gneezy et al., 2018) or
violating social norms (Elster, 1989). As a consequence all $N$ outsiders rate
contestants $h$ and $l$ sincerely. This implies that the size $n$ of the set
of outsiders $N$ affects the effectiveness of any type of strategic behavior
performed by the set of $H$ and $L$ agents: the more outsiders there are, the
lower the effect that any individual strategic rating has. Effectively, $n$
can be considered a modifier in our model that governs how effective strategic
behavior is. Note also that outsiders $n$ are not necessarily agents who never
compete—they simply do not compete in the current contest. We will use this
feature to identify strategic behavior in our empirical analysis where
contestants do not enter every contest: they rate as neutral outsiders in some
weeks and rate as competitors with stakes from idea generation—and incentives
to rate strategically—in the contest in other weeks.

The rating given by agent $i$ to agent $j$ is denoted by $v_{ij}$. The value
of agent $j$’s submission is the sum of all the ratings that this focal agent
receives: $v_{j}=\sum_{i\in N\cup L\cup H}v_{ij}$. Throughout the remainder,
let $v_{h}$ denote the value of some arbitrary high type (she), and $v_{l}$
denote the value of some arbitrary low type (he). As an example, in the case
of sincere rating by everyone (no promotion and no sabotage), the value of a
high type is $v_{h}=b_{h}(n+l+h)$ and that of a low type is
$v_{l}=b_{l}(n+l+h)$. Finally, we also assume that the difference between
contest entries $b_{l}$ and $b_{h}$ is large enough so that the relative order
between high and low types cannot be changed through strategic behavior alone.
That is, $v_{l}<v_{h}$ even if all low types engage in self-promotion and all
high-types are sabotaged by everyone.

###  B.2 Utility Function

The winner of the contest is determined based on the evaluations from $N$,
$H$, and $L$. For simplicity, we model this as a Tullock contest (Tullock,
1980) where the probability of winning is proportional to the aggregate rating
$v_{i}$ plus an error term in relation to all other contest submissions. That
is, each agent $i$’s probability of winning the contest is
$p_{i}=\frac{v_{i}}{\sum_{j\in L\cup H}v_{j}}$. For simplicity, let
$S=\sum_{j\in L\cup H}v_{j}$ denote the total value of all contestants and we
write the Tullock contest success function as $p_{i}=\frac{v_{i}}{S}$. The
winner of the contest receives a prize $M$, which is without loss of
generality normalized to $1$. Given the equilibrium values for productive
effort and the resulting quality bid in the ordinary Tullock contest, agents
maximize their expected winning probability minus their total costs of
sabotage and cost of promotion. The utility function for agents’ strategic
contest behavior can then be written as:

$\displaystyle E\Pi_{i}=Mp_{i}-c_{s}(\sum_{j}sab_{ij})-c_{p}sprom_{ii}),$ (3)

where $M$ is the winner prize, $p_{i}$ is the probability of agent $i$ winning
the prize, $sab_{ij}$ ($prom_{ij}$) is $1$ if agent $i$ sabotages (promotes)
agent $j$ and $0$ otherwise, and $c_{s}$ and $c_{p}$ are the cost of sabotage
and promotion, respectively.

Formally, a strategy of agent $i$ is a list $(r_{ij})$ of rating behavior
towards all agents $j\in L\cup H$, where
$r_{ij}\in\\{\text{sincere},\text{sabotage},\text{promote}\\}$. Since
sabotaging oneself and promoting any agent other than oneself are clearly
strictly dominated and payoffs are symmetric with respect to actions towards
other agents of the same type, we only consider equivalence classes of
strategies of the form $s_{i}=(hsab_{i},lsab_{i},sprom_{i})$, where $hsab_{i}$
is the number of high type agents $i$ sabotages, $lsab_{i}$ the number of low
type agents $i$ sabotages, and $sprom_{i}$ takes the value $1$ or $0$,
indicating whether or not $i$ self-promotes. We now investigate the strategic
behavior for self-promotion and sabotage.

###  B.3 Self-Promotion

Self-promotion increases agent $i$’s value $v_{i}$ and total contest output
$S$ by the same amount and hence increases agent $i$’s chance of winning. The
size of the this increase of course depends on agent $i$’s type, the number of
other high and low types in the contest, and the number of outsiders.

###### Lemma 1

In contests with a sufficiently large performance gap $g$ between low and high
types, the expected gain from self-promotion is higher for a low type than a
high type.

##### Proof.

The expected gain from self-promotion for a currently non-self-promoting high
type agent $h$ is $\frac{v_{h}+\Delta sp_{h}}{S+\Delta
sp_{h}}-\frac{v_{h}}{S}=\frac{\Delta sp_{h}(S-v_{h})}{S(S+\Delta sp_{h})}$,
and $\frac{v_{l}+\Delta sp_{l}}{S+\Delta sp_{l}}-\frac{v_{l}}{S}=\frac{\Delta
sp_{l}(S-v_{l})}{S(S+\Delta sp_{l})}$ for a currently non-self-promoting low
type agent $l$. Considering the worst case value for a high skill agent gives
us $v_{h}\geq b_{h}(n+1)$ (i.e., sincere rating by all $n$ outsiders and $h$
herself, all $l$ and $h-1$ sabotage and rate $0$). Conversely, the best case
value for a low type gives us $v_{l}\leq b_{l}(n+l+h)$ (i.e., sincere rating
by everyone).

In a contest where the performance gap between high and low types is large
with respect to the proportion of neutral outsiders $n$ (who vote truthfully)
and the overall contest size (where $l$ and $h$ might potentially sabotage) we
get $S>v_{h}>v_{l}$. Define $g$ as the performance gap between high and low
types so that $v_{h}=gv_{l}$. Expressing $g$ as the inequality between the
worst case value for a high type (receiving sabotage from all other high types
and all low types) and the best case value for a low type (not being sabotaged
at all)

$\displaystyle g=\frac{b_{h}}{b_{l}}\frac{n+1}{n+l+h}.$ (4)

This expression gives a lower bound on $g$ for a contest with a positive
performance gap. A positive performance gap ensures that the high type values
$v_{h}$ are higher than the low type values $v_{l}$ even if the high types are
being sabotaged by all $H_{-i}$ and $L$ agents. This ensures that the
$b_{l}<b_{h}$ relationship also holds for the values $v_{l}<v_{h}$. Note that
this expresses the necessary size of the performance gap as a product of the
difference in production skill and the proportion of neutral outsiders to
overall contest size. Considering the smallest possible contest will have
$n=3,l=2,h=1$ provides a lower bound for $g$ so that

$\displaystyle
g=\frac{b_{h}}{b_{l}}\frac{n+1}{n+l+h}\geq\frac{b_{h}}{b_{l}}\frac{2}{3}$ (5)

Now consider a contest with a significant performance gap between high and low
types so that $g\geq 1$ holds, then we can compare the expected gain from
self-promotion between high and low types.

$\displaystyle\frac{\Delta sp_{l}(S-v_{l})}{S(S+\Delta sp_{l})}-\frac{\Delta
sp_{h}(S-v_{h})}{S(S+\Delta sp_{h})}=\frac{S((\Delta sp_{l}-\Delta
sp_{h})S+v_{h}-\Delta sp_{l}v_{l})+\Delta sp_{l}(v_{h}-v_{l})}{S(S+\Delta
sp_{h})(S+\Delta sp_{l})}.$ (6)