# Socio-technical constraints and affordances of virtual collaboration - A
study of four online hackathons

Wendy Mendes  0000-0002-1241-2489<EMAIL_ADDRESS>Federal University of
ParáBelémPABrazil ,  Albert Richard  0000-0001-9033-6533
<EMAIL_ADDRESS>Federal University of ParáBelémPABrazil ,  Tähe-Kai
Tillo<EMAIL_ADDRESS>University of TartuTartuEstonia ,  Gustavo Pinto
0000-0001-7598-2799<EMAIL_ADDRESS>Zup InnovationSão PauloSPBrazil Federal
University of ParáBelémPABrazil ,  Kiev Gama  0000-0003-1508-6196
<EMAIL_ADDRESS>Federal University of PernambucoRecifePEBrazil and  Alexander
Nolte  0000-0003-1255-824X<EMAIL_ADDRESS>University of
TartuTartuEstonia Carnegie Mellon UniversityPittsburghPAUSA

(2018)

###### Abstract.

Hackathons and similar time-bounded events have become a popular form of
collaboration in various domains. They are commonly organized as in-person
events during which teams engage in intense collaboration over a short period
of time to complete a project that is of interest to them. Most research to
date has thus consequently focused on studying how teams collaborate in a co-
located setting, pointing towards the advantages of radical co-location. The
global pandemic of 2020, however, has led to many hackathons moving online,
which challenges our current understanding of how they function. In this
paper, we address this gap by presenting findings from a multiple-case study
of 10 hackathon teams that participated in 4 hackathon events across two
continents. By analyzing the collected data, we found that teams merged
synchronous and asynchronous means of communication to maintain a common
understanding of work progress as well as to maintain awareness of each
other’s tasks. Task division was self-assigned based on individual skills or
interests, while leaders emerged from different strategies (e.g., participant
experience, the responsibility of registering the team in an event). Some of
the affordances of in-person hackathons, such as the radical co-location of
team members, could be partially reproduced in teams that kept open
synchronous communication channels while working (i.e., shared audio
territories), in a sort of “radical virtual co-location”. However, others,
such as interactions with other teams, easy access to mentors, and networking
with other participants, decreased. In addition, the technical constraints of
the different communication tools and platforms brought technical problems and
were overwhelming to participants. Our work contributes to understanding the
virtual collaboration of small teams in the context of online hackathons and
how technologies and event structures proposed by organizers imply this
collaboration.

online hackathons, time-bounded collaborative events

††copyright: acmcopyright††journalyear: 2018††doi:
10.1145/1122445.1122456††booktitle: CSCW ’21: 24th ACM Conference on Computer-
Supported Cooperative Work and Social Computing, October 23–27, 2021,
Online††price: 15.00††isbn: 978-1-4503-XXXX-X/18/06

##  1\. Introduction

Hackathons111We will use the term hackathon as a substitute for similar time-
bounded events throughout the remainder of this article. and similar time-
bounded collaborative events have become a popular form of collaboration
(Taylor and Clarke, 2018). They have been adopted in various domains including
corporations (Pe-Than et al., 2020; Komssi et al., 2015; Rosell et al., 2014),
(higher) education (Porras et al., 2019; Gama et al., 2018; Kienzler and
Fontanesi, 2017), civic engagement (Hartmann et al., 2018; Lodato and DiSalvo,
2016; Henderson, 2015), (online) communities and others with the aim to create
innovative technology (Stoltzfus et al., 2017; Briscoe, 2014), tackle civic,
environmental and public health issues (Hope et al., 2019; Taylor et al.,
2018; Hou and Wang, 2017; Birbeck et al., 2017; Porter et al., 2017), spread
knowledge (Porras et al., 2019; Gama et al., 2018; Fowler, 2016; Nandi and
Mandernach, 2016) and expand communities (Nolte et al., 2020b; Huppenkothen et
al., 2018; Taylor and Clarke, 2018). During such events participants typically
form teams and engage in intensive collaboration to complete a project that is
of interest to them (Pe-Than et al., 2019).

One of the defining characteristics of hackathons is radical co-location
(Olson and Olson, 2003), i.e., that the “entire project team [is] in one room
for the duration of the project” (Teasley et al., 2000). Radical co-location
is considered to contribute to the development of social ties (Sias and
Cahill, 1998), an increased perception of familiarity (Kiesler and Cummings,
2002), and help to develop trust (Orlikowski, 2002; Olson and Olson, 2000).
This setting also allows collaborators to access situated knowledge thus
supporting coordination and problem solving (Teasley et al., 2000) and
increasing productivity (Trainer et al., 2014). These aspects are particularly
beneficial in a hackathon setting since teams need to quickly get acquainted
and start becoming productive due to the limited time span of a hackathon
event (Trainer et al., 2016).

However, during the global pandemic of 2020, we saw a rapid move of hackathon
events online as evident by e.g. the global #hackthecrisis movement which
included more than 68 online hackathons over the span of a few weeks across
the globe222https://garage48.org/hackthecrisis. Many more events quickly
followed and organizing online hackathons has been the _de-facto_ norm for
2020 and the beginning of 2021, as also observed in announcements on the
largest global hackathon database, Devpost. While in-person hackathons can be
expected to become more prevalent again after the end of the pandemic
situation, it is also likely that online events will continue due to their
unique affordances (Greeno, 1994). For instance, online hackathons can allow
for a quick setup and support participation by individuals who cannot travel
to a hackathon location due to a lack of resources or potential visa-related
issues.

Research on hackathon events has steadily grown in the HCI and CSCW
communities and beyond, focusing on how to run events in various contexts
(Nolte et al., 2020c; Pe-Than et al., 2019; Huppenkothen et al., 2018),
understanding how hackathon teams self-organize (Trainer et al., 2016), how
code gets used, developed and reused (Imam et al., 2021), running events for
diverse audiences (Paganini and Gama, 2020; Hope et al., 2019; Filippova et
al., 2017) and identifying means to foster specific hackathon outcomes (Nolte
et al., 2020a; Taylor and Clarke, 2018; Pe-Than et al., 2020; Komssi et al.,
2015). Moving such events online fundamentally changes their nature thus
challenging our current understanding of how they function, what can be
achieved and what remains when participants – instead of being radically co-
located – are geographically distributed but still collaborate synchronously.
There are few studies on online hackathons (Powell et al., 2021; Braune et
al., 2021; Bertello et al., 2021; Brereton, 2020). These studies however cover
single events and do not focus on how teams perceive an online event and how
they collaborate. Studying online events from a collaboration perspective
provides a unique opportunity for researchers to better understand the
affordances of synchronous collaboration in a distributed setting. Since the
collaboration of participants in teams is at the core of each hackathon, we
are particularly interested to understand how teams adapted to an online
setting and coped with the challenge to collaborate remotely. We thus ask the
following first research question:

RQ1.  _How do teams collaborate in an online hackathon?_

This question specifically covers how teams were formed, how they decided
which project to work on, how tasks were assigned to team members and how
teams monitored their progress. These are common activities that teams perform
when participating in a hackathon (Nolte et al., 2020c). Moreover, our first
research question also considers which collaboration approach (synchronous or
asynchronous) teams chose and which tools they used for communication since
these are necessary for online collaboration.

In addition to coping with the challenges of an online setting, teams also
have to navigate the specifics of the event they participate in. The
organizers of a hackathon commonly scaffold an event by prescribing a specific
theme and an event agenda, setting rules of when and how teams form, what they
should deliver at different times during a hackathon and – in the case of a
competitive event – which prizes will be given to the winners and how winners
are chosen (Nolte et al., 2020c). We are thus also interested in studying how
different event designs can affect team collaboration and consequently ask the
following second research question:

RQ2.  _How does the event design affect team collaboration?_

Here we particularly focus on how organizers scaffolded team formation, how
they organized checkpoints, how they organized the interaction between mentors
and teams and which tools they provided for teams to collaborate online.

To answer these research questions we conducted a multi-case study of 4 online
hackathon events that took place between July and December 2020 across two
continents. We studied 10 teams conducting 27 interviews with team leads and
team members. Utilizing Olson and Olson’s seminal work on distant
collaboration (Olson and Olson, 2000) as a lens, we analyzed the obtained
data.

Our analysis showed that participants combined the usage of synchronous and
asynchronous communication tools in varied proportions to support their work.
Teams were self-organized and had leaders naturally emerging but with no
apparent influence on task delegation. Tasks were rather self-assigned based
on skills or preferences. Synchronous communication was fundamental to
maintaining awareness of each other’s tasks. Radical co-location is one of the
main affordances of hackathons and it could just be partially reproduced by
some teams who preferred synchronous work keeping audio/video communication
channels continuously open and communicating whenever needed (i.e., shared
audio territories), resulting in a sort of “virtual radical co-location”.
However, other affordances of in-person hackathons such as interactions with
other teams, easy access to mentors, and networking with other participants,
diminished or even disappeared: interactions with mentors became formal, less
spontaneous (i.e., booking) and were rather short and casual contact with
participants from other teams disappeared. The flexibility of asynchronous and
remote work of the online setting allowed participants to engage in activities
of their daily lives in parallel to participating in an event (e.g., work,
birthday, wedding). The technical constraints of the different communication
tools and platforms brought technical issues and were data overwhelming to
participants.

##  2\. Background and Related work

In the following sections, we will situate our study in the context of related
work on team organization and collaboration in hackathons (section 2.1) as
well as synchronous distributed collaboration (section 2.2) and discuss the
conceptual framework on distant work we used as a basis for our analysis
(section 2.3).

###  2.1. Team organization and collaboration in hackathons

Hackathons are highly engaging, time-bounded collaborative events where small
teams engage in intense short-term collaboration to complete a project of
interest under a given challenge or topic (Komssi et al., 2015; Pe-Than et
al., 2019). The motivation for participants to join these events are varied:
prizes, learning, experimenting with technologies, the prospect of a future
job, and others (Briscoe, 2014; Komssi et al., 2015; Gama, 2017a). Whatever
the reasons are to take part in a hackathon, participants will work in teams
that can be formed in different ways (Trainer et al., 2016; Pe-Than et al.,
2019). Team formation can take place before or during an event and with or
without the mediation of organizers, who may want to assign team members e.g.
based on skills or interests. Teams work autonomously in a self-organized way,
with self-assigned tasks and a decentralized role of project manager in the
team (Gama, 2017b) although the role of hackathon mentors can be very
important for supporting teams in technical issues or planning their goals
especially when newcomers are participating (Nolte et al., 2020b).

In the software development domain, self-organized teamwork has been studied
in contexts such as agile software teams (Hoda et al., 2012), global software
development (Beecham et al., 2014) and open-source software (Crowston et al.,
2007). The lack of formally appointed leadership (Fagerholm et al., 2013) and
self-directed task allocation (Masood et al., 2020) are common issues of self-
organized teamwork that can also be found in hackathon teams. However, the
sense of urgency related to the time pressure in hackathons (Lifshitz-Assaf et
al., 2020) brings slight differences from a typical software development
context, introducing a certain similarity to self-organized teamwork in more
time-critical domains e.g. related to emergencies (Zhang and Sarcevic, 2018;
Starbird and Palen, 2011).

A hackathon is typically an in-person physical event but recently, due to the
COVID-19 pandemic and the need for social distancing, many hackathons started
to move online. While many of those events aim to crowdsource the generation
of solutions to problems around COVID-19 (Vermicelli et al., 2020; Braune et
al., 2021), there are several other online
hackathons333mlh.io/seasons/2021/events focused on a wide array of topics.
Online or remote hackathons are not a new phenomenon and have been popular
since the beginning of the 2010s in databases such as Devpost 444devpost.com,
where hackathon organizers can advertise their events and attract
participants. However, hackathon participants commonly prefer face-to-face
interactions that are typical for co-location (Briscoe et al., 2015), instead
of remote collaboration. The radical co-location of participants and the face-
to-face interactions are key characteristics that help participants quickly
advance technical work in hackathons (Trainer et al., 2016). They help to
create a highly focused and interruption-free space that allows participants
to concentrate on one thing at a time and increase their productivity (Pe-Than
and Herbsleb, 2019). Under the perspective of hackathon-focused literature,
these events are often seen as competitive or collaborative styles
(Richterich, 2019; Pe-Than and Herbsleb, 2019; Kos, 2019), although both
approaches can coexist in the same event. In the 90’s hackathons started as
events of a mostly collaborative nature (Richterich, 2019). They developed a
competitive facet and depending on how the hackathon is organized, teams may
vary in their collaboration style which can be competitive – the typical
hackathon style focusing on collaborative work only within the group – or
cooperative, where there is mutual help and participants receive support from
the other teams’ members (Pe-Than and Herbsleb, 2019; Kos, 2019). Fostering a
competitive or collaborative ambience is a design choice made by organizers
(Pe-Than et al., 2019). The traditional competitive hackathon format is common
when there are incentives such as awards and prizes being offered. A
cooperative event can be achieved when social elements are introduced – e.g.,
stimulating participants to pitch project ideas or to wander around the
premises and discuss with other teams – thus helping participants from
different teams to collaborate and network (Pe-Than et al., 2020).

In terms of a more recent perspective from the CSCW community (Brudy et al.,
2018; Neumayr et al., 2018; Wang et al., 2019), collaborative styles involve
dimensions of space (co-located vs. remote) and time (synchronous vs.
asynchronous) as well as a perspective on the coupling of tasks (loosely or
tightly coupled), which may also be of mixed-focus collaboration where
coupling fluctuates. Hackathons are fundamentally team-based events focused on
synchronous co-located collaboration. It is thus not surprising that research
on these events has mainly focused on how teams collaborate in a hackathon or
how the organization can affect collaboration (Trainer et al., 2016; Pe-Than
and Herbsleb, 2019). Literature is still being developed around collaboration
in online hackathon events (Vermicelli et al., 2020; Braune et al., 2021;
Bertello et al., 2021; Powell et al., 2021). These works are mostly based on
surveys or structured as reports and lack a rigorous qualitative analysis. Our
aim is to add to this discussion by conducting a multi-case study to
understand the perspective of participants on how this remote format affects
team collaboration.

###  2.2. Synchronous distributed collaboration

Attempting to replicate similar dynamics in an online compared to an in-person
hackathon is challenging since there is no physical co-location and face-to-
face collaboration is not possible. Participants are limited to interacting
with their teammates, having little or no interaction with members of other
teams. It thus reduces or perhaps eliminates opportunities for knowledge
exchange among teams and serendipitous learning resulting from overheard
conversations that take place in a physical shared space (Pe-Than and
Herbsleb, 2019). In the case of an online setting, teams need to adapt their
work dynamics to overcome the limitations of not being face-to-face, having to
rely on tools to support distributed work.

The idea of software development teams working remotely is not new and can be
traced back at least three decades under the concept of Global Software
Development (GSD), where teams are geographically dispersed (Herbsleb and
Moitra, 2001). Communication is one of the main challenges in that context. In
the early stages of software development, communication is key, especially if
the goal of the collaboration such as the software that is to be developed is
not well defined. So, the absence of ongoing conversation due to a remote
setting can lead to the misalignment of ideas.

A balance between synchronous and asynchronous modes of communication is
important to deal with that challenge. Communication among GSD team members
tends to be asynchronous, yielding to limited opportunities for face-to-face
contact. Although asynchronous communication tools such as e-mail and instant
messaging are useful to convey technical information (e.g., source code) in
more detail, synchronous communication technology (e.g., video conferencing)
allows rich and interactive discussions that allows more understanding and
agreement among participants and is one of the recommendations to overcome the
barriers of GSD (Noll et al., 2011).

More recently there has been an improvement in communication tools, such as
Slack555slack.com, an enterprise social networking tool that relies on instant
messaging between individuals and among groups. Slack has been adopted in GSD,
fostering communication transparency and acting as a hub that allows frequent
communication and collaboration among distant team members (Stray et al.,
2019; Calefato et al., 2020).

Time-bounded collaborative events can also take advantage of other similar
communication tools, such as Discord 666discord.com which is typically used in
game jams. Game jams have similarities with hackathons, consisting of events
where a game is created in a relatively short time frame (Kultima, 2015).
Discord has been successfully used as a team communication tool in jams, being
the primary mechanism for community interaction. A Discord server allows
creating text channels, in which members can chat, and voice channels, which
facilitate synchronous conversations among team members also allowing video
calls and screen sharing. Discord is often used by jam participants to
interact with each other before an event takes place, and also while a jam is
happening to e.g. share results, resources, and seek help on art or technology
(Faas et al., 2019).

The aforementioned communication tools support the collaboration of remote
teams. The usage of Discord has been reported as an enabler for cross-border
teams in the Global Game Jam 2020 (Fowler et al., 2020). Similarly, Slack has
been used as the main form of communication between organizers and
participants in online hackathons taking place during the COVID-19 pandemics
(Brereton, 2020; Temiz, 2021; Braune et al., 2021; Powell et al., 2021).
Discord has been used in those types of hackathons as well, but to a seemingly
lesser extent (Chen et al., 2020). In general, it is difficult to enforce
communication tools in hackathons. Thus, organizers can be flexible with which
support technology can be used. But at the same time having a large number of
synchronous/asynchronous tools and online platforms can bring technical issues
and be overwhelming to participants (Braune et al., 2021). As there is scarce
empirical evidence reported around team collaboration in online hackathons,
our work also aims at investigating the ways synchronous collaboration takes
place and how communication tools influence the work of participants.

###  2.3. Conceptual framework on distant collaboration

To analyze the socio-technical conditions that are necessary for distance
work, our study is framed by the concepts introduced by Olson and Olson in
their seminal work about distant collaboration (Olson and Olson, 2000). The
authors structured four key concepts to succeed in that form of collaboration:

  * •

Common ground: In order for teams to collaborate effectively, they need to
achieve and retain a common understanding of not only the project that they
attempt to complete but also each other’s skills and capabilities. The
knowledge that team members have in common and they are aware that they have
in common is referred to as common ground.

  * •

Coupling of work: This aspect refers to how much communication is required for
a team to collaboratively achieve their goals. Tighter coupling refers to
teams closely collaborating with each other on highly interdependent tasks
while looser coupling refers to collaborative tasks being more independent.

  * •

Collaboration readiness: This concept encompasses individual characteristics
of team members concerning their willingness to share information and
collaborate. Using shared technologies assumes that team members are rewarded
for sharing information.

  * •

Collaboration technology readiness: This aspect covers the adequacy of
technologies to perform the work to be done and the current level of groupware
assimilated by a team. A poor alignment related to the usage of technology may
inhibit their adoption.

##  3\. Empirical method

To answer the two main research questions stated in the introduction we
conducted a multiple-case study (Yin, 2017) of 4 online hackathons. This
approach is reasonable because it allows us to study a new phenomenon – teams
collaborating during online hackathons – that has not been observable before
in the context it occurs and draw insights about “how” it takes place (Runeson
and Höst, 2009). Focusing on teams as the unit of analysis we draw on Olson
and Olson’s seminal work on distributed teamwork (Olson and Olson, 2000) as a
theoretical foundation (section 2.3). This approach allows us to focus on
studying how teams collaborate in an online environment (RQ${}_{\ref{RQ1}}$)
and how the surrounding organization – in our case the design of the event a
team participates in – can affect their collaboration (RQ${}_{\ref{RQ2}}$).

In the following we will first provide a short overview of our study setting
(section 3.1) before elaborating on our data collection (section 3.2) and
analysis methodology (section 3.3) in detail.

###  3.1. Study setting

The 4 online hackathons that we studied took place between July and December
2020 in Europe and the Americas (see Table 1 for an overview). We chose events
that lasted between 48 and 72 hours which can be considered a common time
frame for hackathon events (Nolte et al., 2020c; Cobham et al., 2017). We
focused on events that were open for everyone to participate in since closed
events usually take place in a specific organizational context that can affect
collaboration and thus affect our findings. To achieve broader insights into
collaboration during an online hackathon, we selected the following 4
hackathons that were organized around different themes, goals, and sizes.

Event |  Dates |  Organizers |  Theme |  Goal
---|---|---|---|---
H1 |  July 27 to 29, 2020 |  US |  High performance computing |  Grow the community
H2 |  Dec. 3 to 6, 2020 |  Eastern Europe |  Hacking the post-crisis economy |  Foster entrepreneurship
H3 |  Dec. 12 to 13, 2020 |  Latin America |  Towards a better normal |  Resume economy after the pandemic
H4 |  Oct. 13 to 15, 2020 |  Latin America |  Conservation of the Rain Forest |  Green innovation
Table 1. Overview of study setting

####  3.1.1. Hackathon one (H1)

The first hackathon we studied was organized by members of the high-
performance computing (HPC) community in conjunction with one of their main
conferences which was organized in the US. They organized the event to
increase interest in the community by exposing students to projects in the
domain. The organizers reached out to potentially interested community members
prior to the event and asked if they would be willing to propose a project
idea for the event that is related to their respective interest and serve as a
mentor for the participants that chose to work on this idea through the
duration of the event. The organizers also recruited a total of 14 students
from various study fields including computer science, biology and
environmental science across all three US time zones which registered through
an online form. Before the event, the organizers held training sessions for
mentors and students to prepare them for the upcoming event. They also asked
them to join the common hackathon Slack channel. During the kick-off – which
was held via Zoom – the organizers first elaborated on the event agenda before
the mentors presented their project ideas and participants could choose which
project they would like to work on. The choice was facilitated by separate
breakout rooms for each project in Zoom which participants could join.
Moreover, the organizers also provided separate Slack channels for teams and
their mentors so that they could get in touch between Zoom calls. Checkpoints
were held twice per day during which teams presented their progress and
received feedback from organizers, mentors and members of other teams. The
organizers also had created a Github page which contained all necessary
information for participants including contact information for organizers and
mentors and the event’s agenda. For the final checkpoint, the teams were asked
to prepare a presentation and submit their code via Github. The presentation
was streamed live for all interested community members. A jury consisting of
selected community members then served as a jury to select a winner. In
addition, viewers of the final presentation could vote for their favorite
team. Both the team selected by the jury and the team selected by the
community received a price.

####  3.1.2. Hackathon two (H2)

The second event we studied was organized by an Estonian for-profit
organization that specializes in organizing hackathons. The objective of the
event was to overcome and find solutions to the challenges caused by the
COVID-19 pandemics. The registration for teams and ideas was opened prior to
the event and everyone with an idea or the intention to contribute could sign
up. Before the hackathon, the teams were divided between seven main topics: 1)
future of creating trusted networks, 2) future of education, 3) future of
work, 4) future of healthcare, 5) future of travel, tourism and hospitality,
6) future of entertainment and sports, and 7) future of small and mid-sized
enterprises. One week prior to the hackathon two matchmaking events were held.
These were meant for existing teams to recruit additional members, individuals
to present their ideas and assemble a team or find a suitable idea to
contribute to. After the registration and matchmaking, the organizers chose 41
teams that qualified for the hackathon. All selected teams were invited to
join the hackathon workspace on Slack, which was used as the main
communication channel for the event. The hackathon was kicked off in an
opening ceremony which was live-streamed for interested viewers and during
which teams started working on their ideas. The hacking was organized around
checkpoints, held over Zoom each morning and evening, during which teams had
to present their progress and received feedback from mentors. For the
checkpoints, each batch of ideas had a lead-mentor but throughout the
hackathon, the participants also had on-demand access to 74 additional mentors
from the organizing foundation’s network. The mentors were divided between
topics such as business development, marketing, design and technology. For the
last checkpoint, all teams had to submit a video pitch of their idea. 15
pitches made it to the finals. At the end of the event a 7-member jury of
experts selected 7 top teams who received follow-up mentoring as a prize.

####  3.1.3. Hackathon three (H3)

The hackathon was sponsored by the innovation hub of a multinational
telecommunication company and organized by a startup specialized in the
organization of hackathons in Brazil. It was a mix of hackathon and ideathon,
since it focused on prototyping ideas. The goal of the hackathon was to create
new solutions to help resume economic activities after the impact of the
COVID-19 pandemic. Teams had to propose innovative solutions under one of the
six United Nations (UN) Sustainable Development Goals (SDG), namely: zero
hunger, good health and well-being, quality education, affordable and clean
energy, decent work, and economic growth. The focus of the proposed solutions
should be the social function and the teams should use technologies (e.g.,
Cloud Computing, 5G, IoT) as an enabling platform. There were no pre-event
activities. The opening and closing ceremonies were streamed live on YouTube.
During the event, organizers used Discord for communication with participants
and a proprietary tool for registering projects and receiving jury feedback.
Teams had three to five members. Participants individually enrolled in the
hackathon but they could also join as a team. There were over 200 enrolled
participants. After the event started, the organizers suggested for
participants who did not have a team yet to form teams based on the
participant’s expertise (e.g., programming, design, business). Teams were free
to conduct their ideation process as they wanted, not having any specific
guidelines from organizers. Mentoring was on-demand and managed through a
Discord bot. Participants who wanted mentoring would go to the mentoring
channel on Discord, type a command to list mentors with their availability and
another command to schedule a meeting with the chosen mentor. An email would
be sent to the group and the mentor so they could start a conversation to
choose the communication tool of their preference (e.g., Zoom, Google Meet)
and confirm the 30-minute mentoring session. Mentors came from different
fields and had different expertise and experiences. Their role was to provide
advice and constructive criticism thus guiding groups toward better results.
Since mentoring was on-demand, there was no follow-up from mentors and no
intermediate checkpoints with organizers. The deliverables consisted of a
video that included a solution pitch (3 minutes), presentation slides (10
slides), and a link to the solution prototype. A panel of judges mostly formed
by telecommunications and business experts from the sponsoring company were
responsible for evaluating the solutions according to the event’s criteria
(e.g., creativity, applicability). There were 15 teams awarded with prizes (3
for each of the five categories).

####  3.1.4. Hackathon four (H4)

The hackathon was jointly promoted by a nature conservation foundation and a
scientific foundation in Brazil. It aimed to protect inhabitants and species
and promote the development of a rain forest reserve thus covering three
themes: the conservation of the forest reserve, with three sub-topics; the
development of the conservation unit, and the social and biodiversity
production chain; land and sea mobility. There was a pre-hackathon a week
before the event, which prepared the participants for the hackathon. The pre-
hackathon was held by two experts in the event themes, so participants could
familiarize themselves with the event’s themes and goals. The pre-hackathon
lasted one week. The organizers divided the main event (i.e. the hackathon
itself) into three phases: in the first phase (_ideation_), the teams chose a
problem and worked on an idea to solve it. In the second phase
(_prototyping_), the mentors guided the teams to draft a working proposal. In
the third and last phase (_pitch_), the teams presented their proposal. A
checkpoint marked the end of each phase during which teams needed to deliver
partial results.

The organizers recruited professionals from different fields. Participants
signed up individually for the hackathon but teams could be formed beforehand.
There were more than 1,000 registrations, and teams sizes varied between four
and six participants. During the event, the organizers used Discord for
asynchronous communication. There was also a team of mentors (with different
experiences and backgrounds) that supported teams during the event. The
mentoring session happened in the same way as in hackathon H3 (participants
booked 30m of mentoring using a Discord bot, section 3.1.3). In addition to
mentoring, each team had a “godfather” (mentor) who was assigned to help the
team. The jury (formed by experts from different areas including business,
sustainability, innovation, etc.) was responsible for evaluating the solutions
that teams proposed according to the event’s criteria. Three teams were
awarded and 15 promising proposals were selected for mentoring and
acceleration.

###  3.2. Data collection

We conducted semi-structured interviews with hackathon participants a few
weeks after each event as our primary source of data. The interviews were
conducted in English or Portuguese depending on the preference of the
participants and lasted between 15 and 90 minutes. We used two approaches to
find potential interviewees. First, we started by interviewing participants
that were close contacts to our network. Second, at the end of each interview,
we also asked the interviewee if they were aware of any other colleague (that
was not part of their group) that participated in the same online hackathon,
and whether they could put us in touch with them. We aimed to interview at
least two members per team to complement individual perceptions and not have
to rely on a singular perspective for each team. Moreover, we decided to
interview participants individually in order to allow them to speak freely and
potentially reveal differences in their perception about the way they
collaborated as a team. By using these two approaches, we were able to
interview 27 participants from 10 teams (c.f. Fig. 1 for an overview).

Figure 1. Study participants. We label each participant using H$n$P$n$, where
H$n$ refers to the respective hackathon (sections 3.1.1) to 3.1.4) and P$n$
refers to the respective participant in that hackathon.

The interview consisted of four main parts. We started by first asking the
participants about their prior hackathon experience and their motivations to
participate in the respective event (e.g. “What was your motivation to
participate in [hackathon]?”, “How many hackathons did you participate in
before [hackathon]”). Second, we asked them about how they decided with whom
to work together as a team, which projects they worked on, how they approached
their collaboration, and which tools they used (e.g. ”How did you find your
team?”, “Why did you choose to work on this particular project?”, “What did
you do to stay in touch with your team members?”, ”Was everyone on the same
page?”, ”Which tools did you use as a team?”, RQ${}_{\ref{RQ1}}$). Third, we
asked participants about their perception of the event design. We particularly
focused on their perception about the organizational structure, the mentoring
approach and the selection of winners (e.g. “How did you perceive the
organization of the hackathon?”, “What was your perception about your
mentor?”, RQ${}_{\ref{RQ2}}$). Finally, we asked participants about how
satisfied they were with their project and how their team collaborated (e.g.
“How do you perceive your project after the hackathon?”, “What was
particularly good/bad about your collaboration as a team?”), about their
overall perception of the event (e.g. ”Were you satisfied with the event?”),
and their future plans (e.g. “What are your plans after the hackathon?”,
“Would you participate in a similar event again?”). These questions served as
a basis for us to contextualize the participants’ responses and assess whether
they perceived their participation as successful or not.

###  3.3. Analysis procedure

We started the analysis procedure after we had conducted all 27 interviews by
manually transcribing all interviews first.

Utilizing the four key concepts of distributed team collaboration – as
outlined by Olson and Olson (Olson and Olson, 2000) as a basis – one of the
authors then developed an initial list of codes. It is important to note that
we created this initial list of codes before analyzing the collected data. We
opted for this approach to curate a broader list of codes (for which we could
or could not find occurrences within data). This initial list was created
based on our experience participating in and investigating hackathons and on
related works on the topic. For instance, we noticed that online hackathons
used tools such as Slack, GitHub and Zoom for collaboration (i.e.,
collaboration technology readiness in Olson and Olson (Olson and Olson,
2000)), so we included these and other tools of similar purpose. We also used
seminal hackathon collaboration characteristics that have been explored by
related works as codes (Nolte et al., 2020b; Gama, 2017a; Pe-Than and
Herbsleb, 2019), such as networking, learning, mentoring, etc. (i.e.,
collaboration readiness in Olson and Olson (Olson and Olson, 2000)). We
performed a similar procedure for the other key concepts of distributed team
collaboration (Herbsleb and Moitra, 2001; Noll et al., 2011). After we had
created the first draft of the code list, the other authors discussed and
revised each item. The aforementioned codes were aimed to contribute to
answering RQ${}_{\ref{RQ1}}$. In addition we also included codes related to
the organization of and event (e.g. checkpoints) and the mentoring approach
(e.g., mentoring) to answer RQ${}_{\ref{RQ2}}$.

Afterwards, we applied these initial codes to each interview. This process was
conducted by one researcher (responsible for assigning the codes) and
supported by another researcher (responsible for double-checking whether the
code assignment made sense, i.e., if a code would fit to a given statement).
We revised this list of codes throughout the analysis. If an interview brought
additional information that did not fit in our list of codes, we discussed
among the research team if we should include a new code. During this
procedure, additional codes emerged that were added to our overall coding
scheme related to common ground (e.g., skill / task alignment), coupling
(e.g., leadership, parallel activities) and collaboration technology readiness
(e.g. tools suggested by the organizers, tools utilized by participants). We
iterated this procedure until we did not discover any additional codes.

After applying the final list of codes to all interviews, we clustered codes
within the four aforementioned themes (groups of codes). Although many
different and interesting categories arrived, during this procedure, we
focused on categories intrinsically related to our research questions.

##  4\. Findings

In the following we will outline our findings related to the four dimensions
of distributed collaboration according to Olson and Olson (Olson and Olson,
2000) namely common ground (section 4.1), coupling of group work (section
4.2), collaboration technology readiness (section 4.3) and technology
readiness (section 4.4). In addition we will elaborate on aspects that are
specific to collaboration in hackathons and that are not common for
collaboration in work-related contexts (section 4.5). Under these lenses we
discuss how teams collaborated focusing on team formation, task assignment
within a team, a team’s collaboration approach and the technologies they used
(RQ${}_{\ref{RQ1}}$). Moreover, we discuss the potential influence of event
design on team collaboration focusing on an event’s agenda, mentoring and the
tools provided by the organizers (RQ${}_{\ref{RQ2}}$).

###  4.1. Common ground

In the cases we studied we found different strategies that teams took to
establish common ground related to which project they aimed to complete, how
they distributed tasks and how they maintained a common understanding
throughout the event. We also identified strategies of mentors to support
teams to establish and retain common ground (c.f. Table 2 for an overview).

####  4.1.1. What to do?

We found two main strategies that teams took to establish a common vision
about which project they aimed to complete during an event.

One strategy was based on a team leader proposing an idea that s/he then
shared with the team (“ _So first thing we had to do before the hackathon was
to get the idea super clear to be able to explain to other hackathon
participants so they could join the idea._ ”, H2P5). We mainly found this
strategy during hackathons H2 and H3 which might be related to the design of
the respective hackathons because H2 and H3 suggested teams to propose an idea
when registering for the event. The sharing then took place at the beginning
of the event as evident from the previous quote.

The second strategy we identified was the team jointly discussing about what
they aimed to achieve. This discussion took place at different points in time.
Participants of H2 and H3 had it before the event (“ _before the hackathon we
had a call together with the team […] we discussed what we will do_ ”, H2P7, “
_because we already started exploring the theme a little bit before the event
started, so when the event started we already had the idea_ ”, H3P1) while
participants of H1 and H4 mentioned that they discussed their respective
project ideas at the start of the event itself (“ _in the first meeting we
were trying to figure out the scope of our project_ ”, H1P3, “ _each of us
with our own experience and expertise, we were throwing all the ideas around,
trying to identify the problem and the respective solution_ ”, H4P6).In the
case of H1 each team collaborated with one mentor for the entire duration for
the event that also conducted the ideation phase at the beginning of the event
(“ _we had the kind of the prompt from [the mentors] what our group of three
was looking at […]_ ”, H1P5). In the case of H4, there was a dedicated
brainstorming phase at the beginning of the event during which teams collected
and discussed ideas (“ _we were proposing several ideas and filtering them
according to the experience of the participants_ ”, H4P4).

The way that organizers scaffolded an event thus affected (1) when teams
engaged in discussions about their project idea and (2) whether teams
developed an idea together or took an idea proposed by the team leader in the
cases we studied.

####  4.1.2. Skills and task division

In addition to establishing common ground related to their project goals,
participants also needed to develop an understanding about who within their
team would be willing and able to take over certain tasks. In this context, we
found two main strategies.

The most common strategy was teams distributing tasks based on individual
skills in the cases we studied. We found this strategy across all four
hackathons with H1P1 reporting that “ _I had experienced grabbing datasets off
the internet before, [so] I was one of the people who used requests and to get
it off and into the program_ ” (H1P1). Similarly, H2P9 mentioned that “ _the
task division was rather based on skills_ ” (H2P9), H3P5 elaborated that their
team “ _did the division of tasks according to what each one was better at
doing_ ” (H3P5) and H4P1 mentioned that “ _regarding task distribution […]
each member took the specific part they could work on_ ” (H4P1). Related to
this strategy we also found teams that distributed tasks based on the ability
of individual team members to quickly acquire the skills required to complete
a task (“ _so I knew I would be able to learn more about Plotly if that was
the thing I focused on_ ”, H1P6). The approach of distributing tasks based on
individual skills can be expected to be common especially in an environment
where teams only have a limited amount of time during a hackathon to complete
a project.

The second strategy we identified was teams distributing tasks based on
individual interests. We found this strategy in hackathons H1 and H2 (“ _it
became clear that [team member] was interested in producing a visualization,
that [second team member] was very curious about the data transformation
process_ ”, H1P3, “ _what someone was interested in_ ”, H2P1). This strategy
was more common among teams in hackathon H1 which could be related to most
participants mentioning that their aim was related to learning (“ _I wanted to
learn more about coding_ ”, H1P1, “ _learning experience_ ”, H1P3). We did not
find this approach in either hackathons H3 or H4. The scaffolding of H1 can be
expected to have influenced how tasks would be distributed within teams since
the event was explicitly aimed at engaging newcomers which in turn meant that
participants could not be expected to have all skills required to complete a
project. In comparison, the focus of H2, H3 and H4 was more related to
developing a functioning prototype which meant that teams could be expected to
design their projects in a way that they were doable for them during the short
duration of a hackathon.

We also found one team that utilized both strategies with H2P1 reporting that
task distribution was “ _first of all based on skills of course_ ” (H2P1) but
then also mentioning that tasks were distributed based on “ _what someone was
interested in_ ” (H2P1). This again makes sense in the context of H2 since the
aim was to develop a functioning prototype as discussed before.

Tasks were mainly distributed based on discussions among all team members (“
_We said, what is the task? What has to be done to accomplish the task?_ ”,
H1P4, “ _What do we think we need for this part? And we kind of, so we said,
what is the task? What has to be done to accomplish the task?”_ ”, H2P5).
These discussions were in some cases aided by teams already being familiar
with each other (“ _I actually had worked with H1P3 during [extracurricular
activity]_ ”, H1P4). Team familiarity was based on different common prior
experiences including established friendships (“ _a friend who is a designer
and also has a project related to this topic called me and then we decided to
participate in this hackathon_ ”, H3P4) which for some went as far back as
their childhood (“ _I have known them, H3P1 and H3P3, for quite some time. We
have studied together since kindergarten_ ”, H3P2). Other teams knew each
other from prior school activities (“ _we had pretty much already built out a
team, like people that we wanted to work with, through the coding institute_
”, H1P3) or from participating together in a hackathon before (“ _We kept the
same team from the hackathon we participated in before H4_ ”, H4P1). We thus
did not observe any influence of the hackathon design in the strategy that the
teams used. All teams approached this activity similarly.

####  4.1.3. Retaining a common understanding

After developing a common understanding of what they were aiming to do
(section 4.1.1) and deciding on an initial division of tasks among team
members (section 4.1.2) teams needed find ways of retaining a common
understanding of who was currently working on which task and which tasks were
still open to complete their project.

Teams again showed different strategies to achieve this. One common strategy
was one participant taking the role of a project manager who had a “ _working
document_ ” (H2P5) to keep track of what their team was working on and what
still needed to be done (“ _I kept a rolling task list of everything that
needed to be done and what would be involved in doing that. So it would just
keep getting like, okay, you finished with that thing. Awesome. This thing
still on the list_ ”, H1P4, “ _I have to delegate and micromanage_ ”, H3P3) or
coordinate their activities “ _as H4P2 is more from the technological side,
s/he was able to pass some coordinates to the team and this worked very well
for the development of the project”, (H4P1)_. We will discuss how teams
selected their leader in section 4.2.4.

Other teams had no explicit group leader. In order to retain a common
understanding, they employed different strategies. Some had “ _running calls
throughout the day, just to keep track of where each of us was at and see what
the next roadblock was for that_ ” (H1P6) or “ _worked together all the time,
stayed in Discord [voice channel]_ ” (H3P1) to maintain a constant level of
common understanding of the current state of their collaboration. Other teams
instead had occasional “ _Zoom calls, just to understand who is doing what is
the team_ ” (H2P7) or relied on a self-selected collaboration platform to
retain a common understanding (“ _Miro [a collaborative online whiteboard] was
our point of convergence where we had the tasks that each one could do when we
weren’t on Miro, we stayed connected through Google Meet_ ”, H3P5). Some teams
even reported that they relied almost entirely on asynchronous communication
via Slack (“ _we have a connection through Slack_ ”, H2P4). We will discuss
the different collaboration styles that teams took in more detail in section
4.2.3.

In some cases, participants also reported that checkpoints set by the
organizers aided their common understanding because they utilized them as
opportunities to meet before each checkpoint and discuss about what they had
achieved. This was evident by e.g. H1P2 answering “ _yes, we did. We all
checked in to see where we were_ ” (H1P2) when being asked if they met before
checkpoints as well as by H2P1 mentioning that s/he “ _actually created an
availability spreadsheet […] which would have all the important checkpoints_ ”
(H2P1).

Taking the aforementioned findings together we did not identify one specific
strategy for certain hackathons. The only difference we identified was teams
in H1 and H3 reporting that they stayed on a common call for extended times
during the event. There is however no indication that the design of these
events could have fostered this behavior since all teams were free to
collaborate in any way they wanted. We thus conclude that the event design did
not have a strong influence on the way teams retained a common understanding
in the cases we studied. Instead, teams decided for themselves how they
attempted to retain a common understanding for the duration of the hackathon.

####  4.1.4. Mentoring

All hackathons we studied provided teams with access to mentors to help them
solve issues and provide guidance during the event. Participants of all
hackathons consequently reported that mentors aided them to develop and retain
a common understanding of their project (“ _the mentor gave a lot of tips on
how we could better shape our proposal_ ”, H3P1). We did however identify
different strategies for how mentors engaged teams during the event.

Participants of all hackathons reported that they “ _got a lot of feedback_ ”
(H1P4) during dedicated checkpoints regarding their current project progress
which in some cases was “ _passed on to the team_ ” (H4P1) by the team leader.
These checkpoints took place at multiple set instances during each event and
were led by organizers and in the case of H2 specific mentors (“ _lead
mentors, who was like the same person throughout all the checkpoints and ours
was also very, very nice._ ”, H2P1).

Most mentors were hands-off and waited for teams to contact them before
providing advice such as “ _suggesting […] to do some pair programming_ ”
(H1P5) or “ _[…] giving us some ideas, helping us on polishing what we had
done_ ” (H4P2). This was evident by e.g. H2P5 mentioning that s/he
specifically “ _asked mentors_ ” (H2P5) for support. We found this approach
across all hackathons.

In addition, we also found instances where mentors were more proactive (“ _so
[mentor name] would kind of set checkpoints […] just so we’d internally as a
group, uh, kind of get together and see where everybody’s at_ ”, H1P6). These
instances were limited to hackathon H1 though which can be related to the
design of the event where one mentor was responsible for a specific team. This
was not the case in hackathons H2, H3 and H4.

Aspects |  H1 |  H2 |  H3 |  H4
---|---|---|---|---
What to do? (4.1.1) |  Team jointly developed idea, Ideation phase at the beginning |  Team leader proposed idea, Discussion about idea at the beginning |  Team jointly developed idea, Discussion about idea prior to the event |  Team jointly developed idea, Ideation phase at the beginning
Skills and task division (4.1.2) |  Distribution based on skills and interest, Discussion about distribution, Team familiarity |  Distribution based on skills and interest, Discussion about distribution, Team familiarity |  Distribution based on skills, Discussion about distribution, Team familiarity |  Distribution based on skills, Discussion about distribution, Team familiarity
Retaining common understanding (4.1.3) |  Team leader, Tracking through continuous calls, meetings and messages, Checkpoints as synchronization opportunity |  Team leader, Tracking through meetings and messages, Checkpoints as synchronization opportunity |  Team leader, Tracking through continuous calls, meetings and messages, Checkpoints as synchronization opportunity |  Team leader, Tracking through meetings and messages, Checkpoints as synchronization opportunity
Mentoring (4.1.4) |  During checkpoints, On demand, Proactive mentor |  During checkpoints, On demand |  During checkpoints, On demand |  During checkpoints, On demand
Table 2. Strategies to establish and retain common ground across the four
hackathons we studied.

###  4.2. Coupling of work

Our analysis revealed different strategies of how teams prepared for and
organized their collaboration to complete the project they were aiming for.
Regarding project preparation these activities included different approaches
related to team formation and project scoping. Regarding project organization
they included different collaboration styles, leadership and parallel
activities that teams engaged in while the hackathon they participated in was
ongoing (c.f. Table 3 for an overview).

####  4.2.1. Team formation

We focused on understandig how teams formed as well as whether and how their
approach affected the way they collaborated during a hackathon. Participants
reported two main strategies for team formation. Teams were either formed
before or at the beginning of the event.

In most cases, teams had already formed before the event. In fact this was
scaffolded by the event design in the cases of hackathons H2, H3 and H4 where
organizers suggested for participants to register as a team (“ _our team lead
signed us up, like as a team_ ”, H2P1, “ _the rules stated that there had to
be at least 3 people on the team_ ”, H3P4). Despite this not being mandated in
the case of hackathon H1, we also found instances there where participants had
already formed a team before the event (“ _we had pretty much already built
out a team, like a people that we wanted to work with, um, through the
[extracurricular activity]_ ”, H1P3).

In the case of H1 we also found reports where teams formed during the event.
This was again scaffolded by the event design in that team formation took
place around the project proposed by mentors during the opening of the event
(“ _I watched all the mentors talk about their projects and I found, I found,
[the mentors] project most interesting because […] it seemed like I could
learn some things from their project. So, I kind of signed up for their
project_ ”, H1P1). All hackathons allowed teams to be geographically
dispersed. Among the participants we interviewed we found team members from
different regions of the same country in hackathons H1, H3 and H4. In
hackathon H2 there were people based on different countries participating in
the same team.

Participants also reported that their team recruited additional members during
the event. These activities were often sparked by teams realizing that they
lacked specific skills to complete the project they were aiming for so they
searched for members that would fit (“ _I did like a 15-20 minute talk, to see
if there was a match between what they wanted and the idea we have_ ”, H2P5).
Another motivation for teams to search for additional members was the
hackathon requiring teams to have a specific size (“ _minimum number of
participants for the hackathon were four or five_ ”, H2P5). We only found
recruitment-related activities in the case of hackathons H2 and H4. In both
cases, these activities were scaffolded by the event design. In the case of H4
there was a communication channel on Discord that was dedicated to team
formation by theme (“ _As H4P4 and I had a partnership in hackathons, we
introduce ourselves together. In the H4 hackathon, H4P5 talked to me. She
already had a team almost completed with four people. Since there was a mutual
understanding between us, we decided to join the team_ ”, H4P3). In the case
of H2 the organizers held a matchmaking session at the beginning of the event
(“ _So during matchmaking I explained our idea, and there was a bunch of
people interested_ ”, H2P5).

We did not find any cases of teams recruiting additional members in the cases
of hackathons H1 and H3. In the case of H1 this can be attributed to the
aforementioned strategy where teams were formed at the beginning of the event
and the organizers made sure that everyone had a team. In the case of
hackathon H3 the organizers allowed the registration of single participants
but none of the teams we studied took advantage of this possibility.

####  4.2.2. Project scoping

Related to project scoping we focused on understanding how teams approached
planning their project which can subsequently affect activities that need to
be done. Teams again showed different approaches in this regard. Some teams
had already completed this activity before the event and started to work on
their project right away. We found this approach both for teams in H2 (“ _So
for us, we already participated in another hackathon before that educational
hack. And we came up with an idea to improve children’s self-leadership
skills. And thus, like when I saw this hackathon, I thought it’s a good
opportunity for us to develop our product further_ ”, H2P1) and H3 (“ _For the
last few months, we’ve been concentrated on our project focused on health. […]
As the hackathon was linked to health-related SDGs, it matched perfectly_ ”,
H3P2). These approaches were – similarly to the decision about which project
to attempt (section 4.1.1) – scaffolded by the event design. Since teams had
to propose an idea before the event, they also developed an idea about how to
approach their project prior to the hackathon.

We also found teams that engaged in scoping activities during the event. This
was e.g. the case for the team of H4P4 who said “ _we were proposing several
ideas and filtering them according to the experience of the participants_ ”
(H4P4). Similarly, H1P6 reported for their team that at the beginning they
were trying to “ _figure out what [the team] wanted to do_ ” (H1P6). In the
case of H1 this approach was again scaffolded by the event design because
teams had to develop a project idea at the beginning of the event based on a
suggestion of the mentor they collaborated with. This approach was not
perceived as beneficial by all participants, though. H1P6 mentioned that s/he
would have preferred if “ _projects could be posted any sooner, to be able to
attend to see like, Oh, like data visualization. I can kinda start toying
around with these tools so I can, instead of Googling on or during the
hackathon, you might be able to do it a week or two in advance, kind of poking
around tools that might be used_ ” (H1P6), thus pointing towards their
intention to prepare for the project they would aim to attempt during the
event.

####  4.2.3. Collaboration styles

All teams reported the same strategy related to how they organized their
collaboration at the beginning of an event. They first decided on how they
would approach their project (section 4.2.2) before dividing it into tasks
that could be taken over by single team members (“ _we went from just being
altogether brainstorming to then working on it and then breaking out_ ”, H1P1)
or by a subgroup (“ _We were in slack or Zoom meetings. So we were having
calls. […] I also know that the other team members had like one call between
them, […] other members discuss their things, and then I discuss with [another
team member] my things, and then we put all together, so it was mainly Slack
and Zoom_ ”, H2P6). They thus made sure that at least some of the tasks for
their project could be done in parallel. This approach appeared reasonable
since teams only had a limited amount of time to complete their project during
a hackathon.

As discussed in section 4.1.3 teams utilized different collaboration
strategies during an event. Some teams hardly engaged with each other and
mainly worked in parallel during a hackathon only synchronizing when needed (“
_we have a connection through Slack_ ”, H2P4) while others reported that they
stayed on the same video call all the time (“ _we really didn’t work
independently that much at all_ ”, H1P3, “ _synchronous. We used Google Meet
and Miro._ ”, H3P4). This made it easier for them to have the same common
vision of what each member was doing, as if they were sharing the same
physical room. Most teams utilized a mixed form of collaboration relying on
common calls to “ _just to understand who is doing what_ ” (H2P7) while
utilizing asynchronous means of communication in between to e.g. “ _assign
tasks in WhatsApp_ ” (H2P7). Similar to the previous discussion (section
4.1.3) we did not find any specific aspect about the different hackathon
designs that would have afforded the different approaches. Instead, we found
that the decision of which approach to follow was mainly predicated by the
preferences of the different team members. Some participants simply found it
easier to work alone, as stated by H2P8: “ _I’m good in doing like, say, some
research work or some, like, you know, reading, writing, when I’m alone. when
I have people around, I lose this focus because I need attention to those
details, and it’s hard to concentrate“_ (H2P8), while others preferred to stay
on calls because it allowed them to directly address problems when they
appeared (“ _If someone had a problem, we could just speak up_ ”, H1P1).

Another important aspect to note was the perceived necessity of some team
members to break out from common calls as in the case of H1P3 where they felt
they “ _were just sort of monopolizing the conversation. Cause we were just
like throwing back and forth ideas. That’s like what was wanting to do like on
the back end. So they were like, let’s go focus on the front-end stuff. So
they just went into their own breakout room._ ” (H1P3). In the same way, the
team of H3P1 who “ _worked together all the time, stayed in Discord [voice
channel] talking synchronously and when someone needed to do something alone
or wanted to be calmer, s/he would go to another Discord room, but if a member
called to talk or ask questions, the person would go back to the main room
where everyone was_ ” (H1P3).

We also found one instance where two teams worked together for a short period
of time because they “ _were working on very similar things, our team and
[team of H1P3-P4]. So there was one Zoom call where we helped each other_ ”
(H1P1). This connection however was not made by the teams themselves. It was
suggested by the mentors because they noted that the teams were working on a
similar problem. It can thus be partially related to the design of the event
since mentors closely worked together with their teams which helped them be
aware of what they were working on.

Finally, all teams had to present their project at the end of the event which
meant that they had to synchronize and integrate the different parts they had
worked on in parallel. For some teams, this was a collaborative effort (“ _the
day before the last day, um, we really got everything together_ ”, H1P2, “ _we
set up the live stream, we tested it, we made sure it works_ ”, H2P9) while in
other cases the task of merging everything was taken by a single team member
(“ _then they both gave me their code and it was like three hours until the
end of the hackathon_ ”, H1P5). We again did not find any specific strategy
per hackathon. Instead, teams decided for themselves how they would approach
this.

####  4.2.4. Leadership

Most teams developed a leadership structure for the purpose of this event. In
the case of hackathons H2 and H3, the leader position was often filled by the
participant that initially proposed an idea and registered the team for the
event (“ _Our team lead signed us up, like as a team_ ”, H2P1). This approach
was thus scaffolded by the event design since teams were encouraged to
register with an idea as discussed in section 4.1.1. We also found cases where
individuals took over the role of a team leader because of their experience (“
_I’ve worked as a manager in the past_ ”, H1P4, “ _s/he had experienced before
[…] as a product owner or product manager_ ”, H2P8, “ _there are people who
have a very strong power of speech and decision […] what s/he says has a lot
of respect [from team members] because s/he has a lot of experience_ ”, H3P1).
This was a decision that the team took by themselves without any obvious
outside scaffolding.

In hackathons H2, H3 and H4 the leadership role remained stable throughout the
event. In hackathon H1, however, the leadership role was initially filled by
the mentors who provided different possible directions for teams (“ _he gave
us some kind of vague [guidance], you can go this way. You can go that way_ ”,
H1P4). While in some cases the mentor remained the de-facto leader during the
event – as in the case of H1P4 – other mentors disengaged leaving the team to
fend for themselves (“ _s/he hopped off the video and we just got to work_ ”,
H1P5). The leadership change was thus partially scaffolded by the event design
in the case of hackathon H1.

In addition to different strategies of choosing a leader we also found
different approaches for how leaders fulfilled their role. In most cases, the
leader made sure that the team was on track and decided on the project
direction (“ _she kind of directed us in the right directions, or was telling
what to do. […] she had experience before_ ”, H2P8). There were however also
cases where teams collaboratively decided their project directions (“ _I like
the scheduled zoom meetings_ ”, H1P1). We could again not link these
approaches to the specifics of the event design though.

####  4.2.5. Parallel activities

Some participants pointed out that they engaged in parallel activities while
the hackathon was ongoing. Some of these activities were voluntary like H3P2
who mentioned that s/he “ _was not so integrated into this hackathon […]
because I was finishing other projects_ ” (H3P2). Similarly, H2P2 mentioned
that “ _one of my team members had their twins birthday during the hackathon_
” (H2P2) and H3P3 mentioned that “ _on Saturday, a teammate was at a wedding_
” (H3P3).

Not all parallel activities were voluntary though. Participants rather also
reported that they continued working on their job (“ _I didn’t have that much
time because of my job, so I used to log into Discord at 10 PM_ ”, H4P1), had
school activities (“ _I know that my other team members had their school stuff
and work stuff_ ”, H2P2) or were engaged in a conference in parallel (“ _there
was also the conference going on. So there was a couple of times where like I
had to be in more than one Zoom meeting at once_ ”, H1P4). This in some cases
led to team members not being able to participate as much as they would have
liked due to other activities (“ _The work in our team was well divided, apart
from the fact that our team members continued to work on Friday, it was a full
working day for them. So that is why, of course, they couldn’t participate as
much as they wanted_ ”, H2P7).

Most of the aforementioned activities were not related to the design of the
event but were rather based on individual circumstances. Hackathon H1 however
took place in parallel to a conference which meant that many participants were
also engaged there in some capacity.

Aspects |  H1 |  H2 |  H3 |  H4
---|---|---|---|---
Team formation (4.2.1) |  Team formation prior to event, Team formation at the beginning of the event, Geographically dispersed participants |  Team formation prior to event, Recruitment of additional members, Geographically dispersed participants |  Team formation prior to event, Geographically dispersed participants |  Team formation prior to event, Recruitment of additional members
Project scoping (4.2.2) |  At the beginning of the event |  Before the event |  Before the event |  At the beginning of the event
Collaboration styles (4.2.3) |  Splitting work, Working separately, Coordination on demand, Stayed on common calls, Formed subgroups, Collaboration between teams, Collaborative and individual synchronization |  Splitting work, Working separately, Coordination on demand, Stayed on common calls, Formed subgroups, Collaborative and individual synchronization |  Splitting work, Working separately, Coordination on demand, Stayed on common calls, Utilized different video space for quiet time, Collaborative synchronization |  Splitting work, Working separately, Coordination on demand, Calls to resolve issues
Leadership (4.2.4) |  Chosen during event, First mentor then team member, Decided on direction, Collaborative decision for direction |  Chosen before event, Chosen during event, Remained stable, Decided on direction |  Chosen before event, Chosen during event, Remained stable, Decided on direction, Collaborative decision for direction |  Chosen during event, Remained stable, Decided on direction, Collaborative decision for direction
Parallel activities (4.2.5) |  Voluntary and non-voluntary |  Voluntary and non-voluntary |  Voluntary and non-voluntary |  Voluntary and non-voluntary
Table 3. Strategies to deal with coupling of work across the four hackathons
we studied.

###  4.3. Collaboration readiness

In this section we will discuss different motivations for participants to join
a hackathon as well as their potential influence on how teams collaborated
during an event. In general we found diverse motivations that included
learning and networking as well as the theme of a hackathon and the mentorship
that the event provided (c.f. Table 4 for an overview).

####  4.3.1. Theme

One of the main motivations that participants across all four hackathons
mentioned for joining a hackathon was the theme of an event. This is evident
by e.g. H4P6 mentioning that “ _the theme of the hackathon was my main reason
[to participate]_ ”, H3P1 saying that “ _the theme was my first reason_ ”
(H3P1) and H3P4 elaborating that “ _this hackathon focused on UN’s Sustainable
Development Goals (SDG). I truly liked the theme because it focused on social
aspects. Also in the university, I worked on a project on the same topic_ ”
(H3P4). Similarly in the case of H2 and H3 which focused on entrepreneurship,
participants often considered the hackathon as an opportunity to improve their
projects (“ _it’s a good opportunity for us to develop our product further
[…]_ ”, H2P1, “ _[…] As the hackathon happened at the end of the year, we saw
it as opportunities to improve our project quickly instead of taking a break.
Of course, we take accounting the prize due it could also help to finance the
project_ ”, H3P3) and in the case of H1 to engage with high-performance
computing because the event focused on exposing novice participants to those
technologies (“ _I signed up because I wanted to learn more_ ”, H1P1).

####  4.3.2. Learning

Similarly we also found participants in every hackathon we studied that
mentioned learning as a motivation. In this context most participants focused
on learning technical skills (“ _signed up because I wanted to learn more
about coding_ ”, H1P1, “ _more experience on coding from a professional
standpoint_ ”, H1P2) or skills related to the business side of their project
(“ _I joined the hackathon to […] test my skills, to improve the pitching I
also wanted to improve the understanding of the business and building the
business model_ ”, H2P9). Other participants also mentioned that they
participated in one of the hackathons to learn how to collaborate (“ _having
not had too much practical exposure, hands-on experience, especially as a
group. It was something that I thought would be interesting_ ”, H2P6, “ _I
learned to strengthen teamwork because I had done other jobs, but it wasn’t
like that, so I kind of came back to believe in teamwork_ ”, H3P5, “ _I could
learn a lot about technologies and teamwork_ ”, H4P4) especially in a high-
pressure setting (“ _learning how to work in this, like short deadlines_ ”,
H2P1). What participants were planning to learn was often closely connected to
the theme of the event they joined 4.3.1.

####  4.3.3. Networking

Most participants across all hackathons also mentioned networking as a
motivation to join an event (“ _I was looking for good founders to join and I
also wanted to find like-minded people in the start-up environment_ ”, H2P4, “
_[…[in]a hackathon you can meet people who can be future partners or refer you
to projects_ ”, H3P2, “ _What I liked the most was the networking_ ”, H4P3).
Participants with previous experience in in-person hackathons however noted
that they missed having contact with other teams (“ _I like to network and to
know people. So of course, it would be great also to speak more time with
other people with other teams and to get more of their ideas. And to be
honest, we didn’t have much time to follow other teams [in the online event].
When you’re on the place […] you still listen when people present or people
speak, you still hear their ideas, and you also have a chance to speak with
them._ ”, H2P2). In the case of H3, the organizers attempted to foster
networking by providing a specific channel for such activities but this was
not appreciated by all participants (“ _there is a channel for networking
where you send a message and paste the URL of your LinkedIn [social network]
profile, […] I’m not going to read 60 messages and I’m not going to connect
with a person that I don’t even know. Hackathon networking was zero, so I
think it loses a lot, at least in this [online] model._ ”, H3P3).

####  4.3.4. Mentoring

Besides its importance for creating and maintaining common ground (section
4.1.4), we also found mentoring to be a motivation for participants to join a
hackathon. This was particularly the case in hackathons H2 and H3 where
participants stated that their “ _reasons were: first, the excellent mentoring
provided by people that I recognize_ ” (H3P5) and that they perceived it as a
good opportunity “ _to get some useful insights from mentors and potentially
get some further support_ ” (H2P4). In hackathons H1 and H4 mentoring was not
mentioned by participants as a reason to join an event. This difference can
partially be explained by hackathons H2 and H3 focusing on entrepreneurship.
Participants in this context perceived mentors to be able to support them to “
_develop our product further, and get some get some useful insights from
mentors_ ” (H2P1) and also to “ _get some further support_ ” (H2P1) regarding
the commercialization of that product.

####  4.3.5. Team collaboration

As evident by the discussion in the previous sections we found diverse
motivations for participants to join an event. Despite this diversity most
participants showed a strong willingness to collaborate as e.g. evident by
H1P3 stating that the “ _first night we stayed up til like 2:00 AM_ ” (H1P3)
or by H2P5 mentioning that they “ _kept messaging the people to see how
they’re doing_ ” (H2P5). There were however also occasions where participants
felt isolated within their teams. These were mainly reported by participants
in hackathons H2 and H3 as evident by H3P1 elaborating that s/he “ _hasn’t met
anyone I could exchange information with. I said hi on the channel, but I
didn’t do anything beyond that_ ” (H3P1) and by H2P4 mentioning that “ _I
didn’t know what my team was doing. […] So, I asked them and had to wait for
an answer_ ” (H2P4). These issues appeared to be team-specific rather than
hackathon-specific though. Some teams simply took a more asynchronous approach
to collaboration which could lead to delays in communication as discussed in
section 4.2.3.

In addition, it should be noted that participants also mentioned that they
missed engaging with people outside of their team beyond organized checkpoints
(“ _each team worked in its place without sharing information_ ”, H3P4). This
sentiment was shared between participants of all hackathons in particular
those that had prior experience participating in in-person events (“ _if we
were in person, I could have heard something like seeing something walking by_
”, H1P3).

Aspects |  H1 |  H2 |  H3 |  H4
---|---|---|---|---
Theme (4.3.1) |  X |  X |  X |  X
Learning (4.3.2) |  Technical skills |  Collaboration, Business |  Collaboration, Business |  Collaboration, Technical skills
Networking (4.3.3) |  X |  X |  X |  X
Mentoring (4.3.4) |  |  X |  X |
Team collaboration (4.3.5) |  Collaboration willingness, Missed engagement outside of team |  Collaboration willingness, Isolation, Missed engagement outside of team |  Collaboration willingness, Isolation, Missed engagement outside of team |  Collaboration willingness, Missed engagement outside of team
Table 4. Motivations for participants to join an event and their influence on
collaboration (X indicates that participants in that hackathon mentioned the
respective motivation to join the event).

###  4.4. Collaboration technology readiness

In the hackathons we studied organizers, mentors and participants utilized
various technologies to collaborate. Our findings reveal that the tools
suggested by the organizers were in many cases different from those that the
teams utilized themselves. Participants also reported issues utilizing the
different tools as well as general technical difficulties (c.f. Table 5 for an
overview).

####  4.4.1. Tools suggested by organizers vs. tools used by participants.

For all events, the organizers prescribed tools to communicate with mentors
and participants. In the cases of H1 and H2 the organizers prescribed Slack
for asynchronous communication and Zoom for video calls (“ _there are two
tools, mainly Slack and Zoom_ ”, H1P1). The organizers of H3 and H4 also
prescribed two different tools for synchronous and asynchronous communication
but instead of Slack and Zoom they utilized Discord and Zoom (“ _at Discord,
the event’s official channel, we solved everything we had to solve: mentoring,
talking to the “godmother”, etc_ ”, H4P4).

Instead of sticking to the channels that were proposed by the organizers,
teams instead used a large variety of different tools to communicate including
“ _iMessage_ ” (H1P3), “ _WhatsApp_ ” (H2P7), “ _Facebook Messenger_ ” (H2P1),
“ _GoogleMeet_ ” (H3P5), and others. One of the reasons for that could be
related to some participants reporting that they found certain tools such as
Discord “ _extremely challenging_ ” (H4P5). They thus reverted to tools that
they found to be easier to use. Even when using the same tools, teams often
opted to utilize different workspaces for their communication as evident by
H3P3, who mentioned that “ _the communication was in the Discord’s channel on
our server. There was an event’s server, but it didn’t make sense for us to
stay there. By keeping our channel on our private server, we could play music
and share links_ ” (H3P3). Similarly, H1P3 mentioned s/he had “ _zoom premium
through school_ ” (H1P3) which the team utilized during the event instead of
the Zoom workspace that was provided by the organizers.

In addition to the aforementioned communication tools, participants also
utilized different tools to share code (“ _we’d push the code to the GitHub
repo_ ”, H1P1), documents (‘ _Google Drive made our work easier. If anyone had
any questions, just put it in the document, and someone would respond_ ”,
H4P6), and manage their tasks (“ _we used the Miro tool to manage our work_ ”,
H3P5).

The disparity between tools that participants used to collaborate and the
tools that they used to connect with the organizers also led to issues as
reported by H2P1: “ _I think more issues came with following the communication
of the hackathon organizers and what was happening in Slack. Because we didn’t
really use Slack for our communication at that point, so sometimes we had to
remind someone to check that_ ”, (H2P1).

The tools that teams ended up using to collaborate were thus not driven by the
event design. Their usage was rather based on decisions within each individual
team.

####  4.4.2. Tool usage issues.

Participants also mentioned several issues they faced when using certain tools
during the event. The teams of H2P1 and H2P4 reported technical problems when
uploading their video [pitch] to YouTube after editing it (“ _there were these
technical issues that came up during the pitch because I just couldn’t upload
the pitch properly to YouTube_ ”, H2P1). Participant H2P4 added that “ _the
last video [pitch], it was made like in the last minute, so we were trying to
edit the video and make it nice and then when we uploaded the video on
YouTube, we noticed it had a technical error so we had to upload it again and
that was a bit of trouble_ ” (H2P4). These issues can be partially connected
to the design of the event which required teams to upload a video to
participate in the final competition in the case of hackathon H2. It did not
become clear from the interviews though if the issues were based on
participants not knowing how to upload a video or if there was indeed a
problem with YouTube when they tried.

A participant of H4 reported a similar issue related to the tool that the
organizers utilized to record the pitch video that was part of the final
presentation (“ _The tool to record the pitch video. It’s horrible! […] we had
to record the audio directly into the platform for 4 minutes straight. So any
mistake required redoing the whole audio_ ”, H4P4). This issue was directly
related to the choice of a tool on the part of the organizers of hackathon H4.

Finally, participants of different hackathons reported that they were not
always sure where to find the information that they needed and how to reach
organizers or mentors if an issue arose (“ _What was confusing that all the
resources were kind of in different places. Um, you’d see some things through
email, some things through the website, some things through Slack._ ”, H1P4).
Similarly, H2P1 reported issues when utilizing the tool that the organizers
had chosen for registration. Their team lead registered the team and thought
that this would be sufficient (“ _first of all, our team lead signed us up,
like as a team. So we were expecting that’s enough_ ”, H2P1). However s/he
then noticed that s/he “ _didn’t get any information_ ” (H2P1) and realized
that all team members need to “ _register separately_ ” (H2P1). These issues
can be directly related to the event design in that organizers did not
sufficiently communicate to participants where to find information on how to
register.

####  4.4.3. General technical issues.

In addition to issues related to the tools that were used, participants also
mentioned general technical issues. These included power outages (“ _[one team
member’s] power was down for like six hours_ ”, H1P4), issues with their
personal hardware (“ _so her/his computer did get angry and lose connection
and stop working_ ”, H1P4) or connectivity related issues (“ _In some of our
team calls we had some internet or Zoom problems, but not much._ ”, H2P1).
These were however individual issues which albeit sometimes linked to tools
suggested by the organizers but were not specifically caused by the
scaffolding that the organizers of the different events provided.

Aspects |  H1 |  H2 |  H3 |  H4
---|---|---|---|---
Tools suggested (4.4.1) |  Slack, Zoom |  Slack, Zoom |  Discord, Zoom |  Discord, Zoom
Tools used (4.4.1) |  iMessage, GitHub |  WhatsApp, Facebook Messenger |  WhatsApp, GoogleMeet, Miro |  WhatsApp, GoogleMeet, GoogleDrive
Tools usage issues (4.4.2) |  Unable to find information |  Unable to find information, Issues with required tools |  |  Issues with required tools
General technical issues (4.4.3) |  Power outage, Hardware issues |  Connectivity issues |  |
Table 5. Technologies utilized by teams to collaborate.

###  4.5. Aspects specific to collaboration in hackathons

In addition to strategies related to remote teamwork, we also identified
aspects that can be considered to be specific for a hackathon setting and that
affected the participant experience during the event they joined. These
included aspects related to mentoring, dropout and sustaining motivation
during an event (c.f. Table 6 for an overview).

####  4.5.1. Mentoring

We already discussed mentoring in the context of creating and retaining common
ground (section 4.1.4) and as a motivation to join an event (section 4.3.4).
We did also identified other aspects related to mentoring that shaped the
experience of participants during an event.

As common in hackathons participants reported that mentors helped them solve
issues in all four hackathons we studied. These issues were often related to
technical problems as in the case of H4P3 who mentioned that if they had a
problem with their prototype they “ _would look for a mentor with
technological expertise_ ” (H4P3) or “ _look through our screens at our code_
” (H1P1). Similarly H3P6 mentioned that mentors helped them with “ _technical
issues”_ (H3P6). We found such cases across all four hackathons which can be
expected since all teams developed some form of prototype.

In addition, we also found that mentors provided their expertise related to
the theme of the event. In the case of hackathon H2 this meant providing
advice about the business side of a team’s project since H2 focused on
entrepreneurship (“ _mentors gave good advice […] who was doing the similar
idea before where the idea is already on the market and gave us the contact of
that person_ ”, H2P6). In comparison, the advice that mentors gave during H4
were related to conservation (“ _a biologist mentor would be concerned if the
project would be viable for the conservation unit_ ”, H4P3) and in the case of
H3 they were related to social impact (“ _I thought the mentor was very
prepared and immersed in the context because s/he is associated with these
themes related to social impact_ ”, H3P1). Since the focus of hackathon H1 was
to engage newcomers with technologies they were not familiar with the advice
that mentors provided was mainly related to solving technical issues as
discussed before. Mentor feedback was thus directly linked to the design of
the different hackathons we studied in that each event had a specific theme to
which the projects that teams attempted were related to.

We also found cases where mentors provided support regarding the competition
that was part of all the hackathons we studied. In the case reported by H1P4
their mentor suggested for them to have their project on “ _a public listed
website […] cause that’s something that judges tend to like to see_ ” (H1P4).
Similarly, participants of hackathons H2 and H4 reported that a mentor “
_helped us a lot with the pitch_ ” (H2P7) which was the main deliverable for
the competition during that hackathon (“ _there was a mentor who wanted to go
over the whole text line by line of our pitch_ ”, H4P6).

The general perception about mentoring was mostly positive across all four
hackathons (“ _our mentors were able to help us with pretty much all our
problems_ ”, H1P1, “ _the mentoring was very good_ ”, H3P1) with participants
mentioning that there were “ _a lot of mentors from different fields, and they
were quite, like, happy to contribute_ ” (H2P1). In the case of H4, the
mentors were always available “ _when we asked for mentoring there was always
one available who helped us_ ” (H4P2). The only issue that participants
reported related to the mentors was their availability during the event (“ _it
was a bit hard to schedule with some of the mentors_ ”, H2P1, “ _sometimes
there is no connection between the mentor and the team_ ”, H3P2). We only
found these issues in H2 and H3 though. This might be related to the design of
the event since in the case of H1 one mentor was responsible for one team. The
organizers of H2 had set up a system for participants to book meetings with
mentors but not all mentors used it. Participants thus consequently reported
that they were able to reach mentors who used that system but had issues
reaching others who did not (“ _some of the mentors even had, like a calendar,
so you can do like appoint appointment slots with them […] not all the mentors
had same tools, but the ones who did this were like, super easy to get in
touch with_ ”, H2P5).

####  4.5.2. Dropout

Another aspect that has not been discussed in the context of remote teamwork
but that we found in our study is participants dropping out during an event.
In the hackathons that we studied we only found one such occasion (“ _one of
those people wasn’t around anymore. S/he just kinda like stopped talking to
us_ ”, H1P1) but interviewees mentioned this as a common problem (“ _It is
very common for participants to drop out. Teams dissolve very easily. Many
people enter and few people finish_ ”, H4P4). Although it may be related to
the inherent design of an online event, where one can leave a team by just
disappearing/disconnecting without facing any social friction, it only
happened in one of the hackathons teams we studied while the others just
reported that fact generally.

####  4.5.3. Sustaining motivation

Participants also mentioned the remote format of the hackathon as being less
motivating. Examples for this are participant H2P5 stating that “ _I think
there are certain things you get, like in person like all this mingling and
joking and like just sharing the experience of being on the same place and
basically laughing, and then you lose, you lose all this human connection_ ”
(H2P5). Similarly, H3P4 mentioned a “ _lack of human contact, despite the fact
that the team met for a few hours, but then everyone stayed in their corner_ ”
(H3P4) and H1P2 said that “ _I feel like maybe if we were in person, I could
have heard something like seeing something walking by, whereas like it was
still a great experience and I enjoyed my time, but I think in-person possibly
could have made it more personal_ ” (H1P2). This lack of contact subsequently
led to some participants losing motivation over time as reported by H2P1 (“
_it’s harder to stay focused for a long time […] because I’m very much this
person who gets energy from other people and if I’m alone behind the screen,
it’s so hard to push yourself._ ”, H2P1). This sentiment was shared across all
four hackathons we studied. The different designs thus did not appear to have
an influence on this particular aspect, which might thus be considered a
natural consequence of the online setting.

Aspects |  H1 |  H2 |  H3 |  H4
---|---|---|---|---
Mentoring (4.5.1) |  Technical support, Theme and competition-related guidance, Positive mentoring experience |  Technical support, Theme and competition-related guidance, Positive mentoring experience, Availability issues |  Technical support, Theme related guidance, Positive mentoring experience, Availability issues |  Technical support, Theme and competition-related guidance, Positive mentoring experience
Dropout (4.5.1) |  One participant dropped out |  |  Mentioned as common issue |
Sustaining motivation (4.5.1) |  Human contact missing |  Human contact missing |  Human contact missing |
Table 6. Aspects that affected the experience of participants during the four
studied hackathons.

##  5\. Discussion

In the following subsections, we will provide answers to our research
questions by revisiting some of the findings that were previously discussed
under key concepts of effective distance work (Olson and Olson, 2000) and
positioning it within the existing body of work.

###  5.1. How teams collaborate in an online hackathon (RQ${}_{\ref{RQ1}}$)

The way some teams from our study collaborated in these online hackathons
confirms what is found in literature about collaborative work concerning small
groups between two and five people often engaging in mixed-focus collaboration
(i.e., synchronous and asynchronous work), with team members frequently
shifting between individual and shared activities (Gutwin and Greenberg,
2002). Synchronous calls – either continuous or recurring – were fundamental
to increase awareness, which is another important concept in collaborative
work. Although that term has not been defined and used consistently (Gross,
2013), for our context we will consider that having awareness information is
required to have a shared understanding of each other’s activities and to
coordinate group activities as a whole (Dourish and Bellotti, 1992). Such
information sharing is related to the team’s collaboration readiness. Teams
that kept continuous synchronous communication particularly caught our
attention. Those were teams that had previously established a common ground
(e.g., worked before in another hackathon, knew each other since
kindergarten). They seemed to have more awareness than teams that did work in
parallel and just synchronized occasionally. These continuously open audio
calls mimic radical co-location (Trainer et al., 2014) by giving a sense of
virtual co-location (Olson and Olson, 2000), or perhaps radical virtual co-
location considering the intense synchronous online collaboration. This
approach for communication was also the preference of synchronous verbal
conversation in a controlled study about hybrid collaboration (Neumayr et al.,
2018). However, two hackathon teams complained about the continuous exposure
to these online calls as something occasionally disturbing. This can be
explained as the need that some people may have to create their personal audio
territories so they can temporarily disconnect from shared group audio
territories (Neumayr et al., 2018). The need to break out is real. The
cognitive load of video calls also may lead to what has been called Zoom
fatigue (Bailenson, 2021) – a sort of mental exhaustion due to excessive
exposure to synchronous video calls. Despite the reported problems, we believe
this way of working helps to prepare online hackathons participants who are
less experienced in distant collaborative work, so they can be ready for the
likely scenario of post-COVID hybrid workplaces (Kane et al., 2021), where
members of the same team may be either physically at the place or working
remotely and need to rely on similar tools and approaches to performing
teamwork.

In terms of collaboration readiness, we perceive groups participating in
online hackathons as self-organized teams, where there is not a formally
appointed role of leader and task allocation is self-directed. We found
similarities to what was reported in CSCW literature about the main self-
organized teamwork features in a time-critical context: awareness of task
ownership, task self-assignment, and informal team hierarchy (Zhang and
Sarcevic, 2018). In the teams we studied, task division was self-assigned and
based on individual skills or interests. Depending on their coupling of work
participants may have more frequent video or audio calls to synchronize work.
The awareness of task ownership could be maintained through both synchronous
and asynchronous communication but was increased in groups using more
synchronous communication. Team hierarchy was informal and the role of a
leader naturally emerged, either because of their experience – analogous to
the seniority mentioned in (Zhang and Sarcevic, 2018) – or just because the
person acted as a proxy between the group and the event organizers (e.g., the
person that did the team/project/idea registration in the event). This role
had nothing to do with task assignment, but those more experienced helped
guide the teams toward an objective.

In terms of collaboration technology readiness, although event organizers
suggested pre-defined tools, we found that many participants switched to tools
more appropriate to their preference or collaboration style. Tools for
asynchronous communication varied (Slack, Discord text channels, WhatsApp),
but synchronous communication (Zoom, Discord voice channels, Google Meet) was
key for sharing a common understanding, as this is also pointed out in Global
Software Development literature as a key aspect for developers working
remotely (Noll et al., 2011). The myriad of collaboration tools these days
help in the aspect of _collaboration technology readiness_ , but technical
constraints of using many communication tools and online platforms can
introduce issues and can be overwhelming for participants of online hackathons
(Braune et al., 2021).

###  5.2. How event design affects team collaboration (RQ${}_{\ref{RQ2}}$)

Mentors had a strong influence on team collaboration. Expert mentoring is
reported as an important benefit to participants in both in-person (Nolte et
al., 2020b) and online hackathons (Braune et al., 2021). According to our
analysis, mentors helped in several ways, including: 1) helping the team to
focus on the things that matter the most, 2) bringing some ideas of things to
build, 3) sharing their expertise on technical problems, or 4) providing a
supportive environment to make sure the team finishes their project. In these
aforementioned examples, the mentor followed a more _passive_ approach, that
is, the mentor was waiting for a team member to flag her/his attention.
However, we also noticed cases in which mentors took a more _active_ approach.
In the H1 hackathon, one mentor helped two groups – that were working on
similar goals – to collaborate during a short period. This was only possible
because the mentor had an up-to-date overview of the teams and their projects.

Nevertheless, we noticed that in online hackathons interactions with mentors
became less spontaneous and bureaucratic, now depending on scheduled
reservations and very short mentoring sessions. There are often much more
participants than mentors. Therefore, mentors do not always have the
availability that team members desire. Although most of the participants
concur that it was easy to book a meeting with a mentor, team members only had
a short time to discuss their needs. Moreover, this more formal way to
approach a mentor (booking their time in a calendar app) diminished the social
interaction. One participant provided a great summary by stating that “
_usually, like, in a hackathon, where you’re physically there, you can just
bump into them, like in a coffee machine […] you miss all of that [online]. So
in a way, you need to be very conscious that you need to approach people
instead of just letting it happen naturally_ ”.

Networking is perhaps one of the most recurring motivations to participate in
hackathons (Briscoe, 2014; Gama, 2017a; Medina Angarita and Nolte, 2019). Some
participants mentioned that networking only existed within their teams and
between the team and their mentors. Since many teams were formed before the
event, networking with new peers was minimal. To try to network, participants
would have to join busy text channels during the event and keep track of
dozens of messages sent by strange avatars. Given the project tasks and
parallel activities, this would require non-trivial effort from team members.
We believe this was one of the most compromised aspects of a traditional
hackathon when moving to an online setting. By not overhearing and interacting
with other teams, participants of online hackathons were less exposed to new
problems, ideas, or even colleagues. The serendipitous learning from overheard
conversations (Pe-Than and Herbsleb, 2019) of participants in the same
physical space no longer exist, since in the online model, each team works in
isolation from other teams. Although among our interviewees one team mentioned
collaborating with another team in the hackathon, the intrinsic format of an
online hackathon does not favor inter-team collaboration. Unfortunately, the
competitive scenario might add to turning hackathons less welcoming to
newcomers or less experienced participants.

Hackathons are an environment of time constraints and pressure due to their
short duration (Huppenkothen et al., 2018). Because in-person hackathons are
normally hosted on weekends and participants need to be on-site, some people
avoid participating in such events because of the incompatibility with travel
time and family commitments (Anslow et al., 2016). One advantage of online
hackathons in comparison to the in-person format is that such an environment
of pressure can accommodate those types of social activities because people
are working from their homes and also have some flexibility with time.
Although the studied hackathons had the typical short format (2 to 4 days) and
three of them took place over the weekend, the possibility of asynchronous and
remote work allowed some participants to join ordinary activities of their
daily lives (e.g., work, birthday, wedding) without any affecting their
participation on teamwork. Recent hackathon literature also highlighted online
COVID-19 hackathons as a way to foster international collaborations (Braune et
al., 2021). With some of the participants we interviewed, we notice this
geographical dispersion as another affordance of the remove format that allows
people from different locations to collaborate.

###  5.3. Implications for organizers and participants

Based on our analysis, we identified opportunities for online hackathon
organizers to enhance the design of these events.

####  5.3.1. Online hackathons still need more structuring.

Team formation could take place before the event starts to relieve the
pressure of finding a team in the first hours of a hackathon. Previously
formed teams start with a competitive advantage because they can already focus
on their project while other participants without teams are still under team
formation. Therefore, pre-hackathon online events or even asynchronous
activities could be helpful for networking, so participants still unknown to
each other can identify or build common ground to a minimum extent.

Collaboration technology readiness can be improved by trying to manage
communication tools better and attempting to keep participants on the same
platform as organizers, but without forcing them to do so. This could reduce
the number of tools that participants need to deal with. For instance, not
only using the same tool (e.g., Discord) already familiar to participants but
also giving additional support/customization from organizers (e.g., create
audio and text channels for each group) can be successful unify the same tool
for synchronous (audio, video and screen sharing) and asynchronous (text)
communication in an online hackathon, as reported by participants (Gama et
al., 2021). Organizers can also allow participants to create their own
subchannels in a common communication tool for teams and organizers (e.g.,
Slack) (Braune et al., 2021).

####  5.3.2. Invest in connecting mentors with participants.

Instead of mentors that are available on a tight schedule, event organizers
can try to create a more personal experience where mentors are closer to the
teams and can interact with them more frequently. If the same mentor conducts
the checkpoints regularly, they can be aware of the group evolution and
establish a closer relationship with the team members, bringing positive
effects to participants through a relation of trust.

####  5.3.3. Fostering social interaction outside of teams.

In addition to pre-hackathon networking for team formation, organizers can
also foster interaction among teams during the hackathon. Distractions and
games to amuse participants may not be attractive for those who want to focus
on their professional development and networking though. Collective
checkpoints involving different teams was a successful and natural way of
interacting among participants, where they could provide and get feedback from
other teams as well as get to know each other. A downside of that approach is
that groups may not want to share details if the competitive aspect is too
strong among participants.

###  5.4. Limitations

The goal of our research was to study how teams collaborate during an online
hackathon and how the design of an event can potentially affect their
collaboration. It is thus reasonable to conduct a multiple-case study (Yin,
2017) because it allows us to study the novel phenomenon of collaboration
during online hackathons in the context it occurs and draw insights about
“how” it takes place (Runeson and Höst, 2009). There are however inherent
limitations related to this particular research design. We conducted a total
of 27 interviews across 10 teams that participated in 4 hackathons that took
place over a span of six months across two continents. Despite making a
deliberate selection of hackathons that were organized around different
themes, goals and sizes and randomly selecting teams our findings cannot be
generalized beyond our study context. Studying different teams that
participated in different events organized with different goals might yield
different results. Moreover, we opted for interviews as our main source of
information. The perceptions of participants might however affect their report
of the event. In order to address this limitation, we conducted the interviews
close to the event itself to ensure that the participants would remember most
details about the event and their collaboration as a team. We also interviewed
at least two members of each team we studied to avoid solely focusing on a
single perspective thus inducing bias. Another potential source of bias stems
from the fact that some interviews were conducted in Portuguese and had to be
translated to English for analysis. To mitigate this bias we ensured that two
Portuguese native speakers with excellent command of the English language
handled the translation collaboratively. Furthermore, the analysis was
conducted by a team of researchers. Despite collaborating during the entire
analysis procedure, there is the possibility that different researchers with
different expertise and backgrounds might have interpreted certain findings
differently. We thus abstained from making causal claims instead providing a
rich description of the perceptions of participants based on which we discuss
how teams collaborated and how they perceived the event design to affect their
collaboration. We hope to see more research on collaboration in online
hackathons in the future to complement our findings.

##  6\. Conclusions

Hackathons probably became the most popular form of time-bounded collaborative
event in recent years. Due to the social distancing that had to be followed
worldwide to reduce the spread of COVID-19, online hackathons became the norm
for that kind of event. Instead of being limited to people from a city or
region, participation could involve people from distant places. In-person
hackathons rely on affordances such as the radical co-location of participants
that allow them to exchange knowledge and achieve good results. While
literature is still developing around this recent online hackathon phenomenon,
we focused on exploring different perspectives on collaboration: how teams
collaborate in this online setting and how the event format affects team
collaboration. As a way to collect data to understand such perspectives, we
interviewed 27 participants of 10 teams from 4 different hackathons in Europe
and the Americas. We analyzed the transcripts of the interviews framed by four
key concepts of distributed team collaboration (common ground, coupling in
work, collaboration readiness, and technology readiness).

###  6.1. Contributions

As the main contributions, we believe that this work can be (1) an initial
analysis on how teams collaborate in online hackathons with findings that can
potentially be translated to other online collaboration settings; and (2)
recommendations for organizers and teams on how they can better approach
events like those. In our analysis, we found that certain affordances of in-
person hackathons, such as interactions with other teams, the casual and easy
access to mentors, and networking with other participants, have diminished or
even disappeared in the online versions of hackathons. Interactions with
mentors became formal and less spontaneous since they now have to be booked
and take very little time, and casual contact with participants from other
teams did not take place. Technical constraints of this online hackathon
setting include the multitude of collaboration tools that can confuse
participants. We also noticed that, due to the online format, teams must
establish common ground on their participants’ abilities through synchronous
and asynchronous communication tools. The awareness of each other’s work
mainly relied on synchronous communication through audio and video
conferencing tools, with some groups preferring to leave such channels open
continuously, creating a sort of shared group audio territory, partially
reproducing the radical co-location of hackathons in a sort of virtual radical
co-location. Similar to other time-critical contexts of teamwork, the studied
teams worked in a self-organized way, with tasks being selected based on their
skills or preferences, and leaders emerged naturally. The remote format also
brings benefits such as allowing people from geographically distant locations
to collaborate; and the possibility of asynchronous and remote work bringing
less pressure, allowing participants to engage in other activities of their
daily lives (e.g., work, social events, being close to family members).
Despite being one of the main reasons to participate, we noticed that
networking has become restricted to team members and mentors. From the
participants’ perspective who had some experience in in-person hackathons, the
online format is not very attractive. As recommendations we suggested more
structuring from the organization around team formation and tool support,
investing in the relationship between mentors and participants, and fostering
mores exchanges among teams.

###  6.2. Future Work

For future work, we plan to expand the findings of this research in several
ways. First, it is on our research agenda to better understand dropout in
online hackathons. Although we only observed dropout in one hackathon, we have
reason to believe that this is more common than in in-person events. By having
a better understanding about dropouts, we could support hackathons teams and
organizers. Second, it is in our roadmap to organize and conduct an online
hackathon, which should take into account some of the findings observed in
this research; for instance, since we noticed that access to mentors is not
always straightforward, in this planned event, we will instruct mentors to
have a more _active_ approach towards their role. Results from this study
might provide more evidence on the effectiveness of the findings of this
research. Finally, another question that is on our radar is: since we are
approaching a post-COVID era, it is not clear how hackathons (that moved from
in-person to online) will deal with its potential next move: from an online to
a hybrid event. This change will provoke further challenges in the hackathon
design and organization; we plan to conduct field studies to understand
whether and how our findings hold in this new scenario.

###### Acknowledgements.

The authors gratefully acknowledge support by Omnibond LLC, Science Gateways
Community Institute Workforce Development (funded by the National Science
Foundation under award number ACI-1547611), Elizabeth City State University,
Texas Advanced Computing Center, Garage48, XSEDE, SigHPC, Globus, Intel, and
Google.

## References

  * (1)
  * Anslow et al. (2016) Craig Anslow, John Brosz, Frank Maurer, and Mike Boyes. 2016\.  Datathons: an experience report of data hackathons for data science education. In _Proceedings of the 47th ACM Technical Symposium on Computing Science Education_. ACM, 615–620.
  * Bailenson (2021) Jeremy N Bailenson. 2021\.  Nonverbal overload: A theoretical argument for the causes of Zoom fatigue.  _Technology, Mind, and Behavior_ 2, 1 (2021).
  * Beecham et al. (2014) Sarah Beecham, John Noll, and Ita Richardson. 2014.  Using agile practices to solve global software development problems–a case study. In _2014 IEEE International Conference on Global Software Engineeering Workshops_. IEEE, 5–10.
  * Bertello et al. (2021) Alberto Bertello, Marcel LAM Bogers, and Paola De Bernardi. 2021.  Open innovation in the face of the COVID-19 grand challenge: insights from the Pan-European hackathon ‘EUvsVirus’.  _R &D Management_ (2021).
  * Birbeck et al. (2017) Nataly Birbeck, Shaun Lawson, Kellie Morrissey, Tim Rapley, and Patrick Olivier. 2017.  Self Harmony: rethinking hackathons to design and critique digital technologies for those affected by self-harm. In _Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems_. ACM, 146–157.
  * Braune et al. (2021) Katarina Braune, Pablo-David Rojas, Joscha Hofferbert, Alvaro Valera Sosa, Anastasiya Lebedev, Felix Balzer, Sylvia Thun, Sascha Lieber, Valerie Kirchberger, and Akira-Sebastian Poncette. 2021.  Interdisciplinary Online Hackathons as an Approach to Combat the COVID-19 Pandemic: Case Study.  _Journal of medical Internet research_ 23, 2 (2021), e25283.
  * Brereton (2020) Bernadette Brereton. 2020\.  EUvsVirus Hackathon project: A case study from a mentor’s perspective.  _All Ireland Journal of Higher Education_ 12, 2 (2020).
  * Briscoe (2014) Gerard Briscoe. 2014\.  Digital innovation: The hackathon phenomenon.  _Creativeworks London_ 6 (2014), 1–13.
  * Briscoe et al. (2015) Gerard Briscoe, Tarek E Virani, and Mariza Dima. 2015\.  Hackathons: Why Co-location?  _Creativeworks London Working Papers_ (2015).
  * Brudy et al. (2018) Frederik Brudy, Joshua Kevin Budiman, Steven Houben, and Nicolai Marquardt. 2018.  Investigating the role of an overview device in multi-device collaboration. In _Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems_. 1–13.
  * Calefato et al. (2020) Fabio Calefato, Andrea Giove, Filippo Lanubile, and Marco Losavio. 2020.  A case study on tool support for collaboration in agile development. In _Proceedings of the 15th International Conference on Global Software Engineering_. 11–21.
  * Chen et al. (2020) Felix Chen, Michael Cuddy, Conor Daly, and Mia Velletri. 2020\.  Analyzing Community Building Among First-Year Students at WPI During the COVID-19 Pandemic in the Fall of 2020.  (2020).
  * Cobham et al. (2017) David Cobham, Kevin Jacques, Carl Gowan, Jack Laurel, Scott Ringham, et al. 2017\.  From appfest to entrepreneurs: using a hackathon event to seed a university student-led enterprise. In _11th annual International Technology, Education and Development Conference_.
  * Crowston et al. (2007) Kevin Crowston, James Howison, Chengetai Masango, and U Yeliz Eseryel. 2007.  The role of face-to-face meetings in technology-supported self-organizing distributed teams.  _IEEE Transactions on professional communication_ 50, 3 (2007), 185–203.
  * Dourish and Bellotti (1992) Paul Dourish and Victoria Bellotti. 1992.  Awareness and coordination in shared workspaces. In _Proceedings of the 1992 ACM conference on Computer-supported cooperative work_. 107–114.
  * Faas et al. (2019) Travis Faas, I-ching Liu, Lynn Dombrowski, and Andrew D Miller. 2019\.  Jam today, jam tomorrow: Learning in online game jams.  _Proceedings of the ACM on Human-Computer Interaction_ 3, GROUP (2019), 1–27.
  * Fagerholm et al. (2013) Fabian Fagerholm, Patrik Johnson, Alejandro Sánchez Guinea, Jay Borenstein, and Jürgen Münch. 2013.  Onboarding in open source software projects: A preliminary analysis. In _2013 IEEE 8th International Conference on Global Software Engineering Workshops_. IEEE, 5–10.
  * Filippova et al. (2017) Anna Filippova, Erik Trainer, and James D Herbsleb. 2017\.  From diversity by numbers to diversity as process: supporting inclusiveness in software development teams with brainstorming. In _Proceedings of the 39th International Conference on Software Engineering_. IEEE Press, 152–163.
  * Fowler (2016) Allan Fowler. 2016\.  Informal stem learning in game jams, hackathons and game creation events. In _Proceedings of the International Conference on Game Jams, Hackathons, and Game Creation Events_. ACM, 38–41.
  * Fowler et al. (2020) Allan Fowler, Johanna Pirker, and Ali Arya. 2020.  Jamming across borders: An exploratory study. In _International Conference on Game Jams, Hackathons and Game Creation Events 2020_. 16–21.
  * Gama (2017a) Kiev Gama. 2017a.  Crowdsourced software development in civic apps-motivations of civic hackathons participants. In _International Conference on Enterprise Information Systems_ , Vol. 2. SCITEPRESS, 550–555.
  * Gama (2017b) Kiev Gama. 2017b.  Preliminary findings on software engineering practices in civic hackathons. In _2017 IEEE/ACM 4th International Workshop on CrowdSourcing in Software Engineering (CSI-SE)_. IEEE, 14–20.
  * Gama et al. (2018) Kiev Gama, Breno Alencar, Filipe Calegario, André Neves, and Pedro Alessio. 2018.  A Hackathon Methodology for Undergraduate Course Projects. In _2018 IEEE Frontiers in Education Conference (FIE)_. IEEE, 1–9.
  * Gama et al. (2021) Kiev Gama, Carlos Zimmerle, and Pedro Rossi. 2021\.  Online Hackathons as an Engaging Tool to Promote Group Work in Emergency Remote Learning. In _Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 1_. 345–351.
  * Greeno (1994) James G Greeno. 1994\.  Gibson’s affordances.  (1994).
  * Gross (2013) Tom Gross. 2013\.  Supporting effortless coordination: 25 years of awareness research.  _Computer Supported Cooperative Work (CSCW)_ 22, 4-6 (2013), 425–474.
  * Gutwin and Greenberg (2002) Carl Gutwin and Saul Greenberg. 2002.  A descriptive framework of workspace awareness for real-time groupware.  _Computer Supported Cooperative Work (CSCW)_ 11, 3 (2002), 411–446.
  * Hartmann et al. (2018) Sarah Hartmann, Agnes Mainka, and Wolfgang G Stock. 2018\.  Innovation Contests: How to Engage Citizens in Solving Urban Problems?  In _Enhancing Knowledge Discovery and Innovation in the Digital Era_. IGI Global, 254–273.
  * Henderson (2015) Scott Henderson. 2015\.  Getting the most out of hackathons for social good.  _Volunteer Engagement 2.0: Ideas and insights changing the world_ (2015), 182–194.
  * Herbsleb and Moitra (2001) James D Herbsleb and Deependra Moitra. 2001.  Global software development.  _IEEE software_ 18, 2 (2001), 16–20.
  * Hoda et al. (2012) Rashina Hoda, James Noble, and Stuart Marshall. 2012\.  Self-organizing roles on agile software development teams.  _IEEE Transactions on Software Engineering_ 39, 3 (2012), 422–444.
  * Hope et al. (2019) Alexis Hope, Catherine D’Ignazio, Josephine Hoy, Rebecca Michelson, Jennifer Roberts, Kate Krontiris, and Ethan Zuckerman. 2019\.  Hackathons as Participatory Design: Iterating Feminist Utopias. In _Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems_. ACM, 61.
  * Hou and Wang (2017) Youyang Hou and Dakuo Wang. 2017.  Hacking with NPOs: collaborative analytics and broker roles in civic data hackathons.  _Proceedings of the ACM on Human-Computer Interaction_ 1, CSCW (2017), 1–16.
  * Huppenkothen et al. (2018) Daniela Huppenkothen, Anthony Arendt, David W Hogg, Karthik Ram, Jacob T VanderPlas, and Ariel Rokem. 2018.  Hack weeks as a model for data science education and collaboration.  _Proceedings of the National Academy of Sciences_ 115, 36 (2018), 8872–8877.
  * Imam et al. (2021) Ahmed Imam, Tapajit Dey, Alexander Nolte, Audris Mockus, and James D Herbsleb. 2021.  The Secret Life of Hackathon Code Where does it come from and where does it go?. In _2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR)_. IEEE, 68–79.
  * Kane et al. (2021) Gerald C Kane, Rich Nanda, Anh Phillips, and Jonathan Copulsky. 2021.  Redesigning the Post-Pandemic Workplace.  _MIT Sloan Management Review_ 62, 3 (2021), 12–14.
  * Kienzler and Fontanesi (2017) Hanna Kienzler and Carolyn Fontanesi. 2017.  Learning through inquiry: A global health hackathon.  _Teaching in Higher Education_ 22, 2 (2017), 129–142.
  * Kiesler and Cummings (2002) Sara Kiesler and Jonathon N Cummings. 2002.  What do we know about proximity and distance in work groups? A legacy of research.  _Distributed work_ 1 (2002), 57–80.
  * Komssi et al. (2015) Marko Komssi, Danielle Pichlis, Mikko Raatikainen, Klas Kindström, and Janne Järvinen. 2015\.  What are Hackathons for?  _IEEE Software_ 32, 5 (2015), 60–67.
  * Kos (2019) Brittany Ann Kos. 2019\.  Understanding Female-Focused Hackathon Participants’ Collaboration Styles and Event Goals. In _Proceedings of the International Conference on Game Jams, Hackathons and Game Creation Events 2019_. 1–4.
  * Kultima (2015) Annakaisa Kultima. 2015\.  Defining Game Jam.. In _Foundation of Digital Gaames_.
  * Lifshitz-Assaf et al. (2020) Hila Lifshitz-Assaf, Sarah Lebovitz, and Lior Zalmanson. 2020\.  Minimal and adaptive coordination: How hackathons’ projects accelerate innovation without killing it.  _Academy of Management Journal_ ja (2020).
  * Lodato and DiSalvo (2016) Thomas James Lodato and Carl DiSalvo. 2016.  Issue-oriented hackathons as material participation.  _New Media & Society_ 18, 4 (2016), 539–557.
  * Masood et al. (2020) Zainab Masood, Rashina Hoda, and Kelly Blincoe. 2020\.  How agile teams make self-assignment work: a grounded theory study.  _Empirical Software Engineering_ 25, 6 (2020), 4962–5005.
  * Medina Angarita and Nolte (2019) Maria Angelica Medina Angarita and Alexander Nolte. 2019.  Does it matter why we hack?–Exploring the impact of goal alignment in hackathons. In _Proceedings of 17th European Conference on Computer-Supported Cooperative Work_. European Society for Socially Embedded Technologies (EUSSET).
  * Nandi and Mandernach (2016) Arnab Nandi and Meris Mandernach. 2016.  Hackathons as an informal learning platform. In _Proceedings of the 47th ACM Technical Symposium on Computing Science Education_. ACM, 346–351.
  * Neumayr et al. (2018) Thomas Neumayr, Hans-Christian Jetter, Mirjam Augstein, Judith Friedl, and Thomas Luger. 2018\.  Domino: A descriptive framework for hybrid collaboration and coupling styles in partially distributed teams.  _Proceedings of the ACM on Human-Computer Interaction_ 2, CSCW (2018), 1–24.
  * Noll et al. (2011) John Noll, Sarah Beecham, and Ita Richardson. 2011.  Global software development and collaboration: barriers and solutions.  _ACM inroads_ 1, 3 (2011), 66–78.
  * Nolte et al. (2020a) Alexander Nolte, Irene-Angelica Chounta, and James D Herbsleb. 2020a.  What Happens to All These Hackathon Projects? - Identifying Factors to Promote Hackathon Project Continuation.  _Proceedings of the ACM on Human-Computer Interaction_ 4, CSCW2 (2020), 1–26.
  * Nolte et al. (2020b) Alexander Nolte, Linda Bailey Hayden, and James D Herbsleb. 2020b.  How to Support Newcomers in Scientific Hackathons-An Action Research Study on Expert Mentoring.  _Proceedings of the ACM on Human-Computer Interaction_ 4, CSCW1 (2020), 1–23.
  * Nolte et al. (2020c) Alexander Nolte, Ei Pa Pa Pe-Than, Abasi-amefon Obot Affia, Chalalai Chaihirunkarn, Anna Filippova, Arun Kalyanasundaram, Maria Angelica Medina Angarita, Erik Trainer, and James D Herbsleb. 2020c.  How to organize a hackathon–A planning kit.  _arXiv preprint arXiv:2008.08025_ (2020).
  * Olson and Olson (2003) Gary Olson and Judith Olson. 2003.  Mitigating the effects of distance on collaborative intellectual work.  _Economics of Innovation and New Technology_ 12, 1 (2003), 27–42.
  * Olson and Olson (2000) Gary M Olson and Judith S Olson. 2000.  Distance matters.  _Human–computer interaction_ 15, 2-3 (2000), 139–178.
  * Orlikowski (2002) Wanda J Orlikowski. 2002\.  Knowing in practice: Enacting a collective capability in distributed organizing.  _Organization science_ 13, 3 (2002), 249–273.
  * Paganini and Gama (2020) Lavinia Paganini and Kiev Gama. 2020.  Engaging Women’s Participation in Hackathons: A Qualitative Study with Participants of a Female-focused Hackathon. In _International Conference on Game Jams, Hackathons and Game Creation Events 2020_. 8–15.
  * Pe-Than and Herbsleb (2019) Ei Pa Pa Pe-Than and James D Herbsleb. 2019.  Understanding Hackathons for Science: Collaboration, Affordances, and Outcomes. In _International Conference on Information_. Springer, 27–37.
  * Pe-Than et al. (2019) Ei Pa Pa Pe-Than, Alexander Nolte, Anna Filippova, Christian Bird, Steve Scallen, and James D Herbsleb. 2019.  Designing Corporate Hackathons With a Purpose: The Future of Software Development.  _IEEE Software_ 36, 1 (2019), 15–22.
  * Pe-Than et al. (2020) Ei Pa Pa Pe-Than, Alexander Nolte, Anna Filippova, Chris Bird, Steve Scallen, and James D. Herbsleb. 2020.  Corporate Hackathons, How and Why? A Multiple Case Study of Motivation, Projects Proposal and Selection, Goal Setting, Coordination, and Outcomes.  _Human-Computer Interaction_ (2020).