# Project IRL: Playful Co-Located Interactions with Mobile Augmented Reality

Ella Dagan  University of California, Santa Cruz, USASant CruzCA95064
<EMAIL_ADDRESS>,  Ana María Cárdenas Gasca<EMAIL_ADDRESS>University
of California, Santa Barbara, USASanta BarbaraCA93106 ,  Ava Robinson
<EMAIL_ADDRESS>Snap Inc., Northwestern University, USANew YorkNY ,  Anwar
Noriega<EMAIL_ADDRESS>Wabisabi Design Inc., MexicoMexico City ,  Yu
Jiang Tham<EMAIL_ADDRESS>Snap Inc., USASeattleWA98121 ,  Rajan Vaish
<EMAIL_ADDRESS>Snap Inc., USASanta MonicaCA90405 and  Andrés Monroy-Hernández
<EMAIL_ADDRESS>Snap Inc., Princeton University, USASeattleWA98121

# Project IRL: Playful Co-Located Interactions with Mobile Augmented Reality

Ella Dagan  University of California, Santa Cruz, USASant CruzCA95064
<EMAIL_ADDRESS>,  Ana María Cárdenas Gasca<EMAIL_ADDRESS>University
of California, Santa Barbara, USASanta BarbaraCA93106 ,  Ava Robinson
<EMAIL_ADDRESS>Snap Inc., Northwestern University, USANew YorkNY ,  Anwar
Noriega<EMAIL_ADDRESS>Wabisabi Design Inc., MexicoMexico City ,  Yu
Jiang Tham<EMAIL_ADDRESS>Snap Inc., USASeattleWA98121 ,  Rajan Vaish
<EMAIL_ADDRESS>Snap Inc., USASanta MonicaCA90405 and  Andrés Monroy-Hernández
<EMAIL_ADDRESS>Snap Inc., Princeton University, USASeattleWA98121

###### Abstract.

We present Project IRL (In Real Life), a suite of five mobile apps we created
to explore novel ways of supporting in-person social interactions with
augmented reality. In recent years, the tone of public discourse surrounding
digital technology has become increasingly critical, and technology’s
influence on the way people relate to each other has been blamed for making
people feel “alone together,” diverting their attention from truly engaging
with one another when they interact in person. Motivated by this challenge, we
focus on an under-explored design space: playful co-located interactions. We
evaluated the apps through a deployment study that involved interviews and
participant observations with 101 people. We synthesized the results into a
series of design guidelines that focus on four themes: (1) device arrangement
(e.g., are people sharing one phone, or does each person have their own?), (2)
enablers (e.g., should the activity focus on an object, body part, or pet?),
(3) affordances of modifying reality (i.e., features of the technology that
enhance its potential to encourage various aspects of social interaction), and
(4) co-located play (i.e., using technology to make in-person play engaging
and inviting). We conclude by presenting our design guidelines for future work
on embodied social AR.

Playful, Co-Located, Embodied, Social, Augmented Reality, mobile AR, Games,
Play, apps, RtD.

††copyright: rightsretained††journal: PACMHCI††journalyear:
2022††journalvolume: 6††journalnumber: CSCW1††article: 62††publicationmonth:
4††doi: 10.1145/3512909††ccs: Human-centered computing Ubiquitous and mobile
computing systems and tools††ccs: Human-centered computing††ccs: Human-
centered computing Ubiquitous and mobile computing

Figure 1. Playful co-located mobile AR apps: (1) “Face It” (2); “Treasure
Treat”; (3) “Feeture Films”; (4) “Milky Way”; (5) “Freezing Frenzy.” Device
arrangement is illustrated in pink and enablers in blue.

Five schematic drawings showing players and screenshots of app being used

##  1\. Introduction

In recent years, the tone of public and academic discourse surrounding the
impact of technology on face-to-face interactions has become increasingly
negative. Prior research has shown that although digital technology makes it
easier to stay connected, it can also disrupt and alienate people engaged in
face-to-face social interactions (Oduor et al., 2016; Turkle, 2017; Misra et
al., 2016) to the extent that, even when people are physically together, or
co-located, they still experience a sense of being “alone” in their own
digital bubbles (Turkle, 2017; Rogers, 2014). Similarly, journalists have
argued that “human contact is now a luxury good” and that “separating from
screens is harder for the poor and middle class” (Bowles, 2019). Artists have
echoed these sentiments through photographs (Pickersgill, 2014; Tikunova,
2016), cartoons (Rob, 2010; Boligan Corvo, 2013), and new media (July, 2014),
highlighting the isolating nature of technology.

Alongside the growing concerns about technology and co-location, human-
computer interaction scholars have drawn attention to a gap in the literature
on co-located technology (Olsson et al., 2020). A similar gap exists in terms
of commercial technologies designed for in-person interactions, which are
rare, especially outside the productivity domain. For example, none of the ten
most downloaded iOS apps are explicitly designed for co-location (Inc, 2019).
Although the COVID-19 pandemic has increased our reliance on technology for
remote interaction, it has also fueled interest in activities for in-person
social interactions among people with whom we have close social ties. For
example, sales of board games increased during the pandemic (Kibbe, 2020;
Weisholtz, 2020).

We took these concerns, together with the literature gap, as an opportunity to
reimagine and reinvent ways in which technology can support, rather than
detract from, co-located interactions. In a recent study, Liu et al. suggest
that “people enjoy engaging in everyday activities with individuals with whom
they have strong social ties” (Liu et al., 2022). Along with other designers
and researchers (e.g., (Ferdous et al., 2017)), we aim to identify
opportunities in everyday life where AR can positively influence and provide
rich shared experiences.

In this work, we introduce “Project IRL111IRL is Internet slang for “In Real
Life”,” a suite of five mobile apps we created to support playful co-located
interactions among friends and family (see Fig. 1), adopting a research
through design approach (Zimmerman et al., 2007; Gaver, 2012) in our process.
To evaluate the apps, we used interviews and participant observations with 101
participants who played with them in groups of two or more. The five apps use
augmented reality (AR) to engage people in the following distinct playful
experiences:

  1. (1)

People pass around a phone that instructs them to make faces before time runs
out, as silly effects are applied.

  2. (2)

Dog owners direct their pets to catch virtual coins spread around the physical
space.

  3. (3)

Parents and their children engage in a story in which their feet are augmented
as sock puppets.

  4. (4)

People compete to catch as many virtual cows as they can, using their phones
to beam them up from a planet on a shared TV screen.

  5. (5)

People chase each other to cover their opponents with virtual ice before time
runs out.

The core contributions of this work are as follows. First, we developed a set
of novel co-located AR systems that effectively enabled people to have fun
together. Second, we generated a series of design insights for creating
playful co-located mobile AR experiences organized around four themes: (1)
device arrangement (e.g., whether the experience requires people to share a
single phone or use multiple phones), (2) the role of enablers (i.e., the
physical objects that trigger and are the focus of the AR experience), (3) the
affordances of augmentation (i.e., features of the technology that enhance its
potential to encourage various aspects of social interaction), and (4) the co-
located play experience (i.e., using technology to make in-person play
engaging and inviting). Third, we propose a set of guidelines for designing
playful co-located AR experiences (e.g., nudging physical touch, using
people’s bodies–and even their pets–as enablers, and building on familiar
games). We conclude with a call for more research on embodied social AR, a
generative approach that utilizes enablers to focus on bodies as a means of
purposefully supporting in-person social experiences. Our aim is for this work
to inspire researchers and technology designers to explore new design avenues
for encouraging people to playfully interact in person using mobile AR or
other technologies.

##  2\. Related Work

###  2.1. The need to design for co-location

> “Every time you check your phone in company, what you gain is a hit of
> stimulation, a neurochemical shot, and what you lose is what a friend,
> teacher, parent, lover, or co-worker just said, meant, felt.”
> —Sherry Turkle (Turkle, 2016)

Researchers have empirically demonstrated that the mere presence of mobile
devices can negatively impact communication during face-to-face conversations
(Przybylski and Weinstein, 2013; Misra et al., 2016). Similarly, ethnographic
studies have shown that reliance on technology is making people increasingly
socially isolated, causing them to experience the sensation of being “alone
together” when they are in the presence of others (Turkle, 2017). Researchers
argue that the human-computer interaction literature requires more research
focusing on how to design technology that will encourage co-located
interactions by “not only providing opportunities, but also utilizing
computational features that nudge and stimulate people to take action” (Olsson
et al., 2020). Olsson et al. (Olsson et al., 2020) have identified various
roles technology can take in human interactions, ranging from “enabling” to
“encouraging” social experiences. They argue that “future design endeavors
would benefit from more deliberate choices of specific phenomena, social
settings, target user groups, or type of interaction—particularly those that
aim to actively enhance the quality of social interaction” (Olsson et al.,
2020). Similarly, Isbister calls on designers to consider the space between
people. She argues that we should design interactions that require people to
share devices because they can support interdependent interactions (Isbister,
2019), i.e., interactions that require people to engage with each other in
order to interact with their devices.

Lundgren et al. (Lundgren et al., 2015) encourage designers to take four
perspectives when designing co-located mobile experiences: social,
technological, spatial, and temporal. Further, their research suggests that
co-located interaction is based on collaboration, communication, competition,
or a mix of the three. In our work, we embrace these calls to action by
creating a suite of mobile AR applications to encourage co-located
socialization. Our exploration considers different interaction types,
including competitive and collaborative experiences, and experiments with
various social relations, e.g., parents and children, and close friends.

###  2.2. The value of playful casual interactions

In recent years, researchers have drawn attention to the value of playful
interactions (Altarriba Bertran et al., 2020; Sharp and Thomas, 2019).
Therefore, we decided to design applications to enable these types of
experiences among people who are together in the same space. In our work, we
were interested in creating technologies that support casual social
interactions, without focusing on productivity (as much of the existing HCI
literature have already addressed this, e.g., (Lucero et al., 2010; Dickey-
Kurdziolek et al., 2010; Alavi and Dillenbourg, 2012; Kreitmayer et al.,
2013)). For this reason, we turned to games and playfulness as the driving
force for creating co-located mobile experiences.

In prior work on co-located games, researchers have argued that technology
designers should focus on two core elements: game mechanics and the social
affordances of the game’s interface (i.e., features that encourage various
types of social interaction) (De Kort and Ijsselsteijn, 2008). For example,
one approach to social affordance is to “shape the flow of interpersonal
distance (proxemics) in pro-social way” (Isbister et al., 2018). More broadly,
researchers suggest that game designers who want to enhance socio-emotional
player experiences should focus on creating physical-social play (Segura and
Isbister, 2015). Inspired by prior research on movement-based games (Mueller
and Isbister, 2014), we built on two proposed guidelines that relate to AR:
“construct the player’s actions in a way that gives room for sensor error
without drawing attention to it” and “avoid game mechanics that require
precise control.”

We find the qualities of mobile AR well suited to co-located playful
interaction due to its strong support for embodied interaction (Xu et al.,
2011; Dourish, 2004) and the fact that AR is grounded in the physical
environment (Wetzel et al., 2008). Therefore, we decided to focus on exploring
AR as a design material for such experiences.

###  2.3. Leveraging Mobile AR in co-location

We focused on building our system using AR because, unlike other technologies,
it relies heavily on the physical world, i.e., the reality shared by people
who are inhabiting the same place at the same time. We surveyed the literature
and identified the following five considerations outlining how and why AR
technologies are well suited to our task. Research suggests that AR is capable
of supporting playful co-located social experiences because it is:

  * •

Grounded. AR can cultivate ambiance by stimulating a variety of senses,
overlay content onto the real world environment (Wetzel et al., 2008),
transform the world into a “playground” (Altarriba Bertran et al., 2020), and
support exploration (Paasovaara et al., 2017).

  * •

Embodied. AR can physically and socially facilitate social activities, imbue
meaning through embodiment (Dourish, 2004), and support novel physical
interactions (Xu et al., 2011).

  * •

Playful. AR can facilitate playfulness through experience design and content
that is surprising, humorous, thrilling, or challenging (Altarriba Bertran et
al., 2020).

  * •

Social. AR can facilitate and enhance various types of social interactions and
relationships (Wetzel et al., 2008; Vella et al., 2019; Xu et al., 2008).

  * •

Memorable. AR interactions can be memorable (e.g., (Koskinen et al., 2019))
due to well-crafted experience design (Pine et al., 1999); it can also be
easily recorded and shared (Wetzel et al., 2008).

Although mobile AR is still an under-explored technology that has not been
widely adopted, some inspiring experimental designs support co-located
interaction to varying degrees. For example, researchers have explored co-
located mobile AR in relation to interfaces for group collaboration (Wells and
Houben, 2020), co-creation (Guo et al., 2019; Park, 2020), communication
(Raffle et al., 2007), and shared-world gameplay (Bhattacharyya et al., 2019).
In addition, there are several research projects examining mobile AR game
experiences designed for synchronous and co-located interaction, e.g.,
(Mueller et al., 2015; Zambetta et al., 2020). Beyond this research, there are
a few commercial mobile AR apps that can also be used in co-located
interactions (Within Unlimited, 2020; LeverX, 2020; Paavilainen et al., 2017).
These previous works helped us to explore a different aspect of the co-located
mobile AR design space. However, our ultimate goal was to explore how mobile
AR can be used as a design material to support playful co-located interaction.
With this in mind, we we designed five experiences that leverage AR’s grounded
and embodied qualities.

With this prior work as the backdrop for our research, we explored ways of
facilitating playful co-located experiences by experimenting with mobile AR’s
affordances. Our goal was to provide generative insight. To that end, we took
an exploratory and interpretative approach (Waern and Back, 2015) as part of
our research through design (Zimmerman et al., 2007; Gaver, 2012) process.

##  3\. Project IRL: A suite of five Mobile Augmented Reality apps

Table 1. AR considerations and operationalized design attributes. Mobile AR
Considerations
---
| Grounded | Embodied | Playful | Social | Memorable
Design Attributes |  Enabler |  Device Arrangement Physical Movement |  Augmentation |  Interaction Type |  Camera Recording

###  3.1. Design Process

During our research through design (Zimmerman et al., 2007; Gaver, 2012)
process, we asked ourselves two questions: “what makes this design work only
when people are together in person?” and “what would make this design
enjoyable or meaningful to use when interacting with others in person?” This
approach proved generative and kept our brainstorming sessions focused on co-
located social experiences. We drew inspiration from other researchers’
proposed guidelines for designing AR games that emphasized “experience first,
technology second” (Wetzel et al., 2008). Further, we built on interactions
that are already familiar to people by finding inspiration in existing games
(e.g., “Fetch”(Fec, 2020) or “Tag”(Wise and Forrest, 2003)) and social
activities (e.g., storytelling). We took the following steps (over a three-
month period):

  1. (1)

Formulating design attributes. We identified five design considerations for
mobile AR to support co-located interactions based on our literature review
(section 2.3). We operationalized these design considerations into concrete
techniques by mapping them to design attributes (Table 1). Each has a range of
possibilities; for example, an enabler could be a body part, a pet, or a
physical object like a screen; interaction types could be competitive or
collaborative, etc.

  2. (2)

Generating ideas to prototype. Based on these techniques, our research team
(which included designers, engineers, and researcher-focused team members)
brainstormed both individually and collectively in remote video sessions,
sharing our design ideas via Google slides. We generated a list of over twenty
app ideas with various combinations of the target design attributes.

  3. (3)

Prototyping selected concepts. We selected five concepts from the list of
ideas to develop as apps. When selecting, we tried to have the broadest
coverage of the matrix of combinations of the design attributes (step 1) while
balancing the practical constraint of available time. When we were prototyping
the concepts, we frequently met to discuss design iterations and internally
play-test new versions. The design attributes for each of the five IRL apps
are specified in Table 2.

We designed the augmentation in our apps to be the driving force of the
experience and tied it to enablers. Enablers are physical entities that
trigger–and are the focus of–the AR experience. Enablers are similar to
markers (Fiala, 2005; Hirzer, 2008) in triggering AR objects, but they differ
in that they can be any physical entity (beyond a 2D pattern), and they play
an integral role in the AR experience. We chose to experiment with distinct
enablers, such as pets, parts of people’s bodies (e.g., faces, feet, or entire
bodies), and the use of shared screens (e.g., TVs).

Table 2. Design attributes for each of the AR apps. Design Attributes
---
IRL App |  Device Arrangement |  Enabler |  Augmentation |  Interaction Type |  Physical Movement
Face It |  One phone |  Face |  Face filters (i.e., ”lenses”) |  Competition |  Gesturing with face; passing an object
Feeture Films |  One phone |  Feet |  Sock puppets |  Storytelling |  Gesturing with feet
Treasure Treat |  One phone |  Dog |  Dog’s silhouette and environment coins |  Cooperative Game |  Gesturing to the dog; dog moving around
Milky Way |  Two phones, one TV or laptop |  Video on TV |  3D planet |  Competition |  Swiveling around the screen
Freezing Frenzy |  Two phones |  Full body |  Ice on body |  Competition |  Chasing other players

###  3.2. App Descriptions

Here we describe each of the five apps222See video:
https://youtu.be/_51-IlxdBjg and project website: letsplayirl.com333You can
find access codes to download the apps using Snapchat on Android or iOS in the
supplemental materials, explain the user experience of each, and highlight (in
bold text) all of their associated design attributes (See Fig. 1 and Table 2
for an overview). The apps represent a variety of activities with several
different contexts of use.

####  3.2.1. Face It (FI)

Figure 2.  Players passing the phone in a play session with the FI app (left).
Snapshots of the FI app instructing players to make a facial gesture and pass
the phone (right). Note that we blurred the faces of study participants to
preserve their privacy. Images not blurred are people hired to pose for
videos.

We were inspired by the game mechanic of passing an object between people, an
interaction many people are familiar with from toys like ”Bop-It” (Hasbro,
1996) or games like ”Hot Potato” (Wise and Forrest, 2003). We were also
inspired by popular face lenses on Snapchat (by Snap Inc., 2020b) and filters
on Instagram (Tiffany, 2020), particularly, a co-located game that uses head
tilt as the enabler (Lim, 2020).

To play FI, multiple players pass one phone around to compete with each other.
FI is enabled by using the phone’s front-facing camera to track and augment
players’ facial gestures, and the app detects the facial expressions of one
person at a time, turning the face into the game’s controller (i.e., players’
faces enable the AR experience). Players see their faces augmented
continuously in humorous ways through face distortions. For example, their
face turns into an animated broccoli, among other face filters (see Fig. 2).

The app asks the players to begin the game by pressing a video recording
button that starts a 60-second timer. FI prompts players to perform different
facial gestures through text and audio instructions, e.g., “smile,” “kiss,”
and “look left.” Unexpectedly, the app shows and plays the prompt “pass it” to
ask players to pass the phone quickly to the next player, who then receives
their own set of prompts. We developed FI using Lens Studio face segmentation
and classification models for facial gestures. More specifically, we used Lens
Studio’s face triggers (by Snap Inc., 2020a) to determine if the players
completed their prompts before their time ran out.

The pace of the music and instructions increases over time, and the amount of
time players have to complete each gesture decreases, making it progressively
more challenging. When a player fails to execute their instructions, the game
round ends, and that player is out. The remaining players continue until only
one player is left, and this last player is declared the winner. At the end of
each round, players can view a short recorded video of the gameplay that they
can save to their phones or share with others.

####  3.2.2. Feeture Films (FF)

Several research projects explored augmenting traditional storytelling with
digital technology. For example, researchers developed a room-sized mixed
reality immersive storytelling experience for children to experiment with
(Alborzi et al., 2000). Others explored a storytelling system to foster
creativity and collaboration among children by mixing physical and digital
story elements (Cao et al., 2010). Researchers also explored a storytelling AR
system with finger puppets “to enhance social pretend play” (Bai et al.,
2015). There are also commercial AR storytelling apps for children (e.g.,
(Within Unlimited, 2020)).

All of these prior works, along with children’s flap books (Russo, 2020),
inspired us to create FF as an interactive AR storytelling experience for
children and parents to share. To use FF, a parent and a child share one
phone. The experience is enabled by their feet: the app detects players’ feet,
using them as the target of the augmentation by taking advantage of Snap
Machine Learning’s Foot Tracking, which allowed us to detect players’ right
and left feet and estimate their 3D position. (by Snap Inc., 2020d)) (see Fig.
3).

Figure 3.  An adult and child point a phone at their feet to augment them
using the FF app (left). Snapshots of the FF app showing players’ feet
augmented as sock puppets (right).

FF prompts the parent and child to sit next to each other, side by side
(typically on a couch or bed), and uses one foot from each of them. Their feet
are augmented as sock puppets that go on a jungle-themed adventure together to
find their friends. Foot gestures are used to interact and trigger the AR
story-objects; for example, players hover on overlaid objects to discover
other animals or tap their feet together to reveal their emotions (which are
displayed with emoticons).

FF has four scenes. The parent and child can navigate and experiment with
verbally adding their own stories or enact the story by moving their sock
puppets in the frame. In addition, they can capture parts of the experience by
taking photos or videos; they can also exit the app and return to where they
left off in the story at a later time (we utilized internal storage to store
the scene ID). FF is a unique form of digital storytelling as it encourages
experimentation with body gestures while supporting in-person interaction
between a parent and a child. We designed it to keep the traditional
storytelling interaction of reading stories aloud while injecting bodily play
and experimentation with digital augmentation.

####  3.2.3. Treasure Treat (TT)

Figure 4.  A pet owner and their dog using the TT app (left). Snapshots of the
TT app showing the dog collecting points by colliding with AR coins (right).

This app explores playful co-located interaction between a person and their
dog using mobile AR. Human-pet relationships are an essential part of the co-
located interaction design space. Further, due to COVID-19, people are
spending more time at home and adopting dogs at higher rates (Morgan et al.,
2020) than before the pandemic. AR technology has started to follow this pet
trend too. For example, GoDog (LeverX, 2020) is an AR dog training app, and
Lens Studio has added an API (by Snap Inc., 2020e) that lets developers create
AR apps that use dogs’ and cats’ faces as enablers. More broadly, we found
inspiration in existing games people play with their dogs (e.g., ”Fetch” (Fec,
2020)) as a way of including physical movement.

TT is a pirate-themed cooperative game in which the player and their dog team
up to collect AR coins that appear from a treasure box and then scatter on the
floor. TT is a level-based progression game, which is also inspired by classic
video games such as Super Mario (Nintendo, 1985). The dog’s body enables the
game when it is detected (using the phone’s rear camera). Then TT instructs
the person to tap on the screen to identify the floor’s plane. The dog’s
silhouette is also augmented, and it controls the game: collision detection
evaluates whether the coins and the dog’s body appear to touch. When the dog
is successfully detected, the coins are collected, and the dog-human team
earns points. We applied a dog machine learning model to the camera input to
detect the dog’s position in the camera frame (see Fig. 4) and to allow us to
attach a boundary animation to the dog’s body.

The human player is expected to motivate and guide their dog partner to move
in specific directions in order to collect all of the available coins within a
time limit. People might do this by, for example, gesturing to the dog to move
in whichever way they want, making sounds to attract the dog’s attention,
pointing in a specific direction (e.g., (Kaminski and Nitzschner, 2013)),
throwing toys, and using treats. Each level introduces a new challenge by
changing the number of coins or the time constraints. If the team successfully
finishes collecting all of the coins on time, they proceed to the next level.
If not, the game is over. If they complete all of the challenges, they win a
”pirate” AR filter for the dog.

####  3.2.4. Milky Way (MW)

Inspired by the recent concept of “Augmented Reality Television” (ARTV)
(Vatavu et al., 2020), we explored the use of a second screen (TV or laptop)
for a multiplayer game design that “employs one main screen shared by two
players, each one also using a second (private) screen” (Pagno et al., 2015)).

We drew from the familiar co-located interaction of gathering around a shared
focal point to play (e.g., playing video games with others). MW is a
competitive game and is an intergalactic AR twist on the game ”Whack-a-Mole”
(of TOGO, 1975). Two or more players are expected to gather around a TV or
laptop to scan a YouTube video using the rear cameras on their individual
phones to augment a second screen. The YouTube video enables the AR experience
and allows the game to sync automatically. The screen display serves as a
physical anchor for players to swivel around. To find the video on YouTube,
players need to search for a five-letter keyword.

The video shows a rotating ”black hole.” We used the angle of rotation to
synchronize all of the player’s devices without the need for a network or
Bluetooth connectivity between the devices. The rotation itself becomes a
unique marker that allows the system to determine when to start the game and
make new objects appear on the screen. This enabled players to begin the game
simultaneously without the need for network connectivity. Further, this
approach made tracking consistent between phones, regardless of the time or
angle at which the marker is scanned. Finally, we used 2D markers to track and
augment objects on a video.

Figure 5.  Players using the MW app pointing their phones at a TV (left).
Snapshots of the MW app showing the rotating marker and the AR planet where
cows must be rescued (right).

MW starts when a big 3D planet inhabited by space cows appears on players’
mobile screens, as if they are coming out of the video. The goal is to abduct
as many space cows from the 3D planet as possible (see Fig. 5). When MW
starts, it generates a seed from the start timestamp on the players’ phones.
This seed randomizes the space cows’ appearance in the game sessions (but
maintains consistency during the session) such that players see the space cows
appear at the same positions and times on the planet. Players can move around
the 3D planet, aim, and then tap the screen to abduct the space cows during
the 30 seconds of game play. Players see their scores and receive a prompt to
compare them to determine the winner.

####  3.2.5. Freezing Frenzy (FR)

Figure 6.  Players using the FR app and pointing their phones at each other
(left). Snapshots of the FR app showing the frozen AR effect over players’
bodies when they get hit (right).

FR is an AR battle competitive game inspired by the classic ”Laser Tag”
(Ferret, 2007) game. With this app, we focused on using players’ entire bodies
as enablers, both to trigger the game and to serve as the target of the
augmentation. We also draw ideas from physical AR battle interactions, such as
AR pong battle (Hu and Wang, 2018) and a multiplayer AR shooter game
(Hollister, 2017).

FR allows two players to use their phones in parallel to simultaneously target
each other’s entire bodies with their rear cameras. For a 30-second interval,
the phones are ”transformed” into freezing guns. Players chase each other and
tap on the screen to shoot and ”cover” their opponent’s body with AR ice. We
chose the ”freezing” theme because attaching ice texture to the body rendered
relatively well. If players make a successful ”hit,” chunks of AR ice appear
to cover their opponent’s body and they are awarded points (see Fig. 6).

FR prompts players to manually coordinate the start of the game by pressing a
button simultaneously, and it prompts them again at the end to compare their
scores. We developed this app using Lens Studio’s ”Full Body Attachments” (by
Snap Inc., 2020c) to detect bodies and distinguish between their body parts
(e.g., head, neck, and forearm). This feature relies on 2D body segmentation
models. We also used it to detect target collisions between a tap on the
screen and the player’s body to initiate AR ice shooting.

##  4\. Evaluation

The COVID-19 pandemic required us to adapt our plan to support remote
evaluation. Our study design consisted of one-hour video call sessions. Our
data collection included (i) observational notes on participants using the
apps, (ii) transcribed semi-structured interviews, and (iii) participants’
screen recordings. We based our semi-structured interview questions on
recommendations developed by Fullerton and colleagues (Fullerton, 2014), and
additional open questions about the overall experience of using the apps with
others. We ran four pilot sessions with participants, then implemented small
changes in our protocol before deploying the study. To study mobile AR apps
intended to support playful co-located interaction, we created participant
groups of two co-located people (with the exception of one group of three).

###  4.1. Study Protocol

We presented participants with a short slideshow introducing the study, the
apps, and the planned activities for the hour and asked them to confirm
consent to record. We then sent a link to a short demo video so that they
could watch others using the app. Finally, we gave them an access code to use
the app and asked them to start screen recording on their phones before using
it. We instructed participants to use the app as much as they wanted (while we
kept track of the time, stopping them only if they took over 20 mins). While
they used it, we turned off our video and audio on the call to minimize
interruptions during their play experience. When participants finished
playing, we asked for their feedback in a semi-structured interview (approx 20
mins). Finally, we asked them to share their screen recordings by uploading
them to a shared drive.

###  4.2. Participants

We recruited participants (all unique subjects) by posting on Slack channels
at a technology company, on Facebook groups, and on university mailing lists
(see Table 3). Participants were offered gift cards to compensate them for
their time444We asked for participants’ consent to use their faces and photos
in our research and in future publications..

Table 3. Groups of study participants per app. TT: ten dogs considered participants; FF: One adult and one child per group. App |  Groups |  Participants
---|---|---
Face It (FI) | 10 |  n=21; Age range 14-55
Feeture Films (FF) | 10 |  n=20 (10 children, 10 adults); Age ranges 4-7 & 30-45
Treasure Treat (TT) | 10 |  n=20 (10 dogs, 10 people); Age range 24-43
Milky Way (MW) | 10 |  n=20; Age range 19-61
Freezing Frenzy (FR) | 10 |  n=20; Age range 20-39

###  4.3. Analysis

We used a hybrid qualitative method of thematic analysis (Joffe, 2012),
including deductive and inductive approaches. To analyze our participant data,
we examined how participants used each app in relation to the five AR
considerations (that we identified from the literature in section 2.3) to help
us articulate why mobile AR is effective for co-located interaction. As
mentioned earlier, our data collection included interview transcripts and
participants’ observations.

Three researchers conducted a bottom-up thematic analysis in codebooks on the
interview transcripts to analyze the data. The three researchers developed a
codebook template to organize text for subsequent interpretation. The
codebooks had initial categories relating to the five AR considerations
(grounded, embodied, playful, social, and memorable). For each app, the
researchers discussed and adjusted the codes based on their specific design
attributes. Independently, the researchers performed open coding on the
transcriptions of the audio-recorded interviews. They then met to discuss and
iterate the codebook based on similarities in participants’ data while also
taking note of unique responses. Researchers also added their observational
notes to the codebook, using participants as the core unit of analysis.

##  5\. Results

Mentions of the design attributes (see Table 2) were often evident in
participant responses, and their outcomes were observed in their behaviors.
Below we synthesize the results under four main themes that relate to the
design attributes: (i) device arrangement: one device per group, one device
per player; (ii) the roles of enablers; (iii) affordances of augmentation;
(iv) co-located play. For each, we created subcategories to articulate how we
could use mobile AR to create co-located play experiences (see Table 4).

Table 4. Results: four main themes and subcategories. Device Arrangement |  The Roles of Enablers |  Affordances of Augmentation |  Co-located Play
---|---|---|---
• Shapes proxemics • Coordination (which nudges communication) |  • Social focal point • Moving together |  • Entertaining to watch • Experimenting together |  • Enhancing friendly competition • Room for more • Interactive screen time

###  5.1. Device Arrangement: One Device Per Group, One Device Per Player

####  5.1.1. Device arrangement shapes proxemics (27 participants).

The device arrangement relates to the shaping of proxemics social affordance,
defined as making “use of sensors to shape the flow of interpersonal distance
[…] in pro-social ways, which ”guides mutual attention through strategic use
of feedback to players” (Isbister et al., 2018). In our case, some of the apps
required that participants physically moved closer to each other in order to
play, so the device arrangement creates changes to social proxemics (Marquardt
and Greenberg, 2015). For example, Feature Films participants appreciated the
fact that playing the game required them to be in close proximity to others– a
participant noted that they felt it was special that the app directed them to
sit “side by side” with their child (FF-P9) (see Fig. 7). On top of that,
using a shared device made participants collaborate to play and sometimes
engage in social touch. For example, FF-P3 said “So he was sitting down, I was
doing the voices, and he was holding the camera on his own feet […], I added
my own character, I added my little finger guy in there and tickled him a
little [laughing]”.

In another case, participants also indicated that there were some challenges
associated with needing to be physically close: when playing Milky Way,
participants sometimes felt they had to compete for space (e.g., “the fact
that we were using the same laptop. We’re in each other’s way a little bit.
But, [this] also seems like part of the fun of it” (MW-P19).

Figure 7.  Left: FI participants were physically close to each other in order
to pass their shared phone. Right: FF participants sat close together in order
to share one phone.

####  5.1.2. Coordination nudges communication (20 participants).

When the arrangement of devices relies on multiple players using one phone,
this creates a situation in which players may need to coordinate their bodily
movements. Often, this requires them to find ways to communicate in order to
do so. For example, when we observed participants playing Face It they
communicated to coordinate transferring the phone between them. One noted that
sharing and passing around one phone “encourage[d] more face-like
interpersonal interaction” (FI-P21), and a Feature Films participant noted
that coordinating their movement was “more of a communication challenge”
(FF-P7). We also observed Feature Films participants communicating to adjust
their bodies to fit their feet in the frame and discussing who would hold the
phone; they also needed to communicate to plan the coordination of their foot
movements to control the app.

###  5.2. The Roles of Enablers

####  5.2.1. Enablers can become the social focal point (35 participants).

Figure 8.  Left: FF Screen shots show how participants focused their phone’s
cameras at each other. Right: FF players compare their scores.

An enabler is a physical entity that triggers and is the focus of the AR
experience. In each of the five apps, we explored a different enabler. In our
synthesis, we noticed that specific enablers were better at drawing
participant focus toward other players. In particular, enablers that used
other players’ bodies or body parts were well equipped to facilitate this
social focus.

For example, the game mechanics of Freezing Frenzy encouraged participants to
focus on each other (by using their own phone to augment the other player’s
body, see Fig. 8), e.g., one participant observed “I was just so focused on
freezing him” (FR-P6). Freezing Frenzy participant comments indicated that
they liked this aspect of the app, e.g., “I like that I can actually see my
brother on the screen” (FR-P7). In Treasure Treat, players direct the camera
on their phone to track their dog’s movement. One commented on how they felt
synced: “he looked at me and the […] speed with which he let me have the toy
back–it just really gave me the feeling that he knew that we were working on
something […]. It felt like he knew that we were synchronized on getting
something done” (TT-P2). Another participant observed: “she saw my excitement
and I think she was excited by my excitement” (TT-P5). Participants also
indicated that playing Face It was enjoyable because it included watching
others play, e.g., one noted that they would “rather play it with other people
because it’s kind of fun to watch them do it, too” (FI-P11).

####  5.2.2. Enablers can encourage moving together (40 participants).

Enablers are at the heart of the AR experience and afford a wide range of
bodily play. Depending on their design, they can support players moving
together.

For example, having participants interact by tapping their feet in Feature
Films encouraged social touch and movement. Participants appreciated this
aspect of the app, e.g., one noted that “the cool thing about it was being
able to know when you’re tapping [your] feet together to get those reactions”
(FF-P1). One participant also commented on the associated intimacy of such
interaction, e.g., “putting your feet together [is something you do] with
someone you’re really close to” (FF-P16). In Freezing Frenzy, detecting
another player’s body enabled the augmentation. Participants realized that
they “should move [their] body so that it can be more exciting” (FR-P4).
Participants noted that this got them “moving and interacting with [their]
friends” (FR-P13), they appreciated “that it got [them] up out of [their]
chairs” (FR-P9), and noticed they were “actually moving around for 30 seconds”
(FR-P9). Similarly, in Treasure Treat, a dog’s silhouette enabled the AR
experience while its collision with AR coins was at the center of the game.
Participants appreciated that it encouraged them to move with their dog in
order to play, e.g., “I like that it keeps us both active […], I wasn’t just
sitting down […] I really liked [that] we were both active during the game”
(TT-P7) (see Fig. 10).

###  5.3. Affordances of Augmentation

####  5.3.1. Entertaining to watch (39 participants).

Augmentations can be entertaining. When they are connected to other players’
bodies, they had funny or surprising qualities. This made the experience
enjoyable to watch. For example, the continuously changing face augmentation
in Face It surprised and delighted participants, e.g., “I like that it’s very
random […] when the filters change […] it’s kind of like a fun element to it.
I really enjoyed that” (FI-P1). The design of the augmentations themselves
also contributed to that excitement, e.g., “At first, I was like, ’Oh, these
filters are kind of [laughing] kind of out there.’ But I think that also makes
it more exciting […] you know, it’s just, it’s wacky, it’s funny” (FI-P20).
Participants mentioned this made the experience “fun” and “funny”, e.g., one
participant observed that “adding that [AR] filter definitely makes it more
uh, interesting and fun to play” (FI-P20), while another noted that when they
were using Face It, it was “pretty entertaining and fun to watch other people
make faces” (FI-P16) (see Fig. 9). Treasure Treat participants enjoyed
watching their dogs collect the AR coins, e.g., “I was excited while
[laughing] I was using like, this is super cool. Uh, I was smiling the whole
time watching her win through the rounds” (TT-P7), and “I would record a video
of her, like her tail wagging and catching a coin […] I think that’s adorable”
(TT-P5).

Figure 9.  FI participants were engaged and entertained even when others held
the phone. Figure 10.  TT participants lured their dogs to play and move
around by using treats, toys, and commands.

####  5.3.2. Supporting experimentation together (31 participants).

Augmentations can be surprising, and they can contribute to the enjoyment when
they’re designed to be discovered as the experience unfolds (rather than all
at once). In addition, we found that when participants experiment together to
discover augmentations it adds to the fun. For example, augmenting feet into
sock puppet animal characters was part of the fun discovery of Feature Films
(e.g., “her favorite thing was just being able to turn her feet into animals”
(FF-P20)). We also observed participants playfully experimenting with the AR
tracking, e.g., one pair of participants used their thumbs instead of their
feet. Participants also explored adding finger characters to the mix (e.g., “I
added my little finger guy in there and tickled him a little” (FF-P3)) (see
Fig. 11).

However, participants indicated that “the best part” (FF-P1) was “being able
to engage with it together” (FF-P1). Many Feature Films participants commented
they wanted to be continuously surprised and able to experiment with new
content. They suggested adding a variety of changing stories, backgrounds,
settings, and characters. They found it desirable and thought it could
increase the replay-ability of the app, e.g., one participant stated, “if
there was a different story, like every week or every day, I would go back and
use [it], as long as there’s a different story” (FF-P1). Participants also
experimented while using Treasure Treat; some tried different approaches to
having their dog’s body collect the AR coins. For example, instead of
encouraging the dog to move to collect AR coins, they moved around themselves.
That made the dog’s body collide with the AR coins to collect them, e.g., “I
could basically hack the system and like cover Rocky with a coin, and then we
would get the points” (TT-P6). In Freezing Frenzy, some participants
experimented by trying to augment additional body parts rather than simply
using their feet, e.g., “not being able to, like, put ice on her face. It
always went to her head, and I kept trying” (FR-P11).

Figure 11.  Parents and children used FF in unexpected ways, e.g., tilting
their phones and playing with their hands in the frame.

###  5.4. Co-Located Play

####  5.4.1. Enhancing friendly competition (41 participants).

Participants who played our competitive apps (Face It, Freezing Frenzy, and
Milky Way) mentioned that they experienced the competition as friendly and
that engaging in these digital experiences in person rather than remotely
enhanced them. For example, interacting with the apps “actually like live”
made them “more interactive” (FI-P2) and engaging. Face It fostered a fun
sense of thrill through competition, e.g., “the speeding up of the music as
time went on, like it got more and more urgent. I thought that was really fun”
(FI-P3). Being able “to see the person […], look at them, do the competition
right [t]here in the same space” (FI-P10) enhanced the competition. In Face
It, the core game mechanic of “passing the phone with each other,” (FI-P14)
was identified as “more of a physical thing that you do” (FI-P14) and
“remotely [it] wouldn’t be the same since it’s such a fast-paced game”
(FI-P9). “[P]laying a game with someone right next to you” (FI-P7) was
something participants appreciated and considered “unique” (FI-P3). When
playing Freezing Frenzy and Milky Way comparing scores at the end of each game
round provided an opportunity for players to connect. Otherwise, they would
not have known what the other person was up to, e.g., “we would go back over
and be like, ”what was your score?”” (FR-P12).

Figure 12.  All five apps engaged participants actively with others.

####  5.4.2. Making room for more (55 participants).

Mobile AR apps can be designed for multiple players, and our participants
appreciated the ability to play with multiple co-located people and to allow
more people to join them. Face It participants also mentioned the game is
“more of a party game” (FI-P15), which would be “more fun with multiple
people” (FI-P15). One participant discussed the way sharing one phone with the
mechanic of passing it around was what made it more engaging “because you can
play with multiple people” (FI-P1). Many Freezing Frenzy participants also
imagined how the use of individual phones could support a larger group of
players, e.g., “it’d be really interesting to just have a bigger group of
people” (FR-P9). Participants who played Milky Way thought the second large
screen would allow more players to be added, e.g., “I feel like the TV is for
like, multiple people in the same room” (MW-P8). MW-P7 discussed the advantage
of using the app with “bigger screens and movie theaters”, envisioning the
potential for “hundreds of people” to play it simultaneously. Interestingly,
even small screens were able to achieve this party effect with the Face It
app.

####  5.4.3. Making screen time interactive (43 participants).

All of the apps engaged participants actively with others on some level (see
Fig. 12). For example, participants felt that Face It could work as a kind of
ice-breaker (e.g., “it might be fun, like in the car or something. You know,
[on a] long road trip. You’re just trying to bond with your fellow travelers”
(FI-P20)). For Treasure Treat participants, the app motivated them to engage
with their dogs. One participant said, “I have a reason to try and get my dog
to do something. And it’s an external goal that we’re trying to achieve
together” (TT-P2); this participant mentioned that they thought their dog
“felt like there was more play happening than, than just me tossing a toy away
and not caring what the outcome was. It felt like we were playing” (TT-P2).
Participants discussed how the apps made “screen time” social and interactive.
For example, one participant shared this comment about Feature Films: “it’s
not like watching a TV show where it’s totally passive; It’s something you do
together. It’s like a good five-minute activity for you and your kid” (FF-P3).
Another participant discussed the app’s benefits in terms of supporting them
to be playful with their child: “it’s like a little way to get some of the
sillies out […] knowing that it’s like a share-together activity [and] to not
feel guilty over that screen time” (FF-P20). Participants also discussed the
way having the apps ready on their phones could make it easier for them to
initiate the interactive experience with others. For example, a Feature Films
participant mentioned that using the app on their phone made it more readily
available for them to use with their child than reading a storybook, observing
“there’s an ease to it where you don’t need a lot. You don’t have to go pick a
book off your bookshelf” (FF-P5).

##  6\. Discussion

Earlier, we argued that mobile AR is well suited to supporting co-located
interactions because it is grounded, embodied, playful, social, and memorable.
We then identified a set of design attributes that guided our development of
the apps: (1) device arrangement, (2) enablers, (3) augmentation, (4)
interaction type, and (5) movement.

###  6.1. Key Takeaways

Based on learning from our synthesis of existing articles in the literature
review (Section 2.3) and our user study (Section 5), we present a set of
guidelines that designers can use in making choices related to these design
attributes, which are focused on the four main categories (see Table 5). We
developed these guidelines after completing the user study, and they include
our design-focused reflections on the process. The underlying questions
guiding their development were as follows: how does this experience truly
leverage co-location, and would this be nearly impossible to replicate at a
distance?

1\. Device Arrangement. How are mobile phones distributed among participants?
For example, does each person hold their phone? Do they pass one phone around?
Or something else? In making this decision, designers should consider at least
two approaches to shaping the experience:

  * •

Encouraging touch through proxemics. Create a digital experience that gets
people close to one another so they can touch, for example, by playfully
bumping and pushing each other. Physical human contact makes co-location
valuable because it is hard to replicate in remote interactions (related
sections: 2.3–Emobdied; Social, 5.1.1).

  * •

Nudging person-to-person communication via coordination. Create playful
scenarios that require people to share information. For example, introduce
collaborative tasks that nudge people to talk to one another, especially using
their voices, in order to complete them (related sections: 2.3–Emobdied;
Playful; Social, 5.1.2).

2\. The Roles of Enablers. How are visual triggers used to foster social
interaction? AR systems often relay on visual markers to function. Designers
should explicitly consider the potential for markers to play a social role.

  * •

Use bodies as enablers. Steer people’s attention towards one another by using
their faces, feet, arms, and full bodies as enablers. This could help to make
people, not technology, the focus of social interaction. In addition, consider
using body tracking to nudge people to move together (related sections:
2.3–Grounded; Emobdied; Social, 5.2.1).

  * •

Lean on physically meaningful enablers. Rather than using meaningless codes as
enablers, embrace the value of physical enablers. For example, using pets as
enablers leverages our physical world because, despite advances in
telepresence, pets typically respond best to physical touch, smells, and other
analog signals. Similarly, using enablers from people’s built environment
(e.g., objects in a living room) helps ground the social interaction in their
shared space and can also support movement (related sections: 2.3–Grounded;
Emobdied, 5.2.2).

3\. Affordances of Augmentation. AR experiences involve visually altering the
user’s view of the physical world. Designers can use these alterations to
support co-location in at least two ways.

  * •

Entertaining both the player and others. Utilize the visual effects of the
augmentations as comedic devices for disarming people with laughter, for
example, by turning the user’s face into a potato. In addition, changing the
visual effects throughout a game session can help keep the experiences novel
and engaging (related sections: 2.3–Playful; Social, 5.3.1).

  * •

Fostering opportunities to experiment together. Use augmentations that rely on
people moving around and pointing their cameras at different parts of the
physical space to allow them to explore and experiment with others in their
environment (related sections: 2.3–Grounded; Playful; Social, 5.3.2).

4\. Co-located Play. Although game design is a field on its own, we identified
three ways designers might want to optimize technologies for co-location.

  * •

Leverage play “in real-time”. Whether the playful activity is competitive or
cooperative, finding the right balance between fun and challenge also involves
attention to physical dexterity. Further, designers should consider making the
level of challenge relate to people’s physical skills, as physical skills are
more aligned with co-location than non-physical cognitive skills (related
sections: 2.3–Embodied; Playful; Social, 5.4.1).

  * •

Enable any number of players. Allow people to try out the experience on their
own first, but show them how adding more players would make the experience
more fun. Design interactions so they can be easily set up to make room for
the active involvement of multiple players (related sections: 2.3–Grounded;
Playful; Social, 5.4.2).

  * •

Build on familiar play. Draw inspiration from existing game experiences people
might already be familiar with in the analog world, e.g., board-games and
toys. This could help people get started quickly while still exposing them to
novel game techniques (related sections: 2.3–Grounded; Embodied; Playful,
5.4.3).

###  6.2. Embodied Social AR

When we designed the IRL apps, we prioritized taking into consideration the
context where we envisioned people would use the app. The interaction type
design attribute guided us to explore a variety of use contexts carrying a
range of social dynamics– from collaborative storytelling or working together
with one’s pet– to competitive play experiences among friends.As a result, it
also affected the embodied social experience. For example, when designing
Feeture Films we thought about where and how parent-child interactions occur.
This user context guided us to create a collaborative experience while
sticking one’s feet out because that’s what parents and children often do
(while laying on a bed or a couch together).

However, the enabler design attribute became particularly generative and
important during the brainstorming stage of the design process. Enablers are
not just physical entities that trigger the AR experience–they also play an
integral role as the central augmentation from which the entire experience
unfolds. Brainstorming around the enablers allowed us to ground the
experiences in the physical reality of players. The process of designing and
studying the apps with participants led us to notice that there was value in
focusing on enablers that augmented bodies (e.g., faces, feet, dogs, and the
full human body).

The core idea of embodied social AR suggests that social AR experiences can
benefit from focusing on enablers that create embodied interaction, in
particular, grounding the enablers in other social beings to trigger and drive
interactive experiences and, in the process, explore the potential of the
whole body and its affordances for play and movement. Therefore, embodied
social AR can encourage experiences that include movement and bodily play, and
focus players’ attention on one another. This enables people to (1) trigger
the experience by detecting a body, (2) the body detected is then being
augmented, and (3) the results of the previous points enhance the co-located
social experience of the players. In other words, tracking bodies is the input
that initiates the interaction (i.e., button press). Augmenting those bodies
is the central output (i.e., the result), while the outcome impacts the
players’ social experience. We believe embodied social AR will only grow in
popularity, and like any other type of design material, it should be used for
game design.

Table 5. Design recommendations: Key takeaways. | Design Recommendations
---|---
Device Arrangement |  Encourage touch through proxemics |  Needing to coordinate could nudge communication
The Roles of Enablers |  Center around players’ bodies |  Lean on physically-meaningful enablers
Affordances of Augmentation |  Entertain both the player and others |  Foster opportunities to experiment together
Co-located Play |  Leverage play “in real time” |  Easily to set-up, let multiple players join in

###  6.3. Future Work

The five IRL apps represent our initial exploration, and as design instances,
they only reflect aspects of the potential design space for playful co-located
mobile AR. There are many other ways to explore it:

  * •

Different combinations of design attributes. Experiments combining the same
design attributes we explored in different ways would yield other designs that
could be fun to play.

  * •

Changing details of the design attribute. Researchers could use the same
design attributes we explored but change their specifics. For example, using
furniture as an enabler in place of bodies for a shooter game and studying its
effect on the co-located experience.

  * •

Experimenting with other design attributes. Experiment with design attributes
other than those we explored. For example, familiarity with previous related
experiences; augmentation based on the front vs. the rear camera view;
dimensions of the co-located space; indoor vs. outdoor; and more.

  * •

Exploring embodied social AR. Study the effects of enablers that focus
specifically on augmenting bodies to unpack the value of embodied social AR.

  * •

Focusing on specific relationships. Explore specific types of relationships,
such as inter-generational, peers, parents, grandparents, co-workers, etc.

  * •

Studying interaction with pets. It would be interesting to study co-located
interaction with pets (other than dogs) and interactions between multiple
people and one pet.

  * •

Considering the effects of networked backend or the lack of it. Compare and
study the impact of having an available networked backend vs. not having one
would be valuable (to determine whether players can have shared access to the
same game elements on their devices or not).

  * •

Determining the potential for re-playability. This includes studying how we
could create playful co-located mobile AR interactions that are just as
enjoyable when played over and over again (i.e., finding ways to infuse them
with the qualities that makes board games enjoyable to play and re-play).

  * •

Exploring the effects of memorability. Mobile AR technology could be a great
design material for creating memorable shared experiences. Our study
represents a thin slice in time, so we could not test the impacts of the five
apps in terms of the memorability of the co-located experiences they
facilitate. A more longitudinal approach is required to assess the
memorability of playful co-located AR experiences.

  * •

Developing a richer taxonomy for mobile AR. In our exploration, we faced the
need to discuss the nuances of what triggers AR experiences and their effects.
Therefore, we created our taxonomy using the term enabler to describe the
physical entities that trigger and become the focus of AR experiences. In
analyzing our results, we became aware, once again, of the lack of appropriate
terminology for discussing different types of augmentations (e.g., face
morphing, customizing, filters, object additions, overlays, etc.). Prior work
has begun to discuss some of these nuances (Hugues et al., 2011; Normand et
al., 2012). However, more work is needed to develop richer terminology for
interaction design with mobile AR (while also taking into account co-location)
to allow for more fruitful discussion of its affordances.

##  7\. Limitations

If not for the restrictions imposed by the COVID-19 pandemic, we would have
evaluated the playful co-located mobile AR apps with participants in person.
This study would have been easier to run onsite rather than remotely, and it
may have resulted in additional findings about the players’ social
experiences. However, conducting the study remotely did not affect the quality
of our results. Our participants still engaged with our apps while co-located
with others, and we extracted insightful design takeaways.

In terms of the IRL apps generated in the study, we explored only a subset of
design possibilities and focused solely on mobile AR. Therefore, it is likely
that our findings do not represent a comprehensive analysis of the potential
for co-located interaction. For example, we did not explore how devices other
than mobile phones (e.g., smart speakers) could enhance or encourage co-
located playful interaction, nor did we consider gender and age differences in
our analysis. Moreover, we experienced some usability issues at times.
However, we did not detail them as we felt that they did not add any relevant
insights to our discussion.

##  8\. Conclusion

In this paper, we presented project IRL: a suite of five apps designed for
playful co-located interaction using mobile AR. We took a research through
design approach (Zimmerman et al., 2007; Gaver, 2012) to learning and
exploring how mobile AR can enhance players’ co-located social experiences. We
designed and deployed the apps with 101 participants. From our results, we
identified design insights that we organized under four themes: (i) device
arrangement, (ii) enablers, (iii) augmentations, and (iv) co-located play. For
each theme, we developed design recommendations as the key takeaways (Table
5). We learned that opportunities exist to enhance the co-located experience
with playful mobile AR by engaging players in interactions that encourage them
to move together or by creating a social focal point. The device arrangement
(e.g., shared phone vs. use of multiple phones) can further impact the social
experience by shaping proxemics or encouraging communication for coordination
between players. Augmentations could support playful co-located interaction by
engaging players through funny and curious situations. Mobile AR can serve as
an effective additional design material that creators of co-located play
experiences can use to facilitate friendly competitions, to engage multiple
players, and to make ‘screen time’ more social, entertaining, and interactive.

When we connected AR to bodies as enablers, our apps supported co-located play
by encouraging participants to move together and focus on each other more. We
hope to inspire future work that focuses on designing playful co-located
mobile AR. We suggest following the embodied social AR approach to generate
engaging designs for co-located play experiences and to explore different
enablers (however, such research should by no means be limited to bodies
only). We also believe transforming screen time into a shared and active
experience is one way to recover social engagement in home environments. We
hope designers and researchers continue to challenge existing technologies by
designing interactions that help people re-connect.

## References

  * (1)
  * Fec (2020) 2020\.  Fetch (game).  Retrieved September 17, 2020 from https://en.wikipedia.org/wiki/Fetch_(game)
  * Alavi and Dillenbourg (2012) Hamed S Alavi and Pierre Dillenbourg. 2012.  An Ambient Awareness Tool for Supporting Supervised Collaborative Problem Solving.  _IEEE Transactions on Learning Technologies_ 5, 3 (2012), 264–274.
  * Alborzi et al. (2000) Houman Alborzi, Allison Druin, Jaime Montemayor, Michele Platner, Jessica Porteous, Lisa Sherman, Angela Boltman, Gustav Taxén, Jack Best, Joe Hammer, et al. 2000\.  Designing StoryRooms: interactive storytelling spaces for children. In _Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques_. 95–104.
  * Altarriba Bertran et al. (2020) Ferran Altarriba Bertran, Elena Márquez Segura, and Katherine Isbister. 2020.  Technology for Situated and Emergent Play: A Bridging Concept and Design Agenda. In _Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems_ (Honolulu, HI, USA) _(CHI ’20)_. Association for Computing Machinery, New York, NY, USA, 1–14.  https://doi.org/10.1145/3313831.3376859
  * Bai et al. (2015) Zhen Bai, Alan F. Blackwell, and George Coulouris. 2015\.  Exploring Expressive Augmented Reality: The FingAR Puppet System for Social Pretend Play. In _Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems_ (Seoul, Republic of Korea) _(CHI ’15)_. Association for Computing Machinery, New York, NY, USA, 1035–1044.  https://doi.org/10.1145/2702123.2702250
  * Bhattacharyya et al. (2019) Po Bhattacharyya, Radha Nath, Yein Jo, Ketki Jadhav, and Jessica Hammer. 2019.  Brick: Toward A Model for Designing Synchronous Colocated Augmented Reality Games. In _Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems_ (Glasgow, Scotland Uk) _(CHI ’19)_. Association for Computing Machinery, New York, NY, USA, 1–9.  https://doi.org/10.1145/3290605.3300553
  * Boligan Corvo (2013) Angel Boligan Corvo. 2013\.  _Winner in Footbal Section_.  irancartoon.  http://www.irancartoon.ir/gallery/album765/Winner_in_Footbal_Section_Angel_Boligan_Mexico
  * Bowles (2019) Nellie Bowles. 2019\.  Human contact is now a luxury good.  _The New York Times_ 23 (2019).  https://www.nytimes.com/2019/03/23/sunday-review/human-contact-luxury-screens.html
  * by Snap Inc. (2020a) Lens Studio by Snap Inc. 2020a.  Behavior.  Retrieved September 7, 2020 from https://lensstudio.snapchat.com/guides/scripting/helper-scripts/behavior/
  * by Snap Inc. (2020b) Lens Studio by Snap Inc. 2020b.  Face Filters.  Retrieved September 7, 2020 from https://lensstudio.snapchat.com/guides/face/face-effects-overview/
  * by Snap Inc. (2020c) Lens Studio by Snap Inc. 2020c.  Full Body Attachments.  Retrieved September 7, 2020 from https://lensstudio.snapchat.com/templates/object/full-body-attachments/
  * by Snap Inc. (2020d) Lens Studio by Snap Inc. 2020d.  ML Templates Library.  Retrieved September 7, 2020 from https://lensstudio.snapchat.com/templates/ml/ml-templates-library/
  * by Snap Inc. (2020e) Lens Studio by Snap Inc. 2020e.  Pet.  Retrieved September 7, 2020 from https://lensstudio.snapchat.com/templates/object/pet/
  * Cao et al. (2010) Xiang Cao, Siân E Lindley, John Helmes, and Abigail Sellen. 2010.  Telling the whole story: anticipation, inspiration and reputation in a field deployment of TellTable. In _Proceedings of the 2010 ACM conference on Computer supported cooperative work_. 251–260.
  * De Kort and Ijsselsteijn (2008) Yvonne A. W. De Kort and Wijnand A. Ijsselsteijn. 2008.  People, Places, and Play: Player Experience in a Socio-Spatial Context.  _Comput. Entertain._ 6, 2, Article 18 (July 2008), 11 pages.  https://doi.org/10.1145/1371216.1371221
  * Dickey-Kurdziolek et al. (2010) Margaret Dickey-Kurdziolek, Matthew Schaefer, Deborah Tatar, and Ian P Renga. 2010.  Lessons from ThoughtSwap-ing: Increasing Participants’ Coordinative Agency in Facilitated Discussions. In _Proceedings of the 2010 ACM conference on Computer supported cooperative work_. 81–90.
  * Dourish (2004) Paul Dourish. 2004\.  _Where the action is: the foundations of embodied interaction_.  MIT press.
  * Ferdous et al. (2017) Hasan Shahid Ferdous, Frank Vetere, Hilary Davis, Bernd Ploderer, Kenton O’Hara, Rob Comber, and Geremy Farr-Wharton. 2017\.  Celebratory Technology to Orchestrate the Sharing of Devices and Stories during Family Mealtimes. In _Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems_ (Denver, Colorado, USA) _(CHI ’17)_. Association for Computing Machinery, New York, NY, USA, 6960–6972.  https://doi.org/10.1145/3025453.3025492
  * Ferret (2007) Tag Ferret. 2007\.  Laser Tag History.  https://archive.is/20121220040534/http://home.comcast.net/~Ferret1963/All_Systems.HTML
  * Fiala (2005) Mark Fiala. 2005\.  ARTag, a fiducial marker system using digital techniques. In _2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05)_ , Vol. 2. IEEE, 590–596.
  * Fullerton (2014) Tracy Fullerton. 2014\.  _Game design workshop: a playcentric approach to creating innovative games_.  CRC press.
  * Gaver (2012) William Gaver. 2012\.  What Should We Expect from Research through Design?. In _Proceedings of the SIGCHI Conference on Human Factors in Computing Systems_ (Austin, Texas, USA) _(CHI ’12)_. Association for Computing Machinery, New York, NY, USA, 937–946.  https://doi.org/10.1145/2207676.2208538
  * Guo et al. (2019) Anhong Guo, Ilter Canberk, Hannah Murphy, Andrés Monroy-Hernández, and Rajan Vaish. 2019\.  Blocks: Collaborative and Persistent Augmented Reality Experiences.  _Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies_ 3, 3 (2019), 1–24.
  * Hasbro (1996) Hasbro. 1996.  Bop It.  Retrieved September 17, 2020 from https://en.wikipedia.org/wiki/Bop_It
  * Hirzer (2008) Martin Hirzer. 2008\.  Marker detection for augmented reality applications. In _Seminar/Project Image Analysis Graz_ , Vol. 25.
  * Hollister (2017) Sean Hollister. 2017\.  Can these augmented reality blasters bring back laser tag?  https://www.cnet.com/reviews/skyrocket-recoil-preview/
  * Hu and Wang (2018) Si ying Diana Hu and Niniane Wang. 2018.  Multiplayer Augmented Reality: The Future is Social, Presented by Niantic. In _ACM SIGGRAPH 2018 Virtual, Augmented, and Mixed Reality_ (Vancouver, British Columbia, Canada) _(SIGGRAPH ’18)_. Association for Computing Machinery, New York, NY, USA, Article 21, 1 pages.  https://doi.org/10.1145/3226552.3226585
  * Hugues et al. (2011) Olivier Hugues, Philippe Fuchs, and Olivier Nannipieri. 2011\.  New augmented reality taxonomy: Technologies and features of augmented environment.  In _Handbook of augmented reality_. Springer, 47–63.
  * Inc (2019) Apple Inc. 2019\.  The Year’s Top Apps : App Store Story.  https://apps.apple.com/us/story/id1484100916
  * Inc (2020) Snap Inc. 2020.  The fastest way to share a moment!  http://www.snapchat.com/
  * Isbister (2019) Katherine Isbister. 2019\.  Toward ‘Suprahuman’ Technology. In _Proceedings of the Halfway to the Future Symposium 2019_ (Nottingham, United Kingdom) _(HTTF 2019)_. Association for Computing Machinery, New York, NY, USA, Article 24, 4 pages.  https://doi.org/10.1145/3363384.3363468
  * Isbister et al. (2018) Katherine Isbister, Elena Márquez Segura, and Edward F. Melcer. 2018\.  Social Affordances at Play: Game Design Toward Socio-Technical Innovation. In _Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems_ (Montreal QC, Canada) _(CHI ’18)_. Association for Computing Machinery, New York, NY, USA, 1–10.  https://doi.org/10.1145/3173574.3173946
  * Joffe (2012) Helene Joffe. 2012\.  Thematic analysis.  _Qualitative research methods in mental health and psychotherapy_ 1 (2012).
  * July (2014) Miranda July. 2014\.  Sombody: A Messaging Service By Miranda July.  Retrieved September 30, 2020 from http://somebodyapp.com/
  * Kaminski and Nitzschner (2013) Juliane Kaminski and Marie Nitzschner. 2013.  Do dogs get the point? A review of dog–human communication ability.  _Learning and Motivation_ 44, 4 (2013), 294–302.
  * Kibbe (2020) Kayla Kibbe. 2020\.  Board Game Sales Spike as People Turn to Monopoly During Pandemic.  Retrieved February 2, 2021 from https://www.insidehook.com/daily_brief/games/pandemic-board-game-sales
  * Koskinen et al. (2019) Elina Koskinen, Dale Leorke, Kati Alha, and Janne Paavilainen. 2019.  Player Experiences in Location-Based Games: Memorable Moments with Pokémon GO.  In _Augmented Reality Games I_. Springer, 95–116.
  * Kreitmayer et al. (2013) Stefan Kreitmayer, Yvonne Rogers, Robin Laney, and Stephen Peake. 2013.  UniPad: orchestrating collaborative activities through shared tablets and an integrated wall display. In _Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing_. 801–810.
  * LeverX (2020) LeverX. 2020.  A friendly way to make your dog smarter.  https://godog.me/
  * Lim (2020) Varick Lim. 2020\.  How I created the ”Who Is More?” viral Instagram filter.  https://medium.com/swlh/how-i-created-the-who-is-more-viral-instagram-filter-567f4c51beac
  * Liu et al. (2022) Szu-Yu (Cyn) Liu, Brian A. Smith, Rajan Vaish, and Andrés Monroy-Hernández. 2022.  Understanding the Role of Context in Creating Enjoyable Co-Located Interactions. In _Companion Publication of the 2021 Conference on Computer Supported Cooperative Work and Social Computing_ (Virtual Event, USA) _(CSCW ’22)_. Association for Computing Machinery, New York, NY, USA.  https://doi.org/10.1145/1122445.1122456
  * Lucero et al. (2010) Andrés Lucero, Jaakko Keränen, and Hannu Korhonen. 2010\.  Collaborative Use of Mobile Phones for Brainstorming. In _Proceedings of the 12th international conference on Human computer interaction with mobile devices and services_. 337–340.
  * Lundgren et al. (2015) Sus Lundgren, Joel E. Fischer, Stuart Reeves, and Olof Torgersson. 2015.  Designing Mobile Experiences for Collocated Interaction. In _Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work ‘I &’ Social Computing_ (Vancouver, BC, Canada) _(CSCW ’15)_. Association for Computing Machinery, New York, NY, USA, 496–507.  https://doi.org/10.1145/2675133.2675171
  * Marquardt and Greenberg (2015) Nicolai Marquardt and Saul Greenberg. 2015.  Proxemic interactions: From theory to practice.  _Synthesis Lectures on Human-Centered Informatics_ 8, 1 (2015), 1–199.
  * Misra et al. (2016) Shalini Misra, Lulu Cheng, Jamie Genevie, and Miao Yuan. 2016\.  The iPhone effect: The quality of in-person social interactions in the presence of mobile devices.  _Environment and Behavior_ 48, 2 (2016), 275–298.
  * Morgan et al. (2020) Liat Morgan, Alexandra Protopopova, Rune Isak Dupont Birkler, Beata Itin-Shwartz, Gila Abells Sutton, Alexandra Gamliel, Boris Yakobson, and Tal Raz. 2020.  Human–dog relationships during the COVID-19 pandemic: booming dog adoption during social isolation.  _Humanities and Social Sciences Communications_ 7, 1 (2020), 1–11.
  * Mueller and Isbister (2014) Florian Mueller and Katherine Isbister. 2014.  Movement-Based Game Guidelines. In _Proceedings of the SIGCHI Conference on Human Factors in Computing Systems_ (Toronto, Ontario, Canada) _(CHI ’14)_. Association for Computing Machinery, New York, NY, USA, 2191–2200.  https://doi.org/10.1145/2556288.2557163
  * Mueller et al. (2015) Suzanne Mueller, Andreas Dippon, and Gudrun Klinker. 2015\.  Capture The Flag: Engaging In A Multi-Device Augmented Reality Game. In _Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces_ (Madeira, Portugal) _(ITS ’15)_. Association for Computing Machinery, New York, NY, USA, 277–282.  https://doi.org/10.1145/2817721.2823493
  * Nintendo (1985) Nintendo. 1985.  Retrieved September 17, 2020 from https://mario.nintendo.com/
  * Normand et al. (2012) Jean-Marie Normand, Myriam Servières, and Guillaume Moreau. 2012\.  A New Typology of Augmented Reality Applications. In _Proceedings of the 3rd Augmented Human International Conference_ (Megève, France) _(AH ’12)_. Association for Computing Machinery, New York, NY, USA, Article 18, 8 pages.  https://doi.org/10.1145/2160125.2160143
  * Oduor et al. (2016) Erick Oduor, Carman Neustaedter, William Odom, Anthony Tang, Niala Moallem, Melanie Tory, and Pourang Irani. 2016.  The Frustrations and Benefits of Mobile Device Usage in the Home when Co-Present with Family Members. In _Proceedings of the 2016 ACM conference on designing interactive systems_. 1315–1327.
  * of TOGO (1975) Kazuo Yamada of TOGO. 1975\.  Wack-A-Mole.  Retrieved September 17, 2020 from https://en.wikipedia.org/wiki/Whac-A-Mole
  * Olsson et al. (2020) Thomas Olsson, Pradthana Jarusriboonchai, Paweł Woźniak, Susanna Paasovaara, Kaisa Väänänen, and Andrés Lucero. 2020\.  Technologies for enhancing collocated social interaction: review of design solutions and approaches.  _Computer Supported Cooperative Work (CSCW)_ 29, 1 (2020), 29–83.
  * Paasovaara et al. (2017) Susanna Paasovaara, Pradthana Jarusriboonchai, and Thomas Olsson. 2017.  Understanding Collocated Social Interaction between PokéMon GO Players. In _Proceedings of the 16th International Conference on Mobile and Ubiquitous Multimedia_ (Stuttgart, Germany) _(MUM ’17)_. Association for Computing Machinery, New York, NY, USA, 151–163.  https://doi.org/10.1145/3152832.3152854
  * Paavilainen et al. (2017) Janne Paavilainen, Hannu Korhonen, Kati Alha, Jaakko Stenros, Elina Koskinen, and Frans Mayra. 2017\.  The Pokémon GO experience: A location-based augmented reality mobile game goes mainstream. In _Proceedings of the 2017 CHI conference on human factors in computing systems_. 2493–2498.
  * Pagno et al. (2015) Bruno Pagno, Diogo Costa, Leandro Guedes, Carla Dal Sasso Freitas, and Luciana Nedel. 2015.  Guidelines for designing dynamic applications with second screen. In _2015 XVII symposium on virtual and augmented reality_. IEEE, 42–51.
  * Park (2020) Sihwa Park. 2020\.  ARLooper: Collaborative Audiovisual Experience with Mobile Devices in a Shared Augmented Reality Space. In _Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems_ (Honolulu, HI, USA) _(CHI EA ’20)_. Association for Computing Machinery, New York, NY, USA, 1–4.  https://doi.org/10.1145/3334480.3383172
  * Pickersgill (2014) Eric Pickersgill. 2014\.  REMOVED.  Retrieved August 17, 2020 from https://www.removed.social/series
  * Pine et al. (1999) B Joseph Pine, Joseph Pine, and James H Gilmore. 1999\.  _The experience economy: work is theatre & every business a stage_.  Harvard Business Press.
  * Przybylski and Weinstein (2013) Andrew K Przybylski and Netta Weinstein. 2013.  Can you connect with me now? How the presence of mobile communication technology influences face-to-face conversation quality.  _Journal of Social and Personal Relationships_ 30, 3 (2013), 237–246.
  * Raffle et al. (2007) Hayes Raffle, Cati Vaucelle, Ruibing Wang, and Hiroshi Ishii. 2007.  Jabberstamp: Embedding Sound and Voice in Traditional Drawings. In _Proceedings of the 6th International Conference on Interaction Design and Children_ (Aalborg, Denmark) _(IDC ’07)_. Association for Computing Machinery, New York, NY, USA, 137–144.  https://doi.org/10.1145/1297277.1297306
  * Rob (2010) DenBleyker Rob. 2010\.  _New Phone_.  explosm.  http://explosm.net/comics/2091/
  * Rogers (2014) Yvonne Rogers. 2014\.  Bursting our Digital Bubbles: Life Beyond the App. In _Proceedings of the 16th International Conference on Multimodal Interaction_. 1–1.
  * Russo (2020) Maria Russo. 2020\.  Books That Captivate Babies and Toddlers.  https://www.nytimes.com/2020/01/07/books/review/books-that-captivate-babies-and-toddlers.html
  * Segura and Isbister (2015) Elena Márquez Segura and Katherine Isbister. 2015.  Enabling co-located physical social play: A framework for design and evaluation.  In _Game user experience evaluation_. Springer, 209–238.
  * Sharp and Thomas (2019) John Sharp and David Thomas. 2019.  _Fun, Taste, & Games: An Aesthetics of the Idle, Unproductive, and Otherwise Playful_.  MIT Press.
  * Tiffany (2020) Kaitlyn Tiffany. 2020\.  It’s Cool to Look Terrifying on Pandemic Instagram.  Retrieved September 29, 2020 from https://www.theatlantic.com/technology/archive/2020/05/augmented-reality-instagram-zoom/611494/
  * Tikunova (2016) Paulina Tikunova. 2016\.  Soul-Sucking Photos Show How Phone Addiction Is Stealing Our Souls.  Retrieved September 17, 2020 from https://www.boredpanda.com/screens-stealing-soul-social-media-sur-fake-antoine-geiger/?utm_source=google&utm_medium=organic&utm_campaign=organic
  * Turkle (2016) Sherry Turkle. 2016\.  _Reclaiming conversation: The power of talk in a digital age_.  Penguin.
  * Turkle (2017) Sherry Turkle. 2017\.  _Alone together: Why we expect more from technology and less from each other_.  Hachette UK.
  * Vatavu et al. (2020) Radu-Daniel Vatavu, Pejman Saeghe, Teresa Chambel, Vinoba Vinayagamoorthy, and Marian F Ursu. 2020\.  Conceptualizing Augmented Reality Television for the Living Room. In _ACM International Conference on Interactive Media Experiences_. 1–12.
  * Vella et al. (2019) Kellie Vella, Daniel Johnson, Vanessa Wan Sze Cheng, Tracey Davenport, Jo Mitchell, Madison Klarkowski, and Cody Phillips. 2019\.  A sense of belonging: Pokemon GO and Social Connectedness.  _Games and Culture_ 14, 6 (2019), 583–603.
  * Waern and Back (2015) Annika Waern and Jon Back. 2015.  Experimental game design.  In _Game Research Methods_. 341–353.
  * Weisholtz (2020) Drew Weisholtz. 2020\.  How classic board games are bringing families closer during the pandemic.  Retrieved February 2, 2021 from https://www.today.com/popculture/board-games-enjoy-surge-popularity-during-pandemic-t202377
  * Wells and Houben (2020) Thomas Wells and Steven Houben. 2020.  CollabAR – Investigating the Mediating Role of Mobile AR Interfaces on Co-Located Group Collaboration. In _Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems_ (Honolulu, HI, USA) _(CHI ’20)_. Association for Computing Machinery, New York, NY, USA, 1–13.  https://doi.org/10.1145/3313831.3376541
  * Wetzel et al. (2008) Richard Wetzel, Rod McCall, Anne-Kathrin Braun, and Wolfgang Broll. 2008.  Guidelines for designing augmented reality games. In _Proceedings of the 2008 Conference on Future Play: Research, Play, Share_. 173–180.
  * Wise and Forrest (2003) Debra Wise and Sandra Forrest. 2003.  _Great big book of children’s games: over 450 indoor and outdoor games for kids_.  McGraw-Hill, 186,266.
  * Within Unlimited (2020) Inc Within Unlimited. 2020\.  Wonderscope.  https://wonderscope.com/
  * Xu et al. (2011) Y. Xu, E. Barba, I. Radu, M. Gandy, R. Shemaka, B. Schrank, B. MacIntyre, and T. Tseng. 2011\.  Pre-patterns for designing embodied interactions in handheld augmented reality games. In _2011 IEEE International Symposium on Mixed and Augmented Reality - Arts, Media, and Humanities_. 19–28.  https://doi.org/10.1109/ISMAR-AMH.2011.6093652
  * Xu et al. (2008) Yan Xu, Maribeth Gandy, Sami Deen, Brian Schrank, Kim Spreen, Michael Gorbsky, Timothy White, Evan Barba, Iulian Radu, Jay Bolter, and Blair MacIntyre. 2008.  BragFish: Exploring Physical and Social Interaction in Co-Located Handheld Augmented Reality Games. In _Proceedings of the 2008 International Conference on Advances in Computer Entertainment Technology_ (Yokohama, Japan) _(ACE ’08)_. Association for Computing Machinery, New York, NY, USA, 276–283.  https://doi.org/10.1145/1501750.1501816
  * Zambetta et al. (2020) Fabio Zambetta, William Raffe, Marco Tamassia, Florian ’Floyd‚ Mueller, Xiaodong Li, Niels Quinten, Rakesh Patibanda, Daniel Dang, and Jon Satterley. 2020\.  Reducing Perceived Waiting Time in Theme Park Queues via an Augmented Reality Game.  _ACM Trans. Comput.-Hum. Interact._ 27, 1, Article 3 (Jan. 2020), 30 pages.  https://doi.org/10.1145/3361524
  * Zimmerman et al. (2007) John Zimmerman, Jodi Forlizzi, and Shelley Evenson. 2007\.  Research through Design as a Method for Interaction Design Research in HCI. In _Proceedings of the SIGCHI Conference on Human Factors in Computing Systems_ (San Jose, California, USA) _(CHI ’07)_. Association for Computing Machinery, New York, NY, USA, 493–502.  https://doi.org/10.1145/1240624.1240704

## Appendix

##  Appendix A App Access Codes

The co-located AR apps developed for IRL can be accessed using Snapchat (Inc,
2020), you can follow the instruction on Snapchat’s official documentation.

  1. (1)

Open Snapchat

  2. (2)

Point the camera at the images provided on Fig.LABEL:fig:snapcodes

  3. (3)

press and hold your finger on top of the image

  4. (4)

Select unlock for 24/48 hours

Figure 13.  “Snapcodes” for Face-it (FI).  Figure 14.  “Snapcodes” for Feeture
Film (FF). Figure 15.  “Snapcodes” for Treasure Treat (TT). Figure 16.
“Snapcodes” for Milky Moo (MW). Figure 17.  “Snapcodes” for Freezing Frenzy
(FR).

##  Appendix B Semi-structured Interviews

The following is a list of the potential questions we asked in our semi-
structured interviews

  * •

What was your first impression? OR What were your thoughts about what you just
experienced?

  * •

How would you describe this experience to your friend who has never played it
before? What would you tell them about it?

  * •

Now that you have had a chance to experience it, is there any information that
would have been useful to you before starting?

  * •

Was anything confusing? (Please take me through what you found to be
confusing)

  * •

Was there anything you found particularly frustrating?

  * •

What did you find surprising about experience?

  * •

What do you think is special about this experience? OR what are the things
that make it different from other things you’ve used before?

  * •

What were the main challenges for you? OR What were the most challenging
things for you with this experience?

  * •

What did you like most about the experience?

    * –

If it was fun, what was fun about it?

    * –

If it wasn’t enjoyable: what didn’t work for you? What you wish would be
different?

  * •

What did you enjoy the least when using it?

  * •

What was the most exciting moment?

  * •

In what way did you interact with other people (or with your dog)?

  * •

Do you prefer to use it alone or with other people or with your dog)?

  * •

In person vs online playing

    * –

Comparing it to playing in-person vs. playing something similar on-line or
other ‘similar’ online games.

    * –

Example question: how do you think using the lens like you did, in person is
different than using it online with someone?

  * •

Physical interaction

    * –

Questions about the device configuration (sharing one phone/using different
phones/ including a video on TV)

    * –

Questions related to interaction through required body movements (sharing one
phone/using different phones/ including a video on TV)

  * •

Social interaction

    * –

What do you think about using it with other people vs. playing it alone?

    * –

What did you think of the ‘type of interaction’ (e.g. the competition, using
something to tell a story, playing a levels game)

  * •

Recording

    * –

(if it recorded) What do you think of the fact that it recorded you using it,
and then you were able to save it?

    * –

Would you look at it again?

    * –

Would you want to send it to someone else?

    * –

Who would you share it with? Why?

    * –

Would you want to record the experience?

  * •

Sharing

    * –

What if you could share this experience with someone– who would you share it
with?

    * –

Would you like to share this experience with other people?

    * –

How would you share it?

    * –

Would you send videos to people? Post to on social media?

    * –

Would you talk about with other people?

  * •

Duration and pace

    * –

What about the timing, the length of the experience?

    * –

How did it feel– did it feel too long, too short, or just about right?

    * –

What did you think of the pace of it?

  * •

Demo video

    * –

What did you think when you watched the demo video?

    * –

What did you think about the video?

    * –

Did watching it affect how you interacted?

  * •

How did the experience change when you used it for the second time?

  * •

What elements do you think could be improved?

  * •

Is there anything that you did not like about the experience? What for
example?

  * •

Is there anything that you thought I would ask but I didn’t? Anything you
wanted to talk about?

  * •

Overall, how would you describe this experience’s appeal?

  * •

What parts of the experience attracted you? OR what parts of the experience
you thought worked particularly well?

  * •

What was missing from the experience?

  * •

What if you could change just one thing, what would it be?

  * •

How was it to learn how to use it?

  * •

Would you purchase it?