# Perception in Pixels: Understanding Avatar Representation in Video-Mediated
Collaborative Interactions

Pitch Sinlapanuntakul<EMAIL_ADDRESS>Human Centered Design and Engineering,
University of WashingtonSeattleWashingtonUSA and  Mark Zachry<EMAIL_ADDRESS>Human Centered Design and Engineering, University of
WashingtonSeattleWashingtonUSA

(2024)

###### Abstract.

Despite the abundance of research concerning virtual reality (VR) avatars, the
impact of screen-based or augmented reality (AR) avatars for real-world
applications remain relatively unexplored. Notably, there is a lack of
research examining video-mediated collaborative interaction experiences using
AR avatars for goal-directed group activities. This study bridges this gap
with a mixed-methods, quasi-experimental user study that investigates video-
based small-group interactions when employing AR avatars as opposed to
traditional video for user representation. We found that the use of avatars
positively influenced self-esteem and video-based collaboration satisfaction.
In addition, our group interview findings highlight experiences and
perceptions regarding the dynamic use of avatars in video-mediated
collaborative interactions, including benefits, challenges, and factors that
would influence a decision to use avatars. This study contributes an empirical
understanding of avatar representation in mediating video-based collaborative
interactions, implications and perceptions surrounding the adoption of AR
avatars, and a comprehensive comparison of key characteristics between user
representations.

avatars, collaborative interaction, computer-mediated communication, user
representation, videoconferencing

††copyright: rightsretained††journalyear: 2024††doi:
XXXXXXX.XXXXXXX††conference: Manuscript in submission; February; 2024††ccs:
Human-centered computing Empirical studies in collaborative and social
computing

##  1\. Introduction

The increasingly ubiquitous adoption of remote environments, reliant on
synchronous video-based platforms, has underscored the fundamentals of
collaborative interactions within group settings. These interactions are
crucial for effective communication, teamwork, and productivity. However, the
inevitability of these platforms has introduced a growing challenge known as
“Zoom anxiety” (Falzone et al., 2005; Ramasubramanian, 2020). This phenomenon
is characterized by individuals feeling self-conscious and burdened by the
camera during video-mediated interaction, ultimately affecting their self-
esteem, participation, and experience when using these platforms. “Zoom
anxiety” is often driven by the fear of being watched and can lead to users
turning off their cameras to mitigate discomfort. Such withdrawals have been
shown to further hinder the participants’ full engagement and effectiveness of
the collaborative interactions by reducing self-focused attention and active
participation (Asher and Aderka, 2020; Azriel et al., 2020; Lin et al., 2021;
Vriends et al., 2017).

As technology continues to shape the way we engage, connect, and interact
within digitally-mediated spaces, the role of avatars, which serve as user
representations, has become increasingly relevant. Avatars offer a potential
solution to the challenges posed by “Zoom anxiety” and can enhance the overall
experience of video-mediated collaborative interactions. While employing an
audio-only modality can reduce “Zoom anxiety,” that modality can also lead to
the feeling of exclusion from the group (Panda et al., 2022). Avatars can
bridge this gap between audio and video representation by augmenting the
physical self, thus enhancing presence while mitigating “Zoom anxiety” (Park
et al., 2023; Ratan et al., 2022; Wicks, 2021). Some previous research has
explored the use of VR avatars to alleviate the “Zoom anxiety,” with promising
results, but these studies did not fully consider the unique challenges of
online, video-based communication. Unlike VR or fully digitized avatars, which
tend to obscure important live non-verbal cues, such as accurate hand gestures
and subtle head and body movements, AR or screen-based avatars partially
overlay one’s live-streaming representation without blocking the visibility of
some physical cues (Nowak and Fox, 2018).

Despite the growing body of research on user representations and avatars,
there has not yet been an empirical study of video-based collaborative
interactions among groups using AR avatars. Most prior work on avatar use is
narrowly focused on avatars in VR environments and for discussion-based
interactions with little to no tasks involved in the interactions. As
videoconferencing increases the flexibility of goal-directed activities,
studying the use of AR avatars in video-based collaborative interactions
provides the opportunity to understand how AR avatar representation affects
user perceptions and mediated communication, compared to traditional video
representation. Therefore, we conducted a mixed-methods, quasi-experimental
study with groups of participants to address the following research questions
(RQ):

  * •

RQ1: What is the impact of using avatars as a user representation on self-
esteem and collaboration satisfaction in video-mediated collaborative
interactions compared to using real-time video representations?

  * •

RQ2: How does the use of avatars as a user representation influence the
perceptions of self and others in video-mediated collaborative interactions?

  * •

RQ3: How does the shared experience of using avatars as a user representation
influence their perceived appropriateness and likelihood for use in video-
mediated collaborative interactions?

##  2\. Background and Related Work

To situate our research study on the impact of avatar representation in video-
mediated collaborative interactions, we provide an overview of related work in
three key areas pertaining to the potential use of avatars beyond VR, research
on avatar-mediated communication in videoconferencing, and AR avatar
technologies.

###  2.1. Beyond VR: Potential Use of AR Avatars

The rise of avatar usage is primarily associated with the proliferation of VR
use for gaming and social applications, typically accessible through head-
mounted displays like HTC Vive and Meta’s Quest (Abramczuk et al., 2023;
Maloney and Freeman, 2020). However, despite the potential of VR, its limited
adoption is attributed to various trade-offs, including additional headset and
software costs as well as the potential simulator sickness following prolonged
usage (Saredakis et al., 2020; Vishwakarma et al., 2020). Similarly, avatars
accessible through AR headset devices, such as Magic Leap and Microsoft
HoloLens, are considered virtual objects superimposed into the physical
surroundings. Recent studies suggest that prolonged use of AR headsets may
lead to simulator sickness and that further developments to make interactions
with virtual objects in an AR environment more closely resemble natural
interactions are deemed necessary to enhance usability (Sinlapanuntakul et
al., 2023, 2022).

In contrast, screen-based or AR avatars, such as Apple’s Memoji and Microsoft
Teams’ Mesh avatars, facilitated by smartphone or web camera-based
applications, remain a crucial focus for research due to their widespread
availability and ease of use (Bonfert et al., 2023; Makled et al., 2022). The
majority of AR avatar scenarios in existing literature notably are geared
toward immediate and context-specific objectives, such as participation in
online social events and team-building activities.

The use of AR avatars offers several advantages in practical applications, as
highlighted in prior research. For instance, Liu et al. (Liu et al., 2020)
demonstrated that AR avatars delivered a more realistic representation of
garment try-outs as compared to their VR counterparts. Another study revealed
that users engaging with AR avatars experienced reduced mental workloads when
interacting with immersive information, unlike those using headsets (Li et
al., 2021). This suggests that AR avatars can offer a more comfortable and
less cognitively demanding experience during immersive tasks. Furthermore,
Zibrek et al. (Zibrek et al., 2018) explored the likeability of avatars in
both VR and videoconferencing, considering different levels of realism. They
found that AR avatars, when designed with sufficient visual and behavioral
fidelity, were just as appealing and engaging as their VR counterparts,
indicating their effectiveness in eliciting positive user interactions.

While the majority of avatar research in HCI and CSCW is centered around VR,
these studies are often confined to controlled laboratory settings, which may
limit their real-world applicability (Aufegger and Elliott-Deflo, 2022; Dobre
et al., 2022; Hudson and Hurter, 2016). Additionally, the limited
accessibility of VR devices restricts these investigations to specialized
applications, largely in training and skill development (Bartl et al., 2022;
Grassini and Laumann, 2020; Xie et al., 2021). Conversely, the availability of
AR avatars on smartphones and video platforms has not only broadened the scope
of potential research but also enabled access to larger participant pools
(Phadnis et al., 2023). With the accessibility of AR avatars, potential
research at the interaction of the role of AR avatars and group collaborative
interactions (Yoon et al., 2019) offers a more realistic representation of
real-world applications and a more extensive exploration of human experiences
beyond controlled environments.

###  2.2. Avatar-Mediated Communication in Videoconferencing

The exploration of diverse communication modalities has been a subject of
interest ever since the emergence of VR technology. Nevertheless, there
remains an ongoing need for further research into the foundational principles
and practical implications, particularly as new technologies continue to
develop and reshape mediated communication. In this context, Schroeder
(Schroeder, 2011) examined the fundamental differences in how individuals are
represented via video and avatars (Table 1). While these key characteristics
contribute to key aspects of interpersonal interaction, collaboration, and
enjoyment in digital space, the avatar representation modality only captures
VR avatars. Since VR avatars are different from AR avatars used in this study,
we propose an updated comparison of user representations in our discussion
(see Section 5.4), including key characteristics of video, VR avatar, and AR
avatar. Over the past decade, research focused on group interactions using
avatars within VR environments has gained substantial attention. These
investigations have included a wide range of domains, such as transferability,
performance, presence and co-presence, and trust (Moustafa and Steed, 2018;
Pan and Steed, 2017; Schroeder, 2002; Wu et al., 2021).

Table 1. Key characteristics of video versus avatar-mediated interaction (Schroeder, 2011). Characteristic | Video | Avatar
---|---|---
Representation | True to life | Constructed but can approach realism
Body capabilities | Limited to physical ones | “Super powers”
Environment apart from | Typically limited to context of head | From room-size spaces to
person representation | and shoulders (i.e., table and room) | vast interconnected worlds
Interpersonal interaction | Focus on facial expressions | Focus on spatial encounters with bodies
Number of participants | Two - to small groups | Two - to large populations
|  | (though still focused on the group)
Advantage | Few concerns about deception | Plasticity of self-presentation
Disadvantage | Concerns about one’s appearance | Inability to come across as oneself
| (can be avoided with picture-in-picture) |

In Bente et al. (Bente et al., 2008), they conducted a comparative study
involving text-chat, audio, video conferencing, and avatars. Their findings
highlighted distinctions between text-chat and real-time modalities, but not
among audio, video, or avatars. Comparing three different modalities of user
representations, including VR avatars, video, and audio, Junuzovic et al.
(Junuzovic et al., 2012) examined realism, presence, and non-verbal cues in
small-group settings, where participants engaged in collaborative work within
groups of four for brainstorming activities. They found avatars provided
higher sociability but lower realism, possibly because of the low level of
avatar fidelity. Inkpen and Sedlins (Inkpen and Sedlins, 2011) explored avatar
appearance in work settings, finding varying user feedback, with some avatars
akin to webcam photos and not unsettling, particularly when appearing formal
and realistic. It is essential to acknowledge that the landscape of video
conferencing has drastically evolved since the COVID-19 pandemic, specifically
in work-related contexts. Users’ acceptance and preferences for rich forms of
video communication have likely evolved over time, with increased exposure to
new technologies for everyday communication (Panda et al., 2022).

Although no research has specifically explored group collaboration with AR
avatars in videoconferencing, a few prior studies have explored the use of AR
avatars for realism, presence, and self-affirmation for camera shyness. A
recent large-scale survey study found that avatar realism significantly
influenced acceptability for work-related videoconferencing among knowledge
workers, with less realistic avatars being more acceptable for known
colleagues but less so for managers and unknown colleagues (Phadnis et al.,
2023). Critical attributes included professionalism, credibility, and
seriousness, while some participants highlighted the added benefit of
cuteness, fun, and liveliness when using AR avatars at work. Panda et al.
(Panda et al., 2022) designed a low-fidelity, non-customizable AR avatar used
to represent audio-only participants and conducted a user interview on
presence and co-presence in mixed-modality conferencing environments, which
included audio, avatar, and video representations. Their findings revealed
that participants preferred having an avatar representing themselves and
others when there is a lack of visual representation (i.e., audio-only),
affecting both perceived presence and co-presence. Another recent study
investigated the use of an AR avatar in videoconferencing for self-affirmation
when participating in a one-on-one debate, indicating a positive influence of
the use of AR avatars on active participation in discussions when used in
conjunction with self-affirmation but negatively impacted how they responded
to critical messages from others (Park et al., 2023).

Prior research on VR avatars in the context of group or collaborative settings
has primarily focused on comparing different types of representations, such as
videos, avatars, and audio. However, research exploring on-screen AR avatars
is scarce, and all previous research on AR avatars lacks group-based
interactions and activities. Within the context of video-mediated
collaborative interactions, a significant gap exists in understanding the
dynamics and experiences within goal-directed group activities. To the best of
our knowledge, no prior study has taken the initiative to design and
empirically investigate the complexities associated with the use of video-
based AR avatars for collective activity achievement within a group setting.

###  2.3. AR Avatar Technologies

Avatars, which represent users in digital space, offer a broad spectrum of
fidelity, ranging from abstract to highly detailed. Their applications extend
to various contexts, such as live video calls and gaming. These avatars can
either be automatically generated to resemble a user’s appearance or manually
personalized by a user, allowing for extensive customization of features such
as facial characteristics, body type, clothing, and accessories to align with
their preferences and identity (Panda et al., 2022). The capability to animate
avatars for emotional expression and connection among users is a prominent
feature. This is frequently achieved by rigging a skeletal system, with
control methods including computer vision, speech sentiment analysis, or audio
input for viseme-based animations (Gonzalez-Franco et al., 2020a). Even
avatars with minimal animations, like subtle facial movements, contribute to
users’ identification and embodiment with them, which enhances the overall
experience, engagement, and behavior (Gonzalez-Franco et al., 2020b).

The role of avatars as powerful facilitators of user representation stands as
a cornerstone of their effectiveness and experiences across a spectrum of
applications. In this study, we used Memoji, a widely used AR animated
humanoid avatar developed by Apple, which is known for its multi-
functionality, allowing users to create static images or stickers that capture
personalized expressions. Additionally, Memoji supports recorded motions that
allow complete control over facial expressions and audio recording. Using an
iOS device’s camera, Memoji uses facial mapping and tracking technology for a
real-time dynamic augmentation of three-dimensional graphics onto the user’s
head displayed on the iOS device screen (Figure 1). This process ensures high
fidelity and performance, synchronously replicating facial expressions and
movements, as demonstrated in live video communication through FaceTime (Suda
and Oka, 2021).

Figure 1. Memoji avatars on FaceTime.

2x2 grid FaceTime on mobile with each grid featuring a Memoji avatar.

##  3\. Methods

###  3.1. Study Design

In this study, we aimed to gain insights into the experience of video-based
small-group interaction when considering the use of avatars as a user
representation by answering the following questions:

  * •

RQ1: What is the impact of using avatars as a user representation on self-
esteem and collaboration satisfaction in video-mediated collaborative
interactions compared to using real-time video representations?

  * •

RQ2: How does the use of avatars as a user representation influence the
perceptions of self and others in video-mediated collaborative interactions?

  * •

RQ3: How does the shared experience of using avatars as a user representation
influence their perceived appropriateness and likelihood for use in video-
mediated collaborative interactions?

To address these questions, we adopted a mixed-methods approach to investigate
the effects of avatar use in video-mediated collaborative interactions with
goal-directed group activities. Employing a repeated-measures, quasi-
experimental design, this study examined the independent variable of user
representation conditions (avatar vs. traditional video) with the dependent
variables including self-esteem, collaboration satisfaction, and group
interview outcomes. In this context, avatar representation refers to the 3D
digital embodiment that augments an individual’s head using animated
characters (i.e., Memoji) within a computer-mediated videoconferencing (i.e.,
FaceTime), whereas traditional video representation pertains to the direct
streaming of real-time video, capturing participants’ physical appearances
during videoconferencing. The order in which the user representation
conditions were presented was counterbalanced to mitigate potential order
effects across participant groups.

###  3.2. Design of Game-Based Maps and Activities

We thoroughly designed two game-based maps (Figure 2), including the world and
university campus maps on Miro, a digital whiteboard for real-time remote
group collaboration, to serve as the central element in structured puzzle-
solving activities. These maps were instrumental in engaging participants in
the collective identification of countries and campus buildings based on the
activity prompts. This approach aimed to foster collaborative interactions and
leverage the collective knowledge and engagement of the participants. The
overarching intention behind this design was to simulate collaborative
problem-solving scenarios in a synchronous digital environment.

Figure 2. Game-Based World Map and University Campus Map on Miro.

The design of game-based world map and university campus map on Miro
interface.

We strategically integrated motivational aspects through gamification for
interactive participation in our design, including a scoring with extra
incentives and various game-based mechanics. Each participant group was
informed that, besides their participant compensation, they had the
opportunity to earn an extra $15 per person if they achieved the highest score
across all participant groups. These gamified strategies of the scoring system
and the potential monetary reward for reaching a “winning” condition
introduced competitive elements to the interaction and collaboration.
Additionally, our game-based mechanics, including the time limit, text-based
labels, multiple choices, and interactive actions for moving elements, further
gamified the dynamic interactions, facilitating dynamic and engaging
experiences.

The world map portrayed global geography divided into 5 panels, each
representing a region along with its associated countries and names. In the
first world map activity, participants were prompted to identify the second
most populated country in each designated region by placing star icons on
those countries. Subsequently, the second world map activity presented 5
missing country names across the panels, challenging participants to identify
them by selecting from a pool of 10 choices, with 2 possible names for each
panel. However, only 5 of these provided choices were correctly associated
with the countries lacking names, adding an extra layer of complexity to the
activity.

Similarly, the university campus map was divided into 5 panels, representing
parts of the campus that mirrored recognizable buildings and landscapes. The
first campus map activity asked participants to identify the building with the
largest square footage in each panel by placing star icons accordingly.
Following this, the second campus map activity introduced 5 missing building
names across the panels. Participants were asked to identify these buildings
from a pool of 10 choices. In this case, while 5 of these provided choices
correctly matched the buildings’ missing names, the remaining 5 choices were
fake building names.

###  3.3. Participants

Participants were recruited from a university through a combination of word-
of-mouth and distribution of recruitment advertisements on social media
platforms, including Slack and the university website. Due to the game-based
map design of this study, we intentionally recruited groups of university
students or recent graduates, ensuring some familiarity with the selected
campus. Eligible participants signed up and participated in groups of four. We
included participants who signed up as pre-formed groups of four, were already
acquainted as friends or colleagues, and had individually created personalized
Memoji on their mobile devices prior to the study session. During the
recruitment, participants were informed that they would be engaging in
collaborative activities with their pre-formed group members to solve game-
based activities. A total of 32 participants participated in this study,
resulting in 8 participating groups with 4 participants per group. We present
the self-reported demographics of each participant group in Table 2.

Table 2. Self-reported demographics of our participant groups. Values are count (n). Group # | A | B | C | D | E | F | G | H
---|---|---|---|---|---|---|---|---
ID | A1 - A4 | B1 - B4 | C1 - C4 | D1 - D4 | E1 - E4 | F1 - F4 | G1 - G4 | H1 - H4
Age
18 to 22 | 0 | 0 | 4 | 3 | 1 | 1 | 2 | 4
23 to 27 | 2 | 3 | 0 | 1 | 1 | 3 | 1 | 0
28 to 32 | 2 | 0 | 0 | 0 | 0 | 0 | 1 | 0
33 to 37 | 0 | 1 | 0 | 0 | 2 | 0 | 0 | 0
Gender
Male | 2 | 2 | 2 | 2 | 1 | 2 | 3 | 1
Female | 2 | 2 | 2 | 2 | 3 | 2 | 1 | 3
Race
Asian | 3 | 4 | 1 | 4 | 3 | 2 | 4 | 4
Black | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0
Mixed (Asian | 1 | 0 | 3 | 0 | 0 | 0 | 0 | 0
& White) |  |  |  |  |  |  |  |
White | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 0
Experience Using Avatars in Non-Gaming Video-Based Platforms (hours)
N/A | 0 | 3 | 1 | 2 | 3 | 1 | 1 | 0
¡1 to 3 | 2 | 1 | 3 | 1 | 0 | 3 | 3 | 4
3 to 5 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0
5 to 10 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0
10 to 30 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0

The virtual conferencing platforms for avatar self-presentation included
FaceTime (n = 16), Google Meet (n = 5), Microsoft Teams (n = 1), Social VR
(non-gaming) Platforms (n = 6), Zoom (n = 7). Participants also self-reflected
their Memoji representation or design in three main categories: actual self (n
= 21), ideal self (n = 8), and creative/expressive self (n = 3).

###  3.4. Procedure

Data collection sessions were conducted via video-based platforms using both
Zoom and FaceTime. We controlled group dynamics by recruiting groups of four
and restricting the relationship of each group’s members as prior real-life
acquaintances, such as friends. After all group members had consented to
participate, one participant per group assumed the role of “operator” who was
able to manipulate elements on the game-based maps and received a brief
tutorial on interacting with the Miro interface, which covered methods such as
zooming, navigating the workspace, and moving on-screen elements. During each
session, operators were provided access to Miro containing the game-based maps
and shared their Miro screen on Zoom. The design of the game-based maps and
activities ensured that these additional responsibilities did not hinder the
collaborative process with others. Participants joined the FaceTime link using
their personal mobile devices, keeping their laptop cameras off and audio
muted as the screens featured Miro on Zoom. We pragmatically used FaceTime
with Memoji alongside Zoom. This arrangement was necessary given the current
absence of humanoid avatar technology available on other video-based
platforms, such as Zoom, Microsoft Teams, and Google Meet. Participants were
asked to work together to solve game-based activities on two different
maps—the world map and the university campus map—on Zoom while communicating
via FaceTime (Figure 3).

Figure 3. Device Setup with Miro and FaceTime.

The experimental setup of Miro, featuring the world map, on a laptop screen
and FaceTime, featuring Memoji avatars as the user representation.

During FaceTime sessions, participating groups were presented with user
representation conditions (avatar vs. video), counterbalanced across all
groups. The on-screen layout is represented with a 2x2 grid with each squared-
grid mirroring each user’s real-time video, whether with or without a humanoid
avatar representation. Both game-based maps consisted of two comparable
activities each, resulting in a total of four activities (see Section 3.2).
Before beginning each activity for each map, participants were explained and
prompted on the activity, with a 5-minute time limit for each activity.
Following the completion of each map, participants filled out a set of
questionnaires, including the Rosenberg Self-Esteem Scale and the
Collaboration Satisfaction. Demographic questions were included in the second
set of questionnaires. Following this, participants engaged in group
interviews to share their experiences and perceptions of the collaborative
interaction experience during the study group activities: one map with avatar
representation and the other with video representation. The interview portion
was audio-transcribed for subsequent data analysis. After the group
interviews, participants were debriefed and compensated for their
participation and time. The average duration of study sessions was 60 minutes,
and each participant received a $15 gift card as compensation. The team with
the highest score was awarded an additional $15 gift card for each team
member. This study was approved by the University’s Institutional Review Board
(IRB).

###  3.5. Measures

####  3.5.1. Self-Esteem

The Rosenberg Self-Esteem Scale (Rosenberg, 1965) was used to measure
participants’ self-esteem and self-confidence, assessing both positive and
negative feelings about self-image after engaging in each of the group
activities. This scale is the most commonly used measurement of self-esteem
and consists of 10 items measured on a 4-point Guttman-type rating scale,
ranging from strongly agree to strongly disagree. The total possible score on
the scale was 40, with higher scores indicating higher levels of self-esteem
and self-confidence. Each item was adjusted by adding “during the recently
completed group activity” to capture a more contextually relevant assessment
of participants’ self-reflection within collaborative experiences of each
group activity.

####  3.5.2. Video-Based Collaboration Satisfaction

The Collaboration Satisfaction instrument (Reinig, 2003) measured
collaboration satisfaction with the process and outcomes of group
collaborative work. It comprises 8 items that rate two constructs on a 5-point
scale: satisfaction with collaborative process (SP) and satisfaction with
collaborative decision (SD). The SP construct includes 5 semantic differential
items that represent satisfaction with group dynamics during collaborative
interactions. Due to poor loading in factor analysis (Green and Taber, 1980),
the SD construct includes 3 Likert scale items—instead of the originally
intended 5 items—representing satisfaction with collaborative outcomes. The
total score is calculated as an average across all items, with higher average
scores indicating a greater level of satisfaction with the group
collaboration. SP items were adjusted by adding “during the recently completed
group activity,” and SD items were modified by replacing “group’s solution”
with “group’s decision.” These adjustments aimed to more accurately capture
participants’ perceptions of both the process and decision/outcome within the
group collaboration.

####  3.5.3. Group Interview

Group interviews were structured around a set of main questions designed to
explore participants’ experiences and perceptions with avatar-mediated
communication in group collaborative settings. The main questions included
participants’ overall thoughts on team communication, ability to read others’
thoughts and feelings, sense of group cohesion, engagement and participation,
as well as perceived appropriateness and influential factors to use or not use
avatars. This set of questions was consistent across all groups, regardless of
their background, with varied follow-up questions based on the discussion
topics and available time during interviews.

###  3.6. Data Analysis

####  3.6.1. Quantitative Analysis

Paired-samples t-tests were conducted to examine the within-group differences
in self-esteem and video-based collaboration satisfaction between the study
conditions (avatar vs. traditional video) while collaboratively participating
in group activities. Cohen’s d (Cohen, 1988) was calculated to quantify the
effect size of the observed mean difference. Correlations were calculated
using Pearson correlation (r). Bonferroni corrections were applied to adjust
for the family-wise error rate. These analyses were performed using the R
statistical software (version 2023.09.0+463).

####  3.6.2. Qualitative Analysis

We conducted a combined approach of inductive summary annotation coding and
deductive reflexive thematic analysis (Braun and Clarke, 2006) to explore
participants’ experiences and perceptions with avatars during collaborative
interactions in videoconferencing. Aligning with qualitative analysis
guidelines in CSCW and HCI practice (McDonald et al., 2019), our analytical
procedures focused on the generation of concepts and themes rather than
seeking traditional qualitative coding agreement as uniform agreement on codes
might lead to diverse interpretations of each code’s meaning. In addition,
given that the first author solely conducted the group interview sessions, our
analytical procedure intentionally did not pursue agreement and inter-rater
reliability (McDonald et al., 2019).

We followed specific procedures for thematic analysis, beginning with the
first author reading through all eight group interview transcripts and
dividing each transcript into multiple data segments corresponding to the
questions and follow-up questions posed during each session. In the second
phase, 20 student researchers (7 MS students; 13 BS students) independently
coded the data using summary annotations. Each student researcher was randomly
assigned two interview transcripts, resulting in 40 coding instances, with
each of the eight transcripts coded five times. Instructions were given to the
student researchers to carefully read each data segment and provide a written
summary and interpretation of the data as well as notable agreements and
disagreements within the group. Subsequently, the third step involved the
first author reviewing the summary annotations to gain an overall
understanding and identifying recurrent topics. The fourth step involved the
first author re-reading the eight original group interview transcripts,
revising thematic topics, and developing sub-themes. Lastly, both authors
iteratively examined and refined themes, along with our description and
interpretation concerning the use of avatars in video-based collaborative
interactions.

##  4\. Results

###  4.1. Self-Esteem

There was a statistically significant difference in self-esteem scores between
the avatar representation (M = 31.84, SD = 4.55) and the traditional video
representation condition (M = 26.97, SD = 4.72), t(31) = 4.67, p ¡ .001; d =
1.05, indicating a large effect size (Figure 4). This finding implies that
participants experienced significantly higher levels of self-esteem and self-
confidence when represented with avatars compared to video representation
during the video-based collaboration, potentially influencing their engagement
and experiences in collaborative activities.

Figure 4. Comparison of average self-esteem ratings. Error bars represent +/-
1 standard error. Avatar representation demonstrates significantly higher
self-esteem ratings compared to video representation, p ¡ .001.

Bar graphs featuring avatar vs. video representations in self-esteem.

Although not significantly different, there is a noticeable trend among
participants assigned the “operator” role, rating their self-esteem higher
than other group members in both user representation conditions. In the avatar
representation, operators rated their self-esteem (M = 33.88, SD = 4.79)
higher than other group members (M = 31.17, SD = 4.36), mirroring the pattern
seen in the traditional video representation condition (operators: M = 29.50,
SD = 4.31; other group members: M = 26.13, SD = 4.63). This trend may require
further exploration, with a possible explanation that operators may perceive
their role as having more control and responsibility within the group,
influencing their self-esteem.

###  4.2. Video-Based Collaboration Satisfaction

Significant differences in video-based collaboration satisfaction were found
between user representation conditions. With the highest and lowest possible
scores of 5 and 1, respectively, participants reported significantly higher
overall collaboration satisfaction with avatar representation (M = 4.13, SD =
0.61) compared to video representation (M = 3.66, SD = 0.66), t(31) = 2.81, p
= .008; d = 0.74 (Figure 5). This indicates a medium effect size and suggests
that the choice of user representation has a discernible impact on the overall
satisfaction in video-based collaboration as incorporating avatar
representations can significantly enhance overall collaboration satisfaction
among group members.

Further analyses into specific aspects of video-based collaboration
satisfaction revealed statistically significant differences between the user
representation conditions (Figure 5). The satisfaction with collaborative
process construct was significantly higher in the avatar representation (M =
4.25, SD = 0.60) compared to the traditional video representation condition (M
= 3.81, SD = 0.66), t(31) = 2.68, p = .012; d = 0.71, indicating a medium
effect size. This satisfaction with collaborative processes when using avatar
representation suggests that avatars may positively influence the dynamics of
the video-based collaborative process, which include perceived efficiency,
coordination, fairness, understanding, and satisfaction. Similarly, the
satisfaction with the collaborative decision construct demonstrated a
significant difference between the avatar representation (M = 3.94, SD = 0.73)
and video representation (M = 3.43, SD = 0.88) conditions, t(31) = 2.61, p =
.014; d = 0.63, indicating a medium effect size. This suggests that avatar
representation contributes to higher decision-making satisfaction during
video-based collaborations, which could be related to the extent to which the
group’s final decision reflects their inputs, the extent to which they feel
committed to the group decisions, and the extent to which they believe and
trust that the group’s decisions are correct.

Figure 5. Comparison of average video-based collaboration satisfaction
ratings. Error bars represent +/- 1 standard error. Compared to video
representation, avatar representation demonstrates significantly higher
ratings in satisfaction with overall collaboration (p = .008), with
collaborative process (p = .012), and with collaborative decision (p = .014).

Bar graphs featuring avatar vs. video representations in the satisfaction with
overall collaboration, satisfaction with collaborative process, and
satisfaction with collaborative decision.

###  4.3. Interrelationships between Self-Esteem and Video-Based Collaboration
Satisfaction

Pearson correlation coefficients were used to assess the relationships between
self-esteem, collaboration satisfaction, and decision-making processes in both
avatar and video representation conditions. Particularly noteworthy were the
strong positive correlations observed in the avatar representation between
self-esteem and dimensions of video-based collaboration satisfaction: overall
satisfaction, r(30) = .72, p ¡ .001; process satisfaction, r(30) = .72, p ¡
.001; and decision-making satisfaction, r(30) = .62, p ¡ .001. These findings
suggest that, in the context of avatar representation, higher levels of
perceived self-esteem align with increased satisfaction in group
collaboration, process, and decision-making activities. However, several other
correlations either failed to reach statistical significance after Bonferroni
correction or exhibited weak correlations. The lack of correlations in the
traditional video representation condition suggests that perceived levels of
self-esteem may not necessarily be associated with video-based collaboration
satisfaction.

###  4.4. Avatar Engagement in Video-Mediated Collaboration

Although the effects of avatars, detailed in Sections 4.1, 4.2, and 4.3, are
essential to video-mediated participation and experience, participants
reflected broader perceptions shaping their collaborative interactions with
the use of avatar representation in Sections 4.4, 4.5, and 4.6. Drawn entirely
from group interviews, we discuss observed themes on the ability of avatars to
enhance self-confidence, engagement, and comfort while alleviating social
awkwardness within video-based group settings.

####  4.4.1. Increased Self-Awareness and Confidence

An unexpected positive outcome was the heightened self-awareness and
confidence reported by participants while using avatars. Many participants
reflected on their elevated self-awareness, particularly regarding their
facial expressions: “Yeah, I think I paid more attention to my own
expressions. Because I didn’t wanna come across like I was frowning. And I, it
sort of forced me to. Maybe smile a little more. So just so it shows up” (B4).
This insight underscores the participant’s conscious effort to be more aware
of their non-verbal communication and maintain a positive demeanor during a
specific activity. In addition, F2 explained, “it is kinda like I’m having a
mask on, and so in a way I’m more confident with my own thoughts. Like the
confidence to say it out loud.” This metaphor suggests that their engagement
in a particular activity or self-presentation enabled an increased sense of
confidence in expressing their thoughts openly, which implies that this
newfound confidence allowed them to reveal a more authentic self.

A few others discussed a shift from focusing primarily on the game-based
activity to paying more attention to the actions and behaviors of other group
members mediated by avatars. For instance, H4 noted, “I wasn’t a hundred
percent focused on the task like I would go back to seeing it and look at
other people’s, but I feel like I paid more attention to what others are
doing,” serving as an example of an increased self-awareness regarding their
group dynamics.

####  4.4.2. Enhanced Participation Engagement from Initial Self-Consciousness

Participants discussed the impact of their initial self-consciousness,
primarily in response to avatars, on their own engagement during video-based
group activities. Initially, participants expressed concerns about the
avatar’s positioning and were preoccupied with their avatars. They actively
experimented with the avatar’s ability to replicate their facial expressions
due to their curiosity about its capabilities in real-time facial tracking on
videoconferencing. For example, participant H3 conveyed their heightened self-
awareness when using the avatar, noting that they paid closer attention to
their body language and positioning, concerning the avatar being partially or
completely out of the frame on their device screen:

> “I was more conscious because I was leaning forward when I was looking at my
> screen. So I was like, oh, am I out of frame of my phone so the Memoji is
> not on my face at times. So it’s almost a reverse where I was conscious of
> the Memoji even being on my face in the first place.” (H3)

Some other participants shared similar sentiments that they found themselves
gazing at their digital avatars and actively experimenting with the avatar’s
ability to replicate their facial expressions. These observations underscored
a collective curiosity about exploring the capabilities of the avatar and how
they mirrored real-life expressions. Over time, this initial self-
consciousness lessened, as participants “worried about it less” (E3 and E4),
and evolved into a more casual and integrated aspect of their online
interactions. With less time for self-monitoring and concerns, they had “more
time to talk and think through the activities” (E2), allowing for more
meaningful communication and collaboration.

####  4.4.3. Comfort in Augmented Physical Appearance

With the rise of video conferencing platforms, such as Zoom, our physical
appearance in video-based interactions has become a central concern. For some,
this shift has brought about a sense of unease and self-consciousness,
connecting one’s authentic self and the pressures of presenting in
videoconferencing. Particularly when finding oneself in the spotlight or
position of presenting to a larger audience, the act and need of turning on a
camera can be an anxiety-inducing experience. To enable individuals to
participate without the anxiety of being judged based on their physical
attributes, a participant reflected on their past experiences and suggested
avatars as a solution:

> “When we turn on our camera [in] Zoom, we actually pay a lot of attention to
> our features, like how we look, especially when we are presenting. So, I
> think, [an avatar] will be very useful when one person does not really want
> to show their face in a large [video conference].” (G4)

C3 took this notion further, speculating that, “[an avatar] could be used … if
someone did not really feel confident with their appearance.” This
augmentation empowers individuals to present themselves in a manner that
aligns with their self-image, thus mitigating the burden of self-doubt. Some
participants, drawing on both past experiences and their participation in this
study, pointed out how using avatars could mitigate self-consciousness when
having a camera turned on during video-based group interactions. For instance:

> “Sometimes, I get distracted when I’m doing work, like if my camera is on, I
> get conscious sometimes and I’m like, what is my face doing? So, I thought
> the avatar actually provided that mask kind of where the animation is not
> gonna look bad no matter what angle you’re in.” (H1)

Being aware that someone might be watching you on the screen can easily cause
distraction. The use of avatars provides individuals with a sense of agency
over how they are perceived and, in doing so, fosters a sense of comfort and
self-confidence in videoconferencing.

####  4.4.4. Reduced Social Tension and Awkwardness

Within the context of group video-based collaboration, the adoption of avatars
was viewed as a noteworthy strategy for mitigating social tension and
alleviating any awkwardness. Participants stated that these avatars had the
ability to “create a relaxed atmosphere” (C2, F4) during interactions,
suggesting their utility in enhancing group dynamics and reducing the anxiety
often associated with such collaboration.

These avatars were viewed as instrumental in steering conversations towards
more engaging and diverse topics, which foster connections and conversations
beyond the usual work-related topics. Recurring team avatars were speculated
to serve as potential “ice breakers, particularly in situations where
participants were less familiar with each other” (H2), essentially
“stimulating creativity and fostering camaraderie within the group” (C1). The
influence of avatars extended beyond being mere “ice breaker,” as several
participants reflected on their experience in this study that avatars
transformed the collaboration experience, making it “feel less like work and
more like to a group bonding session” (A1).

Participants, for the most part, did not perceive avatars as an essential
component throughout the activities but rather as “a valuable tool during
specific segments” (G3). They speculated that integrating avatars in the
initial minutes of collaborative activities could set a positive tone,
invoking laughter and elevating the overall atmosphere. Avatars were viewed as
potentially effective in interactions designed for team-building and fostering
group cohesion, such as “fun Fridays” (D3) or “themed meetings” (D2).

Furthermore, these avatars were regarded as “a means of stress reduction”
(A2). The inherent charm of these cute and whimsical representations
effectively alleviated tension and anxiety. Participants found that even when
their avatars displayed serious expressions, the playful and lighthearted
nature of these virtual figures mitigated any self-consciousness (A3). This
amalgamation of visual charm and the opportunity for individual expression was
influential in creating an environment where social tension and awkwardness
receded into the background, facilitating a more uninhibited and enjoyable
collaboration experience.

###  4.5. Challenges in Avatar-Mediated Emotional and Authentic Expression

Despite the benefits that avatars may offer, participants acknowledged the
challenges associated with using avatars in video-mediated communication. In
this section, we discuss the critical aspects of avatar representation in both
expressing and perceiving emotions to maintain effective collaborative
interactions as well as in perceived authenticity for self-expression.

####  4.5.1. Fidelity of Emotion Expression through Avatars

Avatars have emerged as important tools for conveying human emotions and
expressions within virtual communication. A particularly noteworthy challenge
surfaced in participants’ attempts to accurately express emotions through
avatars, contributing to occasional distractions. The limitations in conveying
nuanced emotional states underscore the current technological constraints in
achieving seamless emotional expression. The emotive realism that avatars
bring to mediated interactions is not accurate to that level of emotional
communication as H3 astutely notes how avatars can induce a natural smile,
even when one’s facial expression is relaxed:

> “[Avatars] bring more attention to like certain expressions that you’d be
> making. Like, it gives just a natural smile on your face when your face is
> just resting. [It] kind of accentuates that. I mean it’s literally an
> animation so it’s very animated with whatever you do with your face so
> seeing now and other people as well as yourself is interesting.” (H3)

In contrast, while tone and expressions are fundamental for understanding
emotional states, some participants raised a pertinent concern that avatars
primarily and correctly illustrate head movements and mouth expressions but
often neglect the expressive power of the eyes. This observation raises
substantial questions about the efficacy of avatars as reliable tools for
accurately portraying users’ emotional states in addition to typical facial
movements, such as winking. For instance, G1 mentioned:

> “Sometimes it’s very hard to gauge only from their tone. But you know,
> sometimes expressions give it away and I think you always pick up on these
> social cues and that’s when you’re like, okay, it doesn’t seem like this
> person is, you know, 100% onboard on this idea, but I think avatars don’t
> usually convey too much of an expression, except for the head movements and
> mouth. Like we also express emotions with other parts and it doesn’t capture
> too much of, you know, the eyes and stuff.” (G1)

On a similar note, avatars sometimes incorrectly displayed facial expressions,
for example, displaying a frown when, in reality, they were feeling at ease.
This experience raises the dilemma faced by individuals when navigating the
delicate balance between authentic emotions and the need to maintain a
composed professional appearance, particularly in work-related contexts:

> “Yeah, I was gonna say the same thing because I noticed that it doesn’t
> always translate your expressions well. So sometimes, I saw my avatar
> frowning when in fact I was, I just had a natural resting feel going on. So
> I had to put up a smile so it translates onto the screen. So, I think during
> work, that won’t be acceptable, and it’s just better to have your normal
> face on.” (B4)

There are mixed viewpoints on the fidelity of emotion expression through
avatars. While avatars hold the potential to enhance emotional expression and
facilitate communication, they also bear limitations in faithfully capturing
the intricacies of human emotion.

####  4.5.2. Difficulty in Interpreting Avatar-Mediated Thoughts and Emotions

The immersive experience of avatars posed challenges in accurately deciphering
the thoughts and emotions of others during group communication. Participants
expressed a need for more cues in avatars to enhance the interpretability of
non-verbal communication. Interestingly, E3 found that, if you realize the
limitation through trial and error with your own avatars, reading emotions
through avatars was “quite similar to reading faces. Not exactly at that level
but it did not require extra cognitive effort.” While a few participants were
surprised by how “big Memoji heads were” (A3), and that “having big heads
would make it easier to see who’s at least listening from nodding and what
not” (A4), other participants shared contrasting viewpoints in the movement of
head or nodding. Many participants started relying more on audio in both
delivering and receiving messages due to the limitation of expressing subtle
non-verbal cues when using avatars, as G2 mentioned, “the only difference
would be the subtle nods that people make when they’re talking, which we
couldn’t see through the Memoji avatars. It was hard. You had to even say that
you were still thinking, even if it was as simple as saying yes.” Moreover,
participants mentioned the importance of accurately perceiving the emotions of
others in collaborative activities, for example:

> “I think, especially when you have the Memoji on, it is hard to kind of
> gauge the person’s true feelings about any particular thing. And because
> it’s very subtle, I think the changes in expressions, especially if you are
> looking on another screen, it’s barely noticeable if [you are] not paying
> close attention. So, especially in a collaborative activity, I think it is
> important to know everyone’s true thoughts about a certain thing because I
> think that can help you arrive at a more accurate decision.” (G1)

In this case, avatars were seen as obscuring genuine emotions under the
augmented 3D digital information. F2 also faced challenges in interpreting
emotions through the avatars, saying, “I feel like it’s harder to see their
expressions. So sometimes I’m not really sure if they’re thinking or if the
Memoji is simply not reacting.” Similarly, D4 expressed her difficulty in
discerning people’s emotions through avatars, “I just sense that the pre-
scripted remote responses often transform a small smile into the cheesiest
grin you’ve ever seen. I don’t know, it all feels somewhat off, and I couldn’t
read people as well.” Being able to decode the subtleties of facial
expressions in group settings is a core to collaborative interactions because
“we express our agreements, disagreements, and hunches with our faces” (F4).
It is more challenging to connect and comprehend what others are thinking
without being able to see their actual faces. Likewise, F1 echoed the
significance of visual cues and facial expressions, particularly in time-
sensitive activities:

> “It was harder to tell how other people were feeling, I guess. Like, I
> couldn’t really connect as well with my teammates. I think being able to see
> people’s faces is really important, especially if you are working on
> something that’s timed. And you kind of need all the information there in
> front of you, including visual cues.” (F1)

Even though these avatars may accentuate expressions, they often fall short in
conveying subtle emotional cues. This deficiency can make it difficult to
accurately gauge the thoughts and emotions of others, subsequently impacting
the quality of communication, connection, and mutual understanding among group
members.

####  4.5.3. Avatar Presence and Authentic Self-Expression

Avatar fidelity becomes a pivotal factor in creating a truly authentic
emotional expression. Participants recognized an inherent difficulty in
ensuring that avatars authentically mirrored their emotions and identities,
introducing a tension in video-based collaborative interactions. D1 pointed
out that “it is weird when the Memoji [avatars] don’t really look exactly like
the person … it just doesn’t feel as authentic as just talking face to face.
It almost feels like they’re not really there [but] more like their voice
coming out of an avatar.” When technology tries to replicate human identity,
even slight deviations from authenticity can disrupt the sense of genuine
presence. This sentiment reveals an underlying uneasiness, a feeling that the
other person on the screen is not truly present. E4 mentioned a unique aspect
of the desire for authentic human presence in which it is not always about
replicating one’s appearance, faithfully. Instead, E4 suggested that
sometimes, embracing a whimsical and unconventional avatar absurdly fosters a
more authentic sense of self-expression:

> “From my past experience with avatars, I’m more likely to use the animal-
> like ones because I feel like they really emphasize the overall goofiness of
> me. I feel like using the one that looks exactly like me feels a little off
> because like I’m right here. But if I could look like a giraffe, that’s kind
> of fun.” (E4)

This suggests that embodying an avatar that looks different than your actual
self might allow individuals to drop their guards and reveal their genuine
selves. Authenticity does not always stem from literal representations; at
times, it emerges from the freedom to be unfiltered. For authenticity in our
mediated interactions, technology should complement, not supplant, our ability
to create profound and meaningful connections.

###  4.6. Influential Factors for Avatar Adoption and Usage

In our investigation of the influential factors in the likelihood to use
avatars within video-based collaborative contexts, we discovered several key
insights that provide a nuanced understanding of the adoption and usage of
avatars. Participants were specifically asked whether they would be likely to
use an avatar while video conferencing if others were also using avatars, if
the avatar could be easily customized before joining a video conference, and
if the avatar represented facial expressions with greater accuracy,
respectively.

####  4.6.1. Social Influence

Social influence emerged as a pivotal determinant, with unanimous agreement
from all 32 participants (100%) that they would be more likely to use an
avatar in a video-based collaboration if others were also using them. The
consensus highlighted the significance of social dynamics and peer influence
in shaping an individual’s inclination to use avatars during collaborative
endeavors. This unanimous agreement highlights the powerful impact of social
dynamics on individuals’ choices, emphasizing the collective nature of
technology adoption within collaborative settings.

####  4.6.2. Customization Ability

With 31 out of 32 participants (97%) expressing agreement that they would be
more likely to use an avatar if they could easily customize it before joining
a video conference, the ability to personalize and tailor avatars to align
with individual preferences and identities was strongly valued. The degree of
personalization provided a sense of agency and ownership over one’s virtual
representation, which, in turn, influenced the likelihood of adopting avatars
in collaborative settings. One participant, however, thought otherwise and
mentioned, “I disagree just because I would prefer to actually just not use an
avatar at all” (H3). Despite the general preference for avatar customization,
individual preferences and views towards the use of technology could vary
significantly, emphasizing the diverse perspectives regarding the adoption of
avatars in video-based collaboration settings.

####  4.6.3. Accuracy in Emotion Expression

Across all participant groups, 28 out of 32 participants (88%) agreed that
they would be more likely to use an avatar if it accurately represented their
facial expressions, acknowledging the critical role of accuracy in emotion
expression as a determinant for avatar usage. The ability of avatars to
accurately and authentically mirror emotions at micro level was considered
instrumental in conveying the subtleties of non-verbal communication. This
factor played a significant role in shaping users’ perceptions of the efficacy
of avatars in fostering genuine human connections in virtual spaces. As
participant B3 aptly put it, “I mean like if it could pick up the exact level
of expression, that would be ideal.” This sentiment highlights the
participants’ desire for avatars that excel in portraying emotions with the
utmost precision.

However, one group (n = 4) held a dissenting opinion and argued that pure
faces, unaltered by technology, were supposedly more accurate in conveying the
intended emotion expression. For instance, H1 expressed a preference for using
traditional video representation instead of an avatar with greater emotion
expression accuracy, “I disagree. Yeah, if I am intending on having a real
conversation, I would probably just want my face” (H1). This group believed
that technology reduces the authenticity of emotional expression and that
human interaction in its natural form was the superior medium for accurate
emotional conveyance.

####  4.6.4. Contextual Appropriateness of Avatar Use

Participants imagined possible scenarios to speculate about the
appropriateness of using avatars in a video-mediated environment, considering
their suitability in different contexts. It was evident that the acceptability
of avatars depended on the nature of the interaction. Participants
acknowledged that avatars were more aligned with casual or informal settings,
but the extent to which they were deemed suitable in professional contexts
varied.

In organizations that embrace tradition and formality, avatars may be
perceived as less suitable. Some participants questioned the appropriateness
of avatar use in such environments because it might appear “silly to discuss
serious topics, such as financial or healthcare matters” (A3). In addition,
avatars are deemed as better suited for light-hearted interactions or as a
means to infuse fun into conversations within a familiar context. Participants
noted that avatars are “perfect for a good night out with friends in FaceTime
events” (F2) and that they would use avatars with those whom they feel
comfortable, such as “sisters, mom, and dad, as they feel more casual than
formal” (F4).

Some participants aptly highlighted the pivotal role of organizational norms
and practices on the acceptability of avatars in video-based collaborations.
For example, A2 suggested that “if the team is more traditional or formal,
using avatars might not be appropriate. But using avatars in the team that
values creativity and innovation could be fun and engaging.” Moreover,
participants noted that the use of avatars in more formal work-related
situations is viewed with skepticism, as the presence of avatars can make it
challenging to convey and interpret serious or critical information. For
example, B2 imagined interacting with a manager who uses an avatar
representation while assigning work and providing instructions:

> “[The avatar might be] shouting at you, but [with] a very green hair mohawk-
> style Memoji, [which] would obviously be funny. I wouldn’t be able to take
> their critique. No matter how much [the avatar] replicates their real
> expressions, [like] anger, frowning, and everything, I would still think
> it’s adorable.” (B2)

The appropriateness of using avatars in video-mediated environments is
contingent on factors such as the nature of the interaction. While there was
consensus that avatars are well-suited for casual or informal settings, the
tension between the perceived unseriousness and the potential for enhancing
engagement and creativity contribute to the evolving nature of video-based
communication norms, where context plays a key role in shaping the acceptance
of avatars in collaborative interactions.

##  5\. Discussion

Our findings highlight the impact of avatar representation on self-esteem and
collaboration satisfaction, as compared to traditional video representation,
as well as the benefits, challenges, and influential factors to consider when
using avatars as user representations. Therefore, we discuss the implications
for avatar adoption and the evolving role of avatars in mediating the video-
based collaborative interaction experience. In addition, we propose an updated
user representation comparison with key characteristics.

###  5.1. Avatar Enhancing Video-Based Collaborative Interaction Experiences

Characterized by large effect sizes, our findings emphasize the notable
positive impact of avatars on self-esteem and collaboration satisfaction
during video-based collaborative interactions. The elevated self-esteem among
participants in the avatar condition suggests that the choice of user
representation has a profound impact on self-perception, influencing
participant engagement and experiences in collaborative activities. In
addition, the avatar representation positively affects the satisfaction
experienced by participants of both the collaborative process and decision-
making, contributing to multiple aspects of satisfaction such as increased
collaboration efficiency, coordination, fairness, understanding, contentment
with final group decisions, greater commitment to those decisions, and higher
levels of trust in their correctness.

The distinction in the mechanisms driving higher self-esteem and increased
collaboration satisfaction suggests that these two outcomes may be influenced
by different factors, providing an explanation for their lack of correlation
as demonstrated in Section 4.3. Drawing on prior research, the elevated self-
esteem might be attributed to the overall cute-looking visual of the avatars,
aligning with the Proteus effect’s proposition that physical characteristics
and anticipated behaviors can influence the behavior and cognition of
individuals (Liu, 2023; Praetorius and Görlich, 2021; Yee and Bailenson,
2007). In contrast, the enhanced collaboration satisfaction could be
associated with a degree of visual consistency among avatars across
participants, creating a higher sense of group or shared identity. This aligns
with the social identity theory (Ashforth and Mael, 1989), which posits that
similarity in representations contributes to the formation of a shared social
identity and a sense of belonging within a collaborative setting.

While the Proteus effect (Yee and Bailenson, 2007) was not the primary focus
of our study, it provides additional insights into the underlying rationale
behind our qualitative findings. Participants initially showed a positive
inclination towards using avatars, likely influenced by their agreement to
participate in the study. However, they reported favorable collaborative
experiences with avatars and expressed openness to using them in the future
during the post-study group interview. This positive reaction may be related
to the alignment of participants’ facial expressions with the perceived
friendly and cute attributes of their avatars, adhering to societal
expectations.

###  5.2. Avatars as a Middle Ground Between Camera On and Off

The decision of whether to activate or deactivate one’s camera in
videoconferencing has become a pivotal aspect of video-based interactions.
Balancing between the need for privacy and the necessity of being
“presentable,” offering visual cues, remains a considerable challenge. Some
recent research on Zoom fatigue (Bailenson, 2021; Ratan et al., 2022; Wicks,
2021) proposed avatars as a replacement for the camera-off modality. This
suggestion is supported by previous work comparing different user
representation modalities (Inkpen and Sedlins, 2011; Junuzovic et al., 2012;
Panda et al., 2022). Although the context in which an avatar is employed
remains one of the major considerations, the use of avatars can ultimately
enhance the quality of video-based communication when compared to audio-only
modality. Some participants spoke up about the concept of avatars as a middle
ground between the binary choices of having the camera either on or off,
presenting a balanced and effective method of social connection:

> “It was more like, you know that they’re present, you know that they’re
> there, but you can’t really make out like everything because their faces are
> animated, obviously. So I thought that was kind of nice for times when you
> don’t really want to have your camera on but you [would] just like to have a
> conversation.” (H1)

While avatars may not fully replicate the nuanced spectrum of human
expressions, they provide a degree of visual interaction that would otherwise
be absent, with the animated facial features and expressions of avatars
maintaining a visual link and signifying the presence of the conversation
partner without necessitating full camera engagement.

> “So, I know a lot of times, especially since what you’re working virtually
> and remotely has become a thing, a lot of people tend to keep their cameras
> off. So I think [avatars] could be a good alternative. Rather than having no
> access to somebody’s facial expressions, I guess this could kind of bring in
> that element to some extent at least.” (C1)

While a blank screen offers a degree of anonymity, avatars provide a more
engaging and personalized representation, potentially leading to elevated
self-esteem and increased satisfaction. However, it is imperative to conduct a
thoughtful assessment of the suitability of avatars within their specific
contexts, taking into account factors such as company norms, the nature of the
work, and individual preferences. The acknowledgment of avatars as a middle
ground speaks to the evolving nature of video-based collaboration preferences
and the importance of flexibility and customization in communication tools.
Avatars serve as dynamic elements that facilitate individual agency in shaping
the virtual collaborative experience by bridging the divide between an active
camera and a deactivated camera screen.

###  5.3. Self-Prioritization in Collaborative Interactions

The observation of self-prioritization within collaborative interactions using
avatars departs significantly from prior research. Existing studies (Ferrer
and Fujiwara, 2022; Ratan et al., 2022; Wicks, 2021) suggested that avatars,
as representations of individuals within digital environments that mediate the
real-world physical appearance and actions, diverted attention from the self.
This redirection of focus was perceived to promote shared attention and group
engagement while reducing self-consciousness. However, our study incorporated
avatars capable of accurately tracking facial expressions without obscuring
the physical surroundings, revealing a discernible paradigm shift.
Participants emphasized concerns regarding the presentation of their own
representation (i.e., avatar) rather than that of others, indicating an
unconscious shift toward self-prioritization, even within a group setting with
shared, goal-directed activities.

Notably, this phenomenon occurred within the context of collaborative
interactions among small groups of individuals with pre-established
relationships (i.e., friends or colleagues), contrary to the anticipated
collective focus facilitated by pre-existing social connections and
familiarity (Sainsbury and Walker, 2009). Interview findings in Liu et al.
(Liu et al., 2018) indicate that emotional feedback facilitated through a
mediator, namely an avatar in this study, contributed to a heightened self-
awareness of one’s emotions, particularly with regard to inadvertent concerns
about negative emotion leakage. This provides a plausible explanation for the
observed apprehension of own facial expressions in this study. Additionally,
in line with the Proteus effect (Yee and Bailenson, 2007), employing avatars,
unfortunately, places a potential burden on users to express positive or
“happy” emotions aligned with the avatar’s cute-looking design, thereby
introducing a coercion process where user agency is cajoled by the mediator.
The design and implementation strategy of avatars in various collaborative
settings should take into consideration the mindful balance between specific
mechanisms and triggers behind self-prioritization in avatar-mediated
interactions and the need to maintain collective engagement and group
cohesion.

###  5.4. Toward a Comparison of User Representations

Originally introduced by Schroeder (Schroeder, 2011), the key characteristics
compare and contrast the mediated interactions of video and avatar (i.e., VR
avatar). However, AR avatar is another modality that has been widely adopted
due to its accessibility via mobile and screen-based devices (Phadnis et al.,
2023). The avatar technology used in our study is representative of a new
modality in real-time, video-based environments. Drawing from our experience
observing, interacting with, and researching AR avatars in video-mediated
interactions, we propose an updated comparison of user representations (Table
3).

Table 3. Comparison of key characteristics of video, VR avatar, and AR avatar user representations (adapted from (Schroeder, 2011)). Characteristic | Video | VR Avatar | AR Avatar
---|---|---|---
Representation | True to life | Constructed but can | Certain parts of user are
|  | approach realism | constructed, blending real
|  |  | and augmented elements
|  |  | to approach realism
Body capabilities | Limited to physical ones | “Super powers” | Limited to physical ones
Environment | Typically limited to context | From room-size | Typically limited to context
apart from user | of head and shoulders | spaces to vast | of head and shoulders
representation | (i.e., table and room) | interconnected worlds | (i.e., table and room)
Interpersonal | Focus on facial expressions | Focus on spatial | Focus on facial expressions
interaction |  | encounters with bodies |
Number of | Two - to small groups | Two - to large populations | Two - to small groups
participants | (though still focused |  |
| on the group) |  |
Advantage | Few concerns | Plasticity of | Flexible augmentation
| about deception | self-presentation | of self-presentation
|  |  | in realism & privacy
Disadvantage | Concerns about one’s | Inability to come | Concerns about augmented
| appearance (can be avoided | across as oneself | self-representation
| with picture-in-picture) |  | perceived by others

We envision that this comparison could be valuable to researchers, video
conferencing platform developers, and avatar designers. The current literature
primarily targets VR avatars (Aufegger and Elliott-Deflo, 2022; Bartl et al.,
2022; Dobre et al., 2022; Grassini and Laumann, 2020; Hudson and Hurter, 2016;
Xie et al., 2021) and often use and generalize the term “avatars” when
mentioning VR avatars alone, thereby blurring the distinctions between the VR
and AR avatar representations. This proposed comparison serves as a taxonomy
to systematically understand the different characteristics in user
representations. Furthermore, this comparison could guide the future design
and development of AR avatars, complementing the ongoing efforts in system
development of VR avatars. This, in turn, has the potential to stimulate an
increase in empirical research on AR avatars in various contexts, such as
video-mediated interactions, activity-directed scenarios, and collaborative
settings.

##  6\. Limitations and Future Research

Although our work presents new opportunities for exploring the impact of
avatar representation on video-mediated collaborative interactions, it is
important to address current limitations and outline potential future research
directions. First, our design of game-based maps and activities relied heavily
on collective knowledge and familiarity with the maps, which limited our
participant demographics to the associated university students. To broaden the
scope, future studies could include other types of collaborative activities
that cater to a more diverse participant base. Furthermore, we examined only
two user representations (i.e., avatars and video). Future research should
consider involving multiple modalities, for instance, avatar, video, audio,
and even mixed-modalities that combine these representations. Given the
technology constraints, our study required two separate screens per
participant, potentially leading participants to focus less on one of the
screens or increasing mental workload. Future research could invest in the
development of integrated avatar features for video-based platforms, such as
Zoom, to eliminate the need for additional screens and open up the possibility
to assess performance. The design of avatars, too, plays a significant role in
shaping participants’ perceptions. It is imperative for future research to not
only recognize the impact of distinct avatar design choices but also to remain
attentive to potential exclusions inherent in avatar development tools.

##  7\. Conclusion

This study explores an untapped research area of how the use of screen-based
or AR avatars as a user representation affects collaborative interactions in
synchronous video-based environments when tasked with goal-oriented, group
activities. Our experimental findings indicate that the avatar representation
significantly enhances self-esteem and video-based collaborative interaction
experience, compared to the traditional video representation. In addition, we
identify benefits (e.g., increased self-confidence, participation engagement,
comfort of physical self, and decreased social tension and awkwardness),
challenges (e.g., authenticity in expressing and perceiving emotions), and key
factors for avatar use within video-based collaboration contexts. This work
provides empirical insights into how avatar representation mediates
collaborative interactions in video-based settings as well as implications and
discussions concerning the potential future use of avatars, offering a
comprehensive comparison of essential features among user representations.

###### Acknowledgements.

The authors would like to thank Gary Hsieh and Daniela Rosner for their
thoughtful feedback on the paper draft; Julie Vera, Joseph Schafer, and Soobin
Cho for their feedback on an initial draft of the Methods and Quantitative
Results sections; and Sophie Park, Connie Yang, Jinhan Wu, Cassandra Johnson,
Nikta Farahani, and Parniyan Rahbar for their dedicated effort in the early
iterations of game-based map element design and assistance in piloting the
study.

## References

  * (1)
  * Abramczuk et al. (2023) Katarzyna Abramczuk, Zbigniew Bohdanowicz, Bartosz Muczyński, Kinga H Skorupska, and Daniel Cnotkowski. 2023.  Meet me in VR! Can VR space help remote teams connect: A seven-week study with Horizon Workrooms.  _International Journal of Human-Computer Studies_ 179, Article 103104 (2023), 17 pages.  https://doi.org/10.1016/j.ijhcs.2023.103104
  * Asher and Aderka (2020) Maya Asher and Idan M Aderka. 2020.  Dating with social anxiety: An empirical examination of momentary anxiety and desire for future interaction.  _Clinical Psychological Science_ 8, 1 (2020), 99–110.  https://doi.org/10.1177/2167702619867055
  * Ashforth and Mael (1989) Blake E Ashforth and Fred Mael. 1989.  Social identity theory and the organization.  _Academy of Management Review_ 14, 1 (1989), 20–39.  https://doi.org/10.5465/amr.1989.4278999
  * Aufegger and Elliott-Deflo (2022) Lisa Aufegger and Natasha Elliott-Deflo. 2022.  Virtual reality and productivity in knowledge workers.  _Frontiers in Virtual Reality_ 3, Article 890700 (2022), 10 pages.  https://doi.org/10.3389/frvir.2022.890700
  * Azriel et al. (2020) Omer Azriel, Amit Lazarov, Adva Segal, and Yair Bar-Haim. 2020.  Visual attention patterns during online video-mediated interaction in socially anxious individuals.  _Journal of Behavior Therapy and Experimental Psychiatry_ 69, Article 101595 (2020), 9 pages.  https://doi.org/10.1016/j.jbtep.2020.101595
  * Bailenson (2021) Jeremy N Bailenson. 2021.  Nonverbal overload: A theoretical argument for the causes of Zoom fatigue.  _Technology, Mind, and Behavior_ 2, 1 (2021), 6 pages.  https://doi.org/10.1037/tmb0000030
  * Bartl et al. (2022) Andrea Bartl, Christian Merz, Daniel Roth, and Marc Erich Latoschik. 2022.  The effects of avatar and environment design on embodiment, presence, activation, and task load in a virtual reality exercise application. In _2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)_. IEEE, 260–269.  https://doi.org/10.1109/ISMAR55827.2022.00041
  * Bente et al. (2008) Gary Bente, Sabine Rüggenberg, Nicole C Krämer, and Felix Eschenburg. 2008.  Avatar-mediated networking: Increasing social presence and interpersonal trust in net-based collaborations.  _Human Communication Research_ 34, 2 (2008), 287–318.  https://doi.org/10.1111/j.1468-2958.2008.00322.x
  * Bonfert et al. (2023) Michael Bonfert, Anke V Reinschluessel, Susanne Putze, Yenchin Lai, Dmitry Alexandrovsky, Rainer Malaka, and Tanja Döring. 2023.  Seeing the faces is so important—Experiences from online team meetings on commercial virtual reality platforms.  _Frontiers in Virtual Reality_ 3, Article 945791 (2023), 19 pages.  https://doi.org/10.3389/frvir.2022.945791
  * Braun and Clarke (2006) Virginia Braun and Victoria Clarke. 2006.  Using thematic analysis in psychology.  _Qualitative Research in Psychology_ 3, 2 (2006), 77–101.  https://doi.org/10.1191/1478088706qp063oa
  * Cohen (1988) Jacob Cohen. 1988.  _Statistical power analysis for the behavioral sciences_.  Lawrence Erlbaum Associates.  https://doi.org/10.4324/9780203771587
  * Dobre et al. (2022) Georgiana Cristina Dobre, Marta Wilczkowiak, Marco Gillies, Xueni Pan, and Sean Rintel. 2022.  Nice is different than good: Longitudinal communicative effects of realistic and cartoon avatars in real mixed reality work meetings. In _CHI Conference on Human Factors in Computing Systems Extended Abstracts_. Association for Computing Machinery, Article 437, 7 pages.  https://doi.org/10.1145/3491101.3519628
  * Falzone et al. (2005) Richard L Falzone, Sarah Hall, and Eugene V Beresin. 2005.  How and why for the camera-shy: Using digital video in psychiatry.  _Child and Adolescent Psychiatric Clinics_ 14, 3 (2005), 603–612.  https://doi.org/10.1016/j.chc.2005.02.006
  * Ferrer and Fujiwara (2022) Cesar Daniel Rojas Ferrer and Takayuki Fujiwara. 2022.  Work-in-progress—avatar vs teams: Co-presence and visual behavior in a videoconference cooperative task. In _2022 8th International Conference of the Immersive Learning Research Network (iLRN)_. IEEE, 1–3.  https://doi.org/10.23919/iLRN55037.2022.9815938
  * Gonzalez-Franco et al. (2020a) Mar Gonzalez-Franco, Zelia Egan, Matthew Peachey, Angus Antley, Tanmay Randhavane, Payod Panda, Yaying Zhang, Cheng Yao Wang, Derek F Reilly, Tabitha C Peck, et al. 2020a.  Movebox: Democratizing mocap for the microsoft rocketbox avatar library. In _2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)_. IEEE, 91–98.  https://doi.org/10.1109/AIVR50618.2020.00026
  * Gonzalez-Franco et al. (2020b) Mar Gonzalez-Franco, Anthony Steed, Steve Hoogendyk, and Eyal Ofek. 2020b.  Using facial animation to increase the enfacement illusion and avatar self-identification.  _IEEE Transactions on Visualization and Computer Graphics_ 26, 5 (2020), 2023–2029.  https://doi.org/10.1109/TVCG.2020.2973075
  * Grassini and Laumann (2020) Simone Grassini and Karin Laumann. 2020.  Evaluating the use of virtual reality in work safety: A literature review. In _Proceedings of the 30th European Safety and Reliability Conference and the 15th Probabilistic Safety Assessment and Management Conference_. Research Publishing Services, 3975.  https://hdl.handle.net/11250/2824916
  * Green and Taber (1980) Stephen G Green and Thomas D Taber. 1980.  The effects of three social decision schemes on decision group process.  _Organizational Behavior and Human Performance_ 25, 1 (1980), 97–106.  https://doi.org/10.1016/0030-5073(80)90027-6
  * Hudson and Hurter (2016) Irwin Hudson and Jonathan Hurter. 2016.  Avatar types matter: Review of avatar literature for performance purposes. In _Virtual, Augmented and Mixed Reality: 8th International Conference_. Springer, 14–21.  https://doi.org/10.1007/978-3-319-39907-2_2
  * Inkpen and Sedlins (2011) Kori M Inkpen and Mara Sedlins. 2011.  Me and my avatar: Exploring users’ comfort with avatars for workplace communication. In _Proceedings of the ACM 2011 Conference on Computer Supported Cooperative Work_. Association for Computing Machinery, 383–386.  https://doi.org/10.1145/1958824.1958883
  * Junuzovic et al. (2012) Sasa Junuzovic, Kori Inkpen, John Tang, Mara Sedlins, and Kristie Fisher. 2012.  To see or not to see: A study comparing four-way avatar, video, and audio conferencing for work. In _Proceedings of the 2012 ACM International Conference on Supporting Group Work_. Association for Computing Machinery, 31–34.  https://doi.org/10.1145/2389176.2389181
  * Li et al. (2021) Jie Li, Shishir Subramanyam, Jack Jansen, Yanni Mei, Ignacio Reimat, Kinga Ławicka, and Pablo Cesar. 2021.  Evaluating the user experience of a photorealistic social VR movie. In _2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)_. IEEE, 284–293.  https://doi.org/10.1109/ISMAR52148.2021.00044
  * Lin et al. (2021) Muyu Lin, Xu Wen, Mingyi Qian, Dongjun He, and Armin Zlomuzica. 2021.  Self-focused attention vs. negative attentional bias during public speech task in socially anxious individuals.  _Behaviour Research and Therapy_ 136 (2021), 103766.  https://doi.org/10.1016/j.brat.2020.103766
  * Liu et al. (2018) Miki Liu, Austin Wong, Ruhi Pudipeddi, Betty Hou, David Wang, and Gary Hsieh. 2018.  ReactionBot: Exploring the effects of expression-triggered emoji in text messages.  _Proceedings of the ACM on Human-Computer Interaction_ 2, CSCW, Article 110, 16 pages.  https://doi.org/10.1145/3274379
  * Liu (2023) Yansheng Liu. 2023.  The Proteus effect: Overview, reflection, and recommendations.  _Games and Culture_ (2023).  https://doi.org/10.1177/15554120231202175
  * Liu et al. (2020) Yuzhao Liu, Yuhan Liu, Shihui Xu, Kelvin Cheng, Soh Masuko, and Jiro Tanaka. 2020.  Comparing vr-and ar-based try-on systems using personalized avatars.  _Electronics_ 9, 11, Article 1814 (2020), 25 pages.  https://doi.org/10.3390/electronics9111814
  * Makled et al. (2022) Elhassan Makled, Florian Weidner, and Wolfgang Broll. 2022.  Investigating user embodiment of inverse-kinematic avatars in smartphone augmented reality. In _2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)_. IEEE, 666–675.  https://doi.org/10.1109/ISMAR55827.2022.00084
  * Maloney and Freeman (2020) Divine Maloney and Guo Freeman. 2020.  Falling asleep together: What makes activities in social virtual reality meaningful to users. In _Proceedings of the Annual Symposium on Computer-Human Interaction in Play_. Association for Computing Machinery, 510–521.  https://doi.org/10.1145/3410404.3414266
  * McDonald et al. (2019) Nora McDonald, Sarita Schoenebeck, and Andrea Forte. 2019.  Reliability and inter-rater reliability in qualitative research: Norms and guidelines for CSCW and HCI practice.  _Proceedings of the ACM on Human-Computer Interaction_ 3, CSCW, Article 72 (2019), 23 pages.  https://doi.org/10.1145/3359174
  * Moustafa and Steed (2018) Fares Moustafa and Anthony Steed. 2018.  A longitudinal study of small group interaction in social virtual reality. In _Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology_. Association for Computing Machinery, Article 22, 10 pages.  https://doi.org/10.1145/3281505.3281527
  * Nowak and Fox (2018) Kristine L Nowak and Jesse Fox. 2018.  Avatars and computer-mediated communication: A review of the definitions, uses, and effects of digital representations.  _Review of Communication Research_ 6 (2018), 30–53.  https://doi.org/10.12840/issn.2255-4165.2018.06.01.015
  * Pan and Steed (2017) Ye Pan and Anthony Steed. 2017.  The impact of self-avatars on trust and collaboration in shared virtual environments.  _PloS One_ 12, 12 (2017), e0189078.  https://doi.org/10.1371/journal.pone.0189078
  * Panda et al. (2022) Payod Panda, Molly Jane Nicholas, Mar Gonzalez-Franco, Kori Inkpen, Eyal Ofek, Ross Cutler, Ken Hinckley, and Jaron Lanier. 2022.  AllTogether: Effect of avatars in mixed-modality conferencing environments. In _2022 Symposium on Human-Computer Interaction for Work_. Association for Computing Machinery, 1–10.  https://doi.org/10.1145/3533406.3539658
  * Park et al. (2023) Inyoung Park, Young June Sah, Suhyun Lee, and Daeho Lee. 2023.  Avatar-mediated communication in video conferencing: Effect of self-affirmation on debating participation focusing on moderation effect of avatar.  _International Journal of Human–Computer Interaction_ 39, 3 (2023), 464–475.  https://doi.org/10.1080/10447318.2022.2041897
  * Phadnis et al. (2023) Vrushank Phadnis, Kristin Moore, and Mar Gonzalez-Franco. 2023.  The work avatar face-off: Knowledge worker preferences for realism in meetings. In _2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)_. IEEE, 960–969.  https://doi.org/10.1109/ISMAR59233.2023.00112
  * Praetorius and Görlich (2021) Anna Samira Praetorius and Daniel Görlich. 2021.  The Proteus effect: How avatars influence their users’ self-perception and behaviour.  In _Augmented Reality and Virtual Reality: New Trends in Immersive Technology_. Springer, 109–122.  https://doi.org/10.1007/978-3-030-68086-2_9
  * Ramasubramanian (2020) Sowmya Ramasubramanian. 2020.  Over 70% suffered from ‘Zoom Anxiety’ this year, survey finds.  _The Hindu_ (2020).  https://thehindu.com/sci-tech/technology/zoom-anxiety-in-2020/article33277870.ece
  * Ratan et al. (2022) Rabindra Ratan, Dave B Miller, and Jeremy N Bailenson. 2022.  Facial appearance dissatisfaction explains differences in Zoom fatigue.  _Cyberpsychology, Behavior, and Social Networking_ 25, 2 (2022), 124–129.  https://doi.org/10.1089/cyber.2021.0112
  * Reinig (2003) Bruce A Reinig. 2003.  Toward an understanding of satisfaction with the process and outcomes of teamwork.  _Journal of Management Information Systems_ 19, 4 (2003), 65–83.  https://doi.org/10.1080/07421222.2003.11045750
  * Rosenberg (1965) Morris Rosenberg. 1965.  _Society and the adolescent self-image_.  Princeton University Press.
  * Sainsbury and Walker (2009) Erica Sainsbury and Richard Walker. 2009.  Motivation, learning and group work–the effect of friendship on collaboration. In _Proceedings of the Australian Conference on Science and Mathematics Education_. The University of Sydney, 118–124.  https://openjournals.library.sydney.edu.au/IISME/article/view/6213
  * Saredakis et al. (2020) Dimitrios Saredakis, Ancret Szpak, Brandon Birckhead, Hannah AD Keage, Albert Rizzo, and Tobias Loetscher. 2020.  Factors associated with virtual reality sickness in head-mounted displays: A systematic review and meta-analysis.  _Frontiers in Human Neuroscience_ 14, Article 96 (2020), 17 pages.  https://doi.org/10.3389/fnhum.2020.00096
  * Schroeder (2002) Ralph Schroeder. 2002.  _Social interaction in virtual environments: Key issues, common themes, and a framework for research_.  Springer, 1–18.  https://doi.org/10.1007/978-1-4471-0277-9_1
  * Schroeder (2011) Ralph Schroeder. 2011.  _Comparing avatar and video representations_.  Springer, 235–251.  https://doi.org/10.1007/978-0-85729-361-9_12
  * Sinlapanuntakul et al. (2023) Pitch Sinlapanuntakul, Jenna Korentsides, and Barbara S Chaparro. 2023.  Exploring the user experience (UX) of a multi-window augmented reality environment.  _Frontiers in Virtual Reality_ 4, Article 1194019 (2023), 8 pages.  https://doi.org/10.3389/frvir.2023.1194019
  * Sinlapanuntakul et al. (2022) Weerachet “Pitch” Sinlapanuntakul, Jessyca L Derby, and Barbara S Chaparro. 2022.  Understanding the effects of mixed reality on video game satisfaction, enjoyment, and performance.  _Simulation & Gaming_ 53, 3 (2022), 237–252.  https://doi.org/10.1177/10468781221094473
  * Suda and Oka (2021) Mikihiro Suda and Mizuki Oka. 2021.  Evaluation of the effect of mimicry on facial expression in avatar-mediated communication. In _Artificial Life Conference_. MIT Press, Article 48, 9 pages.  https://doi.org/10.1162/isal_a_00394
  * Vishwakarma et al. (2020) Pankaj Vishwakarma, Srabanti Mukherjee, and Biplab Datta. 2020.  Travelers’ intention to adopt virtual reality: A consumer value perspective.  _Journal of Destination Marketing & Management_ 17, Article 100456 (2020).  https://doi.org/10.1016/j.jdmm.2020.100456
  * Vriends et al. (2017) Noortje Vriends, Yasemin Meral, Javier A Bargas-Avila, Christina Stadler, and Susan M Bögels. 2017.  How do I look? Self-focused attention during a video chat of women with social anxiety (disorder).  _Behaviour Research and Therapy_ 92 (2017), 77–86.  https://doi.org/10.1016/j.brat.2017.02.008
  * Wicks (2021) David A Wicks. 2021.  _Minimizing Zoom fatigue and other strategies for a successful synchronous class experience_.  Cambridge Scholars Publishing, 2–24.
  * Wu et al. (2021) Yuanjie Wu, Yu Wang, Sungchul Jung, Simon Hoermann, and Robert W Lindeman. 2021.  Using a fully expressive avatar to collaborate in virtual reality: Evaluation of task performance, presence, and attraction.  _Frontiers in Virtual Reality_ 2, Article 641296 (2021), 15 pages.  https://doi.org/10.3389/frvir.2021.641296
  * Xie et al. (2021) Biao Xie, Huimin Liu, Rawan Alghofaili, Yongqi Zhang, Yeling Jiang, Flavio Destri Lobo, Changyang Li, Wanwan Li, Haikun Huang, Mesut Akdere, et al. 2021\.  A review on virtual reality skill training applications.  _Frontiers in Virtual Reality_ 2, Article 645153 (2021), 19 pages.  https://doi.org/10.3389/frvir.2021.645153
  * Yee and Bailenson (2007) Nick Yee and Jeremy Bailenson. 2007.  The Proteus effect: The effect of transformed self-representation on behavior.  _Human Communication Research_ 33, 3 (2007), 271–290.  https://doi.org/10.1111/j.1468-2958.2007.00299.x
  * Yoon et al. (2019) Boram Yoon, Hyung-il Kim, Gun A Lee, Mark Billinghurst, and Woontack Woo. 2019.  The effect of avatar appearance on social presence in an augmented reality remote collaboration. In _2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)_. IEEE, 547–556.  https://doi.org/10.1109/VR.2019.8797719
  * Zibrek et al. (2018) Katja Zibrek, Elena Kokkinara, and Rachel McDonnell. 2018.  The effect of realistic appearance of virtual characters in immersive environments-does the character’s personality play a role?  _IEEE Transactions on Visualization and Computer Graphics_ 24, 4 (2018), 1681–1690.  https://doi.org/10.1109/TVCG.2018.2794638


§ INTRODUCTION

§ PROBLEM STATEMENT

§ THEORETICAL FRAMEWORK FOR LINEAR PROBES F

§ PRACTICAL ISSL OBJECTIVES FOR LINEAR PROBES F

§ ISSL FOR NON-LINEAR PREDICTORS F+

§ SUMMARY OF INSIGHTS AND RELATION TO PREVIOUS WORK

§ EXPERIMENTS

§ SUMMARY AND OUTLOOK

We presented a simple conceptual framework to understand, compare and improve ISSL methods.
Inspired by recent work on optimal representations for supervised <cit.> and robust <cit.> learning, we derived such a framework by studying   algorithmic-agnostic goals for ISSL.
In particular, <ref> provides the minimal and sufficient requirements for optimal encoders, and <ref> gives the resulting probing risk as a function of pretraining augmentations.
On the algorithmic side,
<ref> uses our framework to derive simpler and theoretically-motivated variants of prior SSL objectives.
Altogether our framework provides actionable insights for ISSL such as how to choose: the SSL algorithm, the dimensionality of the representations, the projection heads, and the augmentations.
On the empirical side, <ref> shows that each of our prescriptions leads to significant gains.
Our final SSL algorithms outperform strong baselines (SwAV+multicrop trained on ImageNet) on linear probing benchmarks.=-1

There are many limitations that should be addressed for a more realistic
prescriptive framework for ISSL.
See <ref> for more details.
First, our theory is binary, ,  augmentations are either label preserving or not, and encoders are optimal or not.
Although our framework likely captures the right intuition, nuance would be more realistic.
Second, we consider unconstrained encoders $\enc{}$, dimensionality, and infinite unlabeled data.
Such assumptions follow recent empirical trends in increasing data and models <cit.>,
but it means that our framework currently cannot improve the computational or data efficiency of the pretraining phase.
Finally, we consider all invariant tasks $\tasksinv{}$ even though only a subset of those are meaningful.
Optimal representations for this subset will be different, and defining this subset likely requires considering inductive biases of encoders <cit.>.
Despite those limitations, we
showed that our framework's insights lead to substantial improvements in ISSL performance.=-1

§ PRELIMINARIES

§.§ Notation

§.§ Assumptions

§.§ Definitions

§ PROOFS AND ADDITIONAL THEORETICAL RESULTS

§.§ Useful known lemmas

§.§ Linear optimal (Sec. 3.1)

§.§ Augmentations (Sec. 3.2)

§.§ ISSL Log Loss (Sec. 4 )

§.§ Non-linear optimality (Sec. 5)

§ PRACTICAL ISSL OBJECTIVES

§ RELATION TO PREVIOUS WORK

§ LIMITATIONS

§ REPRODUCIBILITY

§ ADDITIONAL EXPERIMENTAL RESULTS