# What Do Privacy Advertisements Communicate to Consumers?

Xiaoxin Shen  Carnegie Mellon UniversityPittsburghPennsylvaniaUSA
<EMAIL_ADDRESS>,  Eman Alashwali  King Abdulaziz University (KAU) and
King Abdullah University of Science and Technology (KAUST)Saudi Arabia
<EMAIL_ADDRESS>and  Lorrie Faith Cranor  Carnegie Mellon
UniversityPittsburghPennsylvaniaUSA<EMAIL_ADDRESS>
###### Abstract.

When companies release marketing materials aimed at promoting their privacy
practices or highlighting specific privacy features, what do they actually
communicate to consumers? In this paper, we explore the impact of privacy
marketing materials on:  (1) consumers’ attitude towards the organizations
providing the campaigns,  (2) overall privacy awareness, and  (3) the
actionability of suggested privacy advice . To this end, we investigated the
impact of four privacy advertising videos and one privacy game published by
five different technology companies. We conducted 24 semi-structured
interviews with participants randomly assigned to view one or two of the
videos or play the game. Our findings suggest that awareness of privacy
features can contribute to positive perceptions of a company or its products.
The ads we tested were more successful in communicating the advertised privacy
features than the game we tested. We observed that advertising a single
privacy feature using a single metaphor in a short ad increased awareness of
the advertised feature. The game failed to communicate privacy features or
motivate study participants to use the features. Our results also suggest that
privacy campaigns can be useful for raising awareness about privacy features
and improving brand image, but may not be the most effective way to teach
viewers how to use privacy features.

privacy awareness, privacy advertisements, user perception

††journalyear: 2024††journalvolume: 4††journalnumber: 2024††doi:
XXXXXXX.XXXXXXX

This document is the author’s manuscript for a paper to appear in Proceedings
on Privacy Enhancing Technologies 2024(4).

##  1\. Introduction

In recent years, an increasing number of companies have used their privacy
practices as a way of differentiating themselves from competitors. Apple
launched an advertisement focusing on the iPhone “Ask App Not to Track”
feature in May 2022 (O’Flaherty, 2022). Experts estimate this new iPhone
feature caused a $12 billion dollar decline in revenue for Meta (O’Flaherty,
2022). The advertisement shows the protagonist, Ellie, stumbling upon an
auction where her information is up for sale. As Ellie toggles various privacy
protection features on her iPhone, bidders begin to disappear until she is the
only one left in the auction room (see Figure 1, and Figure 10 in Appendix A.3
for illustration) (Apple, 2022). Apple is not alone in showcasing their
privacy protection features as a way to gain consumer trust; Samsung, another
smartphone giant, launched an advertisement with similar message in the same
year (Samsung, 2022). Google and WhatsApp also have ads highlighting their
security and privacy features (Google Ads, 2020; WhatsApp, 2022), and Twitter
released an old-school video game in an attempt to help users gain better
understanding of their privacy settings (Twitter, 2022).

While experts have praised some of these privacy campaigns (O’Flaherty, 2022;
Ulanoff, 2022) and criticized others (Hurlerand and Wodinsky, 2022; Winkie,
2022), it is unclear what consumers take away from these campaigns. In this
study we explore what consumers exposed to corporate privacy campaigns are
learning from these campaigns, and what impacts these campaigns have on
consumers’ privacy awareness and attitudes.

We conducted 24 semi-structured interviews to evaluate privacy marketing
materials released by tech firms (Twitter, Apple, Samsung, WhatsApp, and
DuckDuckGo) from a consumer perspective. We explored consumers’ attitudes
towards the organizations providing the campaign before and after exposure to
the campaigns, and each campaign’s impact on consumers’ overall privacy
awareness. We also evaluated the actionability of the privacy and security
advice suggested by the campaigns.

Our exploratory study suggests that awareness of privacy features can
contribute to positive perceptions of a company and its products. The ads we
tested were more effective than the game in communicating privacy features to
our participants. We observed that advertising a single privacy feature using
a single metaphor in a short ad increased awareness of the advertised feature
among our participants. The game performed poorly in both explaining privacy
features and encouraging user engagement with the features. Our results
suggest that privacy campaigns might not be the most effective method for
educating users on how to use privacy features. In addition, despite their
positive impressions, most non-users in our study were reluctant to switch
products after seeing ads due to concerns about switching costs, data
synchronization, and familiarity with their current operating system.
Nonetheless, over time, such campaigns may lead to more interest in a brand’s
products, and might be particularly effective for products and services with
low switching costs.

##  2\. Related Work

In this section, we outline previous research about video advertisements and
elements that may impact ad effectiveness. We also introduce research on
serious games related to privacy and security. Next, we discuss research on
privacy awareness and research on acceptance of privacy and security advice.

###  2.1. Effective Advertisements

In her book, Advertising by Design, Landa described three steps for building
any brand image using advertisement: “get people’s attention, keep their
attention, call them to action.” She added that video advertisements should
also be either entertaining or informative and “be interesting enough to be
viewed again or be shareworthy” (Landa, 2016).

Video advertisements contain scenes and imagery carefully crafted by
advertisers. Mohanty and Ratneshwar examined the factors that influence a
viewer’s subjective comprehension of visual metaphors in graphic ads (Mohanty
and Ratneshwar, 2015). The researchers found that incongruity, which is the
discrepancy between what is actually shown and what it is referring to, could
lead to poor comprehension (Mohanty and Ratneshwar, 2015). Mohanty and
Ratneshwar also conducted a later study investigating the effects of
incongruity on ad effectiveness, and found that a moderate level of
incongruity results in the most effective ads, recommending that advertisers
be cautious when selecting visual metaphors (Mohanty and Ratneshwar, 2016).

Some video ads are narrative ads that contain a story or plot. Kim et al.
found that narrative ads were effective because they generated emotive
responses, offered high entertainment value, and created credibility (Kim et
al., 2017). Laurence conducted a study researching the role of storytelling in
narrative ads and found that when used to induce positive emotions,
participants had more positive brand attitudes (Laurence, 2018). Manyiwa and
Ross looked at the impact of negative emotions in advertising and found that
participants with higher reported self-efficacy had more favourable views
towards ads that used fear (Manyiwa and Brennan, 2012).

Privacy marketing has the potential to reach a large audience but prior work
has not examined its effectiveness. Our research seeks to gain insights into
consumer privacy awareness by examining recent privacy marketing campaigns and
what consumers taking away from them. We investigated how the privacy campaign
impacted viewers’ impression of the organization and how well participants
understood privacy metaphors.

###  2.2. Privacy and Security Games

As one of the campaigns in our study was a web-based video game aimed at
better acquainting players with the organization’s privacy policy, we explored
previous research on privacy and security games. Whether tabletop or online,
games have long been used as educational tools.

Denning et al. designed “Control-Alt-Hack” as a classroom tabletop card game
aimed to increase security awareness. While participating educators reported
that the game increased levels of security awareness, critiques found the game
too hard to play and not very fun (Denning et al., 2013). Barnard-Wills and
Ashlenden received similar feedback on their card game “Privacy,” which
participants felt was too difficult to play. The researchers noted that
educational privacy games should not be designed as a “graphical skin overlaid
on [an] existing game” and recommended that the game mechanisms should
actively demonstrate the theoretical model of the “online privacy ecosystem,”
allowing players to better understand the privacy model (Barnard-Wills and
Ashenden, 2015).

Video games can be an effective way of communicating a simplified, simulated
model of any concept to a player (Persuasive Games, 2023). To educate users
about phishing, Sheng et al. developed “Anti-Phishing Phil,” and found games
to be an effective educational method (Sheng et al., 2007). The researchers
credited the effectiveness of the treatment to the interactive game content
(Sheng et al., 2007). Maqsood et al. investigated whether web based games
could help children aged 11 to 13 practice safer online behaviors. They found
significant improvements in knowledge and intended behaviors after being
exposed to the game and found that the game was “usable, fun, and relatable”
(Maqsood et al., 2018). Google also launched the “Interland” game aimed at
educating children about online safety, along with YouTube videos and other
online resources to help parents and teachers raise these issues with children
(Moscaritolo, 2017). Thompson and Irvine evaluated “CyberCiege,” a security
game that teaches students basic cybersecurity concepts and found that
students did not treat the game as a lab session, often overlooking the lab
manual provided and instead played straight away. Researchers noted this shift
in mentality is something they would keep in mind for future renditions of
“CyberCiege” (Thompson and Irvine, 2011).

###  2.3. Privacy Awareness

Major themes that emerged from the selected campaigns were social media
privacy settings, online privacy and behavioral advertising, and end-to-end
encryption in instant messaging. In this section, we explore privacy awareness
research related to each theme.

Acquisti and Gross conducted one of the earliest studies about privacy
awareness and information sharing practices on Facebook and found that while
most participants claimed to be aware of their profile visibility, there were
“a significant minority of members [who] are unaware of those tools and
options” (Acquisti and Gross, 2006). Another Facebook privacy study, conducted
by Sohoraye et al., focused on users’ knowledge of privacy statements and
privacy regulations in their countries. The researchers found that while many
people are concerned about their personal information, they are relatively
unaware of related privacy information (Sohoraye et al., 2015). Users’ lack of
awareness of social media privacy setting could lead to misaligned expectation
between what they intended to share and reality (Liu et al., 2011; Madejski et
al., 2012).

Common social media privacy concerns include personal safety, reputation
management, social disclosure and how easy it may be for other users to see
their activities (Ahern et al., 2007). Depending on the platform and its
intended audience, users have different privacy concerns. Jeong and Kim found
that Facebook users are more concerned about what others may post on their
timeline, versus Twitter users who are more concerned about the content of
tweets they have posted (Jeong and Kim, 2017). Baruh et al. found that
participants’ privacy concern did not significantly correlate with their
actual social media usage (Baruh et al., 2017). Building on top of Baruh et
al’s findings, Barth et al. investigated the privacy paradox and found that
users prioritized functionality and usability above possible privacy concerns,
regardless of their technical ability or financial situation (Barth et al.,
2019).

Increased privacy awareness may also lead to increased concerns about data
used for behavioural advertising. Researchers have found that while users are
generally aware of tracking, their understanding of how personalized
advertising works may not reflect reality (Rader, 2014). Personalized ads and
behavioral advertising is considered as both a useful feature, but also one
that may bring privacy risks (Ur et al., 2012) or become a nuisance (Dehling
et al., 2019).

When it comes to instant messaging, security and privacy may have minor
influence on why users choose a particular app. DeLuca et al. found that
overall, peer influence had the biggest impact on user choice in messaging
apps (De Luca et al., 2016). Abu-Salma et al. interviewed 60 participants to
investigate barriers in secure communication tool adoption and found that most
participants lacked understanding about end-to-end encryption, potentially
limiting their motivation to adopt secure messaging tools (Abu-Salma et al.,
2017). Other barriers to adoption could include misconceptions that SMS or
email may be more secure (Stransky et al., 2021), or lack of trust in end-to-
end encryption (Dechand et al., 2019). Stransky et al. investigated how
visualization may impact perception of messaging security and found that
perception of security depended more on the reputation of the app itself
(Stransky et al., 2021). When attempting to improve user understanding of end-
to-end encryption, Bai et al. found that it is most effective to use simple
wording with minimal technical detail (Bai et al., 2020).

Privacy advertisements and campaigns are designed to increase privacy
awareness, which can impact consumer choices. Kelley et al. examined privacy
as part of a user’s decision to download or install an app. The researchers
found that users seldom considered privacy information when making a decision
to download apps. However, when the authors introduced privacy information
into the app store, users were much more likely to consider privacy as a
factor in their decision making (Kelley et al., 2013). Tsai et al. found that
when privacy information is prominently and intuitively visible, people would
be willing to pay at a premium for privacy (Tsai et al., 2011). Similarly,
Emami-Naeini et al., found that consumers are willing to pay more for
Internet-of-Things products with better privacy and security practices, such
as de-identified cloud storage (Emami-Naeini et al., 2023).

###  2.4. Acceptance of Privacy and Security Advice

Redmiles et al. developed a set of quality metrics for security and privacy
advice: perceived actionability, perceived efficacy, and comprehensibility.
The researchers found that generally, while all study participants found the
advice actionable, neither the users nor the experts were particularly good at
prioritizing advice (Redmiles et al., 2020). Research in security awareness
campaigns also helps to identify key factors that may influence the success or
failure of a campaign. Bada et al. evaluated security campaigns across the UK
and Africa. The researchers found that security education “needs to be
targeted, actionable, doable and provide feedback,” and that invoking fear is
an ineffective way of promoting security awareness (Bada et al., 2015).
Building on Bada et al.’s work, Das et al. conducted a comprehensive
literature review on research related to security and privacy awareness. Based
on their review, the researchers developed the Security and Privacy Acceptance
Framework (SPAF), identifying three key barriers that could impact a user’s
acceptance or rejection of recommended security and privacy advice, being
awareness, motivation and ability (Das et al., 2022).

Technology has also affected the way in which people learn new information.
Kross et al. surveyed a nationally representative sample of US adults, and
found that online safety, security, and privacy are 3 of the 19 categories of
subjects people tried to learn more about through online resources. The
researchers identified YouTube as the most commonly used learning channel
(Kross et al., 2021). Akgul et al. built on Kross et al.’s work and analyzed
influencer videos on YouTube containing VPN ads. They found that “VPN ads
likely reach billions of viewers, comparable to ambitious industry efforts at
influencing users’ understandings of security and privacy tools.” However,
their analysis of these ads suggested that the information they contain may
not always be correct and may potentially have negative impacts on viewers’
understanding of online safety (Akgul et al., 2022).

##  3\. Methods

We conducted an exploratory study, aimed at investigating the impact of
privacy marketing materials on privacy perception. Specifically, we explored
how privacy marketing materials, such as privacy ads and games, impact
viewers’ attitude towards the organizations providing the campaign, awareness
of privacy features and issues, and the actionability of suggested privacy
advice. In the subsections below, we first introduce the privacy marketing
campaigns that were selected for this study, then outline the recruitment and
screening procedure, as well as the interview protocol and data analysis
method.

###  3.1. Privacy Marketing Materials

We searched for privacy marketing materials on YouTube and the web using the
keywords “Privacy Advertisements,” “Privacy Ads,” and “Privacy Awareness.” We
found five campaigns that were published in 2022 by technology companies. We
selected one video ad from each of four ad campaigns we found. We also
selected the Twitter DataDash game, which had received some media attention in
2022. Here we provide a short summary of each campaign. Table 1 provides more
information about each campaign. Additional images from the campaigns are
presented in Appendix A.3.

####  3.1.1. Data Auction by Apple.

Protagonist stumbles upon a room packed with people, where an auctioneer is
selling off her private information to the highest bidder, shown in Figure 1.
As the information for sale becomes more sensitive, she decides to take
action. By using Apple’s “Ask App Not To Track,” she eliminates half of the
bidders in the room. Next, by using Apple’s “Protect Mail Activity,” she
empties the room completely, including all remaining bidders, the auction
attendants, as well as the auctioneer (Apple, 2022).

Figure 1. Auctioneer facilitating auction of protagonist’s data in Apple ad.

####  3.1.2. You’re in Control by Samsung.

Protagonist knows that much of her life is on her Samsung phone, where she
struggles to get away from floating balloons representing apps, shown in
Figure 2. Utilizing Samsung’s privacy dashboard, she takes control of her
privacy. By using Samsung’s permission manager, she turns off precise
location, illustrated with a yellow umbrella from the Weather app, to blend
into a sea of yellow umbrellas. As she continues using her phone, she chooses
to grant or deny location and microphone permission for apps as needed. Using
Samsung’s privacy dashboard, she is able to review all app permission usage in
a central location. Samsung’s Knox Security is mentioned at the end of the
video (Samsung, 2022).

Figure 2. Protagonist struggles to get away from balloons representing apps in
Samsung ad.

####  3.1.3. A New Era of Personal Privacy with Default End-to-End Encryption
by WhatsApp.

People arrive at the post office wanting to mail their letters. To their
disbelief, the clerk, shown in Figure 3, insists on sending those letters
using a messenger pigeon, assuring his customers that using pigeons as a
delivery method is “mostly” secure. The ad states that 5.5 billion texts per
day are sent without encryption, while with WhatsApp, their conversations
would not be one of them (WhatsApp, 2022).

Figure 3. Post office clerk (right) uses carrier pigeon for sending letters in
WhatsApp ad.

####  3.1.4. Watching you by DuckDuckGo.

Singer in a Google T-shirt singing an altered version of the song “Every
Breath You Take” (The Police, 2023a, b), where lyrics were changed to be
relevant to web tracking (e.g. “Every click you make, I will be watching
you.”) As the singer shoulder-surfs people while they browse the Internet,
they are visually uncomfortable with the singer’s presence, as shown in Figure
4. One person decides to start using DuckDuckGo, causing the singer to be
dragged away by an invisible force. An announcer explains that “The Internet
does not have to be so creepy” and introduces DuckDuckGo products (DuckDuckGo,
2022).

Figure 4. Singer wearing Google T-shirt (right) shoulder-surfs a person
browsing the Internet in DuckDuckGo ad.

####  3.1.5. DataDash by Twitter.

DataDash is an online game simulating the style of 90s old-school platform
games, where players journey through PrivaCity with Data dog as illustrated in
Figure 5. Players must collect five artifacts to pass to the next level. The
artifacts for level 1 and 2 are envelopes containing dog bones. At the end of
each level, players are shown a short explanation referring to a particular
privacy and safety feature, with a “Twitter Setting” button that takes them to
a specific privacy and safety setting on Twitter. The privacy and safety
settings for each level are:  (1) Ad Preference,  (2) Direct Messages,  (3)
Location Information, and  (4) Audience and Tagging . The website where
DataDash is hosted explains the purpose of the game: “The Twitterverse can be
tricky to navigate if you don’t know your way around. So we made a game to
help you understand our privacy policy a little better” (Twitter, 2022).

Figure 5. Start of level 1 of Twitter game. Table 1. Campaign Details. Company |  Campaign name |  Release Date | Length | Source
---|---|---|---|---
Apple |  Data Auction |  Published on YouTube May 18, 2022 | 1 min 35 sec | https://www.youtube.com/watch?v=NOXK4EVFmJY
Samsung |  You’re in Control |  Published on YouTube Feb 9, 2022 | 1 min 35 sec | https://www.youtube.com/watch?v=kZKK80urUZc
WhatsApp |  A New Era of Personal Privacy with Default End-to-End Encryption |  Published on YouTube Oct 17, 2022 | 1 min | https://www.youtube.com/watch?v=zvI4cVGWJhM
DuckDuckGo |  Watching You |  Published on YouTube May 17, 2022 | 30 sec | https://www.youtube.com/watch?v=QWpPyYlZXNI
Twitter |  DataDash |  Estimated to be May, 2022 based on time of media coverage (Winkie, 2022; Hurlerand and Wodinsky, 2022). | Not Applicable | https://twitterdatadash.com

###  3.2. Recruitment and Screening

We recruited participants for our study using convenience sampling, snowball
sampling and Prolific, a research-oriented online crowdworking platform
(Prolific, 2023). To reduce privacy and security priming, we presented the
study as a study on “Consumer Perspective on Tech Advertising.” We omitted any
words related to privacy or security in the study title and purpose, letting
the campaigns deliver relevant privacy components naturally. We also did not
mention any company name related to the campaigns when we advertised the
study. Participants were compensated $1.25 USD for completing a screening
survey and those who subsequently participated in the 25-minute interview were
compensated $10 USD for their time.

We designed four treatments to test the five campaigns: Apple, Samsung,
Twitter, and WhatsApp-DuckDuckGo. With the exception of Twitter which was an
online game, all other campaigns were short videos ranging from 30 to 95
seconds. WhatsApp and DuckDuckGo ads did not introduce actionable privacy
features that we could ask participants about and were the shortest of all
video campaigns, with their combined video length being roughly the same as
that of Apple and Samsung. In the interest of maintaining similar interview
length across all treatments, WhatsApp and DuckDuckGo were grouped together.

We required eligible participants to be aged 18 or above, fluent in English,
and located within the United States (the primary target location for the
versions of the campaigns we tested). With the exception of Twitter, whose
game was primarily intended for its users, all remaining video campaigns were
ads likely aimed at the general public, both users and non-users. Thus we
developed a screening survey, aimed at discerning participants’ product or
service usage that we used to assign participants to treatment groups. We used
purposive sampling to select from among those who qualified for our study and
assign balanced treatment groups. The screening survey is in Appendix A.1.

For Apple, Samsung and WhatsApp-DuckDuckGo treatment groups, we screened for
50% users and 50% non-users of the relevant product or service. The Twitter
treatment group was comprised of 100% Twitter users with some form of online
and/or digital gaming experience. We defined users as those who utilize the
product or service featured in the campaign at least once every week. Due to
low interview response rate from eligible DuckDuckGo participants, we relaxed
the qualifying user condition from using DuckDuckGo at least once a week to at
least once a month. We also asked participants about their technical
backgrounds and distributed those with more technical backgrounds evenly
across treatment groups.

For the Twitter treatment, we screened for Twitter users who participate in
some form of digital gaming. We added this requirement after Twitter pilot
participants without gaming experience repeatedly mentioned that they would
like to stop playing due to frustration. We did not include non-users in this
treatment, as the game was designed to inform existing Twitter users rather
than interest non-users in using Twitter.

Initially, we screened 181 participants from Prolific using our screening
survey, and invited 133 participants who were eligible. Only 24 participants
responded to the invite, including seven who rejected the invite, and three
who missed their appointment. Thus, only 14 participants completed both the
survey and the interview. Due to low response rate for interview invitations
with Prolific participants, we then began recruiting via snowball sampling
using email invitations. The first author emailed acquaintances unaware with
the research topic using the same information as had been presented to
Prolific participants. These acquaintances were also invited to refer others
who might be interested. Using this method, we recruited 10 participants. In
total, 24 participants were recruited for this study, split evenly into four
treatment groups. We ensured that for each treatment group, there were no more
than three participants who indicated they had technical expertise. All
participants who were recruited through Prolific were paid via the platform,
whereas snowball participants were compensated at the same rate, but via
Amazon gift cards.

###  3.3. Interview Procedure

We developed a semi-structured interview procedure, with an estimated
completion time of 25 minutes. All interviews were conducted over the Zoom
video conferencing platform and recorded and transcribed by Zoom. Participants
were asked to keep their cameras off throughout the interview. Interviews were
conducted between March 31 to April 19, 2023.

First, we asked about participants’ general impression of the companies, then
screen-shared the advertisements into Zoom for participants to watch. For
Twitter DataDash, we sent each participant a Twitter login details and the
game link. Once participants had the game open, they were asked to share their
screen over Zoom and given 8 minutes of DataDash playtime while the
interviewer observed them. In the other treatments, participants spent up to
95 seconds viewing a video advertisement. We asked participants a series of
questions aimed at gauging their awareness of privacy features that were shown
in the campaign, as well as discussing any privacy and security concepts
mentioned by participants. We asked participants for their impression of the
companies again after exposure to the campaigns to see whether the campaign
had an impact on this. Then, we asked participants to describe how actionable
they thought the privacy advice provided was, and self-rate their motivation
and confidence in being able to complete the steps shown in the ad or game.
Next, we moved to the actionability exercise where we asked all participants
except those in the WhatsApp-DuckDuckGo treatment to find a specific setting
that was shown in the ad or game, with a total activity time of 3 minutes.
Finally, for ad treatment groups, we re-exposed participants to the ad and
asked for concluding remarks and final observations.

Since the WhatsApp and DuckDuckGo videos did not contain other actionable
steps aside from switching to the advertised service, we did not include the
actionability section of the interview for that treatment, but instead
repeated the entire interview procedure again for the second ad. The order of
the advertisements shown (WhatsApp first, then DuckDuckGo, or vice versa) was
randomized.

The main purpose of the actionability section of the interview, was to assess
perceived ease of actionability, measured through participants’ description of
what steps they thought the advertisement had recommended, as well as self-
ranked motivation and confidence. We shared an iPhone 12 Pro, a Galaxy A13
smartphone, or the Twitter home page over Zoom. Participants were asked to
direct the interviewer on where to go to find the appropriate setting. Prior
to the study, we had restored both smartphones’ desktops to factory default
desktop settings.

Video advertising campaigns are generally designed with the assumption that
viewers will see ads multiple times (Landa, 2016). Therefore, in the three ad
treatments we re-exposed participants to the ads again at the end of the
interview to see if this would generate any new insights.

Before recruiting participants for our study, we conducted eight pilot
interviews, two per treatment group. We made a number of revisions to our
protocol based on issues discovered during the pilots. Pilot interviews are
not included in our results and analysis.

###  3.4. Data Analysis

Interview transcripts were generated automatically using Zoom’s built-in
transcription function and manually reviewed with the voice recordings by a
member of the research team. Two researchers performed qualitative analysis
for each participant interview transcript using Template Analysis, a style of
thematic analysis that combines both inductive and deductive coding (King,
2024; Brooks and King, 2014). Template Analysis emphasizes hierarchical coding
without specific prescription regarding the number of levels required and what
levels represent. To analyze the data, both researchers first familiarized
themselves with the data and the first researcher created an initial codebook
(the template) and coded all the data. Then, the second researcher reviewed
and refined the codes applied by the first researcher. Both researchers met
over multiple sessions to discuss any disagreements in the coding and adjusted
the codebook (added, updated, and deleted codes) until they reached a
satisfiable version of the codebook, agreeable to both researchers. Our coding
started with broader themes, such as “attitudes,” “awareness,” and
“actionability,” which encompass narrower themes, such as “organization
impression pre/post-exposure,” to more specific ones, such as “positive,”
“negative,” etc. For the actionability section (task), after the research team
agreed on a standard criteria (also provided in Appendix A.6) to evaluate task
completion, one researcher recorded whether participants completed the
activity, then recorded the participants’ self-rated scores regarding
motivation and confidence. A diagram summarizing the coding hierarchy is
provided in Appendix A.5. The codebook along with the specific interview
question(s) from which each set of codes were distilled is provided in
Appendix A.6.

###  3.5. Ethical Considerations

This study protocol was approved by [Anonymous Institution]’s Institutional
Review Board (IRB). Before completing the screening survey, we asked all
participants to read and acknowledge their acceptance of the consent form. We
asked all interview participants to keep their cameras off. Before
participants in the Twitter treatment were asked to share their screen over
Zoom we reminded them to avoid accidental information sharing. Since the
Twitter game actionability activity required users to login and navigate
Twitter’s account settings, participants were provided with an account by the
researcher and were not asked to login using their personal accounts. A member
of the research team removed participant names from transcripts along with any
personal information participants shared in the interviews.

Although one of the authors has previously received research funding from two
of the companies whose campaign materials were used in this study, we received
no funding from any company for this project. The authors, and to the best of
our knowledge, the participants, are not associated with any of the companies.

##  4\. Results

In this section, we discuss the main finding of this study. We use the first
letter from each company name to indicate which treatment group the
participants are in, followed by the participant number. While our study is
qualitative, we provide numbers of occurrence of qualitative themes to
highlight the most or least common themes.

###  4.1. Participant Demographics

We interviewed 24 participants, including 13 males, and 11 females.
Participants’ ages ranged from 18 to 65 years old. Of the 24 participants, 10
indicated they have a technical background. All Apple and Samsung participants
who were asked about their smartphone use said they actively use smartphone
apps and browse the Internet using their smartphone. All but one DuckDuckGo
participants said they actively browse the web and perform search queries
using search engines. For more details about participant demographics, refer
to Appendix Section A.4.

###  4.2. Attitudes and Impressions of Companies

We asked participants about their impressions of the companies relevant to
their treatment group both before and after viewing the videos or playing the
game (Q.5 & Q.14 in Appendix A.2). We reviewed the transcripts of these
responses and categorized the sentiments they expressed before watching the
ads or playing the game into five categories: positive, neutral, negative,
mixed-positive (more positive than negative), and mixed-negative (more
negative than positive). We classified participants who mentioned an equal
number of positive and negative points based on whether the first point they
mentioned was positive or negative. We used the same five codes to analyze
responses after viewing the ads or playing the game, with two additional
codes: positive-plus and negative-plus, to categorize responses that indicated
increased degree of positivity or negativity. We then compared each
participant’s sentiment before and after viewing the ad or playing the game,
and classified the change as: more positive, more negative, or remained the
same.

Our results suggest that most of the campaigns we studied were associated with
brands that our participants already viewed fairly positively, with the
exception of Twitter. Exposure to information about privacy features further
improved some participants’ impressions about these companies, although for a
few participants, the campaigns worsened their impressions as they viewed
claims about privacy in the campaigns with skepticism. The Twitter game also
worsened the impression of some participants due to both poor game play
experience and unclear messaging.

####  4.2.1. Attitude and Impression of Companies Before Exposure

Prior to campaign exposure (Q.5), our results show that most participants had
positive or mixed-positive impressions of the companies relevant to their
treatments for all treatment groups except Twitter. Among the participants in
the Twitter treatment group, only one participant was positive (T6) and one
was mixed-positive (T3).

Top reasons for having a positive attitude towards an organization across all
groups were related to having a good product or having a privacy-focused
product. Some participants who were not users of a product still remarked that
they thought the product was good (A2, S2, S6, WD6 for WhatsApp and
DuckDuckGo). S2, a Samsung non-user, remarked: “I think that they come out
with top notch products. I think I would be almost as happy having a Samsung
Android type phone as I would having an iPhone.” DuckDuckGo received the most
mentions of privacy-focused aspects as a reason for positive attitudes towards
the company, mentioned by both users and non-users. For example, WD5 said they
have “positive feelings about [DuckDuckGo]” due to “the privacy features.”

Twitter had the most negative impressions, with two negatives (T1, T2) and one
mixed-negative impression (T4). Participants mentioned false news (T1, T4) and
lack of moderation on the platform (T1, T2). From T2’s perspective, “what
Twitter used to be, was not perfect, but tolerable. And now it’s just a single
rich person’s plaything.” It is interesting to note that when discussing their
views of Twitter, three participants (T2, T4, T5) either directly mentioned
Twitter owner Elon Musk, or implied that the leadership change had some impact
on their impression of the organization. One Apple participant (A4) mentioned
high prices as the reason for their negative impression of the company. A4
explained, “An iMac will cost on average, about twice as much … as a given
windows machine [of] approximately equivalent capability.”

####  4.2.2. Attitude and Impression of Companies After Exposure

After being exposed to the campaigns, 12 of the 30 responses111WhatsApp
responses and DuckDuckGo responses are counted separately, once for each
campaign. reflected the same impression as before while 13, including some
participants from every group, expressed more positive attitudes (Q.14). Seven
of these participants, including three in the WhatsApp group, cited becoming
more informed about privacy features as the reason they became more positive.
S3 said that he now felt “a little bit more comfortable with the privacy
features on my phone, and also as soon as we’re done here, I’m probably gonna
to open up my phone.” S3’s response also highlighted another reason that led
to improved impression: an increased motivation to use the product. Similarly,
WD3 said that the ad made her a “bit more curious … I have no idea what it has
to offer, and how it would compare to Google.”

Participants also recognized that organizations may promote privacy as a
feature of their products and services. A6 described feeling more positively
towards Apple after viewing the ad as “it shows that they are using data
privacy as part of their company brand.” A4, whose impression of Apple stayed
the same, explained that the ad didn’t “change my image of Apple so much …
this is the kind of feature [that] … demonstrate the unique value of its
services to users … This is something you give to users in order to like
convince them that their money is well spent by sticking with you.”
Participants generally discussed promoting privacy as a feature in a positive
or neutral tone.

There were also five participants (from Twitter, DuckDuckGo, and WhatsApp)
who, after being exposed to the campaign, had a more negative view of the
organization. Skepticism was the top reason for having a worsened impression,
mentioned by three participants (WD2, WD4 for DuckDuckGo, WD4 for WhatsApp).
Some of those participants questioned the truthfulness of the message and
whether the organizations will deliver on the messages the ads portrayed. WD2
wanted to do more research on DuckDuckGo, because “well, how does it make
money?” In addition, prior knowledge of negative news about the company such
as data breaches could also lead to skepticism, with WD4 being skeptical about
both WhatsApp and DuckDuckGo due to his exposure to news about those
companies. WD4 felt “Slightly more negative” about DuckDuckGo due to “what
I’ve read online because they’re still reinforcing this, you know, one this
silver bullet of search engines,” likely referring to the Microsoft DuckDuckGo
tracker incident (Fadilpašić, 2022). WD4 admitted that he“wasn’t diligent
enough about following up what happened with WhatsApp about a year ago,”
likely referring to the WhatsApp data breach (Espósito, 2021).

For the two Twitter participants who had a more negative view of the
organization after being exposed to the campaign, their reasoning stemmed from
a poor experience with the DataDash game itself. T1 remarked that “privacy and
safety issues are a very important issue, and and it seems like this, this
game didn’t really do a good job of taking it like very seriously … I think
there are a lot more better ways to educate than this game … I think if they
did it again have like more real life implications, and it was more tied to
real examples of what can happen with the privacy and data.” T5 had a much
more sinister take on what the game represented, concluding that Twitter
“probably [has] no issue selling our data and like exploiting our privacy.”
From T5’s understanding, “the bones [represented] the data pieces, or like
information … the dog was probably just someone scooping up data pieces.”

###  4.3. Communicating Privacy

To assess how well each campaign communicated privacy, we asked participants
to describe the campaign they had just viewed or experienced, how they felt
towards the ad or game, and what they perceived as the purpose of the
campaigns (Q.7, Q.8, and Q.9). From these responses, we explored how well the
campaigns increased awareness of privacy features and issues, how well
participants understood the privacy metaphors used in the campaign, and their
understanding of the purpose of the campaign.

Overall, our findings suggest that participants were able to grasp broader
privacy themes communicated in the campaigns. In general, after being exposed
to the campaign, participants noticed at least one main privacy feature
showcased, and their perception of the privacy metaphors mostly aligned with
that of the research team. The WhatsApp campaign, which appeared to be most
successful at communicating a clear message was short, communicated a single
feature (end-to-end encryption) and used a simple metaphor (pigeon to
represent unencrypted SMS). On the other hand, Twitter’s DataDash game was the
only campaign that faced considerable challenges in communicating its privacy
feature, with some participants failing to recognize the privacy connection
until we explicitly asked about it.

####  4.3.1. Privacy Awareness

Of the 30 responses collected, 25 discussed the privacy features or functions
that were shown in the advertisement or game. The privacy features shown
inspired participants to make specific comments related to privacy, with 14
responses explicitly mentioning the word “privacy” in their replies.

In the Apple ad, there were two features that were displayed: “Ask App Not to
Track” and “Protect Mail Activity.” When we asked participants to describe the
campaign in their own words, in addition to the general synopsis of the
campaign, most Apple participants (A1, A2, A4, A5) included a detailed
description of the “Ask App Not to Track” feature. A5 said “the ad is about
how … using an Apple device can help protect your own data, because there are
lots of other apps that could track and use your data, and through Apple you
can say that I don’t want you to track it.” However, there was only one
mention of the second feature shown in the ad “Protect Mail Activity.” A1
described the protagonist as “going through and selecting options on her
iPhone, asking apps not to track her or protecting her email data.”
Participants who did not describe either feature (A3, A6) spoke about the ad
in much broader terms, with A3 summarizing the features shown in the ad as
“apps or certain settings on the phone to where you keep all your data private
and secure.” Most Apple participants (A2, A3, A4, A5, A6) mentioned issues
surrounding personal data, data sharing, or selling.

For Samsung, the campaign included five privacy features, being privacy
dashboard, location permissions, microphone permissions, camera permissions,
and Samsung’s Knox Security. The campaign mostly featured the privacy
dashboard and location, microphone, and camera sharing permission controls.
All participants in this campaign talked about Samsung’s permission controls
in general. The location sharing control feature captured the most attention,
with four participants (S2, S3, S5, S6) mentioning it in their response, while
the camera sharing control feature was mentioned by two participants (S5, S6),
and the microphone sharing control was mentioned by one (S6). S6 summarized
the ad as “advertising the privacy options that Samsung gives you.
Specifically the option to see which apps are using which features such as
camera, microphone, location, and you could fine-grain control how they’re
using it.”

Each level of Twitter’s DataDash game ends with a button that directs the user
to the privacy setting relevant to that game level. All participants
experienced level 1 with some progressing to latter levels. No participant
clicked on the “Twitter Setting” button at the end of any level, which would
have led them to the corresponding privacy setting for that level. When we
asked participants to describe the campaign, we had not yet asked any
questions containing the words “privacy” or “security.” Thus, two Twitter
participants (T3, T5), had yet to realize the game was related to privacy.
They only made the connection between DataDash and Twitter’s privacy practices
later in the interview, after the interviewer asked: “Were you aware of
Twitter’s “Privacy and Safety” features prior to playing DataDash?”
Immediately after the interview was over, T5 said that she did not realize
there was a privacy connection until halfway through the interview (with her
permission we added this comment to her transcript). For the remaining four
participants who noticed DataDash’s connection to privacy, there were only two
participants who mentioned privacy features in their description of the game.
T4 said she felt the game was trying “to show that there are settings to
protect … I guess the purpose of the game was to show that I have more control
over my data than I realize.” T6 described the purpose of DataDash as “to
learn more about Twitter’s features through like, like a simple game.” Of all
groups, Twitter participants mentioned privacy the least, with T1 commenting
that “I felt like it was a little bit confusing, and it was hard to understand
exactly what the game is representing … It gave me an impression that it can
be getting dangerous to be on Twitter with all the different elements of
privacy concerns.”

The focal point of the WhatsApp campaign was end-to-end encryption. For
example WD3 finished her summary of the ad with this sentence: “in the end we
find out that 5.5 billion messages are sent in an unsecure way. But if you use
Whatsapp, that will not be the case.” When we asked participants to give a
summary of the campaign, encryption as a feature was recognized by nearly all
participants in this group. WD6, the only participant who did not discuss
encryption in WhatsApp specifically, still mentioned unencrypted messages:
“[it] kind of draws that parallel to people sending mail regularly versus just
sending unencrypted instant message.” All WhatsApp participants mentioned the
security or safety of instant messages when discussing the campaign. For
example, W5 mentioned “having messages encrypted versus unencrypted using a
delivery pigeon.”

When we asked DuckDuckGo participants to describe the ad, all participants
discussed tracking and DuckDuckGo’s privacy-friendly search engine and
browser. WD4 explained: “Search engines can see everything you type in, and
all the results that yield from them, and DuckDuckGo is a solution for that.”
However, only two participants (WD1, WD2) were specific about the no-tracking
feature. WD1 explained: “Google keeps track and monitors your search history,
and DuckDuckGo doesn’t.” All participants mentioned DuckDuckGo’s privacy
protective features generally. WD3 described being able to “perform … similar
search queries on [DuckDuckGo] platform, but your privacy won’t be impacted.”
WD5 was the only one who realized the ad also mentioned the DuckDuckGo mobile
app, saying that “I didn’t know before that there was an app for DuckDuckGo.”

We also asked whether participants were aware of the advertised feature (e.g.
“Ask App Not to Track”) or function (e.g. encryption or web tracking) prior to
watching or interacting with the campaign, and how they came across it (Q13 &
Q13B). 18 out of 30 responses indicated prior awareness, including 7 who were
non-users. Participants commonly reported that they became aware of a privacy
features’ existence after being alerted by the app upon first use (A1, A3, S1,
S3, S6) or later when using an app (A2, S6, T2). S6 mentioned both types of
prompts in her answer: “it asks you repeatedly when you install it, and then
when you use the app.” Others discovered the features through exploring the
settings (A2, S2, S4), while some became aware of the feature or concept as it
was prominently displayed in the app or website (T5, WD3 for WhatsApp
regarding encryption). Participants also reported learning about new features
and concepts as a result of updates or policy changes (A5, S5, T4, WD5
regarding DuckDuckGo web tracking feature). T4 described the types of
notifications she received from Twitter: “when they update the terms. It comes
up as like a pop up on the app. Or sometimes, if they change the terms fully,
they send an email.” Interestingly, participants also reported learning about
new features and concepts through their usage of other services and apps. WD5
first becoming aware of encryption through “us[ing] VPN services in the past,
and also PGP.” WD6 described learning about what encryption is through seeing
“a lot of advertising … on Youtube for VPN like TurboVPN and things along
those lines”.

####  4.3.2. Participants’ Understanding of Privacy Metaphors

As all of the campaigns use metaphors to explain privacy concepts, we
investigated what participants understood about these metaphors and whether
they were actually useful in illustrating these concepts. We selected one
prominent symbol from each campaign and asked participants to share what they
thought it might represent (Q.10). We wanted to see how the selected symbol
was perceived by participants for each campaign, and whether participants of
the same campaign felt similarly about what the symbol could represent.
Overall, most participants’ impressions of these metaphors align with the
impressions of the authors of this paper, suggesting that the metaphors are
largely effective in communicating to non-experts a similar message as they
communicate to experts. However, there were occasions where participants
failed to notice the privacy metaphor at all, or had a non-privacy related
understanding. Two of the metaphors seemed fairly difficult for our
participants to interpret correctly.

When we asked participants in the Apple treatment group about the auctioneer,
all of them gave responses related to companies selling personal data.
Participants said that the auctioneer either represented data sales (A2, A3,
A6) or other parties such as app companies, Internet Service Provider (ISP) or
smartphone companies (A1, A4, A5). A1 explained that “what they would are
really hoping is that you associate … the auctioneer with other cell phone
companies … they’re implying that if you are with another cell phone [company]
… that phone company will be able to sell your data.” A5 made similar
comments, theorizing that “the phone company here could be Apple, or it could
be Samsung.”

On the other hand, when we asked participants in the Samsung treatment group
about the yellow umbrella the protagonist used to hide herself from exposing
her exact location, only one participant (S3) said that the umbrella “was
trying to show … whether you’re giving gross location data or precise
location.” Others felt that the umbrella represented a warning (S2, S5) or
protection of information in general such as location or activity (S1). Two
participants (S4, S6) had no idea what the umbrella represented, with S6
saying she “didn’t notice at all.”

We asked participants in the Twitter treatment group about the envelopes
containing bones that were collected in level 1 and level 2 of the game and
received the most diverse interpretations among the campaigns we studied. Two
participants (T1, T5) associated the envelopes with data collection, with T5
saying the envelope represented “pieces of data.” Others felt the envelope
could be a tweet (T2), spam (T4), or opening a direct message (T6). T3 had the
most unique take, stating that he “kind of thought it was a cat, but maybe
there was something deeper with that I don’t know.” (Indeed the co-authors
were not in agreement, and one of our co-authors also thought the envelopes
looked like cats.) Participants’ understanding about the carrier pigeon in the
WhatsApp video were mostly uniform, generally themed around instant messaging
or the communication channel used. Some thought the pigeon represented SMS
message text (WD1 & WD2) or unsafe messages (WD3, WD5). Others felt the pigeon
referred to the communication platform (WD3) or the network (WD4), as well as
outdated technology (WD1, WD6).

DuckDuckGo’s ad followed a singer wearing a Google T-shirt shoulder surfing
others while they browsed the internet. Most participants said the singer
represented Google (WD2, WD3, WD5, WD6) or the search engine (WD4).

####  4.3.3. Participants’ Perceived Purpose of the Campaign

Participants’ thoughts about the main purpose of the campaigns (Q.9) can be
categorized into four groups: advertising privacy services or creating a more
privacy-focused corporate image, promoting the company as being different from
its competitors, providing information, and entertainment.

Using the campaign to advertise privacy services or create a privacy-focused
company image was the most common perceived purpose, with 24 responses across
all campaigns, including all participants in the Apple, Samsung, and
DuckDuckGo groups, citing this as the purpose of the campaigns. A1 described
the main purpose of the Apple ad as “to sell iPhones, to present them as the
most secure option for protecting your identity and your data from
unauthorized use.” Participants also felt companies were using these campaigns
to advertise themselves as being privacy focused. S6 explained that “the main
purpose of video is to make the statement that Samsung very much cares about
your privacy.”

Others also felt that the campaigns were issued by the companies to promote
themselves as being better than their competitors. We saw 15 comments
regarding this across four campaigns (Apple, Samsung, WhatsApp, and
DuckDuckGo). All participants in the WhatsApp group cited this as the reason
behind the campaign, with WD3 stating that “I think it was purely to get more
users to switch on to WhatsApp, instead of using other probably less safe
platforms with instant messaging.” Most participants in this group also viewed
the purpose of the DuckDuckGo campaign as differentiating themselves from
competition. Five DuckDuckGo participants cited this as the main reason behind
the campaign. For example, WD6 said, “The main purpose of the video was to
show consumers that there is another option or a search platform if they feel
like Google is invading their privacy.”

Participants also picked up on potential education efforts and the informative
nature of the campaigns, with 13 responses across all groups commenting on
this intent. Four Twitter participants (T1, T2, T4, T6) said that DataDash was
trying to inform them of some sort of privacy feature or information about
Twitter. T1 said he felt “the main purpose of the game was … intended for
education.” T2 echoed T1’s response, but was worried the message might have
gotten lost: “I think it was to educate people about how Twitter uses our
data, but I’m not quite sure it would get the message across correctly because
I think people would be so focused on actually [navigating the game] and
getting the objects and scoring the points just to beat it that they might
miss the entire point of the game.” Indeed, both T3 and T5 failed to recognize
the privacy connection between DataDash and Twitter until the interviewer
asked a question regarding Twitter’s privacy and safety features. Instead T3
and T5 thought the purpose of the game was entertainment. T3 described the
game as “a diversion … something people would do to pass time” and T5 was “not
super sure” about the intent of the game “but it was fun.” The Twitter
DataDash website explained that the game was designed “to help you understand
our privacy policy a little better…. and learn how to take control of your
Twitter experience along the way.”

####  4.3.4. Impact of Re-Exposure

Since the campaigns we chose were all released in the year before our study,
we also asked whether participants had previously seen the ad or played the
game prior to the study (Q.6). Only four participants (A5, WD4 WD5 for
WhatsApp, WD2 for DuckDuckGo) had previously seen the campaigns. Of those
four, three participants came across the campaign while watching TV or using a
streaming service, with one participant (WD2) stating he saw the ad after
having “read an article about … the Super Bowl ads.”

Previously exposed participants were able to pinpoint specific details that
might be overlooked by others the first time they viewed the campaign during
the study. For example, WD4 made a comment about a scene in the WhatsApp
campaign, stating that: “it’s really funny how the user that they zoomed in on
while texting was actually using iMessage, which to my knowledge, is end-to-
end encrypted.” In addition, A5 commented: “it’s also very clever that Apple
and its other marketing puts the phone over the person’s face, which is also
another way of signaling that Apple is a barrier between what you are doing on
the Internet and what people can see about you.” A5, who was recruited via
snowball sampling, also proactively reached out after the interview and
provided a screenshot of similar Apple billboard marketing she had seen in her
neighborhood, showing the phone-over-face imagery. With her permission, we
included this in our study data.

For non-previously exposed participants, upon second viewing, they were also
able to pick up on more details. After re-watching the DuckDuckGo ad WD6
remarked: “I don’t know if I missed something the first time, but I feel like
I saw more of the advertisement than I did the last time.” S5, a Samsung non-
user, commented on a new detail she observed when watching the ad again: how
Android features seem “more granular in terms of they’re literally giving you
a list of things that are tracking you.”

###  4.4. Actionability

Three of the campaigns we investigated offered actionable advice related to
privacy features in their product or service. We explored participant
perceptions of both perceived (Section 4.4.1) and actual (Section 4.4.2)
actionability of the Apple, Samsung, and Twitter campaigns. We then observed
whether participants exposed to all campaigns reported any motivation to
switch or continue using the products (Section 4.4.3).

Overall we found that the campaigns were not all that effective at teaching
viewers how to use privacy features. When we asked participants to direct the
interviewer to the privacy features, they seemed to rely on their prior
knowledge of the OS and intuition about how to use the interface rather than
information provided in the campaign. Some features, such as Apple’s “Ask App
Not to Track,” are presented differently depending on whether the user is
opening a newly installed app or searching through settings. Thus, the
interface shown in the campaign may not exactly match what a user sees when
they try to use the feature. In addition, whether participants could easily
find the relevant Twitter privacy settings seemed to have little to do with
their experience playing the game.

####  4.4.1. Perceived Actionability of Privacy and Security Advice

Prior to the activity, participants were asked to rank their confidence in
being able to perform the steps or go to the setting shown in the campaign on
a 5-point scale, with 1 being not at all confident and 5 being extremely
confident (Q.18). Apple participants rated themselves as a 4 or 5 in
confidence, with most Samsung participants rating themselves at 5. Most
Twitter participants rated their confidence at 3. Participants were also asked
to rate their motivation to perform the steps shown in the ads or game on a
5-point scale, with 1 being not at all motivated, to 5 extremely motivated
(Q.17). Most Apple participants ranked themselves at 3 or 5, and majority of
the Samsung participants ranked themselves at 4 or 5. Half of the Twitter
participants ranked their motivation at 1. These results could suggest that a
negative interaction with the privacy campaign could diminish the
participants’ confidence and motivation in enacting privacy protections as
recommended by the campaigns.

####  4.4.2. Actual Actionability of Privacy and Security Advice

To test actual actionability of privacy and security advice in the campaigns,
participants were asked to direct the interviewer to a particular privacy
setting or feature that was shown in the campaign (Section 6 & Section 7 of
the interview). We asked participants in the Apple treatment group to turn on
the “Ask App Not to Track” and “Protect Mail Activity” features, shown in
Figure 6. We asked participants in the Samusung group to change location
permissions for the Weather App and change microphone permissions for the Zoom
app, shown in Figure 7. Finally, we asked Twitter participants to go to the
“Ad Preference” setting, shown in Figure 8.

All Apple participants were able to find the “Ask App Not to Track” setting
eventually. However, only two participants (A5 and A6, both users) were able
to successfully complete the task without help. During this process, A3
requested help twice in order to locate the setting, and A4 used Google to
find Apple’s online documentation. In addition, both users and non-users (A1,
A2 and A3) did not realize that they had completed the task. The “Ask App Not
to Track” feature was presented in the ad as a pop-up that is prompted when
opening a newly installed app for the first time. However, it did not show how
someone who had already used an app would find the setting, which would have
been more consistent with the scenario shown in the video since the
protagonist had already installed the apps that were tracking her. Within
Apple settings, the setting can be found under Settings $\rightarrow$ Privacy
and Security $\rightarrow$ Tracking. This discrepancy was noticed by A5 (a
user), who knew she had found the correct setting, adding that “I didn’t know
that there was a settings page to manage it. But it does make sense.”

activity-apple-01

(a) Apple’s “Ask App Not to Track” feature.

activity-apple-02

(b) Apple’s “Protect Mail Activity” feature.

Figure 6. Apple “Ask App Not to Track” and “Protect Mail Activity” features.

Only one Apple participant (A1) was able to confidently locate the “Protect
Mail Activity” setting without help. One participant (A3) did not continue due
to having used up their total allocated activity time (3 minutes for the two
activities). All remaining participants were able to perform the activity, but
either required help or were unsure they had completed the task. During this
activity, two participants, a non-user (A4) and a user (A6), both consulted
Google to find Apple’s documentation for this feature, with A6 also bringing
out their own device to navigate in a more familiar setting. One participant
(A5, user) was also unsure whether she had reached the correct setting, and
was quickly reassured when the feature had the exact same name as shown in the
ad.

All Samsung participants were able to change the location permission for the
Weather App. However, only S5 and S6 were able to complete the activity
without help. S1 was unsure that she had completed the activity. The device
that was used for the Samsung group was a Galaxy A13, and after the desktop
was returned to factory settings, the main setting icon is not immediately
available on the home screen. The setting icon becomes visible either when one
swipes down on the screen, revealing a small gear on the top right corner, or
when one swipes up revealing the master setting icon. This default
configuration stumped S2, S3, and S5. S3 was a self-described “loyal Samsung
customer” who had purchased at least three flagship Galaxy phones in the past.
S2 requested help, and S3 used Google twice in an attempt to locate Samsung’s
master setting icon. While the setting icon was not available on the home
screen, the Google search bar was. S5 directed the interviewer to type
“Setting” into the Google search bar and the OS displayed a “From Your Apps:
Setting” icon at the bottom of the search, allowing S5 to get into the main
setting and progress from there. After the activity, S3 commented that “if I
had never seen the Samsung app ad before I would have gone to the right place
first. Because I saw the Samsung Ad, I wanted to go to the Samsung privacy
first to see if it was in there and then backed out back to the other one.”
Both S2 and S3 were unable to continue through to the next activity due to
having used up their total allocated activity time.

activity-samsung-01

(a) Samsung location permission for the weather app.

activity-samsung-02

(b) Samsung microphone permission for the Zoom app.

Figure 7. Samsung app’s location and microphone permission control features.

Only two Samsung participants (S1, S6) were able to change microphone
permissions for Zoom without help. S4 completed the activity with help. S5
used the Google search bar as she did for the prior activity. However, this
time when “Mi” was entered, the icons displayed under “From Your Apps” did not
include “Microphone,” and S5 was unable to complete the activity.

Finally, for the Twitter group, all participants were able to locate the “Ad
Preference Setting,” with T3 and T4 doing so confidently and without help.
However, there were minimal connections between participants’ ease of being
able to carry out the privacy advice, and what they learned playing the game.
T3, T5 and T6 all credited Twitter’s intuitive design, with T5 stating that
“That was actually pretty easy. I didn’t know that that was a setting but I
feel like where I found it, or where we found it was pretty straightforward.”

Figure 8. Twitter’s “Ads preferences” setting.

####  4.4.3. Motivation to Switch or Continue Using the Products

At the end of the interview, we asked users if the campaigns motivated them to
continue using their product (Q.22), and for non-users, we asked if the
campaigns motivated them to consider switching to the product shown (Q.21).

Eight of the 18 study participants who are already users of the device or
service said the campaigns motivated them to continue or increase their
current usage. A1 said the campaign had no impact on her choice of
smartphones, as her decision is based on “other factors.” She said she would
stay with Apple because: “I’m not really interested in switching to another
brand because it would mess everything up. I’ve got my iCloud calendars and …
all that would have to be redone. It would be a huge pain, so I would have to
really be motivated by something to try to switch from Apple.” Motivations for
continued or increased usage include already being a user and familiarity,
like S4 who explained he is motivated to continue using Samsung smartphones
because he is familiar with it. Other participants who reported that the
campaign motivated them to continue or increase usage referred to feeling
reassured from the privacy focused message the company is conveying, with A6
explaining that “[by] bank[ing] their brand on being the data privacy brand …
even if [Apple is] not selling my data today, they’re making a public
commitment to be the data privacy brand.”

All Twitter participants except one (T4) felt that the game did not motivate
them to continue using Twitter. Two participants followed up their answers
with comments critical of the game. For example, T3 stated that he felt the
game “wouldn’t drive me to action to do anything.” T2 was much more critical
of the game, going on to say that “the game doesn’t quite have bearing on why
I use Twitter to begin with, it might just be something that a few devs made
in their spare time over the course of a couple weeks or months, and they
decided to put a data protection spin on it to get it in front of more
people.”

The campaigns only motivated two non-user study participants to consider
switching (S2, WD6 for WhatsApp). When asked what motivated them to consider
switching, S2 stated “It seemed like it was a simpler process based on the ad
the way the phone was set up.” WD6 said the campaign “has me thinking about
potentially using that to communicate with some of the people I currently talk
to on Snapchat.” In addition, the WhatsApp and DuckDuckGo campaigns inspired a
few non-users (WD3, WD6 regarding DuckDuckGo, and WD5 regarding WhatsApp) to
look into what these services have to offer. WD3 explained that the
advertisement “encouraged me to look into what [DuckDuckGo] is, but not to
switch, just to explore what it is.”

For non-users who are not interested in switching, they offered similar
reasoning to users, with S5 saying that she would not switch due to “exit cost
for me as an iOS user to switch to an Android operating system, like moving
all the contacts, or like photos, or like even memos … with the current iOS
system, I can already turn on all the features that I am aware of, and it’s
just easier for me as a user to keep using my current operating system.”
Again, data synchronization and familiarity with their current device and app
were credited as the main reasons not to switch. S5 and S6 highlighted the
fact that they have similar features in their current smartphones, and the ad
did not motivate them to incur the costs of switching from their current
smartphone.

##  5\. Limitations

Our study’s small sample size, with 6 participants per group, limits our
ability to draw quantitative conclusions or compare across campaigns and
participant characteristics. However, our sample size is typical of
qualitative interview studies that focus on surfacing themes rather than
drawing quantitative conclusions (Distler et al., 2021).

In the screening survey, we did not ask participants whether they are
affiliated with any of the five companies whose campaigns were selected for
this study. However, participants’ open-ended answers did not indicate any
affiliation and we are not aware of any participant affiliated with any of the
companies mentioned in the study.

During the activity phase of the interview, we observed some signs of
Hawthorne effect (McCarney et al., 2007), with one Twitter participant stating
that they chose not to click on the Twitter setting button as they thought we
would like them to focus on the game. Similar observations were also seen in
other groups, where non-users often opted not to ask for help even when they
were stuck. While we gave reassurance throughout the interview by asking
participants to act as they normally would, participants may not have acted
so. After the first few participants we began listing examples of “normal”
actions, such as using Google or looking on one’s own device, which somewhat
alleviated this problem.

We also observed some signs that participants in the WhatsApp-DuckDuckGo
treatment group were primed by the first video when watching the second video.
For example, we observed instances where participants would answer a question
about campaign 2, by comparing it to campaign 1. This is mitigated somewhat by
the fact that we randomized the order in which the videos were presented to
participants.

##  6\. Discussion

Our exploratory study provides insights into the effectiveness of five
privacy-focused marketing campaigns. We distilled themes from our interviews
that led to six actionable recommendations for future privacy campaigns.
Although the campaigns we studied were all for commercial products, most of
our takeaways are likely also applicable to privacy education and awareness
campaigns from privacy advocacy groups and educational organizations.

Increased privacy awareness leads to positive impressions. Our findings
suggest that awareness of privacy features contributes to positive perceptions
of a company or its products. Campaigns that increased awareness of privacy
features improved some participants’ impressions of the associated companies,
although they had no impact on the impressions of other participants. In our
study, being privacy-focused was credited as a reason for a positive
impression of a brand both before and after exposure to the campaigns. This
shows that building a privacy focused brand could help building positive
impressions. Being suspicious of a company’s privacy practice post exposure to
the campaigns could also lead to negative impression, such as T5’s takeaway
from DataDash being that Twitter would sell user data.

Avoid invoking skepticism. Campaigns should strive to avoid raising skepticism
from viewers and avoid giving conflicting messages. Participants who voiced
skepticism towards a company usually expressed concerns about the organization
not being able to live up to claims advertised in the campaign, such as
questioning whether DuckDuckGo could make money without tracking browsing
history. Skepticism towards an organization seemed to be amplified if the
participant had prior knowledge about the organization such as exposure to
negative news (e.g. data breach), which may conflict with the message that is
conveyed in the campaign.

Be careful with visual metaphors. In designing privacy awareness
advertisements using visual metaphors, companies need to ensure that viewer
comprehension of the metaphor will not arouse negative connotations. One
example is Apple’s auctioneer being perceived as representing smartphone
companies and possibly Apple itself. This could become a double edge sword
that causes viewers to question the company’s intent as they are able to
associate the negative connotation as portrayed in the campaign with the
organization that issued the campaign. Another more obvious example is when
DataDash presented Twitter as a company that the user needed to protect
themselves from, which likely goes against any intent Twitter had with
launching DataDash. Prior work in the advertising literature cautions
advertisers to be careful when selecting visual metaphors, and our results
support this (Mohanty and Ratneshwar, 2015, 2016). Creating positive imagery
becomes particularly important when viewers are likely to be exposed to the
campaign more than once. For those who were previously exposed, continually
reinforcing a message can be effective in building a privacy-focused brand
image, or allow viewers to pick up on details that may not match the
campaign’s message.

Effective privacy ads are short and to the point. We found that the most
effective privacy awareness ads were those that were short, focusing on one
privacy feature, using a metaphor that is easy to comprehend. This supports
Bai et al.’s findings, where the concept of encryption was best communicated
when using simple wording with little technical detail (Bai et al., 2020). In
our study, we see that participants often walked away with impressions of one
main feature. For example, all Apple participants noticed the “Ask App Not to
Track” feature that was shown first, while only two noticed the “Protect Mail
Activity” feature shown later. Similarly, most participants discussed the
location permission control feature in the Samsung ad that was shown first,
while only few noticed the camera and microphone permission control features
shown later. WhatsApp and DuckDuckGo campaigns were both short and addressed a
single feature. Participants seemed to focus their attention on this feature
(Encryption for WhatsApp and Private Browsing for DuckDuckGo), and had a good
understanding of the campaign messages. Twitter’s DataDash is on the opposite
end of the spectrum, where too many abstract elements clouded the campaign’s
messages.

DataDash was not effective privacy education. Our results suggests that
DataDash was not an effective privacy education game, mainly due to lack of
connection between the game and Twitter’s privacy features and setting. In
order to use games to educate players about privacy and security concepts, the
game should be fun, relatable, and interactive (Sheng et al., 2007; Maqsood et
al., 2018). While some of our participants found DataDash fun, the game
content might have not been relatable enough to Twitter’s privacy features.
Barnard-Wills and Ashenden advised against reducing privacy education games to
a “graphical skin overlaid on [an] existing game”, with little connection
between the gaming mechanisms and online privacy systems (Barnard-Wills and
Ashenden, 2015). Our results support their view, suggesting that DataDash
likely fell under this category, with little connection between the game
mechanism and Twitter’s privacy features.

Privacy marketing may not be the most effective way to educate users about how
to use privacy features. We found that privacy ads and marketing campaigns may
not be the best way to show viewers how to use privacy tools and features. The
activity segment of the interview highlights the importance of having
demonstrations in the ad match what a user will experience. This may be
difficult when user experiences may vary. On the other hand, if the campaign
makes users aware of a feature and conveys that it is easy to use, users may
be motivated to figure out how to use that feature on their own.

To make privacy advice shown more actionable, what is shown in the campaign
should remain closely aligned with the actual product or service. From the
Apple activity, we see that the difference between the “Ask App Not to Track”
pop-up users receive after installing an app and the “Allow Apps to Request to
Track” setting can lead to confusion should viewers decide to explore the
settings as shown. Given participants’ ease of finding the settings advertised
relied on their familiarity with the OS and whether the settings interface was
intuitive, it highlights the importance of alignment between the feature
advertised and the actual feature.

Privacy campaigns may be most effective for products that don’t incur
switching costs and for building positive brand associations over time.
Privacy campaigns may not motivate non-users to switch, but could motivate
some non-users to explore the product, and could motivate users to continue
usage. Participants said they were not motivated to switch products and
pointed out that switching costs can be high, especially for smartphones.
However, if people associate a particular brand with privacy, it may sway them
to switch brands the next time they need to replace their device. In addition,
for products with lower switching costs that can be used on a trial basis
without giving up an existing service (e.g. messaging apps), privacy campaigns
may have a more immediate impact, encouraging viewers to try a service.

##  7\. Conclusion

We conducted an interview study to explore what participants took away from
five privacy campaigns. Participants gained an increased awareness of privacy
features in the products mentioned in the video ad campaigns. However,
participants exposed to campaigns that demonstrated features being used did
not gain a clear understanding of how to use those features. In addition, the
game did not communicate privacy features effectively to our participants, who
largely viewed the game as more entertaining than educational. The campaigns
that used short, focused videos with a simple metaphor seemed to communicate
most effectively about privacy issues and how consumers might protect
themselves.

##  8\. Acknowledgment

Eman Alashwali acknowledges the financial support of the Ibn Rushd Program at
King Abdullah University of Science and Technology (KAUST). This work was
funded in part by Innovators Network Foundation. The authors thank the
participants for their time and valuable insights.

## References

  * (1)
  * Abu-Salma et al. (2017) Ruba Abu-Salma, M Angela Sasse, Joseph Bonneau, Anastasia Danilova, Alena Naiakshina, and Matthew Smith. 2017.  Obstacles to the Adoption of Secure Communication Tools. In _Proc. Symposium on Security and Privacy (SP)_. 137–153.
  * Acquisti and Gross (2006) Alessandro Acquisti and Ralph Gross. 2006.  Imagined Communities: Awareness, Information Sharing, and Privacy on the Facebook. In _Proc. Privacy Enhancing Technologies_ , George Danezis and Philippe Golle (Eds.). Springer Berlin Heidelberg, 36–58.
  * Ahern et al. (2007) Shane Ahern, Dean Eckles, Nathaniel S Good, Simon King, Mor Naaman, and Rahul Nair. 2007\.  Over-Exposed? Privacy Patterns and Considerations in Online and Mobile Photo Sharing. In _Proc. Conference on Human Factors in Computing Systems (CHI)_. 357–366.
  * Akgul et al. (2022) Omer Akgul, Richard Roberts, Moses Namara, Dave Levin, and MIchelle L. Mazurek. 2022.  Investigating Influencer VPN Ads on YouTube. In _Proc. Symposium on Security and Privacy (SP)_. 876–892.
  * Apple (2022) Apple. 2022.  _Privacy on iPhone — Data Auction — Apple_.  https://www.youtube.com/watch?v=NOXK4EVFmJY Accessed on Nov. 07, 2022.
  * Bada et al. (2015) Maria Bada, Angela M. Sasse, and Jason R. C. Nurse. 2015\.  Cyber Security Awareness Campaigns: Why do they Fail to Change Behaviour?. In _International Conference on Cyber Security for Sustainable Society_. 118–131.
  * Bai et al. (2020) Wei Bai, Michael Pearson, Patrick Gage Kelley, and Michelle L Mazurek. 2020.  Improving Non-Experts’ Understanding of End-to-End Encryption: An Exploratory Study. In _Proc. European Symposium on Security and Privacy Workshops (EuroS &PW)_. 210–219.
  * Barnard-Wills and Ashenden (2015) David Barnard-Wills and Debi Ashenden. 2015.  Playing with Privacy: Games for Education and Communication in the Politics of Online Privacy.  _Political Studies_ 63, 1 (2015), 142–160.
  * Barth et al. (2019) Susanne Barth, Menno D. T. de Jong, Marianne Junger, Pieter H. Hartel, and Janina C. Roppelt. 2019\.  Putting the Privacy Paradox to the Test: Online Privacy and Security Behaviors Among Users with Technical Knowledge, Privacy Awareness, and Financial Resources.  _Telematics and Informatics_ 41 (2019), 55–69.
  * Baruh et al. (2017) Lemi Baruh, Ekin Secinti, and Zeynep Cemalcilar. 2017.  Online Privacy Concerns and Privacy Management: A Meta-Analytical Review.  _Journal of Communication_ 67, 1 (2017), 26–53.
  * Brooks and King (2014) Joanna Brooks and Nigel King. 2014.  Doing Template Analysis: Evaluating an End of Life Care Service.  _SAGE Research Methods Cases Part 1_ (2014).
  * Das et al. (2022) Sauvik Das, Cori Faklaris, Jason I. Hong, and Laura A. Dabbish. 2022.  The Security & Privacy Acceptance Framework (SPAF).  _Foundations and Trends® in Privacy and Security_ 5, 1-2 (2022), 1–143.
  * De Luca et al. (2016) Alexander De Luca, Sauvik Das, Martin Ortlieb, Iulia Ion, and Ben Laurie. 2016.  Expert and Non-Expert Attitudes Towards (Secure) Instant Messaging. In _Proc. Symposium on Usable Privacy and Security (SOUPS)_. 147–157.
  * Dechand et al. (2019) Sergej Dechand, Alena Naiakshina, Anastasia Danilova, and Matthew Smith. 2019.  In Encryption We Don’t Trust: The Effect of End-to-End Encryption to the Masses on User Perception. In _Proc. European Symposium on Security and Privacy (EuroS &P)_. 401–415.
  * Dehling et al. (2019) Tobias Dehling, Yuchen Zhang, and Ali Sunyaev. 2019\.  Consumer Perceptions of Online Behavioral Advertising. In _Proc. Conference on Business Informatics (CBI)_ , Vol. 1. 345–354.
  * Denning et al. (2013) Tamara Denning, Adam Lerner, Adam Shostack, and Tadayoshi Kohno. 2013.  Control-Alt-Hack: The Design and Evaluation of a Card Game for Computer Security Awareness and Education. In _Proc. Conference on Computer & Communications Security (CCS)_. 915–928.
  * Distler et al. (2021) Verena Distler, Matthias Fassl, Hana Habib, Katharina Krombholz, Gabriele Lenzini, Carine Lallemand, Lorrie Faith Cranor, and Vincent Koenig. 2021.  A systematic literature review of empirical methods and risk representation in usable privacy and security research.  _ACM Transactions on Computer-Human Interaction (TOCHI)_ 28, 6 (2021), 1–50.
  * DuckDuckGo (2022) DuckDuckGo. 2022\.  _DuckDuckGo: Watching You_.  https://www.youtube.com/watch?v=QWpPyYlZXNI Accessed on Jan. 24, 2023.
  * Emami-Naeini et al. (2023) Pardis Emami-Naeini, Janarth Dheenadhayalan, Yuvraj Agarwal, and Lorrie Faith Cranor. 2023\.  Are Consumers Willing to Pay for Security and Privacy of IoT Devices?. In _Proc. USENIX Security Symposium_.
  * Espósito (2021) Filipe Espósito. 2021\.  _WhatsApp Vulnerability Could Lead to Sensitive Data Leakage_.  https://9to5mac.com/2021/09/02/whatsapp-vulnerability-could-lead-to-sensitive-data-leakage Accessed on Oct. 09, 2023.
  * Fadilpašić (2022) Sead Fadilpašić. 2022\.  _DuckDuckGo in Hot Water over Hidden Tracking Agreement with Microsoft_.  https://www.techradar.com/news/duckduckgo-in-hot-water-over-hidden-tracking-agreement-with-microsoft Accessed on Oct. 09, 2023.
  * Google Ads (2020) Google Ads. 2020\.  _A Year in Privacy | Privacy Week at Google_.  https://www.youtube.com/watch?v=zmhMA1Xiukk Accessed on Nov. 07, 2022.
  * Hurlerand and Wodinsky (2022) Kevin Hurlerand and Shoshana Wodinsky. 2022.  _Twitter’s New Privacy Policy Is a Video Game That Sucks_.  https://gizmodo.com/twitter-privacy-policy-video-game-data-dash-1848912387 Accessed on Oct. 28, 2023.
  * Jeong and Kim (2017) Yongick Jeong and Yeuseung Kim. 2017.  Privacy Concerns on Social Networking Sites: Interplay Among Posting Types, Content, and Audiences.  _Computers in Human Behavior_ 69 (2017), 302–310.
  * Kelley et al. (2013) Patrick Gage Kelley, Lorrie Faith Cranor, and Norman Sadeh. 2013\.  Privacy as Part of the App Decision-Making Process. In _Proc. Conference on Human Factors in Computing Systems (CHI)_. 3393–3402.
  * Kim et al. (2017) Eunjin Kim, S Ratneshwar, and Esther Thorson. 2017.  Why Narrative Ads Work: An Integrated Process Explanation.  _Journal of Advertising_ 46, 2 (2017), 283–296.
  * King (2024) Nigel King. 2024\.  _Template Analysis_.  Accessed on: Feb. 13 2024.
  * Kross et al. (2021) Sean Kross, Eszter Hargittai, and Elissa M. Redmiles. 2021\.  Characterizing the Online Learning Landscape: What and How People Learn Online.  _Proc. ACM Hum.-Comput. Interact._ 5, CSCW1, Article 146 (2021), 19 pages.
  * Landa (2016) Robin Landa. 2016\.  _Advertising by Design: Generating and Designing Creative Ideas Across Media_.  John Wiley & Sons.
  * Laurence (2018) Dessart Laurence. 2018\.  Do Ads that Tell a Story Always Perform Better? The Role of Character Identification and Character Type in Storytelling Ads.  _International Journal of Research in Marketing_ 35, 2 (2018), 289–304.
  * Liu et al. (2011) Yabing Liu, Krishna P Gummadi, Balachander Krishnamurthy, and Alan Mislove. 2011.  Analyzing Facebook Privacy Settings: User Expectations vs. Reality. In _Proc. Internet Measurement Conference (IMC)_. 61–70.
  * Madejski et al. (2012) Michelle Madejski, Maritza Johnson, and Steven M Bellovin. 2012\.  A Study of Privacy Settings Errors in an Online Social Network. In _Proc. Pervasive Computing and Communications Workshops_. 340–345.
  * Manyiwa and Brennan (2012) Simon Manyiwa and Ross Brennan. 2012.  Fear Appeals in Anti-Smoking Advertising: How Important is Self-Efficacy?  _Journal of Marketing Management_ 28, 11-12 (2012), 1419–1437.
  * Maqsood et al. (2018) Sana Maqsood, Christine Mekhail, and Sonia Chiasson. 2018\.  A Day in the Life of Jos: A Web-based Game to Increase Children’s Digital Literacy. In _Proc. Conference on Interaction Design and Children_. 241–252.
  * McCarney et al. (2007) Rob McCarney, James Warner, Steve Iliffe, Robbert Van Haselen, Mark Griffin, and Peter Fisher. 2007.  The Hawthorne Effect: a randomised, controlled trial.  _BMC medical research methodology_ 7, 1 (2007), 1–8.
  * Mohanty and Ratneshwar (2015) Praggyan (Pam) Mohanty and S. Ratneshwar. 2015.  Did You Get It? Factors Influencing Subjective Comprehension of Visual Metaphors in Advertising.  _Journal of Advertising_ 44, 3 (2015), 232–242.
  * Mohanty and Ratneshwar (2016) Praggyan (Pam) Mohanty and S. Ratneshwar. 2016.  Visual Metaphors in Ads: The Inverted-U Effects of Incongruity on Processing Pleasure and Ad Effectiveness.  _Journal of Promotion Management_ 22, 3 (2016), 443–460.
  * Moscaritolo (2017) Angela Moscaritolo. 2017\.  _Google’s “Interland” Game Makes Online Safety Education Fun_.  https://www.pcmag.com/news/googles-interland-game-makes-online-safety-education-fun Accessed on Nov. 24, 2023.
  * O’Flaherty (2022) Kate O’Flaherty. 2022\.  _Apple Slams Facebook and Google With Bold New Privacy Ad_.  https://www.forbes.com/sites/kateoflahertyuk/2022/05/25/apple-slams-facebook-and-google-with-bold-new-privacy-ad Accessed on Nov. 07, 2022.
  * Persuasive Games (2023) LLC Persuasive Games. 2023\.  _Persuasive Games_.  https://persuasivegames.com/games Accessed on Feb. 15, 2024.
  * Prolific (2023) Prolific. 2023.  _Prolific_.  https://www.prolific.co Accessed on Sep. 11, 2023.
  * Rader (2014) Emilee Rader. 2014\.  Awareness of Behavioral Tracking and Information Privacy Concern in Facebook and Google. In _Proc. Symposium On Usable Privacy and Security (SOUPS)_. 51–67.
  * Redmiles et al. (2020) Elissa M. Redmiles, Noel Warford, Aritha Jayanti, Aravind Koneru, Sean Kross, Miraida Morales, Rock Stevens, and Michelle L. Mazurek. 2020\.  A Comprehensive Quality Evaluation of Security and Privacy Advice on the Web. In _Proc. USENIX Security Symposium_. 89–108.
  * Samsung (2022) Samsung. 2022\.  _Samsung Privacy: You’re in Control_.  https://www.youtube.com/watch?v=kZKK80urUZc Accessed on Nov. 07, 2022.
  * Sheng et al. (2007) Steve Sheng, Bryant Magnien, Ponnurangam Kumaraguru, Alessandro Acquisti, Lorrie Faith Cranor, Jason Hong, and Elizabeth Nunge. 2007\.  Anti-Phishing Phil: The Design and Evaluation of a Ggame That Teaches People Not to Fall for Phish. In _Proc. USENIX Security Symposium_. 88–99.
  * Sohoraye et al. (2015) Mrinal Sohoraye, Vandanah Gooria, Suniti Nundoo-Ghoorah, and Premanand Koonjal. 2015.  Do you Know Big Brother is Watching you on Facebook? A Study of the Level of Awareness of Privacy and Security Issues Among a Selected Sample of Facebook Users in Mauritius. In _Proc. International Conference on Computing, Communication and Security (ICCCS)_. 1–7.
  * Stransky et al. (2021) Christian Stransky, Dominik Wermke, Johanna Schrader, Nicolas Huaman, Yasemin Acar, Anna Lena Fehlhaber, Miranda Wei, Blase Ur, and Sascha Fahl. 2021\.  On the Limited Impact of Visualizing Encryption: Perceptions of E2E Messaging Security. In _Proc. Symposium on Usable Privacy and Security (SOUPS)_. 437–454.
  * The Police (2023a) The Police. 2023a.  _The Police - Every Breath You Take (Official Music Video)_.  https://www.youtube.com/watch?v=OMOGaugKpzs Accessed Oct. 18, 2023.
  * The Police (2023b) The Police. 2023b.  _The Police Official Website_.  https://thepolice.com Accessed on Oct. 18, 2023.
  * Thompson and Irvine (2011) Michael Thompson and Cynthia Irvine. 2011.  Active Learning with the CyberCiege Video Game. In _Proc. Workshop on Cyber Security Experimentation and Test (CSET)_.
  * Tsai et al. (2011) Janice Y Tsai, Serge Egelman, Lorrie Cranor, and Alessandro Acquisti. 2011.  The Effect of Online Privacy Information on Purchasing Behavior: An Experimental Study.  _Information Systems Research_ 22, 2 (2011), 254–268.
  * Twitter (2022) Twitter. 2022.  _Twitter Data Dash: Level Up Your Privacy Game_.  https://twitterdatadash.com Accessed on Nov. 07, 2022.
  * Ulanoff (2022) Lance Ulanoff. 2022\.  _Apple’s data auction privacy ad is only scary because it’s true_.  https://www.techradar.com/news/apples-data-auction-privacy-ad-is-only-scary-because-its-true Accessed on Oct. 28, 2023.
  * Ur et al. (2012) Blase Ur, Pedro Giovanni Leon, Lorrie Faith Cranor, Richard Shay, and Yang Wang. 2012.  Smart, Useful, Scary, Creepy: Perceptions of Online Behavioral Advertising. In _Proc. Symposium on Usable Privacy and Security (SOUPS)_. 1–15.
  * WhatsApp (2022) WhatsApp. 2022\.  _A New Era of Personal Privacy with Default End-to-Encryption_.  https://www.youtube.com/watch?v=zvI4cVGWJhM Accessed on Nov. 07, 2022.
  * Winkie (2022) Lance Winkie. 2022\.  _Data the dog: Twitter turns its privacy policy into an old-school video game_.  https://www.theguardian.com/technology/2022/may/14/twitter-data-dash Accessed on Oct. 28, 2023.

##  Appendix A Appendices

In this section, we present the screening survey (Appendix A.1) and interview
script (Appendix A.2). Text inside [square brackets] was not shown or read to
survey and interview participants. Interview section headings were not read to
interview participants. The nature of semi-structured interviews allowed the
interviewer to slightly deviate from the interview script without affecting
the essence of the interview.

###  A.1. Screening Survey

Section 1 - General
Q.1: What is your gender?
Answers: $\circ$ “Male”  $\circ$ “Female”  $\circ$ “Prefer not to answer”
$\circ$ “Prefer to self describe (please specify)”
Q.2: What is your age group?
Answers: $\circ$ “18 - 25 years”  $\circ$ “26 - 35 years”  $\circ$ “36 - 45
years” $\circ$ “46 - 55 years” $\circ$ “56 - 65 years” $\circ$ “65 -75 years”
$\circ$ “76 years or more”
Q.3: Which of the following best describes your highest achieved education
level?
Answers: $\circ$ “Some high school, no degree”  $\circ$ “High school graduate”
$\circ$ “Some college, no degree” $\circ$ “Associate’s degree” $\circ$
“Bachelor’s degree” $\circ$ “Graduate degree (Masters, Doctorate, etc.”
$\circ$ “Other (Please specify)”
Q.4: Do you have (or are currently studying for) a degree, or work (or have
worked) in one or more of the following areas: Computer Science, Information
Systems, Information Technology, Computer Engineering, or a related field?
Answers: $\circ$ “Yes”  $\circ$ “No”

Section 2 - Phone
Q.5: Do you use a smartphone?
Answers: $\circ$ “Yes”  $\circ$ “No”
[If Q.5 answer is “No”, the participant skips the rest of this section and
moves to Section 3]
Q.6: Approximately, how often do you use your smartphone?
Answers: $\circ$ “At least once a day”  $\circ$ “At least once a week”
$\circ$ “At least once a month”  $\circ$ “Less than once a month” $\circ$ “I
do not remember” $\circ$ “Other (Please specify)”
Q.7: Please indicate your current, most used smartphone brand:
Answers: $\circ$ “Apple smartphone”  $\circ$ “Samsung smartphone”  $\circ$
“Other Android smartphone”  $\circ$ “Other (Please specify)”
Q.8: Approximately, how long have you been using your current, most used
smartphone device?
Answers: $\circ$ “Less than a month”  $\circ$ “At least for 3 months”  $\circ$
“At least for 6 month”  $\circ$ “At least for a year”  $\circ$ “At least for 2
years”  $\circ$ “At least for 3 years”  $\circ$ “More than 3 years”  $\circ$
“Other (Please specify)”
Q.9: Have you switched mobile operating systems (OS), like from Apple to
Android, in the last 5 years?
Answers: $\circ$ “No, I have always used the same OS”  $\circ$ “Yes, from
Apple to Android”  $\circ$ “Yes, from Android to Apple”  $\circ$ “I bounced in
between different OS - e.g., Apple to Android then back to Apple”  $\circ$ “I
owned two phones at the same time, one Apple and one Android.”  $\circ$ “Other
(Please specify)”

Section 3 - Gaming
Q.10: Do you play or participate in any form of electronic, digital, and/or
video gaming?
Answers: $\circ$ “Yes”  $\circ$ “No”
[If Q.10 answer is “No”, the participant skips the rest of this section, and
moves to Section 4]
Q.11: Approximately, how often do you play electronic games?
Answers: $\circ$ “At least once a day”  $\circ$ “At least once a week”
$\circ$ “At least once a month”  $\circ$ “Less than once a month” $\circ$ “I
do not remember” $\circ$ “Other (Please specify)”
Q.12: Please indicate all forms of electronic gaming you regularly participate
in:
Answers: $\circ$ “PlayStation games”  $\circ$ “Nintendo Switch games”  $\circ$
“PC games (e.g., League of Legends, SIM4, Minecraft, browser games like Tetris
and other similar computer games)”  $\circ$ “VR games (e.g. Beat Saber, Super
Hot and other similar games)”  $\circ$ “Other (Please specify)”

Section 4 - Social Media
Q.13: Do you use social media?
Answers: $\circ$ “Yes”  $\circ$ “No”
[If Q.13 answer is “No”, the participant skips the rest of this section, and
moves to Section 5]
Q.14: Please indicate all social media platform you use:
Answers: $\circ$ “Facebook”  $\circ$ “Instagram”  $\circ$ “Twitter”  $\circ$
“LinkedIn”  $\circ$ “Snapchat”  $\circ$ “TikTok”  $\circ$ “Reddit”  $\circ$
“Other (Please specify)”
[If Q.14 answer does not include “Twitter”, the participant skips the rest of
this section, and moves to Section 5]
Q.15: Approximately, how often do you use Twitter?
Answers: $\circ$ “At least once a day”  $\circ$ “At least once a week”
$\circ$ “At least once a month”  $\circ$ “Less than once a month” $\circ$ “I
do not remember” $\circ$ “Other (Please specify)”

Section 5 - Instant Messaging
Q.16: Do you use instant messaging?
Answers: $\circ$ “Yes”  $\circ$ “No”
[If Q.16 answer is “No”, the participant skips the rest of this section, and
moves to Section 6]

Q.17: Please indicate all form of instant messaging service you use:
Answers: $\circ$ “iMessage”  $\circ$ “Facebook Messenger”  $\circ$ “Telegram”
$\circ$ “WhatsApp”  $\circ$ “Discord”  $\circ$ “WeChat”  $\circ$ “Slack”
$\circ$ “SMS text”  $\circ$ “Other (Please specify)”
[If Q.17 answer does not include “WhatsApp”, the participant skips Q.18 and
moves to Section 6]
Q.18: Approximately, how often do you use WhatsApp?
Answers: $\circ$ “At least once a day”  $\circ$ “At least once a week”
$\circ$ “At least once a month”  $\circ$ “Less than once a month” $\circ$ “I
do not remember” $\circ$ “Other (Please specify)”

Section 6 - Internet Browsing
Q.19: Do you use search engines and/or web browsers (e.g. Google Chrome,
Safari)?
Answers: $\circ$ “Yes”  $\circ$ “No”
[If Q.19 answer is “No”, the participant skips the rest of this section, and
the survey is completed]
Q.20: Please select all search engines you use:
Answers: $\circ$ “Google”  $\circ$ “Bing”  $\circ$ “Yahoo”  $\circ$ “Baidu”
$\circ$ “AOL”  $\circ$ “Naver”  $\circ$ “Ask.com”  $\circ$ “DuckDuckGo”
$\circ$ “Other (Please specify)”
[If Q.20 answer does not include “DuckDuckGo”, the participant skips Q.21 and
moves to Q.22]
Q.21: Approximately, how often do you use DuckDuckGo?
Answers: $\circ$ “At least once a day”  $\circ$ “At least once a week”
$\circ$ “At least once a month”  $\circ$ “Less than once a month” $\circ$ “I
do not remember” $\circ$ “Other (Please specify)”
Q.22: Please select all web browsers you use:
Answers: $\circ$ “Google Chrome”  $\circ$ “Microsoft Edge”  $\circ$ “Mozilla
Firefox”  $\circ$ “Safari”  $\circ$ “DuckDuckGo”  $\circ$ “Vivaldi”  $\circ$
“Brave”  $\circ$ “Opera”  $\circ$ “Other (Please specify)”
[If Q.22 answer does not include “DuckDuckGo”, the participant skips to Q.23,
and the survey is completed]
Q.23: Approximately, how often do you use DuckDuckGo?
Answers: $\circ$ “At least once a day”  $\circ$ “At least once a week”
$\circ$ “At least once a month”  $\circ$ “Less than once a month” $\circ$ “I
do not remember” $\circ$ “Other (Please specify)”

###  A.2. Semi-structured Interview Script

Introduction

Thank you for participating in our study. As specified in the consent form, we
will be recording the interview to ensure that we do not miss parts of the
conversation. Your name will not be associated with any data collected during
this interview. Please leave your camera off during this process. The
interview will last around 20 to 25 minutes.

[Specific for Twitter] During the interview, you will be asked to share your
screen for an online gaming activity, which will be screen recorded. We will
not ask you to show any private information, such as photos, mail or any other
related setting.

Before we start, do you have any questions regarding the consent form? Could I
have your permission to start the recording?

Thank you, I have started the recording. If there are short silences during
the interview, please excuse them - I am also taking notes during this
process. There are no right or wrong answers to what I am asking, let’s begin.

[All [Prompt] questions are optional. A company name between square brackets
[Company] means this question is specific for participants in that treatment
group.]

Section 1 - All Campaigns
[Interviewer asks different variations of the question based on participants’
assigned treatment group and screening survey results. For Apple and Samsung
groups we use the term “smartphone”; for WhatsApp we use “instant messaging”;
fro Twitter we use “social media”; and for DuckDuckGo we use “browser and
search engine”. For all smartphone groups we use the term “device”; for apps
groups we use “service.”]

Q.1: What is your most preferred (smartphone — instant messaging — social
media — browser and search engine) (device — service)? Q.1A: Why is [Q.1
answer] your most preferred (smartphone — instant messaging — social media —
browser and search engine)?

[For Q.2 to Q.4, ask the question that aligns with the participants’ treatment
group.]

Q.2: [Twitter/WhatsApp User] What is your main purpose of using (Twitter —
WhatsApp)? Q.2A: [WhatsApp Non-User] Do you actively participate in instant
message chats and group discussions?

Q.3: [Apple/Samsung] Do you actively use smartphone apps and Internet browsing
using your smartphone? Q.3A: To the best of your knowledge, when was the last
time you updated your phone?

Q.4: [DuckDuckGo] Do you actively browse the web and perform search queries in
search engines? Q.4A: Do you use the same browser and search engine on your
desktop and mobile?
[Prompt] Why/Why not?

Q.5: What is your general impression of (Apple — Samsung — WhatsApp — Twitter
— DuckDuckGo) as an organization? Q.5A: [WhatsApp] What about your impression
of Meta?
[Prompt] Is it the same with WhatsApp? [Apple, Samsung, WhatsApp & DuckDuckGo
participants moves to Section 2. Twitter participants moves to Section 3.]

Section 2 - Video Instruction (Apple, Samsung, WhatsApp & DuckDuckGo)

I am now going to share my screen, and show you an advertisement from (Apple —
Samsung — WhatsApp — DuckDuckGo)

[Show video, then participants move to Section 4.]

Section 3 - Game Instruction (Twitter)

Next, please open Twitter and log out of your own account. Once you have
logged out please let me know, I will send you a username and password to log
into an account we have created for the study.

Now that you have logged in, I am going to send you a link to an online video
game hosted by Twitter, called Data Dash. Once you have accessed the web page,
please share your screen with me, and narrate your thoughts as we progress
through each level. Please start from level 1 and we will have about 8 minutes
for this activity.

[Share DataDash Link, allow for 8 minutes of play time.]

Thank you, you can now stop screen sharing, Please return to Twitter and log
out of the account.

[Participants move to Section 4.]

Section 4 - Post Exposure

Q.6: Have you (seen this video — heard about DataDash) before? Q.6A: [If Q.6
answer is “Yes”] Where did you (see this advertisement — hear about DataDash)?
Q.6B: [If Q.6 answer is “No” - Continue to next question.]

Q.7: In your own words, could you describe the (ad — game) you had just
(watched — played)?

Q.8: How do you feel about (the ad we just saw — the game you just played)?
Q.8A: What stood out to you?
[Positive prompt] What did you like about the (ad — game)?
[Negative prompt] What did you dislike about the (ad — game)?
Q.8B: [Twitter] Why did you (click — not click) on the setting button at the
end of the level?

Q.9: What do you think was the main purpose of the (video — game)?

Q.10: Based on your impression of the (ad — game), what do you think (Apple -
Auctioneer — Samsung - yellow umbrella — WhatsApp - Pigeon — Twitter -
envelope avatar — DuckDuckGo - Vocalist) represents?

Q.11: On a scale of 1 - 5, with 1 being Poor and 5 being Excellent, how well
do you think the (ad — game) communicated that message? [Prompt] Why did you
give it that rating?

[Q12A is for the ad campaign participants, 12B for the game participants.]
Q.12A: [Apple / Samsung / WhatsApp / DuckDuckGo] If you came across this
advertisement online while browsing the internet, how long would you have
stayed with the video?
Q.12B: [Twitter] If you came across this game while using Twitter, would you
have played the game? [Prompt] How long would you have played it?
[Prompt] Why / Why not?

Q.13: Prior to watching this video/playing this game, were you aware of (Apple
& Samsung - smartphone features shown — WhatsApp - end-to-end encryption —
Twitter - privacy settings — DuckDuckGo - web tracking)? Q.13A: How would you
describe (Apple & Samsung - the features shown — WhatsApp - end-to-end
encryption — Twitter - privacy settings — DuckDuckGo - web tracking)?
Q.13B: How did you come across it (or across this term for end-to-end
encryption and web tracking)?

Q.14: Having (seen this advertisement — played the game), what is your
impression of (Apple — Samsung — WhatsApp — Twitter — DuckDuckGo)?

[Apple, Samsung, Twitter participants move to Section 5. WhatsApp & DuckDuckGo
participants move to Section 9.]

Section 5 - Actionable Steps [Apple/Samsung/Twitter]

Q.15: Relating back to the ad, What actions do you think the advertisement was
suggesting you could take?

Q.16: What are the steps you would need to take to complete those actions?
[Prompt] How would you go about this?

Q.17: On a scale of 1 not at all motivated to 5, extremely motivated, how
would you rank your motivation to (perform the steps — go to the game-level
setting) shown in the (advertisement — game)?

Q.18: On a scale of 1 not at all confident to 5, fully confident, how would
you rank your confidence in being able to (perform the steps — go to the game-
level setting) shown in the (advertisement — game)?

[Apple, Samsung to Section 6. Twitter to Section 7]

Section 6 - Activity Instruction (Apple/Samsung)

Let’s try a quick exercise. I am going to share my phone screen, next, you can
give me verbal directions on where to go in order to find [Apple (1) Access
App tracking permission page, (2)Turn on Protection Mail Activity; Samsung (1)
Change Location permission for Weather App, (2) Turn On/Off overall microphone
permission for Zoom.] During this process, feel free to try different
approaches, ask to start from the beginning or request hints if needed.

[Link phone to screen share into Zoom - Allow for 3 minute max for both
activty (1) and (2) combined, do not reveal time to participants]

Thank you.

[Proceed to Section 8]

Section 7 - Activity Instruction (Twitter)

I am now going to share my screen and show you the Twitter Home page. Next,
please give me verbal directions on how to reach ad preferences setting.

[Watch for activity time, do not exceed 3 minutes max]

Thank you

[Proceed to Section 8]

Section 8 - Activity Feedback (Apple/Samsung/Twitter)

Q.19: How do you feel about the exercise we just did?

Q.20: Having done the exercise, is there something new you have learned?
[Prompt] How do you feel about the (ad — game) now?
[Prompt] Anything new you remembered about the (ad — game)?

Section 9 - User Motivation

[For non-users]

Q.21: Did the (ad — game) motivate you to consider switching from your current
(Apple & Samsung - smartphone — WhatsApp - instant messaging — Twitter -
social media — DuckDuckGo - browser and search engine) to the (ad — game)’s
(smartphone — instant messaging — social media — browser)? [Prompt] Why?
[Prompt] Why not?

[For users]

Q.22 Did the (ad — game) motivate you to continue using your (Apple & Samsung
- smartphone — WhatsApp - instant messaging — Twitter - social media —
DuckDuckGo - browser and search engine) rather than switching to a different
one? [Prompt] Why?
[Prompt] Why not?

[Twitter skips Q23, interview completed.]

Let’s watch the video together one more time.

[Show advertisement again.]

Q.23: Having seen the advertisement again, did you notice anything new?

[Proceed to End. WhatsApp DuckDuckGo group start again at S1, repeats entire
process]

Thank you, that concludes the end of the interview. I will now stop the
recording. Do you have any concluding remarks or questions regarding what we
did today?

###  A.3. Ads Screens

ddg-01

(a) The singer shoulder-surfs a person with their mobile browsing the
Internet.

ddg-02

(b) A screen showing a search using DuckDuckGo.

ddg-03

(c) A screen showing medical web page using DuckDuckGo showing trackers
blocked.

ddg-04

(d) After the person started using DuckDuckGo, the singer dragged away by an
invisible force.

Figure 9. Examples of DuckDuckGo’s ad screens.

apple-01

(a) Data auction.

apple-02

(b) The protagonist using the “Ask App Not To Track” feature.

apple-03

(c) The protagonist using the “Protect Mail Activity” feature.

apple-04

(d) The auction participants disappearing into smoke after the protagonist
used Apple’s privacy features.

Figure 10. Examples of Apple’s ad screens.

twitter-01

(a) During level 1 of the game.

twitter-02

(b) End of level 1 of the game.

twitter-03

(c) Start of level 2 of the game.

twitter-04

(d) During level 2 of the game

Figure 11. Examples of Twitter’s game screens.

samsung-01

(a) Samsung’s privacy dashboard showing which apps accessing which permission
(e.g. camera, microphone, and location).

samsung-02

(b) Protagonist turned off precise location for the Weather app, portrayed by
a yellow umbrella.

samsung-03

(c) Protagonist turned off precise location for the Weather app.

samsung-04

(d) Protagonist uses location access to control when to allow an app access
their location.

Figure 12. Examples of Samsung’s ad screens.

whatsapp-01

(a) The post office.

whatsapp-02

(b) The post office officer uses pigeons as a message delivery method.

whatsapp-03

(c) The ad states that 5.5 billion texts per day are sent without encryption.

whatsapp-04

(d) The ad states that with WhatsApp, your messages will not be one of them
(the encrypted messages).

Figure 13. Examples of WhatsApp’s ad screens.

###  A.4. Participant Overview

Table 2. Participant Overview. In the “User Type” column, U represent users,
and N represents non-users, W represents WhatsApp, and D represents
DuckDuckGo. The “Tech” column indicates whether participants indicated in the
screening survey that they had any experience learning or working in
information technology or a related field.

ID | Gender | Age | User Type | Tech | Recruitment
---|---|---|---|---|---
A1 | F | 46 - 55 | U | No | Prolific
A2 | F | 18 - 25 | N | No | Prolific
A3 | M | 46 - 55 | N | Yes | Prolific
A4 | M | 26 - 35 | N | No | Prolific
A5 | F | 26 - 35 | U | Yes | Snowball
A6 | M | 26 - 35 | U | Yes | Snowball
S1 | F | 36 - 45 | U | No | Prolific
S2 | M | 56 - 65 | N | No | Prolific
S3 | M | 46 - 55 | U | Yes | Prolific
S4 | M | 26 - 35 | U | No | Snowball
S5 | F | 18 - 25 | N | Yes | Snowball
S6 | F | 18 - 25 | N | Yes | Snowball
WD1 | M | 36 - 45 | W-U; D-N | No | Prolific
WD2 | M | 26 - 35 | W-N; D-U | No | Prolific
WD3 | F | 26 - 35 | W-U; D-N | No | Snowball
WD4 | M | 18 - 25 | W-U; D-U | Yes | Snowball
WD5 | M | 36 - 45 | W-N; D-U | Yes | Prolific
WD6 | M | 18 - 25 | W-N; D-N | No | Prolific
T1 | M | 26 - 35 | U | No | Prolific
T2 | F | 26 - 35 | U | Yes | Prolific
T3 | M | 46 - 55 | U | No | Prolific
T4 | F | 18 - 25 | U | No | Snowball
T5 | F | 18 - 25 | U | No | Snowball
T6 | F | 18 - 25 | U | Yes | Snowball

###  A.5. Diagram for Our Qualitative Analysis Coding

Figure 14. A diagram summarizing our qualitative analysis coding.

direction switch/.style= forked edges, for tree= align=left, edge+=thick,
font=, , where level=1minimum width=13em, where level¡=1draw=red, where
level¿=1folder, grow’=0, ,

direction switch [What Do Privacy Advertisements Communicate to Consumers?
[Preferences [Preferred product
(Table 3) [Reasons for preferred product
(Table 4)] ] ] [Attitudes [Organization impression pre-/post-exposur
(Table 5) [Reasons for positive impression pre-exposure
(Table 6)] [Reasons for negative impression pre-exposure
(Table 7)] [Reasons for neutral impression pre-exposure
(Table 8)] ] [Change in impression post-exposur
(Table 9) [Reasons for change in impression post-exposure
(Table 10)] ] [Prior exposure to the campaign
(Table 11) [Effect of re-exposure to the ad campaign
(Table 12)] ] ] [Awareness [Privacy awareness
(Table 13) [Privacy awareness types
(Table 14) [Privacy issues types
(Table 15)] [Privacy features types
(Table 16)] ] ] [Perceived intent of the campaign
(Table 17)] [Prior awareness of advertised features
(Table 18) [How features encountered
(Table 19)] ] [Understanding Metaphors
(Table 20)] ] [Actionability [Task (activity) Evaluation criteria
(Table 21)] [Non-user motivation
to switch the product
(Table 22) [Reasons to switch
(Table 23)] [Reasons to not-switch
(Table 24)] [Reasons to to maybe-switch
(Table 25)] ] [User motivation
to continue using the product
(Table 26) [Reasons to continue
(Table 27)] [Reasons for no-impact to continue
(Table 28)] [Reasons for neutral motivation
(Table 29)] ] ] ]

###  A.6. Codebook

In this section, we list our codebook divided into tables. Each table
represents the codebook for a question or a set of related questions. All
codes are in small letters only. Figure 14 represents a diagram that draws the
overall picture of the codes hierarchy.

Table 3. Codebook for Q.1 about participants’ preferred product. Note that the
participant preferred products can be different from the product of the
study’s group the participant was assigned to.

Q.1: Preferences $\rightarrow$ Preferred product
---
Code (# occurrence) [non-exclusive] |  Code Definition |  Example
apple (6) |  The Apple smartphone |  “iPhone.” (A1); “Apple.” (A5)
samsung (5) |  The Samsung smartphone |  “Samsung galaxy.” (S3); “Samsung.” (S4)
engine-google (4) |  The Google search engine |  “Google.” (WD3, DuckDuckGo)
instagram (3) |  The Instagram app |  “Instagram, I’d say.” (T1)
browser-google (3) |  The Google web browser |  “I use Google and Safari on my laptops” (WD4, DuckDuckGo)
lg (2) |  The LG smartphone |  “Samsung and LG.” (A4)
facebook-messenger (2) |  The Facebook messenger app |  “I use Facebook messenger usually to communicate with friends.” (WD2, WhatsApp)
twitter (2) |  The Twitter app |  “Probably Twitter.” (T2)
facebook (1) |  The Facebook app |  “Facebook is primarily what I use.” (T3)
tiktok (1) |  The TikTok app |  “Probably TikTok.” (T6)
discord (1) |  The Discord app |  “I use Discord the most.” (WD1, WhatsApp)
browser-firefox (1) |  The Firefox web browser |  “My preferred browser is Firefox” (WD5, DuckDuckGo)
browser-edge (1) |  The Edge web browser |  “I alternate between Microsoft Edge and Google Chrome.” (WD6, DuckDuckGo)
browser-safari (1) |  The Safari web browser |  “I use Google and Safari on my laptops” (WD4, DuckDuckGo)
browser-ddg (1) |  The DuckDuckGo web browser |  “And I use DuckDuckGo and, mostly DuckDuckGo on my phone” (WD4, DuckDuckGo)
engine-ddg (1) |  The DuckDuckGo search engine |  “Sometimes I’ll use DuckDuckGo or Bing” (WD2, DuckDuckGo)
engine-bing (1) |  The Bing search engine |  “Sometimes I’ll use DuckDuckGo or Bing” (WD2, DuckDuckGo)
whatsapp (1) |  The WhatsApp app |  “I prefer using Whatsapp.” (WD4, WhatsApp)
snapchat (1) |  The Snapchat app |  “I generally prefer Snapchat as an instant messaging service.” (WD6, WhatsApp)
telegram (1) |  The Telegram app |  “Telegram.” (WD5, WhatsApp)

Table 4. Codebook for Q.1A about reasons for participants’ preferred product.
Note that the participant preferred products can be different from the product
of the study’s group the participant was assigned to.

Q.1A: Preferences $\rightarrow$ Reason for preferred product
---
Code (# occurrence) [non-exclusive] |  Code Definition |  Example
familiarity (8) |  The participant is familiar with the product, or has been using it for long time |  “because I am used to it.” (A4)
privacy (7) |  The product has good privacy |  “the privacy features.” (WD5, DuckDuckGo)
sync (5) |  The product is easy to synchronize with other products |  “it is a lot easier to sync data.” (A5)
used-by-peers-family (5) |  The product is used by peers and family |  “most of the people I want to contact on that app” (WD3, WhatsApp)
ease-use (3) |  The product is easy to use |  “pretty simple and easy to use” (A3)
long-lasting (2) |  The product lasts for long time |  “The device has pretty long life” (S6)
good-product (2) |  The product is good generally |  “So LGs always worked very well” (A3)
focused-function (2) |  The product is focused on one function |  “it’s just always videos.” (T6)
performance (2) |  The product has good performance |  “It seems to be faster.” (S2)
skeptical (2) |  The participant has skepticism about the campaign |  “end-to-end encryption. Even though I’m not 100% sure about that anymore.” (WD4, WhatsApp)
open-env (1) |  The product is not restricted to the company’s app store |  “I like that you are able to download apps outside of the Google play store” (A2)
security (1) |  The product has good security |  “your browsing, is, is private and secure.” (WD5, DuckDuckGo)
switched-bad-product (1) |  The participant switched product because of previous bad experience with other products |  “I had a Google phone before … it was really slow, and it crashed a lot … so I switched.” (A6)
fun (1) |  The product is fun |  “because it’s very humorous” (T4)
developed-product (1) |  The product is developed |  “it’s kind of the more developed search engine ” (WD2, DuckDuckGo)
popularity (1) |  The product is popular |  “this is the most popular one that i’ve or It’s the one that I sort of am most aware of.” (WD3, DuckDuckGo)
good-interface (1) |  The product has good interface |  “Has a good interface overall.” (S6)
personalized (1) |  The product is personalized |  “it has channels to design toward my interest” (WD1, WhatsApp)
combined-function (1) |  The product combines multiple functions |  “but kind of all rolled up into one with like stories and messages, but also posting.” (T5)

Table 5. Codebook for Q.5 & Q.14 about participants’ impression about the
advertising organization. The codes used for both pre-exposure (Q.5) or post-
exposure (Q.14) to the campaign, except codes prefixed with “-plus” are
applicable only for post-exposure.

Q.5 & Q.14: Attitudes $\rightarrow$ Organization impression pre-/post-exposure
---
Code (# occurrence Q.5; Q14)
[exclusive] |  Code Definition |  Example
positive (13; 15) |  The participant expressed positive impression only |  “I have positive feelings about it.” (WD5, DuckDuckGo)
mixed-pos (7; 5) |  The participant expressed mixed positives and negatives impression, with stronger positive. Stronger positive means either more positive points mentioned, or if equal positives and negative points mentioned, then positive came first |  “They seem better than the other tech companies. Relatively. They sell hardware rather than selling my data. I guess they probably sell my data, but they seem to do it less. So I like that.” (A6)
neutral (4; 0) |  The participant expressed neutral impression |  “I don’t really have an impression.” (T5)
mixed-neg (3; 2) |  The participant expressed mixed negative and positive impression, with stronger negative. Stronger negative means either more negative points mentioned, or if equal negative and positive points mentioned, then negative came first |  “I usually don’t trust the parent company Meta, but they seem to be doing all right.” (WD5, WhatsApp)
negative (3; 3) |  The participant expressed negative impression only |  “it’s not great.” (T2)
pos-plus (n.a.; 4) |  The participant expressed even more positive. This code is applicable only post-exposure to the campaign |  “Probably have a better impression on Apple.” (A5)
neg-plus (n.a.; 1) |  The participant expressed even more negative. This code is applicable only post-exposure to the campaign |  “My opinion Twitter, is slightly worse” (T1)

Table 6. Codebook for Q.5 about reasons for participants’ positive impression
about the advertising organization pre-exposure to the campaign.

Q.5: Attitudes $\rightarrow$ Reasons for positive impression pre-exposure
---
Code (# occurrence) [non-exclusive] |  Code Definition |  Example
good-product (12) |  The product is generally good |  “It’s pretty good.” (WD3, WhatsApp)
privacy-focused (6) |  The product is privacy focused |  “a search engine platform that doesn’t steal your data and doesn’t record your info.” (WD6, DuckDuckGo)
good-company (3) |  The company is generally good |  “Apple as an the organization I think they, they’re pretty good.” (A3)
good-competitor (2) |  The company is a good competitor |  “a well-respected competitors in the marketplace.” (S1)
fun (2) |  The product is fun |  “it’s still a fun place to be.” (T4)
reliable-product (1) |  The product is reliable |  “I know whenever SnapChat is down or something, they’ll always have people go straight to Twitter to talk about it” (T6)
security-focused (1) |  The product is security focused |  “Supposedly security forward.” (WD4, DuckDuckGo)
better-than-other-product (1) |  The product is better than other products |  “probably better than Facebook Messenger” (WD4, WhatsApp)
efficient-product (1) |  The product is efficient |  “pretty efficient app” (WD6, WhatsApp)
good-product-feature (1) |  The product has good features (specific) |  “There is some good features on them … for example, Android has this feature, where, if you message people on either a messenger or text, and they can appear as text bubble on top of your apps.” (S6)
innovative-company (1) |  Innovative company |  “I think of it as a pretty innovative company” (A1)
sell-hardware (1) |  The company sells hardware not data |  “They sell hardware rather than selling my data.” (A6)
needed-product (1) |  The product is a necessity |  “I definitely see the need.” (T3)

Table 7. Codebook for Q.5 about reasons for participants’ negative impression
about the advertising organization pre-exposure to the campaign.

Q.5: Attitudes $\rightarrow$ Reasons for negative impression pre-exposure
---
Code (# occurrence) [non-exclusive] |  Code Definition |  Example
skeptical (3) |  The participant is skeptical about the campaign |  “I don’t know how truthfully accurate that is” (WD6, DuckDuckGo)
false-news (2) |  The product spreads false news |  “being on Twitter requires a little bit more research, and not believing the first thing I see, because news outlets” (T4)
new-leadership (2) |  The company has new leadership that is bad |  “Currently under the new leadership of Elon Musk, it’s a mess” (T4)
lack-moderation (2) |  The product lacks moderation |  “I just think there needs to be some sort of like more sort of like regulation on it.” (T1)
high-price (1) |  The product has a high price |  “an iMac will cost on average, about twice as much as a, like given, as a given windows machine” (A4)
noisy (1) |  The product is noisy or disrupting |  “it can be a little noisy if you follow it too much” (T3)
marketing-strategy (1) |  The company has a bad marketing strategy |  “their marketing doesn’t quite click with me … maximize what they can get out of a smaller user base rather than like necessarily expanding and like making their products more accessible to a wider based users.” (A4)
sell-data (1) |  The company sells their customers’ data |  “I guess they probably sell my data, but they seem to do it less.” (A6)
lack-customer-care (1) |  The company doesn’t care about their customers |  “they don’t really care about their individual customers so much.” (S3)
political-game (1) |  The product is used for political games |  “it can be used they as the platform can be used, for like a political game” (T1)
censoring (1) |  The product applies censorship |  “there’s more censoring in Twitter than there has been before.” (T4)
spam (1) |  The product has spam |  “I do get spam on it that’s always kind of annoying” (WD1, WhatsApp)
lots-ads (1) |  The product has lots of ads |  “it’s just so lame with ads.” (WD2, WhatsApp)

Table 8. Codebook for Q.5 about reasons for participants’ neutral impression
about the advertising organization pre-exposure to the campaign.

Q.5: Attitudes $\rightarrow$ Reasons for neutral impression pre-exposure
---
Code (# occurrence) [non-exclusive] |  Code Definition |  Example
not-user (3) |  The participant is not a user of the product |  “I actually don’t have any impression of it … I don’t know anything about it.” (WD1, DuckDuckGo)
new-leadership (1) |  The participant mentioned the new leadership of the product or company |  “I don’t really have an impression … I know that Elon Musk just bought them so he’s quite a character, so I’m not super sure how I feel about them now.” (T5)

Table 9. Codebook for Q.14 about change in participants’ impression about the
advertising organization post-exposure to the campaign.

Q.14: Attitudes $\rightarrow$ change in impression post-exposure
---
Code (# occurrence)
[exclusive] |  Code Definition |  Example
more-pos (13) |  The participant impression changed towards more positive |  “I guess it’s more positive” (A6)
same (12) |  The participant impression did not change. This either expressed explicitly, or implicitly by giving an impression similar to the pre-exposure impression |  “I’d say it hasn’t changed.” (T3)
more-neg (5) |  The participant impression changed towards more negative |  “Slightly more negative” (WD4, WhatsApp)

Table 10. Codebook for Q.14 about reasons for change in participants’
impression about the advertising organization post-exposure to the campaign.
Note that the number of occurrence of each code here represent the aggregate
number where this code occurred in any combination of impressions change
between pre- and post-exposure to the campaign.