# Staying at the Roach Motel: Cross-Country Analysis of Manipulative
Subscription and Cancellation Flows

Ashley Sheil  Maynooth UniversityIreland ,  Gunes Acar  Radboud UniversityThe
Netherlands ,  Hanna Schraffenberger  Radboud UniversityThe Netherlands ,
Raphaël Gellert  Radboud UniversityThe Netherlands and  David Malone  Maynooth
UniversityIreland

###### Abstract.

Subscribing to online services is typically a straightforward process, but
cancelling them can be arduous and confusing — causing many to resign and
continue paying for services they no longer use. Making the cancellation
intentionally difficult is recognized as a dark pattern called _Roach Motel_.
This paper characterizes the subscription and cancellation flows of popular
news websites from four different countries, and discusses them in the context
of recent regulatory changes. We study the design features that make it
difficult to cancel a subscription and find several cancellation flows that
feature intentional barriers, such as forcing users to type in a phrase or
call a representative. Further, we find many subscription flows that do not
adequately inform users about recurring charges. Our results point to a
growing need for effective regulation of designs that trick, coerce, or
manipulate users into paying for subscriptions they do not want.

Dark Patterns, Deceptive Designs, Roach Motel, HCI

††ccs: Human-centered computing Human computer interaction (HCI)

Draft: September 26, 2023

##  1\. Introduction

In 2023, internal documents leaked to Business Insider suggested that Amazon
had deliberately designed the process of cancelling Prime subscriptions in an
arduous and prolonged manner (Towey and Kim, 2023). It was also revealed that
Amazon were aware of consumers being involuntarily enrolled in their service
(Commision, 2023b). Amazon’s practices had also prompted complaints by
consumer organizations in Europe (Council, 2021; Commission, 2022). This
practice of building obstacles to keep users subscribed however, is not
limited to a single company, other companies have similarly received
complaints in the past (FTC, 2022, 2021; Lyons, 2018; Ritschel, 2020). In
fact, a 2022 survey of 2,500 Americans showed that one-third of Americans are
paying for subscriptions they do not use, with 42% feeling ‘locked in’ to
their current subscription plans (Malhotra, 2022). Frustrated users including
celebrities (Tre, 2023) turn to social media and online forums such as
Reddit111Reddit threads mentioning “cancel subscription”,
https://www.reddit.com/r/assholedesign/search/?q=cancel%2Fsubscription or
Figure 1. to complain about these manipulative practices (nie, 2023; Affairs,
2023). Responding to Nieman Journalism Lab’s call in 2021, several users
listed unexpected credit card charges as a reason to cancel their newspaper
subscriptions (nie, 2023). In 2022, hundreds of one-star reviews left on the
customer review site ConsumerAffairs complained about subscription-related
practices in industries such as dating sites, software subscriptions, radio/TV
services, weight loss and health club memberships (Huffman, 2022).

Figure 1. A 2023 tweet by comedian Trevor Noah ridiculing difficulties of
cancelling subscriptions.

New laws and regulations across the globe have been recently passed or
proposed to better protect consumers against unfair practices related to
subscriptions (DSA, 2022a; Competition and Authority, 2018). A notable example
of these new laws is German Fair Consumers Act which requires a ‘termination
button’ to make it easy to cancel subscriptions (Kreß, 2021). In the US, the
Federal Trade Commission propose a ‘Click to Cancel’ option for cancelling
(Commision, 2023a). Finally, the UK by 2024, aim to ensure, consumers can
easily exit contracts without any unnecessary steps (ideally one click)
(Eastham, 2023).

###  1.1. Objectives & Contributions

Motivated by user complaints, the recent and upcoming regulatory changes, and
the surge of subscriptions as a business model, this paper investigates
problematic practices in the context of online news subscriptions. Existing
research typically focuses on dark pattern taxonomies and/or user studies (Di
Geronimo et al., 2020a; Bergman, 2021; Mathur et al., 2019; Bongard-Blanchy et
al., 2021; Maier and Harr, 2020), but the empirical research on specific dark
patterns such as _Roach Motel_ is limited. In order to fill this research gap,
we characterize the subscription and cancellation paths to news sites from
five different locations. Specifically, we posed as a German, Dutch, English,
Texan and Californian consumer and subscribed to a list of up to ten news
sites for each persona’s country. In addition, our European personas also
subscribed to the same American news sites to observe any differences in
subscribing and cancelling based on subscribers’ location.

The main contribution of this work is a systematic cross-country comparison of
subscription and cancellation flow designs of popular news websites. We study
these flows through the lens of requirements imposed by the relevant existing
and upcoming regulations. In addition, we provide a comparison of subscription
and cancellation flows in terms of efforts necessary to achieve the two
respective goals. We show that in all the jurisdictions studied, the _Roach
Motel_ is still present, though to varying degrees. Finally we contribute a
dataset of over 100 screen recordings of subscription and
cancellations222Cancellation videos of our dataset can be viewed on
https://anonymous.4open.science/r/media-subscription-study-4CF5/. The dataset
also includes emails received during or after the cancellation. This
comprehensive dataset serves as a valuable resource for researchers studying
subscription-based services from various perspectives.

##  2\. Background & Related Work

During the COVID-19 lockdowns, an increasing number of individuals turned to
the internet for shopping, education, entertainment and connection (McClain et
al., 2021). This coincided with a surge in the global subscription e-commerce
market, which is projected to rise from $72.91 billion in 2021 to $904.2
billion by 2026 (Wertz, 2022). However, not everyone is satisfied with their
subscriptions. According to The Guardian, more than two million UK customers
gave up their streaming services last year due to financial hardship (Sweney,
2023).

For most subscribers, hard-to-cancel subscriptions can result in financial
harm and, at the very least, frustration. Such cancellation challenges can
have a more pronounced effect on individuals with mental health conditions. A
2017 survey by Money and Mental Health Policy Institute revealed that
individuals facing mental health challenges are more likely to forget to
cancel their subscriptions and three times more likely to avoid cancelling
when having to do so by phone (Evans, 2017). The same survey also revealed
that 21% of the participants expressed their inclination to postpone
cancelling a service, despite no longer desiring it. The participants cited
concerns that the company would apply pressure tactics to retain their
subscription (Evans, 2017). Moreover, the commonly used tactic of requiring
customers to contact a customer service representative to cancel their
subscription presents an unsurmountable challenge for users with hearing
impairment (Frost, 2022).

Perhaps the most well-known example of easy-to-enroll but hard-to-cancel
subscription practices is used by Amazon Prime, which has more than 200
million subscribers worldwide (Contributors to Wikimedia projects, 2023a). The
recent US Federal Trade Commission (FTC) complaint against Amazon alleges that
Amazon knew that their customers involuntarily enrolled into Prime
subscriptions and that customers were frustrated about the arduous and
perplexing Prime cancellation procedure (of Washington, 2023). The complaint
asserts that it requires a minimum of six clicks to cancel the Amazon Prime
subscription (of Washington, 2023) and the cancellation flow contains several
dark patterns such as _Forced Action_ , _Interface Interference_ ,
_Misdirection_ and _Obstruction_ (Gray et al., 2018; of Washington, 2023). In
July 2022, Amazon agreed to change its cancellation process for the EU and EEA
users, following a complaint made by the European Consumer Organisation
(BEUC), the Norwegian Consumer Council and the Transatlantic Consumer Dialogue
(Commission, 2022). Thanks to these changes, EU and EEA users have been able
to cancel their subscriptions in two clicks with a clear and prominent cancel
button–an option that was not available in the US until April 2023 (of
Washington, 2023).

Amazon’s changes in their cancellation process for the EU and EEA highlight
how regulations regarding subscriptions can vary between countries and
jurisdictions. These differences in practices and regulation partly motivates
our multi-vantage point investigation into subscription and cancellation
flows.

###  2.1. Related work

Manipulative design and dark patterns Dark patterns333As the term dark
patterns is officially used in the Digital Services Act (DSA) (art, 2022), an
EU regulation brought into law in November 2022 (DSA, 2022b), we will use this
term.  are defined by Mathur et al. (2019) as user interface design choices
that benefit an online service by coercing, steering, or deceiving users into
making decisions that, if fully informed and capable of selecting
alternatives, they might not make. To date, there has been a growing number of
papers in the area of dark pattern research (Mathur et al., 2019; Bergman,
2021; Bongard-Blanchy et al., 2021; Di Geronimo et al., 2020b; Maier and Harr,
2020; Luguri and Strahilevitz, 2021). Mathur et al. (2021) review recent
studies on dark patterns and identify that the literature does not have a
consistent definition or singular concern on how dark patterns impact the
user. Observing dark patterns through different diciplines, they propose a set
of normative guidelines that can be used to evaluate dark patterns.

Coined by Brignull (2022), the _Roach Motel_ dark pattern refers to a
situation where signing up for a service is straightforward, but the process
of discontinuing or terminating the service is challenging or complex. Hard to
Cancel category of Mathur et al. (2019) also refers to designs that are
difficult to cancel. Gray et al. (2018) categorized _Roach Motel_ under the
_Obstruction_ category of dark patterns, which is defined as [m]aking a
process more difficult than it needs to be, with the intent of dissuading
certain action(s). Chivukula et al. (2019) analyzed 1002 posts from the
subreddit ‘/r/assholedesign’, where they found _Obstruction_ to be the second
most frequent dark pattern discussed. Despite Chivukula et al.’s observation,
_Roach Motel_ dark patterns appear to be less recognised compared to other
dark patterns from a user’s perspective (M. Bhoot et al., 2020). Finally,
_Preselection_ is another dark pattern, where options that are favourable to
the business are pre-selected, such as options to receive promotional email or
agreeing to recurring charges (Mathur et al., 2021).

Di Geronimo et al. (2020a) examined 240 mobile apps and conducted an online
experiment with 589 users on how they perceive dark patterns. Their analysis
revealed that 95% of the examined apps include one or more forms of dark
patterns, with popular applications typically featuring at least seven
distinct forms of deceptive interfaces, on average. Einav et al. (2023)’s
analysed credit card data used in payment for ten subscriptions, and revealed
that subscription businesses benefit from inattentive customers. Using data
from 23 million accounts, they found that users are more likely to cancel
their subscriptions during the months when their cards are replaced (e.g., due
to theft or loss). In a similar study Miller et al. (2023) studied consumer
inertia, using a news site contract that automatically renews or cancels. They
find that consumers anticipate their inattention and avoid subscribing to
contracts which auto-renew. Recently, Glaser (Glaser, 2022), compared the
cancellation process of four major streaming platforms and found dark
patterns, such as highlighted re-subscribe buttons in all platforms except
Netflix. Disney+ took the least time to cancel at 51 seconds with C More
taking 60 seconds. Glaser also performed a user study and found that users
aged 50-59 had the most difficulty in unsubscribing compared to those in their
20’s (Glaser, 2022).

Another line of academic research on subscriptions focuses on consumer
retention and the underlying factors that motivate individuals to subscribe to
online news services (Fletcher and Nielsen, 2017; Chyi and Ng, 2020). Many
online articles and blogs also provide advice to news media on how to keep
their churn rate low (the annual percentage rate at which customers stop
subscribing to a service) such as tracking what content their consumers are
reading and identifying which consumers are at risk of not renewing their
subscription (Sonderman and Vargo, 2021).

###  2.2. Updates to Regulation on Subscriptions

The countries involved in this study have seen some changes in regulations
regarding subscriptions and dark patterns. These changes are discussed below,
as well as certain relevant surveys conducted in each country.

####  2.2.1. United States

In October 2021, the Federal Trade Commission (FTC) issued an enforcement
policy statement on the Restore Online Shoppers’ Confidence Act (ROSCA) (FTC,
2019), which signalled a ramp up of enforcement on dark patterns in
subscriptions (Commission, 2021). In 2023, the FTC announced their proposed
amendment to the FTC act, which introduces three main requirements businesses
must adhere to (ftc, 2023; Commission, 2006). The requirements consist of 1)
clearly disclose all details about the subscription, including price, renewal
dates and how to cancel; 2) obtain customers expressed consent before charging
them for products or services; 3) provide an easy and simple method to cancel
subscriptions (ftc, 2023).

California: In 2017, an online cancellation provision was added into the
California Business and Professions Code. In accordance with this provision,
if a consumer chooses to subscribe to a service online, the business is
obligated to offer an online cancellation option as well (McCants, 2022). On
October 4, 2021, California amended the Automatic Renewal Law (ARL) of 2010
(Rosati, 2021), which applies to all businesses that offer automatic renewal
or continuous service to Californians. The new ammendment, which went into
effect in 2022, provides more explicit cancellation guidelines compared to the
previous rule. Of the 50 US states, half have also adopted an automatic
renewal law, and eight states with no existing laws are considering adopting
them (Marion, 2019).

Texas: To the best of our knowledge, Texas still lacks specific regulation on
subscription services (Counsel, 2022; Marion, 2019). None of the bills
introduced in 2009, 2011 and recently in 2021 (HB 2259) were enacted into law
(Counsel, 2022). The lack of regulation is indeed one of the reasons why we
include Texas in the study, so we can compare the potential effects of local
(as opposed to federal) regulation, or lack thereof.

####  2.2.2. Europe

In addition to national laws described below, EU-wide legislation, such as the
Unfair Commercial Practices Directive (UCPD), also applies to subscriptions or
cancellations in the e-commerce domain (Unf, 2023). The UCPD prohibits unfair
commercial practices such as untruthful information or aggressive marketing
techniques, and it lists “Dark Design Patterns” as one of the explicitly
prohibited practices (UCP, 2023b). Moreover, the official guidance document on
the interpretation and application of the UCPD mentions several potential
infringements related to subscriptions (UCP, 2023a). These include “not making
it clear to consumers that they may enter into subscriptions by signing up to
a free trial” and “[o]mitting or providing information in an unclear manner on
the recurring costs of a subscription” (UCP, 2023a). Note that the EU
directives need to be incorporated (transposed) into the national laws by the
EU Member States. In the UK, the Consumer Protection from Unfair Trading
Regulations 2008 (CPUT) was introduced to enact the EU UCPD. After the UK’s
exit from the EU, CPUT continues to be effective, with EU references removed
(Bell, 2021).

Germany In Germany, the new Fair Consumer Contracts Act (FCCA) came into
effect in July 2022. The act states that businesses offering subscriptions to
German consumers must make it possible for them to cancel subscriptions
online. In addition, the act requires businesses to have a ‘termination’ or
‘cancel now’ button to easily cancel subscriptions (Gebhard, 2021; Centre,
2022; of Justice, 2021)444Prior to the reform, the inclusion of auto-renewal
clauses in contracts allowed for the extension of the contract for an
additional one-year term. Additionally, the required notice period before the
end of the initial contract period could be three months (Busch, 2022).. This
button can lead to a page where customers are requested to fill in their
details, the contract they wish to cancel and reasons for cancelling (Ger,
2023; of Justice, 2021). This page should also incorporate a clear
confirmation button, which upon clicking, guarantees the termination of the
contract. The consumer should receive a confirmation email immediately
afterwards (Centre, 2022).

The Netherlands The Dutch Authority for Consumers and Markets (ACM) offer
guidelines on the protection of the online consumer (ACM, 2023a) for
businesses involved in e-commerce. With respect to subscriptions, the ACM
states that businesses must provide clear information on how consumers can
terminate a contract, and if a contract is initiated through a website, it
should also be possible to terminate it through the same online platform (ACM,
2023b). The ACM also states that termination via the website should not
involve excessive obstacles such as extensive questionnaires. The guidelines
also suggest that marketing, design, and legal departments coordinate to
comply with regulations and to adhere to the principles of fairness by design
and privacy by design. The guidelines also point out that some groups of
customers are more vulnerable than others such as the elderly (ACM, 2023a).

United Kingdom Responding to a super-complaint555Broadly speaking, a super-
complaint is a complaint submitted by a designated UK consumer body about
practices or businesses that significantly harm consumer interests (Authority
and Markets, 2015), the UK’s Competition and Markets Authority (CMA) launched
an investigation into subscription traps and loyalty penalties in 2019
(Authority and Markets, 2020). Their study focusing on anti-virus software and
online video games resulted in Norton and McAfee committing to changes in
their practices (CMA, 2021c, b). These commitments included making auto-
renewing contracts easier to understand and exit, and ensuring refund rights
(CMA, 2021c, b). Following this study the CMA published guidelines (Oct 2021)
on auto-renewal opt outs and notifications (CMA, 2021a). These guidelines
however are not legally binding (O’Flaherty, 2022).

In April 2022, reforms were announced by the UK government to address online
dark patterns (MP, 2022a). The CMA will be given more power to deal with
companies that break these rules. They will also be able to compensate
consumers who lose out from these dark patterns without getting the courts
involved (MP, 2022a, b). In April 2023, the Digital Markets, Competition and
Consumers Bill was submitted and is currently at the committee stage (for
Business and Trade, 2023). The bill addresses auto-renewal and cancelling
information and highlights that consumers should be able to cancel online at
any time.

####  2.2.3. Summary

The above updated regulations from all jurisdictions share certain themes,
with different levels of detail. In summary, consumers should be informed
about the terms of a service when signing up, and businesses must not erect
unreasonable barriers to cancellation. Consumers should be notified throughout
the process, for auto-renewal and recurring charges. They should also be
provided with a subscription receipt and cancellation confirmation. All
jurisdictions highlight the importance of consumers being made aware of auto-
renewal and to be given the opportunity to cancel online. In short, these
regulations aim to address the _Roach Motel_ problem and lays out the
normative perspective (Mathur et al., 2021) that our study benefits from.

###  2.3. State of subscriptions

Here we mention market and user surveys related to subscriptions and
cancellations. We note that most surveys cited here are not peer-reviewed.

####  2.3.1. United States

Subscriptions to newspapers have increased by 30% in the US during the
pandemic and they appear to still be on the rise (Chandler, 2022). In 2021,
American Press Institute conducted a study on 526 publications’ retention
strategies (Sonderman and Vargo, 2021), and found several strategies appears
to aim at reducing cancellations. For instance 60% of the respondents said
their “customer service reps are trained in tactics for ‘saving’ renewals when
customers ask to cancel”, 28% said they segment subscribers based on their
cancellation risk (Sonderman and Vargo, 2021). On the other hand, 41% said
they make their subscriptions easy to cancel to improve customer experience
(Sonderman and Vargo, 2021).

####  2.3.2. Germany

In 2022, the Federation of German Consumer Organisations (vzbv) examined
around 840 cross-industry websites involving subscriptions to study how the
FCCA was being implemented (Centre, 2022). The study revealed that 349
websites had no cancellation button, 65 had hidden cancellation buttons, and
38 had invalid labels. In the few months after the law came into effect,
consumer watchdogs were reportedly sending cease and desist letters to several
online businesses (Konstantin Ewald and Hilgert, 2022). The vzbv released a
website where consumers can report any website that does not have a compliant
cancel button (Centre, 2022).

According to a survey (Recurly, 2022) conducted by the subscription-management
platform Recurly, the majority of German consumers spend up to €100/month on
subscriptions, with 67% of Germans using one or more subscription services. On
average, German consumers have three subscriptions, and 70% cancel their
subscriptions as soon as any benefits expire (Recurly, 2022). A study
undertaken by the German newspaper Der Spiegel found special offers work
better in attracting and retaining customers than free trials. Paid trial
subscribers overall tend to value the content more and are more likely to keep
their subscriptions than those who had a free trial (Iskorkina, 2021).

####  2.3.3. The Netherlands

The National Institute for Budget Information report (2021) on subscription
consumers in the Netherlands found that Dutch people underestimate how many
subscriptions they have. The average household has 14 subscriptions, with 28%
saying they often forget to cancel their subscriptions (Houtsma, 2021). Of the
respondents taking the survey, 22% said they find it a hassle to cancel
online, with 30% saying they prefer to cancel online and 13% saying they put
off cancelling as long as possible, as they dread the task (Houtsma, 2021).

In 2021 the ACM conducted a study into subscriptions and found websites make
it unnecessarily difficult to cancel a subscription. In their study, they
checked newspapers, magazines and lotteries. It was revealed that none of the
nine businesses in the study provided an adequate range of online subscription
cancellation options. Furthermore, some of the businesses did not offer the
ability to cancel subscriptions online (for Consumers and Markets, 2021).

####  2.3.4. The United Kingdom

Consumers in the UK currently are estimated to spend £1.8 billion per year on
subscriptions, where the average household is spending up to £60 per year on
unwanted subscriptions (MP, 2022b; Kemp, 2022). A 2021 survey conducted on
2,000 UK adults showed that 16% of the respondents hold newspaper and magazine
subscriptions, with 68% feeling their subscriptions were good value for money
(Research, 2023).

Summary The surveys mentioned above shows that across all countries included
in our study, subscriptions constitute a major market where users spend
billions of dollars every year, with many underestimating how many
subscriptions they have. In addition the surveys show the difficulties users
experience when trying to cancel their subscriptions.

##  3\. Methodology

As discussed in Section 1, our main objective in this study is to characterize
the subscription and cancellation pathways for news media sites in the USA,
UK, Netherlands, and Germany. This objective can be achieved through various
methods. One potential approach involves direct inquiries to businesses
regarding their practices, while another entails leveraging crowdsourcing to
gather information from participants about their encounters with the studied
businesses. In our study, we chose the method of employing simulated personas
for subscription and cancellation processes, which enabled us to directly
collect the study data. This approach mitigated the potential for inaccurate
or biased information stemming from businesses or participants.

We choose these origin locations to study practices in countries and states
with different regulations. For each country, we first needed to compile a
list of each country’s news sites. We compiled a list of ten for each, to
observe budget limitations for the project. The news sites were gathered from
the Global Digital Subscription Snapshot (Q4 2022) report compiled by the
International Federation of Periodical Publishers (FIPP) (of Periodical
Publishers, 2022b). Data about all forms of media are included in this report,
including a list of newspapers ranked according to the number of subscribers
they have (of Periodical Publishers, 2022a). From this list, we took the top
ten US and German newspapers. We omitted two US media outlets, which were not
news related666The Weather Channel and Americas test kitchen and replaced
these with the next ranking news sites. We chose to include the newsletter
subscription service Substack, because it offers subscriptions to independent
journalists, subject-matter experts, and media platforms (Contributors to
Wikimedia projects, 2023b). There were only seven British newspapers and no
Dutch newspapers in the Global Digital Subscription Snapshot report. To
account for this, we picked the remainder of the newspapers from Wikipedia’s
lists of Dutch and British newspapers, which are ranked by circulation volume
(Wikipedia, 2022a, b). Since we focus on online subscriptions, instead of
circulation volume, we ordered these newspapers by their Tranco rank and
picked the top ten 777Tranco website https://tranco-list.eu/. Table 1 provides
an overview of the news sites we successfully subscribed to from each country.
Newspapers that are picked from Wikipedia by their Tranco rank are marked with
an asterisk.

Location | Organization |  | Num. of
---
Subscribers
(or Tranco
rank*)
| Subscription
---
Price
| Trial
---
Period
| Trial
---
Price
United States | The New York Times | 8.3M | $4.25/w | 1 y | $1.00/w
Wall Street Journal | 3M | $9.75/w | 1 y | $2.00/w
Washington Post | 2.7M | $12.00/m | 1 y | $4.00/m
The Athletic | 1.1M | $7.99/m | 1 y | $1.99/y
Substack | 1M | $5.00/m | - | -
Medium | 725k | $5.00/m | - | -
The Daily Wire | 600k | $14.00/m | - | -
Barrons | 535k | $5.00/w | 1 y | $1.00/w
Bloomberg Media | 375k | $34.99/m | 3 m | $1.99/m
United Kingdom | Financial Times | 1M | £55.00/m | 1 m | £1.00/m
The Telegraph | 578k | £12.99/m | 3 m | Free
The Guardian | 420k | £3.00/m | - | -
Tortoise | 110k | £12.00/m | - | -
Mail + | 100k | £10.99/m | 3 m | £0.90/m
Spectator | 33k | £33.99/q | 1 m | Free
The Economist | (628*) | £18.90/m | 1 m | Free
iNews | (5k*) | £1.24/w | - | -
Netherlands | Het Algemeen Dagblad | (4k*) | €2.19/w | 2 y | €1.50/w
Telegraaf | (5k*) | €3.00/w | 1 y | €1.75/w
De Volkskrant | (7k*) | €3.12/w | 2 y | €2.50/w
NRC | (9k*) | €4.38/w | - | -
Trouw | (13k*) | €3.12/w | 2 y | €2.50/w
Eindhovens Dagblad | (39k*) | €2.19/w | 2 y | €2.00/w
Noordhollands Dagblad | (44k*) | €3.00/w | 1 y | €1.75/w
Tubantia | (49k*) | €2.19/w | 2 y | €2.00/w
PZC | (65k*) | €2.19/w | 2 y | €2.00/w
Germany | Bildplus | 603k | €7.99/m | 1 y | €1.99/m
WeltPlus | 200k | €9.99/m | 1 m | €1.00/m
Süddeutsche Zeitung+ | 136k | €9.99/m | 1 m | Free
Frankfurter Allgemeine | 91k | €4.95/w | 1 y | €2.95/y
Heise+ | 45k | €12.95/m | 1 m | Free
Rheinische Post+ | 26k | €7.99/m | - | €3.00/m
Kieler Nachrichten+ | 20k | €9.99/m | 1 m | Free
The Pioneer | 17k | €25.00/m | - | -

Table 1. The list of studied news websites, subscribers/rankings, subscription
cost and trial subscription period/prices offered.

For each country, we created a persona to subscribe to our compiled lists of
news sites. Our Dutch, German and English persona subscribed to both their
national and US news media sites. American personas subscribed to American
news sites only. Our rationale behind this decision was based on the
assumption that Europe, in general, has stricter regulations, making European
subscriptions to US sites more likely to yield intriguing results than the
reverse scenario. The limited study budget also impacted this decision.

As mentioned in Section 2.2.1, California has implemented guidelines on
cancelling subscriptions online, but Texas has no guidelines in place as yet
(faegredrinker, 2018). For this reason, when subscribing to American media, we
used a separate persona for California and Texas. This resulted in a total of
five personas, which we created using a fake name generator website (fak,
2023). Using a VPN connection, each persona signed up for subscriptions
appearing to connect from their own country888We used Mullvad VPN (mul,
2023).. The subscriptions were paid for with a virtual credit card with a low
limit, which we monitored for charges.

Following the protocol in Appendix A, we documented the subscription process
by recording the screen during the subscription and cancellation processes. In
addition we documented each subscription and cancellation according to the
parameters listed in Table 2. In compiling these parameters, we aimed to
record as much detail as possible about each subscription and cancellation,
pertaining to our objectives in Section 1.1. Moreover, we included parameters
based on regulatory requirements in the studied countries such as whether
users are informed about auto-renewals. As the study budget was limited, we
kept paid subscriptions for one month only and cancelled all subscriptions
before the end of the payment term, or before trials ran out, to avoid being
charged for a second term. The subscription process started in January 2023
and ended in Fenuary 2023.

• Subscription price. • Term of subscription, charged weekly or monthly. • Are
trials offered, if so, price and length of trial. • Does trial continue after
cancelling? • What payment method is required? • Is the credit card holder’s
name required? • Do they authenticate the credit card? • Is there information
on cancelling both on website and in the subscription flow. • Is there a
cookie paywall? • What personal information is required? • Is there adequate
renewal information given? • Is there an auto-renew button on the interface? •
Is there an auto-renew acknowledgement tick box on the interface? • Are emails
received after subscribing and cancelling? • Is there a termination button on
the interface? • Number of clicks to subscribe and cancel subscription. • Is
there surveys for leaving and are they mandatory? • Are there special offers
presented when cancelling? • How was the subscription cancelled? • What other
ways to cancel are offered? For example, phone, email, chatbot and form. • How
long does it take to cancel when cancelling by phone? • What other barriers or
details were observed?

Table 2. Subscription parameters documented when subscribing to each news
website.

In the cases where it was necessary to cancel by phone, we took notes on the
conversation. For the subscription and cancelling steps, we record how many
‘click next’ buttons we encounter in the subscription and cancellation flow.
The number of ‘clicks’ made during subscription or cancellation flows roughly
correspond to the number of steps in the process. Clicks during account sign-
up, those used to select survey answers from drop-down menus, and clicks made
for personalizing service preferences were all exempt from the click count.

##  4\. Results

###  4.1. Overview

We successfully signed up for 67 subscriptions, for a total cost of €412.92.
Of the 80 subscriptions in total, 13 could not be completed due to issues such
as requirements to provide real addresses, and payment issues. The LA Times
was the only website in which all five personas failed to sign up to,
potentially due to our use of a VPN connection. Table 3 and 4 give an overview
of subscription and cancellation flows studied.

Country | Organization |  | Automatic
---
renewal
disclosure
| Sub.
---
confirm
email
| Info on
---
cancelling
during
subscription
| Info on
---
cancelling
before
subscription
| Clicks to
---
subscribe
Germany | Bildplus | ✓ | ✓ | ✓ | ✓ | 3
WeltPlus | ✓ | ✓ | ✓ | ✓ | 3
Süddeutsche Zeitung+ | ✓ | ✓ | ✓ | ✓ | 5
Frankfurter Allgemeine | ✓ | ✓ | ✓ | ✓ | 5
Heise+ | ✓ | ✓ | ✗ | ✓ | 4
Rheinische Post+ | ✓ | ✓ | ✓ | ✓ | 4
Kieler Nachrichten+ | ✓ | ✓ | ✓ | ✓ | 5
The Pioneer | ✓ | ✓ | ✓ | ✓ | 4
Average |  | 100% | 100% | 87.5% | 100% | 4.1
Netherlands | Het Algemeen Dagblad (AD) | ✓ | ✓ | ✓ | ✓ | 3
Telegraaf | ✓ | ✓ | ✓ | ✓ | 3
De Volkskrant | ✓ | ✓ | ✓ | ✓ | 3
NRC | ✓ | ✓ | ✓ | ✓ | 3
Trouw | ✓ | ✓ | ✓ | ✓ | 3
Eindhovens Dagblad (ED) | ✓ | ✓ | ✓ | ✓ | 3
Noordhollands Dagblad | ✓ | ✓ | ✓ | ✓ | 3
Tubantia | ✓ | ✓ | ✓ | ✓ | 3
PZC | ✓ | ✓ | ✓ | ✓ | 3
Average |  | 100% | 100% | 100% | 100% | 3.0
United Kingdom | Financial Times | ✓ | ✓ | ✓ | ✓ | 3
The Telegraph | ✓ | ✓ | ✓ | ✓ | 3
The Guardian | ✓ | ✓ | ✓ | ✓ | 2
Tortoise | ✓ | ✓ | ✓ | ✓ | 4
Mail + | ✓ | ✓ | ✓ | ✓ | 4
Spectator | ✓ | ✓ | ✓ | ✓ | 4
The Economist | ✓ | ✓ | ✓ | ✓ | 3
iNews | ✓ | ✓ | ✓ | ✓ | 3
Average |  | 100% | 100% | 100% | 100% | 3.3
United States | The New York Times | ✓ | ✓ | ✓ | ✓ | 4
Wall Street Journal | ✓ | ✓ | ✓ | ✓ | 5
Washington Post | ✓ | ✓ | ✓ | ✓ | 4
The Athletic | ✓ | ✓ | ✓ | ✓ | 2
Substack | ✗ | ✓ | ✗ | ✓ | 3
Medium | ✓ | ✓ | ✗ | ✓ | 3
The Daily Wire | ✓ | ✓ | ✗ | ✗ | 3
Barrons | ✓ | ✓ | ✓ | ✓ | 5
Bloomberg Media | ✓ | ✓ | ✓ | ✓ | 3
Average |  | 88.8% | 100% | 66.6% | 88.8% | 3.6

Table 3. Overview of subscription flow features. United States represent both
Texas and California, which had identical results for the scope of this table.

Country | News sites |  | Cancellation
---
method
| Cancellation
---
survey
| Cancellation
---
survey
mandatory?
| Offers
---
to stay
| Post-
---
cancel
email
| 2-step
---
termination
button
| Num. of
---
steps/clicks
to cancel
Germany | Bildplus | Online | ⚫ | ⚫ |  | ⚫ |  | 5
WeltPlus | Online | ⚫ | ⚫ | ⚫ | ⚫ |  | 6
Süddeutsche Zeitung+ | Online |  |  |  | ⚫ | ⚫ | 5
Frankfurter Allgemeine | Online |  |  | ⚫ | ⚫ |  | 7
Heise+ | Online | ⚫ |  | ⚫ | ⚫ |  | 5
Rheinische Post+ | Online | ⚫ | ⚫ | ⚫ | ⚫ |  | 5
Kieler Nachrichten+ | Online |  |  |  | ⚫ | ⚫ | 4
The Pioneer | Online (Form) |  |  |  | ⚫ | ⚫ | 4
Average |  |  | 50% | *75% | 50% | 100% | 37.5% | 5.1
Netherlands | Het Algemeen Dagblad | Online | ⚫ |  | ⚫ | ⚫ |  | 4
Telegraaf | Online | ⚫ |  |  | ⚫ |  | 6
De Volkskrant | Online | ⚫ | ⚫ | ⚫ | ⚫ |  | 5
NRC | Phone |  |  |  | ⚫ |  |
Trouw | Online | ⚫ | ⚫ | ⚫ | ⚫ |  | 4
Eindhovens Dagblad | Online | ⚫ | ⚫ | ⚫ | ⚫ |  | 4
Noordhollands Dagblad | Online | ⚫ |  |  |  |  | 6
Tubantia | Online | ⚫ | ⚫ | ⚫ | ⚫ |  | 4
PZC | Online | ⚫ | ⚫ | ⚫ | ⚫ |  | 6
Average |  |  | 88.9% | *62.5% | 55.6% | 88.9% | 0% | 4.9
United Kingdom | Financial Times | Online | ⚫ |  | ⚫ | ⚫ |  | 6
The Telegraph | Online | ⚫ | ⚫ | ⚫ | ⚫ |  | 8
The Guardian | Online | ⚫ | ⚫ | ⚫ |  |  | 5
Tortoise | Online |  |  |  | ⚫ | ⚫ | 4
Mail+ | Phone | ⚫ | ⚫ |  | ⚫ |  |
Spectator | Online (Form) |  |  |  | ⚫ |  | 4
The Economist | Online | ⚫ | ⚫ | ⚫ | ⚫ |  | 7
iNews | Online |  |  |  | ⚫ | ⚫ | 4
Average |  |  | 62.5% | *80% | 50% | 87.5% | 25% | 5.4
USA Texas | The New York Times | Online | ⚫ |  | ⚫ | ⚫ |  | 7
Wall Street Journal | Phone | ⚫ | ⚫ |  | ⚫ |  |
Washington Post | Online | ⚫ | ⚫ | ⚫ | ⚫ |  | 6
The Athletic | Online | ⚫ | ⚫ | ⚫ | ⚫ |  | 6
Substack | Online | ⚫ |  |  | ⚫ |  | 5
Medium | Online | ⚫ | ⚫ | ⚫ | ⚫ |  | 5
The Daily Wire | Online | ⚫ | ⚫ | ⚫ | ⚫ |  | 4
Barrons | Online | ⚫ | ⚫ | ⚫ | ⚫ |  | 5
Bloomberg Media | Online (ChatBot) | ⚫ | ⚫ | ⚫ |  |  | 10
Average |  |  | 100% | *77.8% | 77.8% | 88.9% | 0% | 6.0
USA California | The New York Times | Online | ⚫ |  | ⚫ | ⚫ |  | 7
Wall Street Journal | Online | ⚫ | ⚫ |  | ⚫ |  | 5
Washington Post | Online | ⚫ | ⚫ | ⚫ | ⚫ |  | 6
The Athletic | Online | ⚫ | ⚫ | ⚫ | ⚫ |  | 6
Substack | Online |  |  |  | ⚫ | ⚫ | 3
Medium | Online | ⚫ | ⚫ | ⚫ | ⚫ |  | 5
The Daily Wire | Online | ⚫ | ⚫ | ⚫ | ⚫ |  | 4
Barrons | Online | ⚫ | ⚫ | ⚫ | ⚫ |  | 5
Bloomberg Media | Online (ChatBot) | ⚫ | ⚫ | ⚫ |  |  | 10
Average |  |  | 88.9% | *87.5% | 77.8% | 88.9% | 11.1% | 5.7

Table 4. Overview of the cancellation flow in each country. *: Percentage
based on news sites which presented cancellation (exit) surveys.

For subscribing to European media, our personas needed to make on average 3.5
clicks. For our American media subscriptions, our American personas needed on
average 3.6 clicks to subscribe. To cancel, the European and American sites
required on average 5.1 and 6.2 clicks, respectively. The average number of
clicks required to subscribe or cancel in each location is given in Figure 2.

Figure 2. Average number of clicks it takes to subscribe and cancel for each
location. Extra clicks indicate the difference between the number of clicks
for cancelling and subscribing.

Auto-renewal disclosure Substack (US) was the only website that did not
mention auto-renewal in their subscription flow999According to its help page,
Substack sends reminders one week before automatic renewal of both monthly and
yearly subscriptions (sub, 2023). . All other sites informed users about the
auto-renewal of their subscriptions. Auto-renewal information is predominantly
presented in fine print on the payment page (see Figure 5(b)). This message
informed users that by purchasing a subscription, users acknowledged and
agreed to the terms of automatic renewal.

On 14 of the 34 individual news sites, this auto-renewal message included a
tick box, whereupon ticking indicates that you understand the subscription’s
auto-renewal terms and conditions. Three news sites included an auto-renew
button that can be toggled to disable subscription from auto-renewing at the
end of the month. This design allows users to pause their subscriptions and
saves them from the burden of cancelling.

Information on cancellation All European media provide information on
cancelling subscriptions on the website, either on the homepage or inner
pages. In the US, the situation is similar with eight out of nine websites
providing some form of information about cancellations.

Offers to stay & Exit survey Overall 51.9% of the European media we subscribed
to presented special offers to incentivise staying and an average of 67.1%
asked reasons for leaving the subscription through the means of a survey.
Roughly 72.5% of the European websites made it mandatory to answer. For the US
roughly 77.8% of the websites offered discounts to encourage staying, and
almost all of the US websites (for Substack California no survey was
requested) asked for reasons for cancelling the subscription in the form of a
survey, with 82.7% making it mandatory you answer.

Figure 3. Selected subscription and cancelation parameters (averaged) for
European vs. US news sites. The averages were computed based on subscriptions
from the same country.

Post-cancellation email After cancelling all subscriptions, 88.9% of the
American and 92.1% of the European news subscriptions provided a post-
cancellation email. Exceptions to this were the Dutch news site Noordhollands
Dagblad and the UK’s Guardian. Overall subscription and cancellation results
comparing Europe and USA can be seen in Figure 3. Other aspects of the
cancellation flow are summarised in Figure 4. Once subscriptions were
cancelled for all accounts, our personas continued to receive emails asking
them to re-subscribe and providing news updates.

Figure 4. Average percentages per country measuring whether we could cancel
online, if we were presented with offers to stay during cancellation, if we
were asked to fill an exit survey, and if the exit survey was mandatory.

Below, we discuss the results in more detail for each country and we give an
overview of dark patterns encountered during the subscription and cancellation
processes.

###  4.2. United States of America

####  4.2.1. Subscribing

We successfully subscribed to nine of the ten American news sites using our
Texan and Californian personas. Our European personas could subscribe to eight
of the ten US news sites. Eight of the nine news sites informed our users
about the automatic renewal of their subscriptions, Substack provides auto-
renewal information only on their help centre page. Six of the nine US news
sites provide information on cancelling during the subscription flow. Only
four out of the nine US websites subscribed to from our Californian persona,
featured an auto-renew ‘opt in’ box to tick. For our Texan consumer, Barron’s
and Wall Street Journal did not have this tick box, which was available to our
Californian consumer. The remaining websites informed their consumers of their
renewal policy with a notification in the payment form with small print. The
New York Times’ notice, despite not having a tick box was at least more clear
and easier to read (Figure 5(a)).

(a) New York Times

(b) The Athletic

Figure 5. (a) New York Times auto-renew notice at payment stage for all the
countries in the study. Writing is in bold and notably bigger than the
majority of news media sites studied. (b) The Athletic checkout page with
their renewal information in small print at the end of the page.

The Daily Wire, Substack and Medium provided no information on cancelling in
their subscription flow. All websites provided a post-subscription email in
which they notified consumers of their renewal policy. Washington Post, Medium
and The Daily Wire were the only websites that clearly state the next billing
date in their post-subscription email. After subscribing to the Daily Wire’s
monthly subscription, we received several promotional emails inviting us to
upgrade to their annual plan.

Comparing the number of clicks required to subscribe, we find very similar
results for US and EU personas that subscribed to US newspapers. On average,
US personas made 3.5 clicks, while European personas made 3.2 clicks when
subscribing to American websites.

####  4.2.2. Cancelling

It was possible to cancel all Californian subscriptions online. In order to
cancel Wall Street Journal for our Texan and British consumers we had to make
a phone call. The calls were no more than three minutes long and we were
required to give email, a choice of giving address or the last few digits on
the credit card. In all but one case (NRC), the customer representative asked
about our reasons for terminating the subscription. We were not given any
discount offer to stay during the calls. Shortly following the calls, a
cancellation confirmation email was received. Barron’s and Wall Street Journal
are both published by Dow Jones & Company. When purchasing both a Barron’s and
Wall Street Journal subscription, they both appear together on either
website’s settings. Despite this, it is possible to cancel Barron’s online
both in Texas and California, whereas Wall Street Journal requires calling to
cancel from Texas. The Bloomberg subscription required cancellation via a
chatbot for all personas, which required a total of ten clicks to cancel. The
chatbot offered promotional discounts and asked for our reason for leaving
(Figure 6).

(a) Bloomberg response after choosing to cancel from a list of options

(b) After cancelling, Bloomberg requesting you answer a survey.

Figure 6. Bloomberg chatbot cancelling procedure

On average it took 5.8 clicks to cancel for Texas and California personas.
With the exception of Substack for California, all American news sites asked
for reasons for cancelling the subscription. Of these, 77.7% made answering
mandatory. Seven of the nine Texas USA news accounts offered discounts to keep
subscription, whereas for California, six out of the nine news sites offered
discounts to stay. When cancelling Barron’s from Texas, discount offers are
presented, but not when cancelling from California. For the UK and Germany,
Substack offered discounts.

Nineteen days after cancelling The Daily Wire, all personas received a
promotional email with an offer to re-subscribe. The initial email was
followed by another email two days later, reminding the offer with the line no
pressure just wanted to bump this promotion. Yet again nine days later,
another email is received notifying all personas that the offer is expiring.
The Daily Wire was the only website which sent third-party promotions via
email. Some of these emails requested donations. All media apart from
Bloomberg sent a confirmation of cancellation email. The Washington Post’s
cancellation flow stood out from others, with the emphasis more on cancelling
rather than convincing the customer to keep subscription. The ‘Continue to
cancel’ button was presented more prominently in all dialogs in the
cancellation flow (see Figure 7).

Figure 7. A bright pattern: Washington Post’s cancelllation interface presents
the “Continue to cancel” more prominently, contrary to the majority of media
cancellation interfaces.

###  4.3. Germany

####  4.3.1. Subscribing

Our German persona was successful in subscribing to eight German media
websites. Five of the news websites displayed a cookie paywall, where the user
is presented with a paid subscription option that will limit tracking and
advertisements. Morel et al. (2022) explored cookie paywalls and noted they
are mostly used in news sites and do not prevent all tracking. In Europe at
present, there is a lack of consensus among DPAs as to whether cookie paywalls
comply with GDPR requirements (Morel et al., 2022).

Figure 8. The cookie paywall (Morel et al., 2022) on Rheinische Post’s
subscription page. The user is asked to choose between (left) paying for a
subscription with reduced ads and tracking, and (right) free access with more
ads and tracking. In our study, when presented with this decision, we always
opted for the paid subscription.

All websites provided notice of automatic renewal, and information on
cancelling was provided during the subscription flow. The exception to this
was Heise, who provided no information on cancelling in their subscription
flow. There were no auto-renewal acknowledgment/terms and conditions tick
boxes at the payment stage for any of the German websites.

All websites provided emails after subscribing, Kieler Nachrichten+ was the
only news website to state a renewal date in their post-subscription email.
Bildplus and the Pioneer attach receipts with account information in their
emails. The average number of clicks to subscribe was 4.1.

####  4.3.2. Cancelling

It was possible to cancel all subscriptions online on a monthly basis and all
websites provided adequate information for cancelling on their interface. One
of the eight websites (The Pioneer) required filling in a form to cancel the
subscription. This form requested full name, email and invoice number, the
last of which was not mandatory. The Fair Consumer Contracts Act (FCCA),
states it should take no more than one extra step after pressing the ‘cancel
now’ button to successfully cancel your subscription (Centre, 2022). In our
study of German websites, we noted how many ‘clicks’ were required to cancel
once the ‘cancel now’ button was pressed. Only three of the eight websites
satisfied this requirement. KN+ was the only German website which featured an
auto-renew button which could be switched off. Figure 9 shows an example of
cancelling a subscription in two clicks, part (a) shows a clear ‘cancel now’
button, after clicking this, the button changes to a ‘confirm cancellation’
button (b). The subscription is cancelled when the ‘confirm cancellation’
button is clicked.

(a)

(b)

(c)

Figure 9. Süddeutsche Zeitung+ two click termination button

The Consumer Fair Contracts Act also stipulates that the ‘termination button’
must be prominently displayed and appropriately labelled. Certain links
encountered on German news sites were not clear or obvious, compared to an
clearly visible button. For instance, while Pioneer’s cancel link in Figure 10
is underlined, it is unclear whether it constitutes a ‘prominently displayed’
button.

(a) The Pioneer Figure 10. The link for cancelling Pioneers subscription is
underlined, but otherwise harder to distinguish compared to a button.

Out of the eight German websites, four inquire as to the reason for
cancellation through the means of a survey, with three of these requiring
mandatory responses. Four out of the eight websites offer discounts to stay.
All provided emails confirming the cancellation, some of these contained
further offers. On average the number of clicks to cancel was 5.1.

###  4.4. The Netherlands

####  4.4.1. Subscribing

From our list of Dutch websites our Dutch persona was successful in
subscribing to nine of the ten Dutch news sites on our list. All websites
provided sufficient information on automatic renewal and cancelling both in
the subscription flow and the website interface. All websites provided a tick
box acknowledging auto-renewal or terms and conditions. All Dutch websites
required direct debit payments and sent post-subscription emails, none of
which mentioned billing dates. It took on average three clicks to purchase a
subscription.

####  4.4.2. Cancelling

It was possible to cancel all subscriptions online with the exception of the
NRC news site. Cancelling NRC required a phone call, during which we were on
hold for ten minutes. When cancelling our Dutch media sites, all sites apart
from NRC, requested a reason for leaving. De Telegraaf and Noordhollands
Dagblad required us to fill in an online form that included an optional survey
about reasons for leaving. On average, 55.6% of Dutch news sites presented
special offers to stay, but notably, these offers were often positioned on the
same page as the ‘cancel now’ button. This contrasts with practices in other
countries where such offers typically appear after starting the cancellation
process. We received cancellation emails for all subscriptions but one
(Noordhollands Dagblad). On average it took 4.9 clicks to cancel.

###  4.5. United Kingdom

####  4.5.1. Subscribing

Our British persona successfully subscribed to eight out of the ten British
websites. All websites gave adequate information with regards to cancelling
and automatic renewal, and all websites sent a post-subscribing email. Three
websites provided a tick box to acknowledge auto-renew/terms and conditions.
Emails varied in the amount of details given regarding subscription. The Daily
Mail, The Guardian and Tortoise provide no details about renewal dates.
Regarding auto-renew information, we found all media sites gave adequate
information. On average it took 3.3 clicks to subscribe.

####  4.5.2. Cancelling

With the exception of The Daily Mail, it was possible to cancel all accounts
online. For three of the websites, it was possible to cancel with two clicks.
The Spectator required us to fill in an online form to cancel. This form
requested full name and email which was mandatory. The user’s address and
subscriber number were also requested but they were not mandatory. Tortoise
and iNews both had an auto-renew button that could be toggled to disable auto-
renewal without cancelling the subscription. Out of the eight British websites
our British persona subscribed to, five requested a reason for cancelling in
the form of a survey, and four of these websites made it obligatory to provide
an answer. Four out of the eight websites offered discounts before confirming
cancellation. All news sites apart from The Guardian, provided confirmation
emails after cancelling accounts and it took on average 5.4 clicks to cancel.

###  4.6. Dark Patterns

In this section we present some example dark patterns encountered during our
subscriptions and cancellation. We do not claim to present an exhaustive list
of dark patterns, which is out of scope for our study. Instead, we aim to
share representative examples based on the taxonomy classified by Mathur et
al. (2019). The patterns found in subscribing were _Obstruction_ , _Forced
Action_ and _Misdirection_. Some of these dark patterns are further
categorised into types, which is seen in _Misdirection_. The three
_Misdirection_ types encountered in this study were _Visual Interference_ ,
_Pressured Selling_ and _Confirm Shaming_. _Hard to cancel_ or _Roach Motel_
are categorized under the _Obstruction_ category. The most notable barrier to
cancelling was having to click through numerous steps of (often mandatory)
surveys and promotional offers before finally cancelling the service.

Mathur et al. (2019) characterizes _Visual Interference_ in cases where an
interface uses style and visual presentation to influence users’ choices. We
noted many cases of alternating colours between the ‘cancel’ button and the
‘keep subscription’ button, which is categorized as Bait and switch (Brignull,
2022). An example of this can be seen can be seen in Figure 11 from the German
newspaper Rheinische Post. The ‘Cancel subscription’ button starts of as white
and the ‘Manage subscription’ button is yellow. After pressing ‘Cancel
subscription’ however, the next page now displays the ‘Cancel subscription’
button in yellow and the ‘Return’ button in white. This is a form of
misdirection (Mathur et al., 2019) and can potentially lead a consumer to
inadvertently give up cancellation by pressing the wrong button

(a) ‘Cancel subscription’ button here is white

(b) Now ‘Cancel subscription’ is yellow

Figure 11. RP+ cancellation flow displaying colour switching buttons

Another example of _Visual Interference_ is misleading buttons in a
promotional offer from the Financial Times when cancelling. Our British
persona is presented with both her current subscription and a yearly
subscription, her options are to keep her current offer or to upgrade to a
yearly offer. The ‘Cancel subscription’ button is in black at the end of the
page. The ‘Confirm change’ button could be mistaken for ‘Confirm cancel’ (see
Figure 12).

Figure 12. The ‘confirm change’ button may lead users to believe they are
confirming the cancellation, but in fact it confirms purchasing an annual
offer. The actual cancel button is in black at the bottom of the page, and
depending on the vertical screen estate it can be missed if the user does not
scroll down.

Although the majority of websites provided information on auto-renew,
oftentimes the information was in small print and could be overlooked. Cancel
buttons were often hard to locate, hidden amongst plain text (Figure 10) or
placed at the bottom of the dialogs where they can missed (Figure 12).

_Confirm Shaming_ dark patterns were observed when cancelling subscriptions
and in post-cancellation emails. Certain cancellation emails opened with a
line such as What a pity you are leaving! (see Figure 15(a)). The Pioneer’s
enticement email stated Journalism is a participatory event and they would
like us to be a part of the party again… In some cases, the tone varied
calling our decision to cancel our subcription a ‘mistake’ (see Figure 13).

(a) The Pioneer

(b) De Welt

Figure 13. Messages received after cancelling subscriptions from The Pioneer’s
news sites and De Welt

_Forced Action_ refers to interfaces that coerce users to take a certain
action to complete a task. While requiring users to create an account is
understandable and does not constitute forced action, certain newspapers
required more than necessary information during subscription. Overall, the
type of data collected during subscription varied across our countries. Table
5 in Appendix B shows what data each website requested. The US and the UK news
websites seem to demand a greater amount of user data compared to their Dutch
and German counterparts. Several websites asked for extensive personal
details, including full address, date of birth, occupation, and phone number.
Newspapers such as The Financial Times, The Economist, and Barrons requested
significant personal information, such as occupation, under the pretext of
”improving” or ”tailoring” the user experience. In certain instances,
providing this information is mandatory (refer to Figure 14(a)), leading to a
cumbersome and long registration process.

(a) Financial Times

(b) Daily Wire

Figure 14. (a) Mandatory information required to sign up for the Financial
Times subscription. (b) The Daily Wire makes it mandatory to answer survey
questions during cancellation.

Many websites had pre-ticked boxes for promotional emails or newsletters. The
Daily Wire required our consumer to type in ‘CONFIRM’ to cancel their
subscription (see Figure 15(b)).—which constitutes a _Forced Action_.

(a) De Volkskrant

(b) Daily Wire

Figure 15. (a) Message received after cancelling De Volskrant and (b) the
Daily Wire required users to type ‘CONFIRM’ to proceed with cancellation.

(a) NRC

(b) Het Algemeen Dagblad (AD)

Figure 16. (a) NRC is the only Dutch newspaper that requires a phone call for
cancelling subscriptions. (b) Het Algemeen Dagblad (AD) cancel notice with
equal colours

##  5\. Discussion

An overview of the results shows that more European news sites provide
information relating to auto-renewal and cancelling compared to the US. On the
other hand, the US news sites display more exit surveys and special offers to
stay (Figure 3). European news sites require fewer clicks to cancel in
comparison to their American counterparts (Figure 2). Differences were noted
between some countries, for example the Netherlands subscriptions were all
paid via direct debit and displayed their special offers on their manage-
subscription page as opposed to after pressing cancel. Certain German news
websites used cookie paywalls, which was not encountered in any of the other
countries’ news sites. While our results show differences across
jurisdictions, we cannot precisely attribute these changes to the differences
in regulations. Other factors including market dynamics could have played a
role, among others.

Throughout the process of investigating the regulatory framework in our
selected countries, a commonly noted notion was that ‘websites should make it
as easy to cancel, as it was to subscribe’. Although cancelling online was
available for 63 of the 67 subscriptions made in the study, it was not an
effortless process. When cancelling, websites often took us through many
different steps before reaching the end goal of exiting their contract. These
obstacles included finding the cancellation button or link, answering
mandatory survey questions, and typing in a specific phrase. While subscribing
was usually less circuitous, not all were straightforward and certain
subscriptions required considerable amount of personal information disclosure
( Table 5 ).

As mentioned, certain subscriptions could only be cancelled by making a phone
call to the customer representative. Calling to cancel, in most cases, was
straightforward. Often the phone number was easier to locate than a cancel
button. Three of the four phone calls we made took only about three minutes.
The remaining call (NRC, NL), however, took more than ten minutes. As
mentioned in Section 1, requiring users to make phone calls is often
associated with cognitive burdens insofar as anticipating and engaging in
phone conversations can evoke anxiety in certain individuals. While offering
the choice to cancel via phone call can be beneficial, it may not be a
suitable option for all users. It should not be regarded as a one-size-fits-
all solution or as a sufficient substitute for an easy online cancellation
process.

To conclude this section, we would like to discuss possible paths to improve
the state of user experience related to subscriptions and cancellations. From
a regulatory perspective, we echo the three-fold route that Busch suggests
(Busch, 2022): (1) clear choices at the beginning of the subscription process
(2) reminders about auto-renewals and prolonged inactive subscriptions and (3)
a simple and unambiguous cancellation protocol. Given the variety and
complexity of cancellation flows observed in our study, a combination of high-
level principles and specific regulations may simplify the process. The newly
introduced German regulation is an example of unambiguous cancellation process
as it requires a ‘termination button’. This regulation provides details as to
what this button should look like and where it should be placed. These precise
requirements may make it harder to circumvent regulations, but on the long
term they may potentially suffer from being too technology-specific, and may
need to be updated.

###  5.1. Limitations

The news sites which we investigated in this study are highly popular, and
therefore may be subject to higher scrutiny and may be less likely to use
manipulative design. While a random and varied list of newspapers could
uncover other practices, studying popular websites has the advantage of
studying the experience faced by millions of users.

Screen captures were recorded and coded by the lead (junior) researcher whose
main language was English, in order to identify dark patterns. In order to
prevent potential biases and mistakes, the results were reviewed and discussed
in several meetings with three senior researchers in the field of law,
computer science and HCI.

For non-English websites we used Google Chrome’s internal translator and
Gmail’s translation option for emails. When these translators failed, we
manually copied and pasted text to Google Translate, or used used a smartphone
app to translate non-English text. While the accuracy of these translations
are generally high, there may be minor mistranslations.

Our study is based on a snapshot of the subscription and cancellation
processes at the start of 2023. These practices may change in time due to
upcoming regulations, user feedback or for other reasons. Moreover, businesses
that use A/B testing might offer alternative cancellation options that may not
be captured in our recordings.

The metrics that we use to characterize user experiences, such as the number
of clicks, do not fully capture all difficulties posed by the cancellation
interfaces. For example, in certain cases, finding a cancellation link hidden
within a maze of menus may be a larger challenge than the actual cancellation
process. For instance, several times the cancellation link was found at the
bottom of the page, which could have been missed if the user did not scroll.
Studying user experience metrics that consider the mental effort caused by
these designs could help us understand the challenges of subscription and
cancellation interfaces better.

###  5.2. Future Work

We limited ourselves to five locations to stay within the study budget, but
future studies may use our methods to study news websites from different
countries, including the global south. Another direction for further research
would be to study different types of subscriptions, such as streaming
services, online games, and physical deliveries (e.g., cosmetic products,
cleaning supplies, groceries, clothes). A longitudinal study, considering
changes over time, might also shed light on the effects of regulatory changes
or industry trends. Buying multiple subscriptions for the same site could
identify A/B testing and help investigate the consistency of the subscription
and cancellation process over multiple attempts. A user study could
investigate what users find particularly challenging in subscription and
cancellation flows, and what dark patterns are particularly effective in
preventing cancellations against users’ will. The emails sent by the
subscription services also offer an opportunity for further research, exposing
the tactics employed to win back unsubscribed users.

Ethics During our study we took appropriate measures to avoid interfering with
the regular operation of the businesses we subscribe to. By using simulated
personas with fictitious names and addresses, we aimed to protect the identity
and security of our research team. Only in a few cases, we had to provide the
real name of the lead co-author, because the name on the credit card was
required. Since we subscribed to popular news sites with thousands of
subscribers, we believe any negative effect due to our use of the simulated
personas will be minimal. Moreover, using simulated personas (Roesner et al.,
2012; Barford et al., 2014) and mystery shopping (Turner, 2012) are
established methods for evaluating business practices. We only engaged with
real customer representatives when we were required to call to cancel (four
times in total), during which we focused solely on cancelling subscriptions
quickly. These conversations were brief and resembled typical interactions
with potential customers. Given our research’s focus on business practices, we
did not disclose that these calls were part of a research study. We did not
record our calls with the representatives.

##  6\. Conclusion

While subscriptions can benefit online businesses by providing a sustained
revenue, they may also pose a challenge when it comes to the complexities and
obstacles users encounter while attempting to cancel them. To the best of our
knowledge, our study is the first attempt to comprehensively characterize the
difficulties encountered by users when trying to cancel online subscriptions
across a range of news websites, from multiple vantage points.

Our results show that overwhelming majority (63/67) of the subscriptions can
be cancelled online, with a few exceptions where we were required to make a
phone call. Nevertheless, our findings indicate that online cancellation
processes were not without their share of barriers and complexities, including
dark patterns such as _Misdirection_ , _Obstruction_ and _Forced action_.

Comparing results across European and US news websites, we find that compared
to their European counterparts, US news sites require more clicks to cancel,
they ask more information when subscribing, and they are more likely to force
users to fill in a mandatory exit survey.

While we believe that upcoming regulatory interventions rightfully focus on
preventing problematic practices related to subscriptions, their success
depends on effective enforcement. In the meantime, we believe businesses
should focus on gaining and retaining users’ trust instead of short-term
gains.

## References

  * (1)
  * FTC (2019) 2019\.  Restore Online Shoppers’ Confidence Act.  https://www.ftc.gov/legal-library/browse/statutes/restore-online-shoppers-confidence-act, Accessed 18-7-2023.
  * FTC (2021) 2021\.  Children’s Online Learning Program ABCmouse to Pay $10 Million to Settle FTC Charges of Illegal Marketing and Billing Practices.  https://www.ftc.gov/news-events/news/press-releases/2020/09/childrens-online-learning-program-abcmouse-pay-10-million-settle-ftc-charges-illegal-marketing, Accessed 17-07-2023.
  * CMA (2021a) 2021a.  Compliance Principles for anti-virus software businesses that use auto-renewing contracts.  https://www.gov.uk/government/publications/compliance-principles-for-anti-virus-software-firms/compliance-principles-for-anti-virus-software-businesses-that-use-auto-renewing-contracts, Accessed 08-08-2023.
  * DSA (2022a) 2022a.  Digital Services Act: agreement for a transparent and safe online environment $|$ News $|$ European Parliament.  https://www.europarl.europa.eu/news/en/press-room/20220412IPR27111/digital-services-act-agreement-for-a-transparent-and-safe-online-environment, Accessed 11-07-2023.
  * DSA (2022b) 2022b.  Digital Services Act (DSA).  https://enterprise.gov.ie/en/what-we-do/the-business-environment/digital-single-market/eu-digital-single-market-aspects/digital-services-act/, Accessed 03-08-2023.
  * FTC (2022) 2022\.  FTC Action Against Vonage Results in $100 Million to Customers Trapped by Illegal Dark Patterns and Junk Fees When Trying to Cancel Service.  https://www.ftc.gov/news-events/news/press-releases/2022/11/ftc-action-against-vonage-results-100-million-customers-trapped-illegal-dark-patterns-junk-fees-when-trying-cancel-service, Accessed 17-07-2023.
  * art (2022) 2022\.  Preamble 61-70, Digital Services Act (DSA).  https://www.eu-digital-services-act.com/Digital_Services_Act_Preamble_61_to_70.html, Accessed 03-08-2023.
  * Unf (2023) 2023\.  EUR-Lex - 32005L0029 - EN - EUR-Lex.  https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A32005L0029, Accessed 13-07-2023.
  * UCP (2023a) 2023a.  EUR-Lex - 52021XC1229(05) - EN - EUR-Lex.  https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A52021XC1229%2805%29, Accessed 13-07-2023.
  * nie (2023) 2023\.  Expensive, boring, and wrong: Here are all the news publications people canceled and why.  https://www.niemanlab.org/2021/10/expensive-boring-and-wrong-here-are-all-the-news-publications-people-canceled-and-why, Accessed 18-7-2023.
  * Ger (2023) 2023\.  Germany Fair Consumer Contracts.  https://www.iubenda.com/en/help/60579-germany-fair-consumer-contracts, Accessed 02-08-2023.
  * fak (2023) 2023\.  Get a whole new identity at the Fake Name Generator.  https://www.fakenamegenerator.com, Accessed 18-7-2023.
  * ACM (2023a) 2023a.  Guidelines on the protection of the online consumer.  https://www.acm.nl/en/publications/information-for-companies/acm-guideline/guidelines-protection-online-consumer, Accessed 02-08-2023.
  * mul (2023) 2023\.  Mullvad VPN - Privacy is a universal right.  https://mullvad.net/en, Accessed 18-7-2023.
  * ftc (2023) 2023\.  Negative reinforcement? FTC proposes amending Negative Option Rule to include click-to-cancel and other protections.  https://www.ftc.gov/business-guidance/blog/2023/03/negative-reinforcement-ftc-proposes-amending-negative-option-rule-include-click-cancel-other, Accessed 14-9-2023.
  * ACM (2023b) 2023b.  Rules for cancelling contracts online $|$ ACM.nl.  https://www.acm.nl/en/publications/information-for-companies/acm-guideline/guidelines-protection-online-consumer/rules-cancelling-contracts-online, Accessed 14-9-2023.
  * Tre (2023) 2023\.  Trevor Noah on Twitter.  https://web.archive.org/web/20230717145148/https://twitter.com/Trevornoah/status/1616164835992944647?lang=en, Accessed 17-07-2023.
  * UCP (2023b) 2023b.  Unfair and blacklisted commercial practices - Your Europe.  https://europa.eu/youreurope/citizens/consumers/unfair-treatment/unfair-commercial-practices/index_en.htm, Accessed 13-07-2023.
  * sub (2023) 2023\.  Will my subscription automatically renew?  https://support.substack.com/hc/en-us/articles/360045091251-Will-my-subscription-automatically-renew-, Accessed 18-7-2023.
  * Affairs (2023) Consumer Affairs. 2023.  Consumers still complain that some companies make it hard to cancel a service.  https://www.consumeraffairs.com/news/consumers-still-complain-that-some-companies-make-it-hard-to-cancel-a-service-111522.html, Accessed 11-07-2023.
  * Authority and Markets (2015) Competition Authority and Markets. 2015.  What are super-complaints?  _GOV_ (2015).  https://www.gov.uk/government/publications/what-are-super-complaints/what-are-super-complaints, Accessed 17-07-2023.
  * Authority and Markets (2020) Competition Authority and Markets. 2020.  ‘Loyalty penalty’ super-complaint.  _GOV_ (2020).  https://www.gov.uk/cma-cases/loyalty-penalty-super-complaint, Accessed 17-07-2023.
  * Barford et al. (2014) Paul Barford, Igor Canadi, Darja Krushevskaja, Qiang Ma, and Shan Muthukrishnan. 2014.  Adscape: Harvesting and analyzing online display ads. In _Proceedings of the 23rd international conference on World wide web_. 597–608.
  * Bell (2021) Rachel Bell. 2021.  eCommerce and the Post-Brexit State of Play.  https://www.charlesrussellspeechlys.com/en/news-and-insights/insights/commercial/2021/ecommerce-and-the-post-brexit-state-of-play/, Accessed 09-08-2023.
  * Bergman (2021) KB Bergman. 2021.  _Dark Patterns: Malicious Interface Dedign From A Users’Perspective_.  Master’s thesis.  Utrecht University.
  * Bongard-Blanchy et al. (2021) Kerstin Bongard-Blanchy, Arianna Rossi, Salvador Rivas, Sophie Doublet, Vincent Koenig, and Gabriele Lenzini. 2021.  ” I am Definitely Manipulated, Even When I am Aware of it. It’s Ridiculous!”-Dark Patterns from the End-User Perspective. In _Designing Interactive Systems Conference 2021_. 763–776.
  * Brignull (2022) Harry Brignull. 2022.  Deceptive Design.  https://www.deceptive.design/, Accessed 16-08-2022.
  * Busch (2022) Christoph Busch. 2022.  Updating EU Consumer Law for the Digital Subscription Economy.  (2022).
  * Centre (2022) Consumer Centre. 2022.  Problems with the cancellation button?  https://www.verbraucherzentrale.de/probleme-mit-dem-kuendigungsbutton-76779, Accessed 1-12-2022.
  * Chandler (2022) Susan Chandler. 2022.  Subscriptions Still on the Rise at Most Newspapers in the Medill Subscriber Engagement Index.  https://localnewsinitiative.northwestern.edu/posts/2022/11/21/subscriptions-increasing/index.html, Accessed 29-11-2022.
  * Chivukula et al. (2019) Shruthi Sai Chivukula, Chris Watkins, Lucca McKay, and Colin M Gray. 2019.  ” Nothing Comes Before Profit” Asshole Design In the Wild. In _Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems_. 1–6.
  * Chyi and Ng (2020) Hsiang Iris Chyi and Yee Man Margaret Ng. 2020.  Still unwilling to pay: An empirical analysis of 50 US newspapers’ digital subscription results.  _Digital journalism_ 8, 4 (2020), 526–547.
  * CMA (2021b) CMA. 2021b.  CMA secures refund rights for McAfee customers.  https://www.gov.uk/government/news/cma-secures-refund-rights-for-mcafee-customers, Accessed 28-11-2022.
  * CMA (2021c) CMA. 2021c.  Norton extends refund rights after CMA action.  https://www.gov.uk/government/news/norton-extends-refund-rights-after-cma-action, Accessed 28-11-2022.
  * Commision (2023a) Federal Trade Commision. 2023a.  Federal Trade Commission Proposes Rule Provision Making it Easier for Consumers to “Click to Cancel” Recurring Subscriptions and Memberships.  https://www.ftc.gov/news-events/news/press-releases/2023/03/federal-trade-commission-proposes-rule-provision-making-it-easier-consumers-click-cancel-recurring,Accessed 03/07/2023.
  * Commision (2023b) Federal Trade Commision. 2023b.  FTC Takes Action Against Amazon for Enrolling Consumers in Amazon Prime Without Consent and Sabotaging Their Attempts to Cancel.  https://www.ftc.gov/news-events/news/press-releases/2023/06/ftc-takes-action-against-amazon-enrolling-consumers-amazon-prime-without-consent-sabotaging-their, Accessed 27-06-2023.
  * Commission (2022) European Commission. 2022.  Consumer protection: Amazon Prime changes its cancellation practices to comply with EU consumer rules.  https://ec.europa.eu/commission/presscorner/detail/en/ip_22_4186, Accessed 07-07-2023.
  * Commission (2006) Federal Trade Commission. 2006.  Federal Trade Commission Act.  https://www.ftc.gov/legal-library/browse/statutes/federal-trade-commission-act, Accessed 30-11-2022.
  * Commission (2021) Federal Trade Commission. 2021.  FTC to Ramp up Enforcement against Illegal Dark Patterns that Trick or Trap Consumers into Subscriptions.  https://www.ftc.gov/news-events/news/press-releases/2021/10/ftc-ramp-enforcement-against-illegal-dark-patterns-trick-or-trap-consumers-subscriptions, Accessed 10-8-2022.
  * Competition and Authority (2018) Finnish Competition and Consumer Authority. 2018.  FCCA report: Consumers loose millions of Euros each year due to subscription traps.  https://www.kkv.fi/en/current/press-releases/fcca-report-consumers-lose-millions-of-euros-each-year-due-to-subscription-traps/, Accessed 17-08-2022.
  * Contributors to Wikimedia projects (2023a) Contributors to Wikimedia projects. 2023a.  Amazon Prime - Wikipedia.  https://en.wikipedia.org/w/index.php?title=Amazon_Prime, Accessed 14-9-2023.
  * Contributors to Wikimedia projects (2023b) Contributors to Wikimedia projects. 2023b.  Substack - Wikipedia.  https://en.wikipedia.org/w/index.php?title=Substack&oldid=1164027880, Accessed 17-07-2023.
  * Council (2021) The Norwegian Consumer Council. 2021.  You can log out, but you can never leave.  https://fil.forbrukerradet.no/wp-content/uploads/2021/01/2021-01-14-you-can-log-out-but-you-can-never-leave-final.pdf, Accessed 11-07-2023.
  * Counsel (2022) Cobb & Counsel. 2022.  Why is it so hard to cancel subscription services?  https://cobbxcounsel.com/2022/03/why-is-it-so-hard-to-cancel-subscription-services/, Accessed 31-05-2023.
  * Di Geronimo et al. (2020a) Linda Di Geronimo, Larissa Braz, Enrico Fregnan, Fabio Palomba, and Alberto Bacchelli. 2020a.  UI dark patterns and where to find them: a study on mobile applications and user perception. In _Proceedings of the 2020 CHI conference on human factors in computing systems_. 1–14.
  * Di Geronimo et al. (2020b) Linda Di Geronimo, Larissa Braz, Enrico Fregnan, Fabio Palomba, and Alberto Bacchelli. 2020b.  UI dark patterns and where to find them: a study on mobile applications and user perception. In _Proceedings of the 2020 CHI conference on human factors in computing systems_. 1–14.
  * Eastham (2023) Chris Eastham. 2023.  Changes to Consumer Subscription Contracts.  https://www.fieldfisher.com/en/insights/changes-to-consumer-subscription-contracts, Accessed 14-09-2023.
  * Einav et al. (2023) Liran Einav, Benjamin Klopack, and Neale Mahoney. 2023.  _Selling Subscriptions_.  Technical Report. National Bureau of Economic Research.
  * Evans (2017) Polly Mackenzie & Katie Evans. 2017.  Subscription retail: an expensive trap.  http://www.moneyandmentalhealth.org/wp-content/uploads/2017/04/20170306-Subscription-retail-paper-3.pdf, Accessed 28-11-2022.
  * faegredrinker (2018) faegredrinker. 2018.  Automatic Renewal Laws in All 50 States: An Updated Guide.  https://www.faegredrinker.com/en/insights/publications/2018/8/automatic-renewal-laws-in-all-50-states-an-updated-guide, Accessed 30-11-2022.
  * Fletcher and Nielsen (2017) Richard Fletcher and Rasmus Kleis Nielsen. 2017.  Paying for online news: A comparative analysis of six countries.  _Digital Journalism_ 5, 9 (2017), 1173–1191.
  * for Business and Trade (2023) Department for Business and Trade. 2023.  Digital Markets, Competition and Consumers Bill.  https://publications.parliament.uk/pa/bills/cbill/58-03/0294/220294.pdf, Accessed 29/06/2023.
  * for Consumers and Markets (2021) The Netherlands Authority for Consumers and Markets. 2021.  Consumers should be able to cancel online any subscriptions that have been taken out online.  https://www.acm.nl/en/publications/acm-consumers-should-be-able-cancel-online-any-subscriptions-have-been-taken-out-online, Accessed 22/11/2022.
  * Frost (2022) Natasha Frost. 2022.  Why call-to-cancel policies are an accessibility nightmare.  https://www.modernretail.co/retailers/why-call-to-cancel-policies-are-an-accessibility-nightmare/, Accessed 14-07-2023.
  * Gebhard (2021) Alexander Gebhard. 2021.  Germany: New Provisions for Consumer Contracts.  https://brsa.lawfed.com/germany-new-provisions-for-consumer-contracts/, Accessed 30-11-2022.
  * Glaser (2022) Lais Glaser. 2022.  _What is the optimal unsubscribing journey for paid video streaming services?_ Master’s thesis.  Tampere University.
  * Gray et al. (2018) Colin M Gray, Yubo Kou, Bryan Battles, Joseph Hoggatt, and Austin L Toombs. 2018.  The dark (patterns) side of UX design. In _Proceedings of the 2018 CHI conference on human factors in computing systems_. 1–14.
  * Houtsma (2021) Annette Groen & Nanne Houtsma. 2021.  Subscription Report 2021.  https://www.nibud.nl/onderzoeksrapporten/rapport-abonnementen-2021/, Accessed 15-06-2023.
  * Huffman (2022) Mark Huffman. 2022.  Consumers still complain that some companies make it hard to cancel a service.  https://www.consumeraffairs.com/news/consumers-still-complain-that-some-companies-make-it-hard-to-cancel-a-service-111522.html, Accessed 12-07-2023.
  * Iskorkina (2021) Mariia Iskorkina. 2021.  Are free subscription trials more effective than paid ones?  https://whatsnewinpublishing.com/are-free-subscription-trials-more-effective-than-paid-ones/, Accessed 25-04-2023.
  * Kemp (2022) Nicole Kar & Elisha Kemp. 2022.  The trial period is over: updated consumer laws for the digital economy.  https://www.linklaters.com/en/insights/blogs/linkingcompetition/2022/june/the-trial-period-is-over_updated-consumer-laws-for-the-digital-economy, Accessed 28-11-2022.
  * Konstantin Ewald and Hilgert (2022) Leonie Schneider Konstantin Ewald and Felix Hilgert. 2022.  Enforcement Alert: “Termination Button” in Germany.  https://gameslaw.org/enforcement-alert-termination-button-in-germany/, Accessed 1-12-2022.
  * Kreß (2021) Kristina Ehle & Stephen Kreß. 2021.  New “Two-Click Cancellation” Button – German Exceptionalism for Subscription Terminations.  https://www.mofo.com/resources/insights/211006-new-two-click-cancellation-button, Accessed 04-09-2023.
  * Luguri and Strahilevitz (2021) Jamie Luguri and Lior Jacob Strahilevitz. 2021.  Shining a light on dark patterns.  _Journal of Legal Analysis_ 13, 1 (2021), 43–109.
  * Lyons (2018) Jenna Lyons. 2018.  eHarmony to pay out millions in settlement to customers.  (2018).  https://www.sfgate.com/bayarea/article/eHarmony-to-pay-out-millions-in-settlement-to-12485546.php, Accessed 17-07-2023.
  * M. Bhoot et al. (2020) Aditi M. Bhoot, Mayuri A. Shinde, and Wricha P. Mishra. 2020.  Towards the identification of dark patterns: An analysis based on end-user reactions. In _IndiaHCI’20: Proceedings of the 11th Indian Conference on Human-Computer Interaction_. 24–33.
  * Maier and Harr (2020) Maximilian Maier and Rikard Harr. 2020.  Dark design patterns: An end-user perspective.  _Human Technology_ 16, 2 (2020), 170.
  * Malhotra (2022) Anil Malhotra. 2022.  Subscription wars: the subscriber strikes back.  https://mobile-magazine.com/articles/72-believe-there-are-too-many-subscription-services-72, Accessed 03-05-2023.
  * Marion (2019) Laura Koewler Marion. 2019.  Automatic Renewal Laws in All 50 States.  https://www.faegredrinker.com/-/media/files/collateral-pieces/automatic-renewal-laws-in-all-50-states-march-26-2019.pdf, Accessed 30-11-2022.
  * Mathur et al. (2019) Arunesh Mathur, Gunes Acar, Michael J Friedman, Eli Lucherini, Jonathan Mayer, Marshini Chetty, and Arvind Narayanan. 2019.  Dark patterns at scale: Findings from a crawl of 11K shopping websites.  _Proceedings of the ACM on Human-Computer Interaction_ 3, CSCW (2019), 1–32.
  * Mathur et al. (2021) Arunesh Mathur, Mihir Kshirsagar, and Jonathan Mayer. 2021.  What makes a dark pattern… dark? Design attributes, normative considerations, and measurement methods. In _Proceedings of the 2021 CHI conference on human factors in computing systems_. 1–18.
  * McCants (2022) Carter McCants. 2022.  Canceling Difficult Cancellation: An Analysis of Recent Regulatory Efforts to Make Canceling Subscriptions Easier.  _Wm. & Mary Bus. L. Rev._ 14 (2022), 463.
  * McClain et al. (2021) Colleen McClain, Emily A. Vogels, Andrew Perrin, Stella Sechopoulos, and Lee Rainie. 2021.  The Internet and the Pandemic.  https://www.pewresearch.org/internet/2021/09/01/the-internet-and-the-pandemic/,Accessed 30/06/2023.
  * Miller et al. (2023) Klaus M Miller, Navdeep S Sahni, and Avner Strulov-Shlain. 2023.  Sophisticated consumers with inertia: Long-term implications from a large-scale field experiment.  _Available at SSRN 4065098_ (2023).
  * Morel et al. (2022) Victor Morel, Cristiana Santos, Yvonne Lintao, and Soheil Human. 2022.  Your consent is worth 75 euros a year-measurement and lawfulness of cookie paywalls. In _Proceedings of the 21st Workshop on Privacy in the Electronic Society_. 213–218.
  * MP (2022a) Paul Scully MP. 2022a.  New rules to protect consumers’ hard-earned cash.  https://www.gov.uk/government/news/new-rules-to-protect-consumers-hard-earned-cash, Accessed 22-11-2022.
  * MP (2022b) Rt Hon Kwasi Kwarteng MP. 2022b.  Reforming competition and consumer policy: government response.  https://www.gov.uk/government/consultations/reforming-competition-and-consumer-policy/outcome/reforming-competition-and-consumer-policy-government-response, Accessed 23-11-2022.
  * of Justice (2021) Federal Office of Justice. 2021.  German Civil Code.  https://www.gesetze-im-internet.de/englisch_bgb/, Accessed 30-11-2022.
  * of Periodical Publishers (2022a) International Federation of Periodical Publishers. 2022a.  Global Digital Subscription Snapshot.  https://www.fipp.com/wp-content/uploads/2022/06/GDS-Snapshot-Q2-2022.pdf, Accessed 02-01-2023.
  * of Periodical Publishers (2022b) International Federation of Periodical Publishers. 2022b.  Global Digital Subscription Snapshot (2022 Q4).  https://www.fipp.com/resource-type/global-digital-subscription-snapshot/, Accessed 02-01-2023.
  * of Washington (2023) United States District Court Western District of Washington. 2023.  Complaint for Permanent Injunction, Civil Penalties, Monetary Relief, and Other Equitable Relief.  https://www.ftc.gov/system/files/ftc_gov/pdf/amazon-rosca-public-redacted-complaint-to_be_filed.pdf, Accessed 28-06-2023.
  * O’Flaherty (2022) Carlton Daniel & Ailin O’Flaherty. 2022.  Consumer Law Focus: Subscription Auto-Renewals, New CMA Guidance.  https://www.iptechblog.com/2022/02/consumer-law-focus-subscription-auto-renewals-new-cma-guidance/, Accessed 28-11-2022.
  * Recurly (2022) Recurly. 2022.  Study on subscription services: Large majority of German consumers pay up to EUR 100 a month.  https://recurly.com/press/german-consumers-pay/, Accessed 31-05-2023.
  * Research (2023) Opinium Research. 2023.  Usage of subscription services: key findings of survey (Opinium, November 2021).  https://www.gov.uk/government/publications/subscriptions-types-offered-by-business-and-numbers-held-by-consumers/usage-of-subscription-services-key-findings-of-survey-opinium-november-2021, Accessed 02-06-2023.
  * Ritschel (2020) Chelsea Ritschel. 2020.  ’Really and truly scammed me’: Rihanna’s lingerie company Savage X Fenty accused of deceptive marketing.  _Independent_ (2020).  https://www.independent.co.uk/life-style/rihanna-lingerie-savage-x-fenty-scam-reviews-a9334071.html, Accessed 17-07-2023.
  * Roesner et al. (2012) Franziska Roesner, Tadayoshi Kohno, and David Wetherall. 2012.  Detecting and Defending Against $\\{$Third-Party$\\}$ Tracking on the Web. In _9th USENIX Symposium on Networked Systems Design and Implementation (NSDI 12)_. 155–168.
  * Rosati (2021) Wilson Sonsini Goodrich & Rosati. 2021.  Amended California Automatic Renewal Law (ARL) Takes Effect in 2022.  https://www.jdsupra.com/legalnews/amended-california-automatic-renewal-6661081/, Accessed 03-07-2023.
  * Sonderman and Vargo (2021) Jeff Sonderman and Gwen Vargo. 2021.  What news publishers do to retain subscribers.  https://www.americanpressinstitute.org/publications/reports/survey-research/what-news-publishers-do-to-retain-subscribers/, Accessed 22-11-2022.
  * Sweney (2023) Mark Sweney. 2023.  UK homes cancelled 2m streaming services last year as cost of living soared.  https://www.theguardian.com/media/2023/jan/30/uk-homes-cancelled-2m-streaming-services-cost-of-living-netflix, Accessed 31/01/2023.
  * Towey and Kim (2023) Hannah Towey and Eugene Kim. 2023.  Amazon used a sneaky tactic to make it harder to quit Prime and cancellations dropped 14%, according to leaked data.  https://www.businessinsider.com/amazon-project-iliad-made-cancel-prime-membership-harer-leaked-data-2022-3, Accessed 20-04-2023.
  * Turner (2012) Helen Turner. 2012.  Mystery shopping.  _Market Research Handbook_ (2012), 333–346.
  * Wertz (2022) Jia Wertz. 2022.  The Growth Of Subscription Commerce.  https://www.forbes.com/sites/jiawertz/2022/07/15/the-growth-of-subscription-commerce/, Accessed 11-07-2023.
  * Wikipedia (2022a) Wikipedia. 2022a.  Dutch Newspapers.  https://en.wikipedia.org/wiki/List_of_newspapers_in_the_Netherlands, Accessed 06-11-2022.
  * Wikipedia (2022b) Wikipedia. 2022b.  English Newspapers.  https://en.wikipedia.org/wiki/List_of_newspapers_in_the_United_Kingdom, Accessed 06-11-2022.

##  Appendix A Subscription Study Steps

In order to make sure that the subscription process ran smoothly and without
mistakes, we compiled a standard procedure. This prevents any mistakes in the
process, for example forgetting to change your VPN location when subscribing
from Germany, and also saves time on inputting details.

  1. (1)

Before subscribing to a website, have your personas details ready for copying
and pasting. The card details should also be on same page, along with
passwords needed for accounts.

  2. (2)

Another spreadsheet should be created containing all media links and details
should also be open.

  3. (3)

For each individual persona, set up a Chrome account.

  4. (4)

When subscribing from each individual personas country, turn on VPN for their
location and open their Chrome account.

  5. (5)

Switch to the persona’s Chrome account, click on the media link in spreadsheet
for each newspaper and start screen recording.

  6. (6)

Scroll down through the website to document everything on all pages you click
through.

  7. (7)

Record in original language.

  8. (8)

Go through the process of subscribing to the account using Google auto
complete to save time and stop recording when done.

  9. (9)

Note details under each column in spreadsheet.

  10. (10)

Repeat for cancelling subscriptions.

  11. (11)

When phoning to cancel, have subscription details ready for verification.

  12. (12)

Save recordings in separate folders for subscribing and cancelling.

##  Appendix B Tables

Country | Media | Data
---|---|---
Germany | Bildplus | ENO
WeltPlus | ENO
Süddeutsche Zeitung+ | ENO
Frankfurter Allgemeine | ENO
Heise+ | ENO
Rheinische Post+ | EO
Kieler Nachrichten+ | ENOZ
The Pioneer | ENO
Netherlands | Het Algemeen Dagblad (AD) | EN
Telegraaf | DENP
De Volkskrant | EN
NRC | AENZ
Trouw | EN
Eindhovens Dagblad (ED) | EN
Noordhollands Dagblad | EN
Tubantia | EN
PZC | EN
United Kingdom | Financial Times | ENOP
The Telegraph | EN
The Guardian | EN
Tortoise | EN
Mail + | DENO
Spectator | ACENOZ
The Economist | ACENOZ
iNews | EN
United States | The New York Times | ENOZ
Wall Street Journal | ACENOZ
Washington Post | EOZ
The Athletic | EN
Substack | E
Medium | E
The Daily Wire | ACENOZ
Barrons | ACENOZ
Bloomberg Media | CENO

Table 5. Data requested by news websites during subscription sign up. Letters
indicate required details — E: Email, N: Name, O: Country, Z: Zip code, D:
DOB, A: Address, P: Phone, C: City

US Media | Texas USA | California USA | Germany | Netherlands | United Kingdom
---|---|---|---|---|---
|  | Clicks to
---
Subscribe
| Clicks to
---
Cancel
| Extra
---
Clicks
| Clicks to
---
Subscribe
| Clicks to
---
Cancel
| Extra
---
Clicks
| Clicks to
---
Subscribe
| Clicks to
---
Cancel
| Extra
---
Clicks
| Clicks to
---
Subscribe
| Clicks to
---
Cancel
| Extra
---
Clicks
| Clicks to
---
Subscribe
| Clicks to
---
Cancel
| Extra
---
Clicks
The New York Times | 4 | 7 | 3 | 4 | 7 | 3 | 5 | 7 | 2 | 4 | 7 | 3 | 5 | 7 | 2
Wall Street Journal | 5 | * | * | 5 | 5 | 0 | 4 | 5 | 1 | 4 | 5 | 1 | 4 | * | *
Washington Post | 4 | 6 | 2 | 4 | 6 | 2 | 3 | 6 | 3 | 3 | 6 | 3 | 3 | 6 | 3
The Athletic | 2 | 6 | 4 | 2 | 6 | 4 | 2 | 6 | 4 | 2 | 6 | 4 | 2 | 6 | 4
Substack | 3 | 5 | 2 | 3 | 3 | 0 | 3 | 6 | 3 | 3 | 6 | 3 | 3 | 6 | 3
Medium | 3 | 5 | 2 | 3 | 5 | 2 | 3 | 5 | 2 | 3 | 5 | 2 | 3 | 5 | 2
The Daily Wire | 3 | 4 | 1 | 3 | 4 | 1 | 3 | 4 | 1 | 3 | 4 | 1 | 3 | 4 | 1
Barrons | 5 | 5 | 0 | 5 | 5 | 0 | * | * | * | * | * | * | * | * | *
Bloomberg Media | 3 | 10 | 7 | 3 | 10 | 7 | 3 | 10 | 7 | 3 | 10 | 7 | 3 | 10 | 7
Average | 3.5 | 6 | 2.6 | 3.5 | 5.6 | 2.1 | 3.3 | 6.1 | 2.9 | 3.1 | 6.1 | 3 | 3.3 | 6.3 | 3.1

Table 6. Number of clicks to subscribe & cancel from US news sites by
different personas.