# Advancing a Consent-Forward Paradigm for
Digital Mental Health Data

Sachin R. Pendse  Logan Stapleton  GroupLens Lab, Department of Computer
Science & Engineering, University of Minnesota - Twin Cities, Minneapolis,
Minnesota, United States, 55455  Neha Kumar  School of Interactive Computing,
Georgia Institute of Technology, Atlanta, Georgia, United States, 30309
Munmun De Choudhury  School of Interactive Computing, Georgia Institute of
Technology, Atlanta, Georgia, United States, 30309  Stevie Chancellor
GroupLens Lab, Department of Computer Science & Engineering, University of
Minnesota - Twin Cities, Minneapolis, Minnesota, United States, 55455

###### Abstract

The field of digital mental health is advancing at a rapid pace. Passively
collected data from user engagements with digital tools and services continue
to contribute new insights into mental health and illness. As the field of
digital mental health grows, a concerning norm has been established—digital
service users are given little say over how their data is collected, shared,
or used to generate revenue for private companies. Given a long history of
service user exclusion from data collection practices, we propose an
alternative approach that is attentive to this history: the consent-forward
paradigm. This paradigm embeds principles of affirmative consent in the design
of digital mental health tools and services, strengthening trust through
designing around individual choices and needs, and proactively protecting
users from unexpected harm. In this perspective, we outline practical steps to
implement this paradigm, toward ensuring that people searching for care have
the safest experiences possible.

## Introduction

The field of digital mental health is rapidly expanding. Digital modalities of
mental health support have created new means for underserved populations to
access care 1. They have also supported the collection of novel data via
social media 2, mental health applications and wearables 3; 4, and online
support groups 5. These data sources allow for insights into daily lived
experiences of distress outside of clinical settings, with the potential to
deeply impact the lives of those in need. Data-driven insights from past
research include the prediction of symptom progression 5, matching individuals
in distress to relevant resources 6; 7, and identifying moments of crisis for
intervention 8. In parallel, novel intervention tools are being designed by
non-clinical organizations. These new tools for mental health support are
being created and used at a scale and pace unparalleled by any other time in
history 9; 10.

New digital approaches to support have been motivated by the potential to
bring mental health research out of ivory towers and into the hands of people
in distress. Diverse groups from both clinical and non-clinical domains have
enthusiastically developed digital mental health tools and services, including
initiatives from researchers, not-for-profit organizations, health systems,
insurance companies, and technology companies. The creators of some of these
tools have even applied for or received approval from the United States (U.S.)
Food and Drug Administration (FDA), and are expected to get approval from the
European Union’s (E.U.) Medical Device Regulation (MDR) and the United
Kingdom’s Medicines and Healthcare Products Regulatory Agency (MHRA) 11. For
many, digital mental health tools and services are a promising business
opportunity, with record high investments from venture capitalists in 2020 and
2021 12.

Digital mental health tools are fundamentally powered by data, whether through
analyzing individual data to support users, identifying patterns of mental
health experiences, or as a means to profit from the sale of data. Mental
health data from most online tools and services is legally considered non-
clinical and is thus not subject to standard health data regulations around
informed use 13; 14. Instead, data from digital mental health tools and
services is often collected without the active and explicit consent of
individuals in distress. Users have little say over how sensitive data about
their mental health is collected or used, who their data is seen by, and
whether their data is used to generate revenue for private actors, such as
data brokers 13. Recent examples that have garnered press attention include
public outcry over large language models guiding peer supporters on the Koko
support platform, 15 strong negative reaction to Crisis Text Line sharing
“anonymous” data to a for-profit spin-off company 16, U.S. Senate inquiries
into data privacy practices by teletherapy platforms 17, and U.S. Federal
Trade Commission action against teletherapy companies sharing data to third
party advertisers without clear consent from consumers 18. These issues will
only increase in scope and across domains—recent U.S. congressional hearings
have raised concerns about sensitive mental health data gathered and sold by
social media companies 19.

Informed consent for data collection is a cornerstone of mental health
research, in both digital and traditional formats—however, these ethical
principles have not been translated to the design of many digital mental
health products. Data collection from digital mental health applications is
done with little oversight, a practice that is extremely different from the
research protocol review that might be done by an ethical review board, such
as a U.S. Institutional Review Board (IRB) 20. If consent to collect data is
obtained at all, it is done through opaque Terms of Service agreements or
Privacy Policies 21; 22, which may not constitute true consent 23. The issues
around consent for non-clinical digital mental health data are complex, as the
field is largely unregulated, and data is created as a byproduct of
application or service use. These applications or services may not be mental
health focused (such as social media platforms) but may contain information
that can be used to infer mental health 24.

Due to a lack of regulation or standardization, individual organizations
determine how they approach consent in their tools and services. Without
standards around consent, organizations may rely on existing laws to decide
their privacy, security, and consent practices, and in many cases, existing
laws can still be misunderstood by organizations or poorly enforced by
regulators 25. An unrestrained use of data could have severe consequences for
the very users that care aims to protect, including discrimination in health
coverage or employment, and targeted advertisements by organizations that prey
on the vulnerable 14. However, designing data collection practices around
active and affirmative consent could increase the level of trust that a user
has in digital mental health tools and services, and allow users to feel safer
in expressing distress and seeking support—thereby improving their care and
outcomes of assistance.

In this perspective, to spur greater discussion around the role of consent and
agency in digital mental health data collection, we propose an alternative
approach to current practices—a consent-forward paradigm to digital mental
health data. This paradigm embeds the principles of affirmative consent in the
design of digital mental health tools and services, in which a digital mental
health technology must “ask for—and earn—enthusiastic approval” 26; 27; 28
from a user before taking any action that involves their data. A consent-
forward paradigm is attentive to the history of service users having their
consent and agency ignored in data collection, may strengthen user trust
through designing around individual choices and needs, and may proactively
protect individuals from unexpected harm. This article outlines how new
technological innovations from computer security, data privacy, and critical
data studies can be leveraged to implement a consent-forward paradigm. We
propose new design approaches to help realize a future in which user choice is
a core part of engagements with diverse digital modalities. These approaches
are rooted both in our own lived experiences engaging with mental healthcare
systems, work from authors who have described their own experiences evaluating
the risks of non-consensual interventions 29; 30, and work from authors
profiling non-consensual experiences with mental healthcare systems 31.

We believe that a consentful infrastructure for data collection within digital
mental health spaces could allow people in distress to seek care and self-
disclose freely without fear of harm, and use digital mental health technology
to its fullest potential. A consent-forward paradigm could also lay the
groundwork for increased trust and communication between users and
practitioners, towards responsibly gaining deeper insights from data and
improving care. A consent-forward paradigm is one potential approach that asks
us to deeply consider the risks of technology-mediated care and how we could
mitigate them. We hope that this Perspective acts as a foundation for greater
discussion (and empirical investigation) of the lived experiences of service
users as they engage with digital mental health tools – and what ways that
consent-forward paradigms may serve their needs. Digital mental health tools
and services do not have to continue along the disempowering norms of history
and present practice—rather, we argue that a different future can be realized
through a consent-forward paradigm to digital mental health data collection.

## Historic Perspectives on Data Consent for Mental Health

Our consent-forward approach to digital mental health data collection is
rooted in the history of how consent has been approached in mental health
settings, and furthermore, legal decisions that institutionalized certain
approaches to consent. This historical context highlights gaps in current
approaches, as well as how those gaps may influence digital mental health data
collection practices. Historical data collection largely ignored the consent
of people with mental illness, with consent often only critically considered
via legal proceedings around liability or medical negligence.

The first systematic data collection about mental health was conducted by
Western asylums and the U.S. Census in the late 18th 32 and early 19th 33; 34
centuries. Asylum administrators and staff would collect data when an
individual in distress was involuntarily committed by a family member. This
process happened without patient consent, and patients needed their family’s
or a practitioner’s advocacy to leave 35. Collected data was used for
organizational needs or to justify the involuntary commitment of patients
(such as medical certification or warrants 35; 36) and was unrelated to
patient welfare. On a national level, the U.S. Census first tracked data
around mental illness in 1840. Data collectors would ask the head of each
family to report the number of “insane” or “idiotic” individuals in a
household (these terms are pejorative today but were the official language of
the Census in 1840) 37. Contemporary medical professionals noted that data
collectors would overcount free Black people as insane, and supporters of
slavery would use the presumed “objectivity” 38 of systematic data collection
to argue that enslavement was good for mental health 39. In both data
collection processes, the voices and consent of people with mental illness
were excluded by design, stemming from a belief that patients did not have the
mental capacity to voluntarily contribute data.

The idea that data could be used for individual welfare was introduced with
the advent of psychoanalysis—however, data was intentionally collected without
consent out of the belief that it might interfere with data quality. In A Case
of Hysteria, 40; 41, Freud argued that using patient data for research that
could help others in distress was a core part of a physician’s duty, so long
as the physician “avoid[ed] causing direct personal injury to the single
patient concerned” 42, which set precedents around how case studies were
analyzed and reported. However, Freud was clear in his belief that research
benefits outweighed individual consent, and intentionally did not ask for
consent when collecting patient data. Freud believed that patients may “never
have spoken [to him] if it had occurred to them that their admissions might
possibly be put to scientific uses” 42, but that the value of data to society
outweighed the individual harm of not asking for consent.

Freud’s emphasis on avoiding direct personal injury when collecting data also
set a precedent for future legal protections against non-consensual data
sharing. This process largely occurred through legal decisions around the
nature of consent, as part of criminal proceedings in which clear harm was
demonstrated due to data being shared non-consensually. For example, in the
U.S., the legal right to the confidentiality of mental health data was first
recognized through state courts in Binder v. Ruvell only in 1952 43 and
federally by the Supreme Court in Jaffee v. Redmond in recent times (1996) 43,
both drawing on psychoanalytic theory on the value of confidentiality. Legal
authorities and professional organizations later built on these cases to
integrate ideas around privacy, consent, and confidentiality from
psychoanalysis into proactive laws and professional codes around the world 44,
such as The Mental Healthcare Act 2017 (India) 45 or the Philippine Mental
Health Act of 2017 (the Philippines) 46.

These standards were further institutionalized through new privacy laws around
digital health data. For example, the medical privacy rule of the U.S. Health
Insurance Portability and Accountability Act (HIPAA), announced in 2000,
included protections for clinical mental health data 43. Similarly, the
European General Data Protection Regulation (GDPR) has special protection for
data “related to the physical or mental health of a natural person” 47. The
EU’s GDPR protections are more wide-ranging than the clinical specificity of
the US’s HIPAA; however, it is unclear whether the law covers data that can be
used to infer health but is not explicitly health data 24. There are
exceptions to the confidentiality and consent of these laws, such as the duty
to warn in case of imminent harm to self or others 48, which ties data
confidentiality laws to other actions where consent can be violated to treat
those experiencing crisis, such as involuntary hospitalization (also called
civil commitment). Experiences with these non-consensual treatments can be
deeply violating—as Saks notes: “I know, better than most, how the law treats
mental patients, the degradation of being tied to a bed against your will and
force-fed medicine you didn’t ask for and do not understand” 30.

In research spaces, the oversight of ethical review boards anchored to widely
agreed upon principles (beneficence, justice, and respect for persons) 49; 50
guide researchers to ensure that future research studies do not propagate past
harms and proactively consider participant consent 51. In contrast, consent in
data collection from mental health tools and services has often only been
considered as a reaction to harm that already occurred, a consequence of court
proceedings. As we describe below, a commercial, non-clinical, and mostly
unregulated digital mental health landscape has propagated this reactive
(rather than proactive) approach from history, and harmed users as a result.
We propose an affirmative consent paradigm for data collection in digital
mental health, one that builds an infrastructure for users to have greater
ownership and choice in how their data is used. Affirmative consent is the
“idea that someone must ask for—and earn—enthusiastic approval before
interacting with another person” 26 and argues that consent should be
voluntary, informed, revertible, specific, and unburdensome. An affirmative
consent framework centers the context of a decision, providing all relevant
information about the decision and its impacts on the individual. It also
centers the lived experience of the person the decision affects by ensuring
their experience is voluntary and not burdensome, and that decisions can be
reverted. We describe each part of this framework in Table 1, presenting how
current approaches to data collection violate these principles, and the harm
of those violations.

## Data Driven Violations of Consent

Current digital mental health tools and services leverage technological
innovations to have a greater reach than past mental health services, using
novel data sources to tailor services. However, these technologies can amplify
non-consensual aspects of data collection in mental health services. Below, we
use two contemporary case studies that demonstrate these harms—the use of
algorithmic inference to support non-consensual intervention, and
discriminatory targeting from monetizing digital mental health data. We draw
attention to where design practices and approaches violate principles of
affirmative consent (additional details in Table 1).

### Algorithmically Mediated Intervention

Online spaces, such as social media platforms or search engines, are used by
individuals in distress to seek support and learn more about resources. While
online platforms are not designed to provide mental health services, as a
consequence of their scope, platforms are put in the position of deciding how
to best connect individuals in distress to resources. Due to limited moderator
resources, online social media platforms use artificial intelligence-based
analyses of social media posts to identify users at risk of harm 8. In
collaboration with existing services, platforms will then deploy emergency
services or law enforcement 14, paralleling the non-consensual active rescue
52; 14 crisis protocol undertaken by suicide hotlines. However, unlike
interventions initiated by suicide hotlines, platform-based intervention can
be initiated without the knowledge, let alone consent, of people in distress.
Wipond describes the anecdote of an individual participating in a private
Facebook support group meant for separated and divorced women, and expressing
her suicidal ideation—at midnight, the police showed up at her door, with the
woman “never knowing how or why it happened” 31. Non-consensual active rescue
is also used by digital tools and services that are explicitly designed for
mental health support— non-consensual active rescue is initiated for users of
Crisis Text Line 28 times per day on average 53. Non-consensual active rescue
is grounded in the idea that “anyone who is suicidal deserves aggressive
intervention to keep [them] alive” 54; 55, independent of an individual’s
“willingness or ability to provide consent” 55. These protocols are enacted as
a last resort and done by crisis services when all other support methods have
been exhausted. Similar to laws sanctioning involuntary hospitalization, and
continuing a long history of denying the agency of people with mental illness,
these protocols are founded on the legal doctrine of parens patriae. This
doctrine understands an individual experiencing a mental health crisis to be
unable to make decisions in their own self-interest, and the state to play a
primary role in determining when intervention supersedes individual rights 56.

Though potentially life-saving, non-consensual active rescue can also be
terrifying and deadly. In the U.S., individuals who are seriously considering
suicide are likely to be armed, which can and has been justification for
shooting, traumatizing, and even killing the individual needing help 53. 23
percent of police fatalities from January 2015 to August 2020 in the U.S.
resulted from police action during a mental health crisis 14. Similar risks
exist globally—from 1990 to 2016, 27% of firearm deaths globally were from
suicide 57; 58, and suicide deaths via firearm are highly concentrated in
countries in the Americas 57; 58 and Greenland 59. Crisis intervention
training for police is also limited and voluntary, with police officers
expressing discomfort with their role as “street corner psychiatrists” 60 from
a lack of knowledge of what might best help a person in distress. In their
survey of people who have used suicide hotline services, Leppert and Jervert
61 describe how respondents reported difficulties in understanding public-
facing information from helplines around non-consensual active rescue and
named the lack of transparency as a core reason for why they could not be
honest about the extent of their suicidal ideation with hotline volunteers.

Algorithmically mediated active rescue protocols initiated by social media
platforms, health services, government agencies, and other actors 14 are
similarly not transparent. One of the only available statistics on
algorithmically mediated active rescue comes from a 2018 report from Meta,
describing how 1000 wellness checks were initiated by the Facebook platform in
2018 14; 8, but little was shared about the algorithms that triggered these
checks. Similar to non-consensual active rescues that happen via traditional
means, emergency responders still enter crisis situations with little context
about what precipitated the individual’s mental health crisis, and what they
can do to best assist a person. Individuals in distress do not voluntarily
consent to the intervention, nor are they informed about what the intervention
might look like or how wide-ranging it might be.

There is a new opportunity for algorithmic crisis prediction systems to
incorporate a consentful approach to their deployment, ensuring that an
individual has sufficient context for their intervention and choice for what
that intervention looks like. As one supporting tool for a consent-forward
approach, we describe how digital mental health researchers and practitioners
could use digital psychiatric advance directives to ensure that this process
centers individual consent, even when an individual is in crisis.

### Monetization, Reidentification, and Discrimination

Throughout history, people with mental illness had little agency,
contribution, or even awareness of how their data was being used. Digital data
collection continues to passively involve users, buoyed by the modern
Internet’s reliance on an individual’s data for targeted advertising. The sale
of this data can be poisonous, enabled through a lack of enforced regulatory
frameworks, expansive Terms of Service policies, and unclear standards on what
constitutes mental health data 24. People with mental illness have little
ability to voluntarily consent to their data being shared or sold, little
ability to retract that consent after they have shared data, and derive little
immediate benefit from the revenue generated from their data.

This data is often not anonymized because it is not considered confidential
health data, even if a user may believe otherwise 62. This happens through the
sale of data from digital mental health applications to advertisers by data
brokers and can include the first and last names of individuals, their
psychiatric diagnoses, the medications they take, their likelihood of mental
illness, and even their contact information and home address 13. While
advertisers do primarily buy this data, anyone with sufficient funds can buy
this data, as demonstrated by Kim 13. Access to this data is facilitated
through the opaque and expansive Terms of Service (TOS) or privacy policies
found in digital mental health applications 21; 20; 63, which allow the
sharing of most data from the application.

Users may falsely believe that their non-clinical mental health data is
confidential, similar to how it is in other clinical contexts. This belief is
augmented by the fact that digital mental health interfaces have been found to
mislead users into thinking that all application data is confidential or not
being shared, as documented in the FTC complaint against BetterHelp 62 or the
FCC letter to Crisis Text Line 16. Users cannot opt out of this data
collection if they want to access services. Consent is thus taken with a
substantial burden associated with refusing to share data, and users are not
sufficiently informed about how their data is shared or about the security
practices of an application. Parker at al. 22 found that nearly half of the 61
digital mental health applications they reviewed had no privacy policy
whatsoever, and Iwaya et al. 64 found poor security practices among the top
mental health apps in the Google Play Store.

As the FTC complaint against BetterHelp notes, deanonymization and sharing of
this sensitive data can lead to impacts on an individual’s ability to “obtain
and/or retain employment, housing, health insurance, or disability insurance,”
along with additional “stigma, embarrassment, and/or emotional distress.” 62
There is the potential that this data may also be used to limit the kind of
treatment that an individual can access. Data around past engagements with
crisis services have been used to deny people additional care in the hospital,
as part of the Serenity Integrated Mentoring (SIM) program formerly used in
the UK by the NHS 65, and in trial in several U.S. states 14. It is also
possible data collection from digital mental health applications could be used
for similar purposes. For example, individuals with frequent application usage
may be triaged away from further accessing more involved crisis resources,
especially in light of stereotypes that frequent service users are “time-
wasters” 66 in clinical literature. Conversely, discrimination may happen due
to a lack of data or previous use—digital mental health services may be
designed such that they do not function effectively without providing user
data or expansive ToS agreements. Once data is sold, users may have little
recourse to reclaim their data or share in its revenue. In the following
section, we describe how a consent-forward paradigm can ensure that users have
full consent over how their data is used and that any shared data is
anonymized and private by design.

## Mitigating Current Harms Through a Consent-Forward Paradigm

As awareness around the harms of non-consensual intervention and data sharing
has grown, policy initiatives have been proposed and enacted to limit user
harm. Both Kim 13 and Bossewitch et al. 14 argue that a comprehensive data
privacy law or expansion of HIPAA could prohibit non-consensual data sharing.
In April 2023, the U.S. state of Washington enacted the My Health My Data Act,
which regulates the collection, sharing, and sale of private health data,
including the right to withdraw consent to data sharing 67. Similarly, in May
2023, the FTC proposed amendments to the Health Breach Notification Rule that
would include health tools and services not covered by HIPAA to notify users
of data breaches 68. To address the potential for harm in non-consensual
intervention, initiatives have been proposed that limit the use of police for
wellness checks and more use of trained and unarmed mental health crisis teams
69. We argue that these policies are sorely needed.

The success of current and future data policy initiatives is also tied to the
design of digital tools and services that implement their protections. We
believe these initiatives also present an opportunity for clinicians,
designers, and researchers in digital mental health to be proactive about
consent within the design of tools and services. Data policy initiatives and
technical design choices that protect consent could lay the groundwork for a
broader infrastructure that centers user consent and agency for digital mental
health data, across research and industry domains. Below, we begin by
describing our vision for how principles from affirmative consent anchor a
consent-forward paradigm and what an infrastructure centered on consent may
look like within digital mental health data collection. We then introduce
related areas of data privacy, security, and policy practices from computer
science that underlie this paradigm and ensure a more consentful experience
for people in distress. In each subsection, we describe how practices from
these related areas could be implemented across the diverse domains in which
digital mental health data collection happens. We also summarize these
practices in Table 2.

We believe that the consent-forward paradigm we present provides one potential
approach to more deeply consider user agency and current risks when designing
digital applications and data collection tools. However, our recommendations
should not be taken as a blanket prescription for any case of mental health
data collection. Consent is highly contextual, and other approaches may be
more amenable to a user (and are crucial to further study). Our perspective is
highly tied to our own lived experiences and research paradigms, and might be
very different from those of a population with identities and mental health
needs that are very different from our own. We propose the following paradigm
and associated practices as a beginning point for practitioners and
researchers to more deeply consider issues around consent and agency in their
design processes and implementations.

### Infrastructuring Consent: Trustworthy Data Collection Landscapes

It is clear that data from digital mental health applications and services is
valuable to understand lived experiences with mental illness and identify
where services may not meet user needs. For example, first looking to
healthcare applications, sensor data from wearable health applications can
point to precursors to suicide risk 4. Looking to research domains,
demographic information may be used to understand how underrepresented groups
use platforms differently 70. Looking to commercial applications, engagement
data may be used to understand whether tools and services create sustained and
long-term value 71 for users. Current norms around data collection from these
services are based on opt-in or opt-out models of consent, in which users are
prompted to decide whether they want to share (opt-in) or not share (opt-out)
their data when they begin to use a service. User preferences around data
collection are often stored long-term, even with the EU’s GDPR data
protections in place 72, and it has been debated whether this one-time
interaction this truly constitutes consent 23. Opt-in and opt-out models of
consent are also widely used in medical settings—for example, the UK’s
national data opt-out policy allows “[patients] to choose if they do not want
their confidential patient information to be used for purposes beyond their
individual care and treatment” 73. Similarly, experts debate whether opt-in
and opt-out models constitute actual consent in health settings, such as in
the case of organ donation 74.

Changing a landscape in which digital mental health data is largely collected
and sold without user consent requires changes in technology design, policy
reform, and norms around how users and their data are understood across
diverse domains—what we call changing the infrastructure of consent. A
consent-forward approach rooted in affirmative consent is one path to ensure
that data is collected with the active participation of users in contributing
their data. As we present in Table 1, a consent-forward approach is one in
which digital service users have five criteria: users can voluntarily provide
their consent to sharing data, are fully informed about how their data will be
used before it is shared, can reverse their decision to share their data, can
choose what specific types of data they want to share, and are not required to
agree to a burdensome terms of service that requires data sharing to access
services. Creating a landscape of digital mental health tools and services
that accommodates these principles will require policy interventions 13,
empirical research to understand what forms of requests for consent are not so
burdensome that they dissuade people from accessing care, and innovative
technology design to ensure that users can manage how their data is being
used, should they choose to use a service. We understand these design, norms,
and policy shifts around consent in digital mental health to broadly lay the
infrastructure for a consent-forward approach.

We believe that a core part of transitioning to a consent-forward approach is
greater communication between digital mental health service providers
(including both research organizations and commercial entities) and potential
service users. This could naturally happen individually, with users able to
quickly communicate their preferences for how their data is collected and
used. However, it is also important that this happens on an institutional
level, through lived experience advisory boards. Several mental health service
providers and organizations (such as the U.S. 988 Suicide and Crisis Lifeline
75 and the U.S. Substance Use and Mental Health Services Administration 76)
have lived experience advisory boards that they consult to ensure that service
provision is acceptable to diverse service users.

In transitioning to a consent-forward approach, it is possible that
underrepresented groups may feel uncomfortable voluntarily sharing their data
because of a lack of trust in researchers, a pattern observed in participation
in research studies 77. Similarly, in commercial domains, users may not trust
private corporations to safeguard their data, particularly given data breaches
and violations of consent in the past 62; 16, as well as the profit associated
with the sale of sensitive data. However, we understand a consent-forward
approach to be an opportunity for the creators of digital mental health tools
and services to build stronger trust among potential digital service users in
light of past harms and violations, such that users feel safe and comfortable
voluntarily sharing their data. As George et al. 77 note, one method to
mitigate the underrepresentation of minority individuals in research is
through consent being treated as “an ongoing process—a dialogue—rather than a
discrete act of choice that takes place in a singular moment in time.” A
similar dialogue-based approach may be successful in enabling data collection
from digital mental health tools and services without compromising user safety
or experience.

Another practice that could help build trust at an institutional level is
institutional support for the formation of patient advocacy groups and lived
experience advisory boards. Lived experience advisory boards to facilitate
dialogue, helping creators of tools and services understand where repeated
requests for consent may be burdensome and not worth the risk of dissuading
people from seeking support. Though a lived experience panel is out of scope
for this Perspective piece, an advisory board or panel could assist in
empirically validating what modifications are needed to our approach to be
successful and encourage people to feel safer accessing support, and ensure
that data is responsibly learned from (in line with member checking approaches
in qualitative research 78; 79). Along with the mechanisms we describe below
(and outlined in Table 2, an affirmative consent approach that treats consent
to data contribution as a dialogue between service providers and users could
allow for safer access to digital mental health support services.

### Encoding Consent: Digital Psychiatric Advance Directives (DPADs)

Individuals may engage with care under severe distress with impaired decision-
making capabilities 80. Users in severe distress may also engage with multiple
forms of care online, including online crisis chat services, social media-
based resources, or search engines. We propose digital psychiatric advance
directives (DPADs) as one way that users have full consent over their
engagements with online tools and services, including what data is collected
from those engagements. Traditional psychiatric advance directives are legal
documents that “[allow] a person with mental illness to state their
preferences for treatment before a crisis.” Along similar lines, DPADs could
encode consent into the design of digital tools and services. A DPAD could
contain information similar to a traditional psychiatric advance directive,
including close contact information, designated legal proxies, and exact
information about treatment plans and their execution. Similar to mobile
Medical IDs, 81 DPADs could be stored privately on mobile phones, with
functionality enabling directives to be accessed by health providers when an
individual indicates that they are in crisis or are incapacitated.

DPADs also allow users to determine what data they are comfortable sharing
before engaging with digital tools, services, and commercial applications. We
envision that applications may ask an individual for their DPAD before
engagement and collect data as per the directive—if a directive is not
provided, a consent-forward paradigm would suggest that data not be collected.
DPADs thus fulfill the principles of affirmative consent by allowing an
individual to voluntarily choose the specific forms of data they are
comfortable sharing and do so while not in a burdensome context (such as being
in crisis) and while informed of how data might be used.

Secure methods could be used to implement DPADs, ensuring that individuals can
revoke or revert their previously provided consent around data collection.
Smart contracts are one such example, consisting of digital contracts that
enable the contract creator to share private information with any individual
or organization based on the terms of the contract 82; 83 without intervention
from a third party. Smart contracts could be used to ensure that later changes
in preferences around data collection in a DPAD are automatically reflected in
data collected from a user’s past engagements with care. We envision that
DPADs would thus take on a standardized and interoperable form across
services, but with additional service-by-service questions relevant to new and
future digital mental health support forms. However, empirical research is
needed to understand what a base standardized DPAD should include, based on
what kinds of data could be collected and user perspectives on collecting that
data.

To accommodate a widespread use of interoperable DPADs, digital mental health
services would have to design their data collection and analysis techniques to
be sensitive to the directives of users, in line with changes made to comply
with the “right to erasure” 84 in the GDPR. Doing so would encode consent into
the design of digital tools and services, ensuring that users fully consent to
their engagements with care. While the integration of DPADs into digital
mental health tools and services might be complex, there is a strong ethical
and commercial incentive to leverage them—users may feel more safe expressing
distress freely when using mental health applications, as they would not have
to fear action being taken without their awareness or consent.

### Fortifying Consent: Security Practices for Large Scale Analyses

One motivation for collecting mental health data is the potential to identify
patterns of distress that traditional research methods may not catch or to
better understand whether digital interventions are working well to ease user
distress. However, data collection comes with risks. Data can be collected
from commercial applications without sufficient anonymization 13 and can be
integrated with other publicly available data sources (such as social media
data) to reidentify users 85, even if an application may assure a user that
data is deidentified. DPADs may allow users to delineate what engagement data
they are comfortable sharing. Additionally, ensuring that users trust the data
protection processes of tools and services could make them more willing to
contribute their data, even if they have poor experiences. To protect users,
we describe three approaches from the field of usable security and privacy
that ensure that data can be collected and analyzed while minimizing risk. The
field of usable security and privacy investigates these human-focused (or
“usable”) practices in trust, security, and safety to make interactions more
secure, private, and transparent. The efficacy of each of the data security
approaches we describe below can also feasibly be audited by independent
organizations, further strengthening trust in services across research and
industry domains.

Differential privacy. Differential privacy is an algorithmic approach to
anonymization that ensures that overall patterns that describe the data stay
consistent, while individual data points do not yield sufficient information
for re-identification. This is done by adding noise or altering feature values
so predictions are still accurate, but individual datapoints are not.
Differential privacy approaches are effective at anonymizing health data
broadly 86; 87; 88 and digital mental health data specifically 89; 90.

Federated learning. Digital mental health tools and services often collect
data from centralized or third-party servers 64. This centralization of data
and poor security practices risk privacy violations, particularly given the
sensitivity of mental health data. With federated learning, predictive models
could be trained locally on user devices—instead of sharing user data, model
weights from users are sent to a central hub for cross-user analysis. A
federated learning approach has been found to be helpful in healthcare broadly
91; 92, and in digital mental health applications 90.

End-to-end encryption. Confidential communication is of the utmost importance
in mental healthcare. However, many mental health applications (including some
facilitating access to support groups) do not encrypt user communications 64.
End-to-end encryption ensures that the platform owners cannot read or make
inferences from messages, protecting sensitive data from being exposed in
breaches—strictly senders and recipients can access messages locally. It is
important that the meaning of encryption be clear to users via application
interfaces, as users may not have an understanding of how end-to-end
encryption works (even if they are told that a platform is end-to-end
encrypted) 93.

Together, these three approaches guard affirmatively consentful approaches by
ensuring that users can trust that the most sensitive parts of their data
cannot be shared with others by design.

### Systematizing Consent: Data Governance Strategies

Research on the willingness of individuals to share their health data for
scientific research has demonstrated widespread public support for data
sharing 94. However, as Kalkman et al. 94 note, this support can be dependent
on whether there exists transparency around how data is being used by
researchers, privacy and security protections, and some ability for
contributors to govern how their data is used. Looking specifically to digital
mental health contexts, recent work 95; 14 has also emphasized the importance
of participatory data governance in the voluntary contribution of mental
health research data. For example, Sieberts et al. 95 found that young users
of digital mental health applications in India, South Africa, and the United
Kingdom are enthusiastic about contributing their data to research, so long as
they can govern why and how that data is being used. The value these service
users associate with data governance is a counterpoint to historical patterns
of exclusion from mental health data collection practices.

Historically, users have had little ability to consent to their data being
used to support policy initiatives. A consent-forward paradigm encompasses not
only an individual’s ability to consent to specific parts of their data being
shared, but the ability of users to collectively consent to how their data is
used. This is a core role that data governance, or determining “who has
authority and control over data and how that data may be used” 96; 97, plays
in a consent-forward paradigm. We argue that mental health data governance is
crucial to ensuring that data use immediately benefits people in need and that
benefits between users and digital mental health companies are symmetric. Core
parts of governance in historical service user movements included “equal power
of members, horizontal decision-making, democratic structure” and “restoring
individual and collective power” 98. Mental health data governance can be
designed to support these values, such as how peer supporters could use
computational tools (such as Decentralized Autonomous Organizations) for
consensus-based decision-making around code of conduct 99. Peer support groups
could function similarly to data cooperatives 100, using similar tools around
democratic decision-making to collectively determine whether their data will
be shared, to whom, and the boundaries to that data sharing. Feygin et al. 101
have proposed that data dividends could be paid to people who voluntarily
contribute their data. A similar approach could be used to compensate user
groups who do decide to contribute their data, to academic or industry
researchers.

We provide these examples of potential data governance structures as starting
points that demonstrate it is valuable to have people with lived experience
govern how their data is used, for users, designers, and practitioners.
However, the most optimal structure is highly dependent on the values of the
users and groups contributing data. Integrating these considerations of mental
health data governance into the design of digital tools and services ensures
that users can consent to not only the data they share individually, but also
how their data is used, and by whom, without them suffering a lack of service
provision or resources.

## Summary and Synthesis

In this perspective, we describe how a consent-forward paradigm to data
collection in digital mental health can be one starting point to ensuring that
users have greater choice and are protected from harm as they engage with
digital tools and services. Digital mental health tools and services are
increasing in popularity around the world, and though our perspective
primarily discusses Western legal frameworks, we believe that aspects of a
consent-forward paradigm to data collection can be adapted to bring value to
diverse geographical and cultural contexts. A consent-forward paradigm is
responsive to service user marginalization in past data collection practices,
and endeavors to incorporate consent into every part of data collection,
analysis, and governance. There will continue to be tradeoffs between the
risks of digital mental health data collection, and the benefits provided by
analyses of collected data. However, through a greater integration of consent-
forward practices, we believe that users in distress can feel more comfortable
engaging with digital mental health tools and services, and have the safest
experiences possible when searching for care.

## References

  * 1 Lo, J. _et al._ Telehealth has played an outsized role meeting mental health needs during the covid-19 pandemic.  KFF (2022).
  * 2 Chancellor, S. & De Choudhury, M.  Methods in predictive techniques for mental health status on social media: a critical review.  _NPJ digital medicine_ 3, 43 (2020).
  * 3 Abd-Alrazaq, A. A., Rababeh, A., Alajlani, M., Bewick, B. M. & Househ, M.  Effectiveness and safety of using chatbots to improve mental health: systematic review and meta-analysis.  _Journal of medical Internet research_ 22, e16021 (2020).
  * 4 Alam, M. G. R., Cho, E. J., Huh, E.-N. & Hong, C. S.  Cloud based mental state monitoring system for suicide risk reconnaissance using wearable bio-sensors.  In _Proceedings of the 8th International Conference on Ubiquitous Information Management and Communication_ , 1–6 (2014).
  * 5 De Choudhury, M., Kiciman, E., Dredze, M., Coppersmith, G. & Kumar, M.  Discovering shifts to suicidal ideation from mental health content in social media.  In _Proceedings of the 2016 CHI conference on human factors in computing systems_ , 2098–2110 (2016).
  * 6 Pruksachatkun, Y., Pendse, S. R. & Sharma, A.  Moments of change: Analyzing peer-based cognitive support in online mental health forums.  In _Proceedings of the 2019 CHI conference on human factors in computing systems_ , 1–13 (2019).
  * 7 Canady, V. A.  Tiktok launches mh guide on social media impact on teens.  _Mental Health Weekly_ 31, 5–6 (2021).
  * 8 Gomes de Andrade, N. N., Pawson, D., Muriello, D., Donahue, L. & Guadagno, J.  Ethics and artificial intelligence: suicide prevention on facebook.  _Philosophy & Technology_ 31, 669–684 (2018).
  * 9 Sinha, C., Meheli, S., Kadaba, M. _et al._ Understanding digital mental health needs and usage with an artificial intelligence–led mental health app (wysa) during the covid-19 pandemic: Retrospective analysis.  _JMIR Formative Research_ 7, e41913 (2023).
  * 10 Wind, T. R., Rijkeboer, M., Andersson, G. & Riper, H.  The covid-19 pandemic: The ‘black swan’for mental health care and a turning point for e-health.  _Internet interventions_ 20 (2020).
  * 11 Carl, J. R. _et al._ Regulating digital therapeutics for mental health: Opportunities, challenges, and the essential role of psychologists.  _British Journal of Clinical Psychology_ 61, 130–135 (2022).
  * 12 DeAngelis, T.  As funding cools, venture capitalists shift investments in mental health.  _American Psychological Association 2023 Trends Report_ 54, 83 (2023).
  * 13 Kim, J.  Data brokers and the sale of americans’ mental health data: The exchange of our most sensitive data and what it means for personal privacy.  _Duke Sanford Cyber Policy Program_ (2023).
  * 14 Bossewitch, J. _et al._ Digital futures in mind: Reflecting on technological experiments in mental health & crisis support.  https://automatingmentalhealth.cc/ (2022).
  * 15 Ingram, D.  A mental health tech company ran an ai experiment on real users. nothing’s stopping apps from conducting more.  _NBC News_ (2023).
  * 16 Hendel, J.  Crisis text line ends data-sharing relationship with for-profit spinoff.  _Politico_ (2022).
  * 17 Office of U.S. Senator Elizabeth Warren.  Warren, booker, wyden call on mental health apps to provide answers on data privacy and sharing practices that may put patients’ data at risk of exploitation.  _U.S. Senate_ (2022).
  * 18 Federal Trade Commission.  Ftc to ban betterhelp from revealing consumers’ data, including sensitive mental health information, to facebook and others for targeted advertising.  _Press Release, Federal Trade Commission_ (2023).
  * 19 Chan, K., Hadero, H. & Amiri, F.  Tiktok ceo testifies before house committee over security concerns, connection to china.  In _PBS NewsHour_ (2023).
  * 20 Wykes, T., Lipshitz, J. & Schueller, S. M.  Towards the design of ethical standards related to digital mental health and all its applications.  _Current Treatment Options in Psychiatry_ 6, 232–242 (2019).
  * 21 Jilka, S. _et al._ Terms and conditions apply: Critical issues for readability and jargon in mental health depression apps.  _Internet Interventions_ 25, 100433 (2021).
  * 22 Parker, L., Halter, V., Karliychuk, T. & Grundy, Q.  How private is your mental health app data? an empirical study of mental health app privacy policies and practices.  _International journal of law and psychiatry_ 64, 198–204 (2019).
  * 23 Obar, J. A. & Oeldorf-Hirsch, A.  The biggest lie on the internet: Ignoring the privacy policies and terms of service policies of social networking services.  _Information, Communication & Society_ 23, 128–147 (2020).
  * 24 Ienca, M. & Malgieri, G.  Mental data protection and the gdpr.  _Journal of Law and the Biosciences_ 9, lsac006 (2022).
  * 25 Gooding, P.  Mapping the rise of digital mental health technologies: Emerging issues for law and society.  _International journal of law and psychiatry_ 67, 101498 (2019).
  * 26 Im, J. _et al._ Yes: Affirmative consent as a theoretical framework for understanding and imagining social platforms.  In _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_ , 1–18 (2021).
  * 27 Friedman, J. & Valenti, J.  _Yes means yes!: Visions of female sexual power and a world without rape_ (Seal Press, 2019).
  * 28 Hilgert, N.  The burden of consent: Due process and the emerging adoption of the affirmative consent standard in sexual assault laws.  _Ariz. L. Rev._ 58, 867 (2016).
  * 29 Saks, E. R.  _Refusing care: Forced treatment and the rights of the mentally ill_ (University of Chicago Press, 2010).
  * 30 Saks, E. R.  _The center cannot hold: My journey through madness_ (Hachette UK, 2007).
  * 31 Wipond, R.  _Your Consent is Not Required: The Rise in Psychiatric Detentions, Forced Treatment, and Abusive Guardianships_ (BenBella Books, 2023).
  * 32 Porter, R.  Foucault’s great confinement.  _History of the Human Sciences_ 3, 47–54 (1990).
  * 33 Andrews, J.  Case notes, case histories, and the patient’s experience of insanity at gartnavel royal asylum, glasgow, in the nineteenth century.  _Social History of Medicine_ 11, 255–281 (1998).
  * 34 Reid-Green, K. S.  The history of census tabulation.  _Scientific American_ 260, 98–103 (1989).
  * 35 Hanganu-Bresch, C. & Berkenkotter, C.  _Diagnosing madness: The discursive construction of the psychiatric patient, 1850-1920_ (Univ of South Carolina Press, 2019).
  * 36 Wright, D.  Asylums and alienists: The institutional foundations of psychiatry, 1760–1914.  In _The Palgrave Handbook of the History of Human Sciences_ , 1–20 (Springer, 2022).
  * 37 Forret, J.  " deaf & dumb, blind, insane, or idiotic": The census, slaves, and disability in the late antebellum south.  _The Journal of Southern History_ 82, 503–548 (2016).
  * 38 Horwitz, A. V. & Grob, G. N.  The checkered history of american psychiatric epidemiology.  _The Milbank Quarterly_ 89, 628–657 (2011).
  * 39 Deutsch, A.  The first us census of the insane (1840) and its use as pro-slavery propaganda.  _Bulletin of the History of Medicine_ 15, 469–482 (1944).
  * 40 Freud, S.  _Dora: An analysis of a case of hysteria_ (Simon and Schuster, 1997).
  * 41 Lipton, E. L.  The analyst’s use of clinical data, and other issues of confidentiality.  _Journal of the American Psychoanalytic Association_ 39, 967–985 (1991).
  * 42 Freud, S. & Strachey, A.  _Fragment of an analysis of a case of hysteria (1905 [1901])_ , vol. 7 (Hogarth Press, 1962).
  * 43 Mosher, P. W. & Swire, P. P.  The ethical and legal implications of jaffee v redmond and the hipaa medical privacy rule for psychotherapy and general psychiatry.  _Psychiatric Clinics_ 25, A575–A584 (2002).
  * 44 Leach, M. M. & Harbin, J. J.  Psychological ethics codes: A comparison of twenty-four countries.  _International Journal of Psychology_ 32, 181–192 (1997).
  * 45 Bondre, A., Pathare, S. & Naslund, J. A.  Protecting mental health data privacy in india: The case of data linkage with aadhaar.  _Global Health: Science and Practice_ 9, 467–480 (2021).
  * 46 Lally, J., Samaniego, R. M. & Tully, J.  Mental health legislation in the philippines: Philippine mental health act.  _BJPsych international_ 16, 65–67 (2019).
  * 47 European Parliament & Council of the European Union.  Regulation (EU) 2016/679 of the European Parliament and of the Council.
  * 48 Borecky, A., Thomsen, C. & Dubov, A.  Reweighing the ethical tradeoffs in the involuntary hospitalization of suicidal patients.  _The American Journal of Bioethics_ 19, 71–83 (2019).
  * 49 Beauchamp, T. L. & Childress, J. F.  _Principles of biomedical ethics_ (Oxford University Press, USA, 2001).
  * 50 for the Protection of Human Subjects of Biomedical, U. S. N. C. & Research, B.  _The Belmont report: ethical principles and guidelines for the protection of human subjects of research_ , vol. 1 (Department of Health, Education, and Welfare, National Commission for the …, 1978).
  * 51 Spellecy, R. & Busse, K.  The history of human subjects research and rationale for institutional review board oversight.  _Nutrition in Clinical Practice_ 36, 560–567 (2021).
  * 52 Zeavin, H.  _The distance cure: a history of teletherapy_ (MIT Press, 2021).
  * 53 Zeavin, H.  The third choice: Suicide hotlines, psychiatry, and the police.  _Somatosphere_ (2020).
  * 54 American Association of Suicidology.  Organization accreditation standards manual (2006).
  * 55 Lifeline, N. S. P.  National suicide prevention lifeline: Policy for helping callers at imminent risk (2010).
  * 56 Testa, M. & West, S. G.  Civil commitment in the united states.  _Psychiatry (Edgmont)_ 7, 30 (2010).
  * 57 Naghavi, M. _et al._ Global mortality from firearms, 1990-2016.  _Jama_ 320, 792–814 (2018).
  * 58 Rivara, F. P., Studdert, D. M. & Wintemute, G. J.  Firearm-related mortality: a global public health problem.  _Jama_ 320, 764–765 (2018).
  * 59 Ilic, I., Zivanovic Macuzic, I., Kocic, S. & Ilic, M.  Worldwide suicide mortality trends by firearm (1990–2019): A joinpoint regression analysis.  _PLoS one_ 17, e0267817 (2022).
  * 60 Teplin, L. A.  Keeping the peace: Police discretion and mentally ill persons.  _National institute of justice journal_ 244, 8–15 (2000).
  * 61 Leppert, M. & Jervert, K.  Suicide hotlines and the impact of non-consensual interventions.  _Mad in America_ (2022).
  * 62 Federal Trade Commission.  FTC v. BetterHelp Inc.  FTC Complaint (2023).
  * 63 Powell, A., Singh, P., Torous, J. _et al._ The complexity of mental health app privacy policies: a potential barrier to privacy.  _JMIR mHealth and uHealth_ 6, e9871 (2018).
  * 64 Iwaya, L. H., Babar, M. A., Rashid, A. & Wijayarathna, C.  On the privacy of mental health apps: An empirical investigation and its implications for app development.  _Empirical Software Engineering_ 28, 2 (2023).
  * 65 Kendall, T.  Nhs england position on serenity integrated mentoring (sim) and similar models.  https://www.england.nhs.uk/long-read/nhs-england-position-on-serenity-integrated-mentoring-and-similar-models/ (2023).  Publication reference: PRN00317.
  * 66 Middleton, A., Gunn, J., Bassilios, B. & Pirkis, J.  Systematic review of research into frequent callers to crisis helplines.  _Journal of Telemedicine and Telecare_ 20, 89–98 (2014).
  * 67 Engrossed substitute house bill 1155: Consumer health data (2023).  Effective date: July 23, 2023.
  * 68 Federal Trade Commission.  Ftc proposes amendments to strengthen and modernize the health breach notification rule.  https://www.ftc.gov/news-events/news/press-releases/2023/05/ftc-proposes-amendments-strengthen-modernize-health-breach-notification-rule (2023).
  * 69 Mubarak, E., Turner, V., Shuman, A. G., Firn, J. & Price, D.  Promoting antiracist mental health crisis responses.  _AMA Journal of Ethics_ 24, 788–794 (2022).
  * 70 Rochford, B., Pendse, S., Kumar, N. & De Choudhury, M.  Leveraging symptom search data to understand disparities in us mental health care: demographic analysis of search engine trace data.  _JMIR Mental Health_ 10, e43253 (2023).
  * 71 Borghouts, J. _et al._ Barriers to and facilitators of user engagement with digital mental health interventions: systematic review.  _Journal of medical Internet research_ 23, e24387 (2021).
  * 72 Sanchez-Rola, I. _et al._ Can i opt out yet? gdpr and the global illusion of cookie control.  In _Proceedings of the 2019 ACM Asia conference on computer and communications security_ , 340–351 (2019).
  * 73 NHS Digital.  Understanding the national data opt-out.  https://digital.nhs.uk/services/national-data-opt-out/understanding-the-national-data-opt-out (2023).
  * 74 MacKay, D.  Opt-out and consent.  _Journal of Medical Ethics_ (2015).
  * 75 988 Suicide & Crisis Lifeline.  Lived experience committee.  https://988lifeline.org/lived-experience-committee/ (2024).
  * 76 Substance Abuse and Mental Health Services Administration (SAMHSA).  Advisory councils.  https://www.samhsa.gov/about-us/advisory-councils (2024).
  * 77 George, S., Duran, N. & Norris, K.  A systematic review of barriers and facilitators to minority research participation among african americans, latinos, asian americans, and pacific islanders.  _American journal of public health_ 104, e16–e31 (2014).
  * 78 McKim, C.  Meaningful member-checking: A structured approach to member-checking.  _American Journal of Qualitative Research_ 7, 41–52 (2023).
  * 79 Lincoln, Y. S. & Guba, E. G.  _Naturalistic inquiry_ (Sage, 1985).
  * 80 Sastre-Buades, A., Alacreu-Crespo, A., Courtet, P., Baca-Garcia, E. & Barrigon, M. L.  Decision-making in suicidal behavior: A systematic review and meta-analysis.  _Neuroscience & Biobehavioral Reviews_ 131, 642–662 (2021).
  * 81 Sandler, R. D.  Mobile medical id: A resource for the off-duty clinician.  _The Journal of Emergency Medicine_ 59, 141–142 (2020).
  * 82 Gupta, M., Jain, R., Kumari, M. & Narula, G.  Securing healthcare data by using blockchain.  _Applications of blockchain in healthcare_ 93–114 (2021).
  * 83 Yue, X., Wang, H., Jin, D., Li, M. & Jiang, W.  Healthcare data gateways: found healthcare intelligence on blockchain with novel privacy risk control.  _Journal of medical systems_ 40, 1–8 (2016).
  * 84 Sarkar, S., Banatre, J.-P., Rilling, L. & Morin, C.  Towards enforcement of the eu gdpr: enabling data erasure.  In _2018 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData)_ , 222–229 (IEEE, 2018).
  * 85 Simon, G. E. _et al._ Assessing and minimizing re-identification risk in research data derived from health care records.  _eGEMs_ 7 (2019).
  * 86 Niinimäki, T., Heikkilä, M. A., Honkela, A. & Kaski, S.  Representation transfer for differentially private drug sensitivity prediction.  _Bioinformatics_ 35, i218–i224 (2019).
  * 87 Liu, X., Zhou, P., Qiu, T. & Wu, D. O.  Blockchain-enabled contextual online learning under local differential privacy for coronary heart disease diagnosis in mobile edge computing.  _IEEE Journal of Biomedical and Health Informatics_ 24, 2177–2188 (2020).
  * 88 Lv, Z. & Piccialli, F.  The security of medical data on internet based on differential privacy technology.  _ACM Transactions on Internet Technology_ 21, 1–18 (2021).
  * 89 Roy, T. S., Basu, P., Priyanshu, A. & Naidu, R.  Interpretability of fine-grained classification of sadness and depression.  _arXiv preprint arXiv:2203.10432_ (2022).
  * 90 Basu, P. _et al._ Benchmarking differential privacy and federated learning for bert models.  _arXiv preprint arXiv:2106.13973_ (2021).
  * 91 Antunes, R. S., André da Costa, C., Küderle, A., Yari, I. A. & Eskofier, B.  Federated learning for healthcare: Systematic review and architecture proposal.  _ACM Transactions on Intelligent Systems and Technology (TIST)_ 13, 1–23 (2022).
  * 92 Brauneck, A. _et al._ Federated machine learning in data-protection-compliant research.  _Nature Machine Intelligence_ 5, 2–4 (2023).
  * 93 Dechand, S., Naiakshina, A., Danilova, A. & Smith, M.  In encryption we don’t trust: The effect of end-to-end encryption to the masses on user perception.  In _2019 IEEE European Symposium on Security and Privacy (EuroS &P)_, 401–415 (IEEE, 2019).
  * 94 Kalkman, S. _et al._ Patients’ and public views and attitudes towards the sharing of health data for research: a narrative review of the empirical evidence.  _Journal of medical ethics_ 48, 3–13 (2022).
  * 95 Sieberts, S. K. _et al._ Young people’s data governance preferences for their mental health data: Mindkind study findings from india, south africa, and the united kingdom.  _PloS one_ 18, e0279857 (2023).
  * 96 Olavsrud, T.  Data governance: A best practices framework for managing data assets.  _CIO_ (2021).
  * 97 Ada Lovelace Institute.  Participatory data stewardship: A framework for involving people in the use of data (2021).
  * 98 Campbell, J.  The historical and philosophical development of peer-run support programs.  _On our own, together: Peer programs for people with mental illness_ 17, 66 (2005).
  * 99 Thrul, J., Kalb, L. G., Finan, P. H., Prager, Z. & Naslund, J. A.  Web3 and digital mental health: Opportunities to scale sustainable mental health promotion and peer support.  _Frontiers in Psychiatry_ 1608 (2022).
  * 100 Network, T. M. R. M. D. S.  Principles of māori data sovereignty (2018).
  * 101 Feygin, Y. _et al._ A data dividend that works: steps toward building an equitable data economy.  _Berggruen Institute_ (2021).

## Acknowledgements

Part of this work was supported by funding from the National Institutes of
Mental Health (R01MH117172 and P50MH115838), from the National Science
Foundation (1952085 and 2000782), from an unrestricted gift from Google, from
the American Foundation for Suicide Prevention, and from the Microsoft AI for
Accessibility Initiative. Any opinions, findings, and conclusions or
recommendations expressed in this material are those of the authors and do not
necessarily reflect the views of Google.

## Contributions

The authors contributed in the following manner in the preparation of the
manuscript: Concept and design: Pendse, Stapleton, Kumar, De Choudhury,
Chancellor. Drafting of the manuscript: Pendse, Stapleton, Kumar, De
Choudhury, Chancellor. Critical revision of the manuscript for important
intellectual content: Pendse, Stapleton, Kumar, De Choudhury, Chancellor.

## Competing Interests

The authors declare no competing interests.

Table 1: The affirmative consent framework, with specific examples of where violations of the framework occur in algorithmically-mediated interventions and monetization of data Principle of Affirmative Consent |  Description |  Case Study 1: Algorithmically Mediated Interventions |  Case Study 2: Impacts of Monetization
---|---|---|---
Voluntary |  Digital service users provide consent freely and enthusiastically |  Users in distress do not choose to have their data analyzed for signs of crisis, and may not even know such analyses are occurring. |  Users may not be aware that data from their engagements is being collected, and may even be misled to believe their engagements are confidential by digital interfaces.
Informed |  Digital service users are informed about the context of their decision before providing consent |  Users are provided little information or transparency around what happens after non-consensual active rescue protocols are engaged. |  Many digital tools and services do not have privacy policies or ToS agreements—if they do, these are too opaque and difficult for an average user to understand.
Revertible |  Digital service users can revoke consent at any time |  Non-consensual active rescue protocols do not ask for consent, nor do they give opportunity for it to be revoked. |  Once data is shared, there is little opportunity for users to unshare the data, as ToS agreements can often be signed as part of one-time engagements (such as with crisis services).
Specific |  Digital service users consent to specific actions, rather than a series of actions |  Per the doctrine of parens patriae, users are not understood to be able to make specific decisions around the kind of care they want, and may be involuntarily hospitalized. |  Users have little ability to choose exactly what data they want to share (or not share).
Unburdensome |  Digital service users should not feel a burden to provide consent when they would rather say no |  Users often engage with digital tools and services in a state of extreme and burdensome distress, which can affect their perception and decision-making processes. |  Users sign ToS agreements or review privacy policies while in a state of extreme distress, and service provision may be contingent on their willingness to contribute data.
Table 2: Data privacy, security, and policy practices for consentful engagements with digital tools and services Data Privacy, Security, and Policy Practices |  Examples of Usage
---|---
Digital Psychiatric Advance Directives (DPADs) |  DPADs are documents that describe an individual’s preferences around their treatment if they are in crisis. They include clear guidelines around what data an individual feels comfortable sharing, as decided by the individual prior to any experience of crisis or severe distress. DPADs can be utilized by digital tools and services to ensure that any subsequent interventions or data collection center the consent and choice of the user.
Differential Privacy |  Differential privacy is an algorithmic approach to anonymization that ensures that individual data points are difficult to re-identify, while broader trends in the data stay constant. Differential privacy techniques can be used to ensure that data cannot be easily reidentified, even in case of data breaches.
Federated Learning |  Federated learning is an approach to machine learning in which predictive analyses are done at the user level, with model weights being used for large-scale analyses (instead of user data). This approach ensures that data used for machine learning analyses is similarly resilient to data breaches.
End-to-End Encryption |  End-to-end encryption is an approach to securing communications that ensures that platform owners cannot read or make inferences from messages, and that strictly senders and recipients can access messages locally. The use of end-to-end encryption secures private and sensitive communications between users, limiting them from potential harm.
Data Governance |  A consideration of data governance, or who has authority over how and why data is used, can ensure that data is used towards the benefit of users. Data governance strategies can be based on past service user movements, including democratic methods of decision-making and centering individual and collective power.