# A Taxonomy of Challenges to Curating Fair Datasets

###### Abstract

Despite extensive efforts to create _fairer_ machine learning (ML) datasets,
there remains a limited understanding of the practical aspects of dataset
curation. Drawing from interviews with 30 ML dataset curators, we present a
comprehensive taxonomy of the challenges and trade-offs encountered throughout
the dataset curation lifecycle. Our findings underscore overarching issues
within the broader fairness landscape that impact data curation. We conclude
with recommendations aimed at fostering systemic changes to better facilitate
fair dataset curation practices.

##  1 Introduction

Persistent concerns from academia, government, industry, and the public sphere
center on the disparate impact and unfairness in machine learning (ML)
[buolamwini2018gender, hofmann2024dialect, cheng2023marked,
tamkin2023evaluating, luccioni2024stable, coe2022, hundt2022robots,
world2021ethics, hern_2018, hern_2015, hern_2020]. Data is often viewed as a
primary culprit, perpetuating biases and compromising fairness
[geiger2020garbage, dehkharghanian2023biased, zhao2021captionbias,
ladhak2023pre]. In response, substantial attention has been directed towards
_fair_ dataset collection practices [fabris2022algorithmic, ding2021retiring,
yang2020towards, schumann2021step, zhao2021captionbias, hazirbas2021casual,
porgali2023casual, gustafson2023facet, ramaswamy2023geode]. However, there
remains a significant gap in understanding both the practices and
practicalities of fair dataset curation.

To address this gap, we shift from theoretical, guideline-focused scholarship
[gebru2018datasheets, andrews2023ethical, hanley2020ethical,
diaz2022crowdworksheets, pushkarna2022data, hutchinson2021towards,
jo2020lessons, miceli2022studying, denton2020bringing] to empirical inquiry,
exploring the grounded practices of fair dataset curation. Following a well-
established tradition in human-computer interaction (HCI)
[holstein2019improving, varanasi2023currently, sambasivan2021everyone,
madaio2020co, heger2022understanding, orr2024data], we conducted interviews
with 30 dataset curators from both academia and industry who have experience
curating fair vision, language, or multi-modal datasets. Through these
interviews, we uncover practical challenges and trade-offs to ensuring
fairness in dataset curation. Our use of qualitative methodology allowed us to
surface nuanced challenges and trade-offs that regularly appear throughout the
curation process and gain insights into considerations that may otherwise
remain undisclosed. Refer to Appendix A for further methodological details.

We first provide three dimensions of fairness— _composition_ , _process_ , and
_release_ —that participants considered during curation. Fairness is not only
a property of the final artifact—the dataset—but also a constant consideration
curators must account for throughout the curation process. Through our
empirical findings, we identify various challenges that obstruct different
fairness goals. Building on hutchinson2021towards’s conception of the dataset
lifecycle, we contribute a taxonomy of challenges dataset curators encounter,
both during the dataset lifecycle (Section 2) and within the broader landscape
of fairness in ML (Section 3). By conducting in-depth interviews with those
engaged in fair dataset work on the ground, we provide empirical support for
prior work [paullada2021data, liang2022advances, jo2020lessons,
andrews2023ethical, hutchinson2021towards, diaz2022crowdworksheets,
luccioni2022framework, google2019pair, ibm2019design], which has focused on
identifying implicit challenges in the fairness literature (see Appendix B for
additional background). We conclude with recommendations aimed at fostering
systemic changes to better facilitate fair dataset curation practices (Section
4).

Our work aligns with existing recommendations for fair dataset curation
[andrews2023ethical, jo2020lessons, scheuerman2021datasets, pushkarna2022data,
gebru2018datasheets, hutchinson2021towards, miceli2022studying,
miceli2021documenting, diaz2022crowdworksheets, paullada2021data,
birhane2021large, mcmillan2024data] and aims to deepen stakeholders’
understanding of the specific challenges involved. By illuminating these
issues, we hope to expedite more effective solutions and promote further
investigation into the complexities of fairness in dataset curation.

##  2 Challenges During the Dataset Lifecycle

We present challenges participants encountered across the dataset lifecycle,
taxonomizing them into requirements, design, implementation, evaluation, and
maintenance phases (see LABEL:fig:lifecycle and LABEL:table:examples1).
Recognizing the multi-faceted nature of _fairness_ , we did not impose a
specific definition during our interviews. Instead, we empowered participants
to articulate their own definitions. Based on these definitions, we identified
three dimensions of fairness: _composition_ , which is achieved through
diverse representations; _process_ , which includes equitable compensation for
data subjects and workers as well as recognition for curation efforts; and
_release_ , which emphasizes the importance of transparent and openly
accessible data. The challenges we surface span all three dimensions of
fairness.

###  2.1 Requirements

The _requirements phase_ involves establishing a dataset’s purpose (e.g.,
intended tasks such as image tagging) and defining the fairness criteria to be
operationalized within the dataset (e.g., group fairness).

Scoping a dataset. Participants sought to balance fairness with utility
(P8111To protect participant anonymity, we use the shortened “PX” to refer to
individual participants, where “P” stands for “Participant” and “X” is the
participant’s ID number (e.g., P8)., P23, P26, P30). On the one hand, careful
curation can lead to more nuanced insights compared to general-purpose
datasets. As P26 explained, they would ideally “design smaller datasets for
smaller models for specific applications, nothing that is deployed on a [South
Asian] scale, because that definitely won’t work properly because of the
[region’s] geographical diversity.” Moreover, datasets containing billions of
entries, such as LAION [schuhmann2021laion, schuhmann2022laion], make
oversight difficult and, as a result, may include “unfair” data (P18)
[birhane2023hate]. Nonetheless, participants also had to consider utility. P13
noted ML is “in this age of scale,” making them “a bit skeptical as [to]
whether people are going to openly use fair datasets for training unless
they’re very large.” P21 highlighted a similar tension between “technical
reasons why you need large open datasets” and “ethical reasons on why that
shouldn’t be the case.” Fairness trade-offs pushed some (P12, P13) towards
focusing on smaller evaluation datasets.

Determining fairness definitions. Nearly all participants stressed the
_contextual_ nature of fairness. Key factors shaping their definitions
included domain (e.g., healthcare), task (e.g., sentiment analysis), and
cultural context. For example, P2 highlighted the importance of cultural
specificity, stating, “you see a lot of work that talks about fairness in
gender or in race. But for a [South Asian] country, race does not manifest
like it manifests for America.” Participants also made trade-offs due to the
multitude of fairness definitions available [mitchell2021algorithmic] (Section
3.5). P19 noted that “there’s more than two dozen different fairness
definitions … used in the literature.” This diversity necessitated sacrifices
in other dimensions, as emphasized by P18, who illustrated this with the “‘no
free lunch theorem”, stating, “You can’t have complete diversity with respect
to, say, races,…geographies,…times of the day, and other domains. Everything
is not possible. Once you clamp on one, the other one goes away.”

###  2.2 Design

In the _design phase_ , curators determine how to operationalize dataset
requirements, including defining the dataset’s taxonomy. For example, curators
specify attributes for measuring fairness (e.g., skin tone) and the categories
within those attributes [thong2023beyond, heldreth2024skin, porgali2023casual,
hazirbas2021towards]. This phase also involves decisions on data collection
and annotation methodologies (e.g., web scraping, hiring vendors).

Creating fair taxonomies. Participants struggled to find a fair taxonomy under
the inherent unfairness of categorization. For example, P18 devised a
geographic taxonomy featuring categories for the U.S. and Asia, acknowledging
that the regions “are not homogeneous, they’re very heterogenous.” P2 also
noted a theoretically ideal taxonomy is as granular as possible, but practical
constraints, such as data availability (Section 2.2) and time (Section 3.3),
necessitated using coarser categories. Finally, the challenge of creating a
fair taxonomy was compounded by the inadequacies of existing domain
taxonomies. For example, P1 and P5 pointed out that the common binary
operationalization of gender in medical data erases many gender identities.
Nonetheless, participants felt compelled to utilize inadequate taxonomies due
to practical constraints, even if it contradicted their personal beliefs.
Participants were forced to align their notions of fairness with disciplinary
norms (Section 3.2).

Data availability in taxonomy design. Similar to when designing taxonomies,
participants had to balance their ideal data collection methods with practical
constraints. For example, P3’s dataset only included Spanish and Arabic even
though they “wanted to look at other languages, but … didn’t have training
data.” Participants questioned prevailing data collection paradigms, such as
web scraping [andrews2023ethical], which were seen as unethical when performed
indiscriminately. For legal compliance, P25 manually collected data for two
years: _“I was downloading, like clicking and clicking, because they didn’t
allow me to do web scraping or didn’t have an API._ ”

###  2.3 Implementation

The _implementation phase_ marks the execution of plans formulated in the
design phase, where curators collect, annotate, and package the data into a
dataset. This phase broadly encompasses two subphases: data collection and
data annotation.

####  2.3.1 Data Collection

_Data collection_ involves gathering relevant data to fulfill dataset
requirements. Challenges during this subphase prevented participants from
attaining fairness goals relevant to dataset diversity.

Diverse data availability. Similar to concerns raised regarding dataset
taxonomies (Section 2.2), participants raised concerns about data availability
for creating a fair dataset. For example, P28 described how sexist stereotypes
permeate web data, such as _“women [being] associated with nurse more often
than men.”_ Additionally, P18 encountered difficulties sourcing web data from
_“Middle Eastern”_ and “ _African countries”_ but found “ _lots and lots and
lots of images from India, Japan and [the] U.S., which are like the three most
dominant geographies in uploading pictures._ ” Participants also lamented the
inaccessibility of specialized or proprietary data, such as medical records or
data from private companies, which could significantly improve the creation of
fair datasets. P4 stated that _“because people don’t own large e-commerce
platforms or social media platforms, or whatever, we just kind of have to deal
with things that we can gather from existing systems.”_

Interestingly, synthetic data, sometimes presented as a potential solution to
biased data [bae2023digiface, smith2023balancing, ramaswamy2021fair], was met
with skepticism as it could perpetuate stereotypes or inadequately represent
underrepresented groups [whitney2024real]. As P19 pointed out, _“You might
address some of the missing data points [with synthetic data] but at the end
of the day it’s still the same underlying data distribution, right?”_

Data collector availability. Many participants associated fairness with
geographically diverse data. For example, P22 expressed how they would
_“proactively sample more data from underrepresented regions.”_ Yet,
actualizing this objective proved challenging, as P12 highlighted the
difficulty in _“get[ting] hold of people … from very, very small regions.”_
Infrastructure hurdles, such as limited internet and mobile phone access,
further complicated the process [jo2020lessons, andrews2023ethical]. Equipping
data collectors with necessary equipment is costly (Section 3.3) and
logistically challenging, as _“you might have to give people smartphones to
start and you’d also need more labor on the ground … who are working in these
different regions to come together and do this”_ (P12).

####  2.3.2 Data Annotation

_Data annotation_ involves labeling data with attributes specified during
design. Participants faced challenges recruiting annotators who had requisite
expertise or came from diverse backgrounds. Upholding fair labor practices
(Section 2.3.3) during annotation also presented challenges.

Data annotator diversity and expertise. The interpretation and application of
annotation categories can vary based on an annotator’s perspective
[barrett2023skin, cambo2022model, kapania2023hunt, andrews2023view]. P22
described finding annotators for labelling building styles across different
geographies: _“You give this same image to a local labeler who is in that
culture, who is an expert in, you know, their architecture … then you get a
much better label.”_ Yet, participants had difficulty hiring annotators that
met their desired aims. While P2 highlighted the value of diverse annotator
backgrounds or beliefs to ensure annotations reflected a wide range of
experiences, accessing diverse annotators was challenging, _“because some of
the attributes of [annotators’] personal lives might even be illegal to ask
about in a particular country.”_ Participants also confronted challenges in
recruiting annotators with specialized expertise. For example, despite
offering “ _$75 or $100 per hour,_ ” P1 faced difficulties finding and
incentivizing medical experts to annotate radiology data. Annotators who lack
diversity or expertise in data concepts may lead to issues with data quality,
including inaccuracies [hsueh2009data], biases [sap2022annotators,
ding2022impact, fan2022socio], and overly homogeneous annotations
[prabhakaran2021releasing, fan2022socio]. Notably, P13 highlighted that
crowdsourced annotators regularly embed gender biases into datasets such that
_“researchers [need] to make sure that annotators represent everyone because
[if] not, you’re just gonna have a skewed pool of annotations as well.”_

####  2.3.3 Implementation Processes

Participants expressed challenges not only with dataset content but also with
the _implementation_ of data collection and annotation. We provide three main
considerations discussed by participants.

Vendor transparency. Collaborating with data vendors introduced transparency
challenges, hindering fairness efforts. First, as prior research documented
[scheuerman2024walled], vendors may prohibit access to data worker identities,
such as demographic details or location (as described by P2 in Section 2.3.2).
Thus, it is impossible to evaluate potential biases or expertise linked to
identity characteristics, such as how an annotator’s cultural identity may
influence their engagement with data concepts. Second, participants had little
oversight into worker compensation or encountered communication restrictions
imposed by vendors. As P12 said, _“I think [pay] was fair in terms of [being]
calibrated across different countries … but we weren’t able to get exact
numbers, because that was confidential.”_ P6 described how vendor platform
design inhibited direct collection of feedback from data workers, impeding
efforts to improve fairness in dataset creation and labor conditions (e.g.,
[miceli2021documenting, miceli2020between]) (Section 3.5).

Language barriers. Curating fair datasets often involves collecting
geographically diverse data, which may require data workers proficient in
languages different from those of curators. Language barriers can hinder
effective communication, necessitating fairness concepts established in the
design phase (Section 2.2) to be accurately translated into the workers’
native languages. Improper translations can result in misinterpreted labels or
instructions and may even lead to contract breaches, particularly concerning
subject consent. Addressing language barriers often involves resorting to
translation services, which may be constrained by cost (Section 3.3) or
introduce its own fairness concerns. Further, participants had to ensure
translations accurately reflected their intentions, but as P3 noted, _“We
relied on our translators to come up with those sorts of decisions in terms of
Spanish.”_

Fair data labor. Several participants (P6, P11, P12, P14, P16, P24, P28)
expressed concern about engaging in fair labor practices when working with
data workers, but systemic organizational (Section 3.3) and regulatory
(Section 3.4) issues made achieving these standards difficult.

###  2.4 Evaluation

The _evaluation phase_ involves assessing data quality and testing dataset
utility. Challenges here can lead to homogeneous annotations, difficulties in
benchmarking, and spurious correlations.

####  2.4.1 Assessing Data Quality

_Assessing data quality_ entails validating and refining the data and its
annotations to ensure clarity and consistency with project requirements.
(Re)alignment of data and annotations with the guidelines from the design
phase is often referred to as _quality assurance_.

Gold standard paradigms. Participants often sought to capture a diversity of
perspectives across annotators. Thus, prevailing practices for validation and
cleaning, such as majority voting and annotator agreement metrics, may be
unsuitable. As P24 emphasized, majority voting can “squash or stifle diverse
opinions when it comes to subjective tasks.” When disagreement is integral to
the objective, annotator agreement metrics become inappropriate, making it
difficult to “validate” annotation quality. Gold standard paradigms are
intrinsically tied to disciplinary challenges (Section 3.2); if submitting a
publication involving dataset creation, reviewers might still call for
annotator agreement metrics and believe the quality of the data is poor if
agreement is low.

Similarly, common practices used to clean or filter data can perpetuate
dominant cultural beliefs. Data that might appear noisy or incorrect can hold
significance for certain communities. P14 explained how quality filters
resulted in “get[ting] rid of vernacular that’s not perfect English but is
maybe like African-American vernacular or like Hispanic-American vernacular,
and that also introduces bias and lowers the diversity of the dataset.” This
echoes prior work [bender2021dangers] which found that standard data filters
might disproportionately exclude content from already marginalized groups.

####  2.4.2 Evaluating Data Utility

To ensure dataset utility, curators must evaluate its effectiveness, often
through _requirements testing_ to confirm its suitability for the intended
purpose. Participants aimed to align the dataset with fairness definitions and
mitigate any potential biases present in the data.

Lack of benchmarking datasets. Curators often seek to benchmark their datasets
to showcase their utility. However, since many participants aimed to create
unprecedented fair datasets to address existing gaps, this norm posed a
challenge as comparable datasets were non-existent. Reflecting on the
struggles with a novel geodiverse dataset, P12 explained, “We couldn’t measure
it unless we had a dataset that actually was fair. Since we don’t have a
dataset that is fair…, you are arguing in circles.” Furthermore, even if
comparable datasets exist, they may harbor fairness issues of their own.

Evaluating immeasurable constructs. Evaluating whether a dataset aligns with
fairness definitions presupposes that fairness is a construct amenable to
measurement. While some participants offered quantifiable indicators of
fairness, such as demographic diversity, others argued that fairness defies
quantification. P14 criticized measurement-oriented perspectives, stating,
“They also assume that fairness can be measured, can be evaluated, and can be
improved. And I think that all of this is a more positivist mindset. ” Even
with a definition in mind, testing may feel incomplete. As P28 said, “Even
when you provide a way to measure fairness, you’re probably overlooking
something.”

Spurious correlations. Several participants (P6, P23, P28) aimed to avoid
introducing spurious correlations that affected the fairness of the dataset’s
composition [katzman2023taxonomizing, geirhos2020shortcut, calude2017deluge].
While these correlations may not be “connected with any demographic or social
variable” (P23), they can still influence downstream models and result in
biased decisions. However, as recent research [meister2022gender] has
revealed, spurious correlations with demographic attributes are ubiquitous.
Thus, enumerating and removing all possible correlations is virtually
impossible.

###  2.5 Maintenance

In the _maintenance phase_ , curators must consider both how their dataset is
released and strategies for ensuring its ongoing utility over time. Challenges
at this stage often linked back to participant concerns around fairness in
dataset release (i.e., ensuring the data is transparent and openly
accessible).

Unstable infrastructural ecosystems. Digital data is intrinsically
impermanent. Some participants (P1, P8, P30) emphasized the risk of data
instances disappearing due to broken links or shifts in platform popularity or
ownership, as observed with platforms like Twitter. Therefore, curators must
then not only monitor for missing data but also find suitable replacements
that match the original dataset’s distribution. This can be particularly
burdensome when the data was expensive (Section 3.3) or difficult to collect
(Section 2.3.1). As data goes missing, datasets can become unbalanced and thus
“unfair,” demonstrating how fairness issues with data release are linked to
concerns about composition.

Dataset traceability mechanisms. The challenge of dataset stewardship is
exacerbated by inadequate traceability mechanisms [scheuerman2023human,
peng2021mitigating]. Participants underscored their inability to track users
and usage patterns of their datasets. One commonly used proxy is citations in
academic papers, but it was hard to “distinguish citations that use the data
versus citations that use the broader idea of the paper” (P2). This is
concerning, especially if fair datasets are repurposed in unintended ways.
While prior works [andrews2023ethical, peng2021mitigating,
scheuerman2023human] have suggested data usage policies to mitigate such
risks, enforcing them becomes impractical when curators are unaware of actual
data users.

##  3 Challenges Overarching the Broader Landscape of Fairness

Many challenges that were surfaced cut across all phases of the lifecycle,
impacting the broader _landscape_ of dataset fairness. We identified five
levels within this broader landscape. Challenges may arise from one or more
levels of this landscape to impact dataset curation at every phase of the
dataset lifecycle (see LABEL:fig:ecological and LABEL:table:examples2).

###  3.1 Individual Level

The _individual level_ of the dataset curation landscape refers to the
contributors of fair datasets, such as data curators, data subjects, and data
workers.

Individual contributor positionality. Decisions made by contributors were
inevitably influenced by their own unique perspectives
[santy2023nlpositionality, scheuerman2024positionality]. As P24 said,
_“There’s this stuff we swim in that we don’t really realize is even there.”_
Despite recognizing this influence, assessing its tangible impact on the
dataset remained elusive. Addressing and diversifying contributor
positionality is further complicated by other challenges within the dataset
curation landscape, such as cost and power differentials. Positionality was
evident in instances where participants felt they had to make trade-offs
during processes like designing taxonomies that may erase others’ experiences.
P27 encouraged reflecting on personal values: “Is this [research] actually in
line with your life philosophy? Was it in line with your gender, with your
sexuality… If it’s not, would you still want to be doing this?”

###  3.2 Discipline Level

The _discipline level_ of the dataset curation landscape centers on the norms
and practices governing specific academic disciplines, particularly ML
[birhane2022values, dotan2020value, raji2021whole, scheuerman2021datasets].

Recognition for fair dataset work. Despite the growing demand for data in ML,
according to participants, fair dataset curation efforts were not seen as
significant contributions to the field. P11 described a _“lack of general
disciplinary value of datasets as contributions.”_ While some major
conferences like NeurIPS [db2021announcement] have introduced dataset tracks,
few venues prioritize dataset-focused work. This lack of appreciation
discourages efforts to ensure dataset stability and longevity
[scheuerman2021datasets].

Incentive mechanisms. Incentives in ML do not align well with the costs of
fair dataset curation. According to P11, there’s _“just [a] total lack of
resources and time to actually deeply engage with labeling and sourcing those
labels and getting people who are representative of those labels to be the
data workers.”_ Participants echoed well-documented observations that model
work is valued over data work [sambasivan2021everyone, scheuerman2021datasets,
rondina2023completeness], with P21 stating that _“data is kind of a second-
class citizen in ML research.”_ Consequently, P25 felt _“people are [not]
seriously talking about fairness … people are still just get[ting] whatever
[data] they get to do their research, or publish, or whatever.”_

Awareness of existing resources and guidelines. Participants had limited
awareness of existing guidance for fair dataset curation. This lack of
awareness may be attributed to some of these resources (e.g.,
[diaz2022crowdworksheets, jo2020lessons, hutchinson2021towards,
pushkarna2022data]) being disseminated outside of traditional ML venues (e.g.,
NeurIPS, *CL, ICML, CVPR). As P29 admitted, _“I don’t remember any explicit
guidelines that I’ve stumbled through for fair dataset collection. Honestly!”_
Promoting interdisciplinary awareness of fairness efforts among those
primarily involved in ML is challenging due to highly disciplinary norms that
prioritize novelty in ML methods over discussions on fair dataset curation.

Responsibility for fairness. The burden of responsibility for fairness weighs
most heavily on individuals aware of fairness concerns in ML. Participants
echoed findings from prior research [birhane2022values] that document how
fairness is not a top priority for many ML researchers. For example, P25 said
that “[in] the team I work with… I never heard them talking about [how] the
dataset has to be fair.” In P25’s experience, the norm was to cursorily engage
with fairness issues without substantive changes to research practices. Given
the lower prioritization of fairness in ML, the onus falls on individual
researchers who “have a strong sense of justice and fairness” (P24) or are
part of fairness-oriented communities to elevate these concerns. However, this
commitment often lacks external recognition and may hinder resource allocation
and research progress. Participants recognized that collecting fair data is
more challenging and resource-intensive compared to conventional methods: “If
you want to build a fair dataset, maybe the most efficient way to do that is
to scrape the web, but getting really diverse data in an ethical way is really
hard and really expensive” (P11).

###  3.3 Organization Level

The _organization level_ refers to the organizations where individuals conduct
fair dataset curation work, which could vary in size or nature, such as
academic or industry settings.

Lack of resources. Insufficient resources were a significant challenge across
all phases of the dataset lifecycle. As P1 declared: ‘ _‘Money?! (laughs) If
you have money, you can have a very high quality of data.”_ Fair data
collection methods are costly, especially concerning data quality and
annotation, which often require hiring experts. Convincing funders or
stakeholders of the value of investing in fair datasets proved difficult, as
noted by P24: _“It’s hard to convince somebody to spend thousands and
thousands to collect [a] dataset of recordings.”_ Moreover, participants aimed
to compensate data subjects and workers fairly, _“not just the minimum wages
that many times academia gives”_ (P29). Longterm maintenance costs added to
the financial burden, with difficulties in securing ongoing funding. P1 stated
no academic or industry organization _“[wants] to spend another millions of
money every year … to maintain those products.”_

Ethics washing. Participants disapproved of organizations that superficially
promote fair ML but fail to meaningfully integrate fairness into their
practices [wang2024strategies]. According to P16, the _“language of fairness
is simply external lip service [that] ultimately boils down to looking at the
maximization of other imperatives, such as economic ones.”_ Resource
constraints exacerbate this issue, leading organizations to prioritize
efficiency and cost-effectiveness over fairness. As P22 noted, _“A lot of big
companies do responsible AI shenanigans … for marketing … And then a new shiny
thing comes down the road, and then they join that instead.”_ When fairness is
valued primarily for its marketing appeal rather than its impact on product
development, it is not prioritized for monetary or labor investment.

###  3.4 Regulatory Level

The _regulatory level_ concerns laws and policies governing dataset curation
and use. Participants expressed anxieties about violating regulations they
were not necessarily equipped to fully understand.

Differing legal practices. Contextual laws and regulations posed a challenge
for participants. P2 described how _“laws in America or laws in Europe … might
not be directly applicable to a [South Asian] country that has a very
different societal situation.”_ Contextually contingent laws and policies
further complicated efforts to obtain data from diverse, underrepresented
populations (Section 2.3.1).

Legal risk. Throughout the dataset lifecycle, participants faced the looming
risk of unintentionally violating laws and regulations, potentially leading to
breaches of privacy, labor, or data ownership laws. Instances of inadvertent
violations are not uncommon, as highlighted by participants’ experiences with
web scraping practices. For example, P21 was aware that _“people discovered
links to child pornography”_ in a widely used benchmark dataset
[birhane2024into, thiel2023identifying]. In another instance, P5 described
working on a clinical dataset only to learn that releasing it was _“not
possible because it’s not consistent with the privacy laws in France.”_ To
mitigate these risks, some participants adopted highly cautious practices,
such as exclusively collecting royalty-free or Creative Commons images, and
storing only image URLs to avoid any copyright violations. However, these
strategies can result in dataset instability, as observed by P8, who faced
issues with broken URLs.

Limited regulatory literacy. Insufficient understanding about navigating the
law intensified concerns about legal risk. P8 described it as _“a big learning
curve to understand what we were allowed to store and what we weren’t.”_ As a
result, P8 consulted an intellectual property lawyer. However, depending on
the other constraints dataset curators are under, such as discipline (Section
3.2) or organization (Section 3.3) level constraints, hiring legal counsel may
be untenable.

###  3.5 Socio-Political Level

The _socio-political level_ covers the shifting social and political contexts
around fairness in which curators operate. These challenges can be
conceptualized as thorny, fluid, and arguably insoluble.

Evolution and contestability of fairness. According to P3, fairness will
_“always be up for debate,”_ making it _“sort of impossible for there to be
like a gold standard.”_ Fairness is subjectively perceived, influenced by
individual contexts, experiences, and beliefs [scheuerman2024positionality].
This subjectivity fuels ongoing scholarly debates [diaz2022crowdworksheets,
Lebovitz2021IsAG, sen2015turkers]; it also fueled diverse perspectives among
participants. As P30 pointed out, _“There are people from the audience who say
that we have a good definition [of fairness], and there are some people who
say that we have a terrible definition. And there’s no way to make everyone
happy.”_ The absence of a universally accepted definition complicated
participants’ efforts to operationalize fairness in dataset curation. Further,
existing guidelines may not suit every notion of fairness, leading to
divergent curation methodologies. As P14 highlighted, _“It’s kind of like a
philosophical question … while the quantitative method says that fairness can
be achieved, contrast it to qualitative that we are just trying to understand
the experience here.”_ Beyond disagreements about what fairness means (or
should mean), participants also noted that current definitions are not stable.
As P16 put it, fairness _“should be a notion that is able to evolve within
society, and certain forms of injustice that were not considered injustice[s]
in the past now are … there might be other evolution towards the future that
we currently do not incorporate in our definition of fairness, and we need to
account for that.”_ This perpetual evolution presents challenges for dataset
curators. They must decide whether to regularly update datasets or retract
them as definitions evolve. However, both approaches have limitations in
addressing the continued use of previously released datasets
[peng2021mitigating, luccioni2022framework].

Social realities versus model realities. P8 described how the real world is
different than “ _what’s experimentally valid and testable._ ” Due to the
complexity of the real world, certain groups inevitably remain
underrepresented, misrepresented, or overlooked entirely despite best efforts.
For example, P12 mentioned that while they wanted to collect images from
underrepresented countries, data collector availability constrained their
options (section 2.3.1). Participants also questioned whether balanced
representation was even the best approach. As P1 pointed out, _“The problem is
when you actually apply such a model to the real world, the real world is
imbalanced, right?”_ This echoes the classic trade-off between fairness and
accuracy in algorithmic fairness work [corbett2017algorithmic]. Curators must
wrestle not only with the impossible task of how to best account for every
human experience in a dataset, but also whether or not they should be.

Power differentials. Power imbalances contribute to fairness issues during the
curation process that are not visible in the dataset’s composition.
Participants noted how more elite institutions and companies dominate efforts
to create fair datasets, largely owing to their access to resources (Section
3.3). Similar to findings from prior work [koch2021reduced], P21 described how
most public datasets are not used, with the majority of _“the datasets that
get used in ML research [being] created by a very, very small elite cadre of …
academic institutions that have close affiliations with top industry
researchers.”_ Similarly, P16 felt it was problematic that the _“most
important tools”_ remain in the hands of a few companies, _“yet they are given
the freedom to define what is fair, and their definition is used, and then the
safeguards that do exist might not always align or ensure protection.”_.
Thinking on a geopolitical scale, P2 noted that _“the field of algorithmic
fairness has been dominated by the Western perspective.”_ This imbalanced
representation exacerbates other challenges previously outlined, including
those at the implementation, disciplinary, and organizational levels.

Power differentials also permeate the relationship between dataset curators
and other stakeholders, such as between curators and data subjects or workers
[wang2022whose]. For example, P6 described how curators have complete
oversight over worker compensation: _“So many platforms don’t actually ensure
that you’re fairly compensating workers. And it’s really up to the individual
researchers which is a crazy system that sets absolutely the wrong
incentives.”_ P10 compared the impulse to collect data cost-effectively, at
the expense of data subjects, as _“a particular kind of colonial impulse,
like, this is just up for grabs.”_ Similarly, curator decisions have profound
implications downstream. P22 described the difficulty of _“fighting”_ clients
who do not prioritize model performance on heavily under-resourced
populations, given they are not central to business incentives: _“It’s like,
‘99% of my customer[s] will be fine, why do I need to care about that last
1%?”_ Overall, dataset curation was seen as _“a very unfair process, no matter
how you do it … unless you’re going to literally tackle society”_ (P8).

##  4 Considerations for Enabling Fair Dataset Curation

Finally, we highlight recommendations across the three dimensions of fairness
for facilitating fair dataset curation. We focus on top-down efforts,
reflecting the need for systemic changes rather than relying solely on
individual contributions. See Appendix D for more details.

Composition. To better enable fair dataset _composition_ , we encourage
interventions for more flexible and robust data practices. For example, at the
design phase (Section 2.2), flexible taxonomies can facilitate different
operationalizations rather than forcing curators to use only one taxonomy
(e.g., protected attributes can include self-reported and third-party labels).
At the discipline level (Section 3.2), we advocate for more communication
across academic communities. Papers published outside traditional ML venues
(e.g., CHI, FAccT, CSCW) have provided guidance on data curation, such as
annotation practices [kazimzade2020biased, diaz2022crowdworksheets,
whiting2019fair, collins2022eliciting] or considerations on taxonomies
[barrett2023skin, khan2021one, hanna2020towards, xiang2024mirror].

Process. A change in the fair dataset curation _process_ requires not only
norm-setting within fairness communities, but also legal and policy
interventions. For example, at the implementation phase (Section 2.3.3),
participants were concerned about labor rights for data workers. As a
discipline, we should have norms about compensating workers, at least at the
local minimum wage, for their labor and support efforts to introduce policies
that offer codified protection for data workers. Furthermore, at the
regulatory level (Section 3.4), rather than expecting curators to develop
legal expertise, we advocate for the creation of accessible resources on legal
practices regarding dataset collection.

Release. We encourage interventions that allow for fairness post-_release_.
For example, at the maintenance phase (Section 2.5), efforts to build tools
and policies to enable better dataset traceability could alleviate concerns
with dataset misuse. Additionally, at the organization level (Section 3.3),
funding entities should invest in maintenance, rather than solely focusing on
modeling research. Monetarily valuing long-term maintenance plans as research
contributions may help shift perspectives about revision, maintenance, and use
policies at the discipline level (Section 3.2).

##  5 Conclusion

Through interviews with fair dataset curators, we developed a taxonomy of
challenges appearing across the dataset lifecycle and the fairness landscape.
Our participants navigated difficult trade-offs between ideal fairness goals
and practical concerns, such as data availability, resources, and time. While
we acknowledge limitations of our methodologies (see Appendix E), taxonomizing
these challenges is a crucial first step in developing long-lasting solutions
to support fair dataset curation.

Addressing these challenges requires effort from individual data curators as
well as systemic changes at the organizational, disciplinary, and regulatory
levels. Beyond providing dataset curators grounded evidence to refer to in the
quest for building fair datasets, our taxonomy offers stakeholders a pathway
to tackling each challenge individually and opens avenues to further, more
targeted investigation into the many challenges to curating fair datasets.

## Checklist

##  Appendix A Methods

To understand the challenges in collecting fair datasets, we conducted 30
semi-structured interviews with ML dataset collectors, each lasting between
45–60 minutes. Interviews were conducted between November 2023 and March 2024.
Participants were asked to define fairness in ML datasets, describe their
process for collecting fair datasets, highlight challenges faced, and discuss
any trade-offs encountered during the dataset collection process. The study
was approved by Arizona State University’s Institutional Review Board.

###  A.1 Participant Recruitment

To qualify for our study, participants must have previously collected at least
one “fair” ML dataset. Considering the extensive discourse on language and
vision dataset practices [andrews2023ethical, paullada2021data, jo2020lessons,
birhane2024into, birhane2021large], we prioritized participants specializing
in these domains. To accommodate diverse perspectives, we refrained from
prescribing a specific definition of “fair”. Participants were initially
recruited through purposive sampling, involving outreach to authors from a
list of public datasets. To expand our sample, we advertised on social media
platforms and relevant mailing lists, and employed snowball sampling. In
total, we reached out to 204 individuals with 95 no responses, 51 declines,
and 28 who did not meet our participant criteria. We concluded recruitment
with 30 participants as we had reached thematic saturation. As shown in
LABEL:table:participant_summary and LABEL:table:participants, the 30
participants represent diverse backgrounds and experiences, with a predominant
presence from academia. Compensation consisted of a $75 Amazon gift card, or
the equivalent in the participant’s local currency.

###  A.2 Participant Anonymity

To preserve participant anonymity, participant recruiting and interviews were
conducted only by members of the research team from Arizona State University.
The interviews were then transcribed and any identifying details were removed
from transcripts. Only the redacted interviews were shared with other members
of the research team for analysis.

###  A.3 Thematic Analysis

To analyze our interviews, we use an abductive approach [thompson2022guide],
building upon existing knowledge about data curation with our qualitative
interviews. We start with an initial set of codes related to challenges in
fair data collection processes that arose from our literature review. Then,
all authors independently coded the same interview surfacing additional
themes, followed by a subsequent interview to create a refined codebook.
Themes evolved iteratively, resulting in high-level codes on dataset taxonomy,
fairness definitions, motivation, and collection processes. Remaining
interviews were equally divided among the research team for qualitative
analysis.

###  A.4 Interview Protocol

We provide the protocol used to guide the semi-structured interview process
conducted with participants. The interview questions were designed based on
considerations around fair dataset curation that had been raised in the
existing literature. Depending on the answers that the participants provided,
the interviewers asked relevant follow-up questions. The questions are as
follows:

##  Appendix B Background

Concerns over the disparate impacts or unjust outcomes associated with machine
learning (ML) continue to persist [buolamwini2018gender, hofmann2024dialect,
cheng2023marked, tamkin2023evaluating, luccioni2024stable]. One of the central
concerns underscoring the pursuit of fair ML remains the datasets used to
develop ML systems [paullada2021data, birhane2021large, birhane2021multimodal,
buolamwini2018gender, luccioni2021s]. Yet obtaining fair and ethically-sourced
datasets remains a challenge. Data is often perceived as the scourge of ML
models and a source of for downstream biases [geiger2020garbage,
dehkharghanian2023biased, zhao2021captionbias, ladhak2023pre]. Here, we
provide background on prior scholarship documenting the current issues with
dataset curation, as well as work focused on improving those practices.

Issues with existing dataset curation practices. Poor training data can lead
to representational harms [buolamwini2018gender, hirota2022gender,
garcia2023uncurated], such as stereotyping [schwemmer2020diagnosing,
caliskan2017semantics, bolukbasi2016man, gallegos2023bias], spurious
correlations [meister2022gender, wang2019balanced, zhao2017mals], and poor
performance or total erasure of certain populations [wilson2019predictive,
buolamwini2018gender, zhao2021captionbias]. Poor evaluation data means harmful
model outcomes may be overlooked or missed, especially as they cascade into
various (often unintended) domains [sambasivan2021everyone]. Beyond data’s
impact on models directly, ML datasets are increasingly scrutinized for
violating the ethical values of privacy and consent [andrews2023ethical,
mittelstadt2016ethics, de2016new, politou2018forgetting, paullada2021data],
reinforcing disputable social constructs [scheuerman2020we, hanna2020towards,
khan2021one, benthall2019racial, blili2022making], including highly offensive
content [birhane2021large, zhao2021captionbias, birhane2023hate], and
exploiting vulnerable populations for both data and annotations
[smart2024discipline, gray2019ghost, wang2022whose].

Practices for collecting large-scale data, such as web scraping, have
consistently failed to meet many legal standards at the local and national
level, violating copyright laws [grynbaum2023sue], biometric laws
[hill2019photos, yew2022regulating], and even including child exploitation
content [birhane2024into, thiel2023identifying]. The difficulty of authoring
and maintaining a comprehensively “fair” dataset is exacerbated by
differential definitions of fairness and how to measure it (or whether it can
be measured at all) [mehrabi2019survey, tomasev2021fairness, andrus2021we].
Current approaches to dataset documentation also obscure the inherently
collaborative work that dataset authors must engage in and negotiate
[miceli2021documenting, miceli2020between].

Improving dataset curation practices. Given the vast and varied issues with ML
datasets, there has been a extensive line of work focused on improving dataset
collection practices. These efforts have evolved substantially beyond _ante
hoc_ calls for more transparent and robust documentation of existing datasets
[gebru2018datasheets, pushkarna2022data, chmielinski2022dataset,
diaz2022crowdworksheets, papakyriakopoulos2023augmented, bender2018data,
mcmillan2024data], such as datasheets, which often result in a _ante hoc_
approach, thus failing to capture decisions and trade-offs which might have
occurred prior to and during data collection. Thus, scholars are attempting to
provide frameworks at different levels of granularity of considering the
responsibility of dataset authors leading to frameworks or design guidelines
for both _pre hoc_ and _per hoc_ dataset curation [andrews2023ethical,
scheuerman2021datasets, park2021designing, gray2024building,
gondimalla2024aligning, wilcox2023ai, orr2024data].

For example, at a higher level, scheuerman2021datasets proposed a value-
centric framework that centers values like positional expertise and
contextually-relevant annotations. andrews2023ethical released a comprehensive
set of considerations for responsibly curating human-centric computer vision
datasets, covering topics like consent, human diversity, and subject
revocation. Recent work from orr2024data distilled seven recommendations from
interviews with 18 dataset curators. Their work highlights high-level themes
such as advocating for more dataset auditing, ensuring participant privacy,
and encouraging more documentation. Scholars are also increasingly providing
highly contextual and specific guidance for collecting data on certain
subgroups and vulnerable populations, such as children [wang2024challenges,
hill2019photos, wilcox2023ai], who are increasingly ending up in large web-
scraped datasets [birhane2024into, thiel2023identifying].

The scholarship focused on providing considerations and guidance for ethical
dataset curation has been invaluable. However, how authors actually approach
curating fair datasets is still opaque—especially given documented gaps
between guidance and practice. Prior work has uncovered numerous barriers to
incorporating fairness into practice [holstein2019improving,
heger2022understanding, deng2022exploring, madaio2020co, madaio2022assessing,
wang2024strategies], including misalignments between available toolkits and
product needs [deng2022exploring], organizational trade-offs that make
auditing methods less effective [deng2023understanding, ojewale2024towards],
and difficulty negotiating expectations across roles [deng2023investigating,
madaio2022assessing]. Yet literature on the challenges to creating fair
datasets currently lacks a holistic framing of fairness that involves not only
the composition of datasets, but the practices of producing and maintaining
them. Identifying the challenges currently facing dataset curators focused on
creating fair datasets is crucial to enabling fairer dataset curation in both
industry and academic settings.

##  Appendix C Additional Figures and Tables

We provide an overview of challenges discussed in Section 2 that occur across
the dataset lifecycle in LABEL:fig:lifecycle. To illustrate each of the
challenges we identified, in LABEL:table:examples1, we provide an example
quotation from our interviews. We similarly map out the overarching landscape
of fairness challenges using a social ecological model
[bronfenbrenner1994ecological] and provide detailed examples (see
LABEL:fig:ecological and LABEL:table:examples2).

##  Appendix D Detailed Recommendations for Enabling Fair Dataset Curation

Recommendations are aimed at diverse stakeholders influencing fair dataset
curation, including—but not limited to—individual contributors, academic
institutions and venues, industrial organizations, policymakers, and the
affected public. Unlike in the main body of the text, where we describe the
challenges with the dataset lifecycle first and the challenges with the
overarching landscape of fairness second, here, we present considerations with
the overarching landscape foremost. We also begin with the highest level of
the dataset landscape, the _socio-political level_ , rather than the lowest,
the _individual level_. Our goal is to underscore how top-down changes can
have broader impacts downstream on individual data curators and the dataset
lifecycle. We advocate for more systemic changes rather than placing the onus
of fairness onto individuals. The following recommendations in Sections D.1
and D.2 are examples. We imagine there are many more interventions which would
be effective in improving fair dataset curation.

###  D.1 Recommendations Overarching the Broader Landscape of Fairness

####  D.1.1 Socio-Political Level

Evolution and contestability of fairness. As conceptualizations of fairness
inevitably change, curators should aim to keep datasets up-to-date. For
example, we recommend that curators revise and amend datasets to comply with
new conceptualizations of fairness. For example, yang2022enhancing obfuscated
faces in ImageNet after release as an effort to mitigate concerns about data
subject privacy. Furthermore, when datasets cannot be aligned with new
standards, norms, laws, or policies surrounding fairness, they ought to be
deprecated and no longer used. Curators can refer to luccioni2022framework’s
framework for retracting and deprecating datasets to better understand this
process.

We also recommend that data curators clearly document the decisions that were
made about contextually and temporally relevant definitions of fairness. Thus,
even if the original curator cannot afford to update the dataset, others can
continue to maintain its documentation pointing toward new research showing
the issues with past fairness operationalizations.

Social realities versus model realities. We recommend dataset curators engage
with affected communities to understand the needs and potential impacts
datasets and downstream models have on the lives of real people. This includes
situating data curation decisions in the experiences and perspectives of
affected communities. For example, kuo2024wikibench introduced WikiBench, a
system for creating community-driven evaluation dataset on Wikipedia. Using
WikiBench, community members can work together to select, label, and discuss
instances for an evaluation dataset. Adopting a more participatory and bottom-
up approach allows dataset curators to ensure that they are capturing the
concepts most relevant to impacted communities.

Power differentials. First, we recommend incentivizing dataset curation with
fairness perspectives outside of the West and Global North
[septiandri2023weird, chan2021limits]. For program committees or conference
chairs, potential actions can include having special tracks for these datasets
or offering travel scholarship for researchers to the conference. We advocate
for approaches that empower researchers from the Global South to create their
own datasets.

Another power differential participants discussed was between researchers and
data subjects or annotators. To address this, we urge curators to center the
agency and consent of data subjects as well as the expertise of data workers.
Rather than treating data workers as “ghost workers” [gray2019ghost], curators
should ensure that data workers are meaningfully involved throughout the data
curation process and thought of as contributors rather than solely as a labor
source.

####  D.1.2 Regulatory Level

To help minimize legal risk, our first recommendation is for the the
discipline to develop ethical review processes to assess for potential legal
implications of dataset collection. Venues, such as NeurIPS, have instituted
impact statements and paper checklists for submitted works
[neurips2021checklist]. We recommend that this reviews extends to include
legal risks. We advocate for this discipline-wide approach as it can defray
potential concerns regarding resource mismatches when it comes to consulting
legal counsel. By developing a standardized procedure for legal compliance
across datasets, it also reduces the burden on individual curators who may
have limited regulatory literacy.

Nonetheless, we still recommend that individual curators pay particular care
when collecting data containing people or about people. One alternative here,
such as that taken by asano2021pass and ramaswamy2023geode, is to ensure there
are no people in the dataset. Of course, there is still a need for human-
centric datasets. In this case, we urge curators to recognize that using
royalty-free or Creative Commons licenses does not absolve the data of
potential ethical or legal issues regarding privacy or consent
[andrews2023ethical]. Instead, curators ought to obtain informed consent from
data subjects following well-established protocols from human subjects
research [faden1986history].

####  D.1.3 Organization Level

Ethics washing. Participants were disillusioned by organizations that treated
fairness as “lip service” and engaging in the practices of ethics-washing.
Echoing wang2024strategies, we recommend institutional efforts to keep
organizations liable for the ethical AI promises that they make. In addition
to relying on individual contributions from researchers and journalists,
having watchdog organizations monitor for ethics-washing. This recommendation
draws from existing practices of monitoring companies for “greenwashing”, or
manipulative promises from companies that they are engaging in environmentally
friendly actions [de2020green].

####  D.1.4 Discipline Level

Lack of recognition and incentives. Since 2021, there have been efforts to
introduce more dataset-focused tracks, such as the Datasets and Benchmarks
track at NeurIPS [neuripsAnnouncingNeurIPS] or the Journal of Data-Centric
Machine Learning Research (DMLR). We recommend building on this trend and
encouraging more dataset-focused tracks, including some that have specific
sub-areas dedicated to fairness-oriented datasets. This can help address the
lack of recognition and incentives for fair dataset work.

Responsibility for fairness. Fairness-oriented changes ought to be widely
adopted amongst ML dataset creators, not only those who may be more “fairness’
or “justice” oriented. To encourage this shift, we recommend adopting more
educational training on these subjects. Universities can include fairness and
ethics into computer science courses. An example of this are the Embedded
EthiCS programs at universities which encourage students to think critically
about the technology they are learning about in their computer science courses
[grosz2019embedded]. Beyond university courses, AI ethics review processes can
also mandate certifications that researchers must complete prior to getting
approval similar to the trainings that researchers must complete before
receiving IRB approval.

####  D.1.5 Individual Level

Individual contributor positionality. Contributor biases are inevitable. Our
recommendations here focus not on removing all individual biases but rather on
encouraging curators to get multiple perspectives and reflect on what biases
they may be bringing prior to data collection. One recommendation is to
institute a “pre-registration” system similar to what social scientists have
in place [p2021pre]. Pre-registration requires social scientists to publicly
state their hypotheses, methods, data collection process, and analysis plans
prior to beginning their experiment. Filling out a pre-registration prior to
data collection could encourage curators to think through design biases and
justify the choices they have made in a transparent and standardized manner.

###  D.2 Recommendations During the Dataset Lifecycle

####  D.2.1 Requirements

Determining fairness definitions. Participants considered fairness to be
highly contextual. To ensure that the definitions of fairness match those of
impacted communities, we recommend that curators solicit and incorporate
community feedback into the design and evaluation of fairness criteria
[blodgett2020language]. This can help ensure that the dataset reflects the
needs and values of diverse populations. As an example for how this can be
done, curators can look to works such as shen2022model which aimed to involve
community members in deliberative processes for defining AI systems. Similar
participatory processes can be adapted for determining fairness definitions in
datasets.

####  D.2.2 Design

Creating fair taxonomies. When designing a label taxonomy, we encourage
curators to evaluate trade-offs associated with adopting coarser categories,
such as loss of granularity versus feasibility and practicality. Data curators
should report both their ideal data collection scenario and the actual
approach taken. This information is useful not only from a transparency
perspective but also for other researchers who may face similar issues in the
future.

In addition, these taxonomies should be designed with scalability in mind.
Curators should make provisions to ensure the taxonomy is flexible enough to
incorporate new data if collected. For example, the OpenImages dataset
[kuznetsova2018open] has had several new versions and additions since its
initial release, including schumann2021step’s new demographic annotations
which are aimed to aid with fairness research.

####  D.2.3 Implementation

Vendor transparency. As third-party vendors offer an alternative path for data
collection, we recommend curators prioritize transparency both in negotiations
with these vendors and when reporting their results. In negotiations with data
vendors, curators should prioritize transparency clauses in the contract. For
example, curators should advocate for transparency in data worker identities
and compensation handling. This can help to ensure that they have access to
necessary information for evaluating dataset fairness. During the collection
process, data vendors should be held accountable for transparency practices
through regular monitoring and evaluation. This could involve conducting
audits or assessments to ensure compliance with transparency agreements and
guidelines.

To reduce the burden on individual curators, there should be a discipline-wide
effort to evaluate and benchmark data vendors based on transparency and
ethical data collection practices. From management studies, there is a line of
work on vendor evaluation systems and vendor scorecards that can be adapted
for third-party data curation services [luzzini2014designing].

Language barriers. When faced with language barriers, the data curation team
should ensure that they have members who have an understanding of the data
collection project’s context, goals, and data requirements such that they can
provide more contextually appropriate translations. If this is not possible,
we recommend establishing partnerships with local community organizations or
language schools to access language resources at reduced costs.

Fair data labor. To ensure fair data labor practices, we recommend curators
create clear guidelines and protocols for hiring, training, and evaluating
data workers to promote fairness and prevent exploitation. Following prior
works [whiting2019fair, andrews2023ethical, chan2021limits,
toxtli2021quantifying], we also advocate for transparent and equitable
compensation structures for data workers. When possible, curators should
provide opportunities for professional development and advancement for data
workers.

Diverse data availability. Curators should consider using alternative data
sources beyond web data, such as community-driven platforms or public
repositories, to supplement dataset diversity. To support this, organizations
should invest in creating public data trusts [chan2023reclaiming] or data
consortia [jo2020lessons] as an alternative source for large-scale data.

Data collector availability. To address a lack of data collector availability,
we recommend curators form partnerships with universities, organizations
(e.g., NGOs, non-profits), or community groups, operating in underrepresented
regions. These partnerships can help the recruitment of data collectors from
the target regions, leveraging existing networks and/or local expertise to
overcome challenges. For example, rojas2022the partnered with Gapminder and
individual photographers to collect geographically diverse images for the
DollarStreet dataset.

Data annotator diversity and expertise. When recruiting data annotators,
curators should research and understand which personal attributes are legally
protected and cannot be asked about during the hiring process. Further, they
should be cognizant of cultural nuances. For example, disclosing sexuality can
potentially endanger workers. Thus, rather than directly asking sensitive
personal attributes, curators can utilize alternative methods for assessing
annotator qualifications and suitability for the project. This is especially
helpful when annotators may not want to disclose certain attributes. Finally,
curators should offer training and resources to annotators to help them
understand the cultural significance of the data they are annotating.

####  D.2.4 Evaluation

Gold standard paradigms. Dataset curators can adopt evaluation methods that
embrace diverse perspectives rather than only using consensus-based methods,
which may only showcase the viewpoint of the majority. Works from both machine
learning [davani2022dealing, leonardelli2021agreeing] and human-computer
interaction [gordon2021disagreement, gordon2022jury] have encourage using a
multiplicity of annotations, which can showcase disagreement, rather than
using majority voting. For example, a curator may capture a diversity of
annotations from each annotator, with qualitative explanations as to why the
annotator chose each label. Prior works [prabhakaran2023framework,
wan2023everyone] have also provided frameworks for quantifying disagreement
across diverse groups of annotators that can be used as an alternative measure
to consensus-based approaches.

Evaluating immeasurable constructs. When it comes to evaluating immeasurable
constructs, curators can supplement quantitative metrics with qualitative
approaches. This could include interviews with data workers to better
understand their point of view and reveal any potential biases or ethical
issues that arose during the collection process. Furthermore, as
miceli2021documenting advocate for in their work, there should be more
reflexivity in the data collection process. Concretely, refereed publications
should require more critical reflection on the limitations and trade-offs of
the dataset by the curators.

####  D.2.5 Maintenance

Unstable infrastructural ecosystems. To manage unstable infrastructural
ecosystems, we recommend building standardized methods for checking the
availability of data sources and creating protocols to replace instances if
they have been deprecated. For example, P8 mentioned developing automated
scripts that would periodically check whether their dataset instances were
still available. Rather than waiting for dataset users to notify curators that
certain instances are no longer available, this allows for proactive
maintenance.

Going hand-in-hand with this, once curators are aware that certain instances
are deprecated, they should have a plan for replacing them in a way that
maintains the overall composition of the dataset. This can be challenging,
especially for datasets where compositional fairness is prioritized. We
recommend that dataset curators create a protocol for identifying alternative
data sources that match the distribution and characteristics of the original
dataset at the design phase. For example, dataset curators can keep a portion
of collected data as “backup” that they can use to replace instances that are
deprecated or removed.

Dataset traceability mechanisms. One challenge curators faced was tracing how
their dataset was used after release. Often they relied on citation metrics as
a proxy; however, it was difficult to disambiguate whether the citation meant
the authors were using the dataset or referring to concepts in the paper. As
an alternative, we recommend curators require users to register or
authenticate their identity before accessing datasets, enabling better
tracking and accountability. For example, ImageNet [deng2009imagenet] requires
users to sign in to their platform before downloading data. Another option can
be to use permanent digital identifiers, such as DOIs, which is already a
standard for some journals such as
Nature.222https://www.nature.com/ncomms/editorial-policies/reporting-standards
Similarly, curators can use centralized data repositories (e.g., Hugging Face,
Kaggle, Zenodo, Harvard Dataverse, Mendeley data).

##  Appendix E Limitations

The participant sample we recruited is skewed both in terms of location and
organization type. In total, three of the 30 participants we recruited were
from regions outside of North America or Europe. In part, our sample may
reflect the Western-centric nature of machine learning and algorithmic
fairness research [septiandri2023weird, koch2021reduced]. Given the challenges
participants raised about creating culturally contextualized datasets and
power differentials across geographies, future work that draws from a more
geographically diverse sample can provide more nuanced insight. We also note
that most of our participants were situated in academia. There may be specific
challenges to dataset curation in an industry context (compared to an academic
one) that we are not highlighting in our taxonomy.

Overall, a consistent theme from our interviews is that fairness is contextual
and the datasets curators collect are often designed specifically for their
task or domain. Inherently this means that the qualitative data we collect
will be constrained to the slices of perspective that our participants have.
While we do distill our findings from shared themes across interviews, these
challenges are neither necessarily applicable nor entirely representative for
all dataset curators.

##  Appendix F Broader Impacts

Our work focuses on understanding the challenges with fair dataset collection
by conducting on-the-ground interviews with dataset curators. We provide a
taxonomy of challenges that curators face throughout the dataset lifecycle and
an exploration into the broader landscape of challenges curators face. For
dataset curators, this work provides valuable insight into the nuance and
trade-offs related to dataset creation that may not appear in publications. By
formalizing this otherwise tacit knowledge, we hope to make the process of
fair dataset collection more accessible for curators. More broadly, we intend
for our work to have an impact on machine learning as a discipline. We seek to
emphasize the importance of dataset curators’ labor, which often is
undervalued [sambasivan2021everyone, scheuerman2021datasets,
rondina2023completeness]. Furthermore, we provide an extensive set of
recommendations that can be implemented by either individual contributors or
from the top-down. By using these recommendations and the challenges we have
surfaced, we hope to help facilitate better fair dataset curation practices
within the machine learning community.