citenum

# Quantifying the Impact of Biobanks and Cohort Studies

Rodrigo Dorantes-Gilardi  Million Veteran Program (MVP) Coordinating Center,
VA Boston Healthcare System, Boston, MA, USA  Network Science Institute,
Northeastern University, Boston, USA  Kerry Ivey  Million Veteran Program
(MVP) Coordinating Center, VA Boston Healthcare System, Boston, MA, USA
Division of Aging, Department of Medicine, Brigham and Women’s Hospital,
Boston, MA  Department of Medicine, Harvard Medical School, Boston, MA  Lauren
Costa  Million Veteran Program (MVP) Coordinating Center, VA Boston Healthcare
System, Boston, MA, USA  Rachael Matty  Million Veteran Program (MVP)
Coordinating Center, VA Boston Healthcare System, Boston, MA, USA  Kelly Cho
Million Veteran Program (MVP) Coordinating Center, VA Boston Healthcare
System, Boston, MA, USA  Division of Aging, Department of Medicine, Brigham
and Women’s Hospital, Boston, MA  Department of Medicine, Harvard Medical
School, Boston, MA  John Michael Gaziano  Million Veteran Program (MVP)
Coordinating Center, VA Boston Healthcare System, Boston, MA, USA  Division of
Aging, Department of Medicine, Brigham and Women’s Hospital, Boston, MA
Department of Medicine, Harvard Medical School, Boston, MA  Division of
General Internal Medicine, Department of Medicine, Brigham and Women’s
Hospital, Boston, MA  Albert-László Barabási  Million Veteran Program (MVP)
Coordinating Center, VA Boston Healthcare System, Boston, MA, USA  Network
Science Institute, Northeastern University, Boston, USA  Department of
Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, USA

###### Abstract

Biobanks advance biomedical and clinical research by collecting and offering
data and biological samples for numerous studies. However, the impact of these
repositories varies greatly due to differences in their purpose, scope,
governance, and data collected. Here, we computationally identified 2,663
biobanks and their textual mentions in 228,761 scientific articles, 16,210
grants, 15,469 patents, 1,769 clinical trials, and 9,468 public policy
documents, helping characterize the academic communities that utilize and
support them. We found a strong concentration of biobank-related research on a
few diseases, where 20% of publications focus on obesity, Alzheimer’s disease,
breast cancer, and diabetes. Moreover, collaboration, rather than citation
count, shapes the community’s recognition of a biobank. We show that, on
average, 41.1% of articles miss to reference any of the biobank’s reference
papers and 59.6% include a biobank member as a co-author. Using a generalized
linear model, we identified the key factors that contribute to the impact of a
biobank, finding that an impactful biobank tends to be more open to external
researchers, and that quality data—especially linked medical records—as
opposed to large data, correlates with a higher impact in science, innovation,
and disease. The collected data and findings are accessible through an open-
access web application intended to inform strategies to expand access and
maximize the value of these valuable resources.

## Introduction

In 2009, Time magazine listed biobanks among the ten ideas changing the world
[1]. Indeed, biobanks, repositories of biological samples and curated data
pertaining to individuals and disease, are fundamental resources for
biomedical research [2, 3, 4], being supported by both public and private
research centers and funding agencies, as well as the research community at
large [5, 6].

One of the first biobanks, The Framingham Heart Study (FHS), was established
as a cohort study in 1948 to document the health of 5,209 adult residents from
Framingham, Massachusetts, helping define the models still used today for
cardiovascular and heart disease risk prediction [7, 8]. Currently monitoring
the third offspring of the original cohort, FHS is one of the most influential
resources in modern medicine [9, 10]. Equally influential is a relatively new
UK Biobank, founded in 2006 to collect the genetic information, lifestyle,
diet, and medical records of 500,000 adults from the United Kingdom [11]. The
datasets arising from the UK Biobank are widely used to advance the
understanding of the genetic basis of disease, genetic epidemiology, and
public health [12, 13, 14, 15]. Following the release of these results,
biobanks like the Million Veteran Program in the United States [16], or the
Kadoorie Biobank in China [17], and the HUNT biobank in Norway [18, 19], among
others, have been established to link participants’ genetic information to
their electronic health records.

Although much attention has focused on the FHS and the UK Biobank, these
highly visible resources coexist with thousands of biobanks worldwide [20].
Quantifying and understanding the scientific impact of biobanks is a
challenging task, given the great diversity in their goals, usage policies,
and cohort characteristics. As a result, currently, we lack a summary-level
understanding of the breadth and the diversity of biobanks, nor do we
understand how biobanks individually and collectively have impacted our
understanding of human health and disease. Here, we fill this gap by relying
on big data and the tools of Science of Science [21, 22, 23, 24] to identify,
catalog, and analyze the characteristics of 2,663 biobanks, mapping out all
the publications, grants, patents, clinical trials, and public policy
documents where these resources are textually mentioned. We use this dataset
to track the research footprint of each biobank, offering the first
quantitative analysis of biobank usage and impact across multiple dimensions,
including research, innovation, public policy, and disease.

We show that biobank-related research over-represents certain disease areas
like aging and obesity, while under-representing others like immunotherapy or
rare diseases. Moreover, we find that the true impact of biobanks is not
visible to the traditional measures of scientific impact. Indeed, On average,
only 58.8% of papers mentioning a biobank cite its reference articles,
reducing measurable impact. Moreover, we find that multidimensional impact of
biobanks is better tracked through sources like patents, clinical trials, and
policy documents, which often do not correlate with their publication counts.
To measure the true impact of a biobank, we introduce the Biobank Impact
Factor (BIF), tracking a biobank’s impact across research & funding, patent
applications, clinical trials, and public health initiatives. We then develop
a generalized linear model to uncover the features that explain the visibility
and impact of a biobank, finding that biobanks with reputable scientific
leaders, open data access for external researchers, and diverse genetic data
like genome sequencing or gene-environment interactions tend to have a larger
impact.

### The Dataset

Biobanks and cohort studies are closely intertwined, often leading to
interchangeable use. Typically, a cohort study is initiated along an
associated biobank, maintaining the study’s original name. Conversely, some
biobanks are designed to support multiple cohort studies, showcasing their
versatile application in research. Here, we use the term biobank independently
from the process and history responsible to its creation. Estimating the total
number of biobanks worldwide is challenging due to the lack of a
comprehensive, centralized data source. A recent study estimated around 2
biobanks per 1 million people globally, suggesting approximately 660 biobanks
in the USA alone [20]. Other estimates place the global number in the realm of
just a few hundred biobanks [25]. The existing biobank directories are often
limited in scope, covering specific regions or research projects with varying
methodologies [26]. Further complicating biobank tracking is the issue of name
multiplicity, where typing errors or references to secondary collections lead
to inconsistent naming conventions [27]. As well as inconsistencies on what
constitutes a biobank [28]. Despite these challenges, attempts to quantify and
catalog biobanks remain crucial for assessing their overall impact on research
and healthcare.

Based on the definition of a biobank as a “large collection of human
biological material linked to relevant personal and health information” [29,
28], our dataset includes resources that provide physical or digital human
biological data associated with lifestyle, demographic, or health information,
such as cohort studies, cancer registries, and large surveys with biological
data, as well as tissue, blood, and brain banks.

To identify the true corpus of biobanks, we integrated 16 biobank catalogs and
expanded this list by systematically scanning the full scientific literature
for mentions of human biobanks (SI Section 1). We also filtered all documents
(academic publications, patents, grants, clinical trials, and policy
documents) mentioning a biobank in the Dimensions relational database [30].
Finally, we employed natural language processing and co-citation similarity
techniques to remove duplicate biobank entries (SI Section 2).

Through this computational approach, we identified 2,663 unique biobanks that
originated from and were utilized in 139 countries (Figure 1A,B). These
biobanks were mentioned across 228,761 scientific articles, 16,210 grants,
1,769 clinical trials, 15,469 patents, and 9,468 public policy documents
(Figure 1C). Based on these documents, we extracted multiple features related
to the biobank’s cohort composition, data offered, and its overall impact (SI
9 and Table 1).

The full dataset, as well as the code to produce it, are available at
github.com/Barabasi-Lab/quantifying_biobanks and 10.5281/zenodo.11671294.

## Results

### The Disease Focus of Biobanks

The most studied diseases by each biobank reflect not only their focus but
also the research interests of the scientific community using them. To capture
the impact areas of biobanks, we constructed a co-citation network, whose
nodes represent individual biobanks, and connections between nodes occur when
biobanks’ corresponding publications are cited together (Figure 2A, SI 4).
Additionally, we identified the diseases studied by each biobank by analyzing
the medical subject headings (MeSH) from publications mentioning these
resources (SI 7.1). From this analysis, we identified 2,910 unique conditions
across 20 disease categories based on 111,539 research publications. The
network is visibly modular, with each community characterized by the focal
disease category of its biobanks, as extracted from the MeSH classifications.

The disease category “Pathological Conditions, Signs, and Symptoms”, which
covers a broad range of anatomical or physiological states, is the most
represented in our network, accounting for 21.3% of biobanks (540 biobanks).
This prominent category is composed of general-purpose biobanks like the UK
Biobank. The next largest groups are “Nervous System Diseases” (391 biobanks,
15.4%), “Neoplasms” (cancer-related biobanks, 315, 12.5%), and “Infections”
(237, 9.3%). Biobanks focusing on specialized categories like infections or
nervous system diseases typically connect with others within the same
category, indicating a high degree of specialization (SI Figure 6). In
contrast, general-purpose and cancer biobanks often act as bridges across
different biobank communities, exemplified by the nodes corresponding to The
Cancer Genome Atlas and the Nurses Health Study, together having 307
connections.

In general, our analysis shows that a high number of biobanks belong to a few
categories. Specifically, seven out of ten biobanks are classified as either
general-purpose, cancer, nervous system, infections, or cardiovascular disease
biobanks. Combined, these categories account for 85% of all mentions in
disease-related studies and target 58% of all conditions researched. Within
these communities, we observe a strong focus on a few conditions—namely
obesity, Alzheimer’s disease, breast cancer, and diabetes—together being the
primary research interests of 20% of biobank-related publications (Figure 2C).
Despite this high concentration, biobanks demonstrate flexibility in
responding to emerging research needs, captured by the rapid attention focus
on COVID-19, displayed by infectious disease and respiratory tract biobanks.

### Alignment of Funding and Research in Biobank Studies

To analyze disease representation in biobank research, we extracted the NIH
Research, Condition, and Disease Categorization (RCDC) codes of 228,984
publications mentioning biobanks. The RCDC compiles the NIH funding amount for
each of the 315 disease categories included from 2010 to 2022. For each RCDC
code, we identified its number of biobank-related publications during this
time period along with its yearly funding. We limited our analysis to 40
categories, whose average yearly funding exceeds 1 billion US dollars (USD)
and 100 publications.

Our analysis revealed a strong linear correlation between the NIH funding of
RCDC categories and the number of related biobank articles ($r=0.71$,
$p<10^{-8}$, Figure 2D), suggesting that higher funding levels generally lead
to more research output using biobanks. In particular, clinical research, as
the top-funded category with an annual average of 12.8 billion USD, also leads
in the number of biobank studies with 73,715 papers. Women’s and minority
health categories show similar patterns, ranked 12th and 20th in funding, and
10th and 25th in biobank publications, respectively. Conversely, aging is
significantly over-represented in biobank research; it ranks third in
publication volume with 57,030 papers, yet only 15th in funding at 3.5 billion
USD.

To evaluate the disparity between expected and actual research outputs in
biobank studies based on NIH funding for RCDC categories, we applied a linear
fit to the funding data and calculated the residuals for each category (Figure
2E). A positive residual indicates that the actual number of papers exceeded
the expected number based on funding, whereas a negative residual indicates
fewer papers than expected. Notably, nutrition-related research shows the
highest over-representation, with 30,673 actual papers versus 8,209 expected,
an increase of 73%. Other significantly over-represented areas include aging
with an increase of 69%, mental illness at 68%, diabetes at 58%, and
cardiovascular disease at 57%. Across 20 RCDC categories with positive
residual values, the mean over-representation is +41%, translating to an
average surplus of 11,599 publications per category.

The extent of under-representation in biobank papers across 32 RCDC categories
is considerable, with an average shortfall of 7,249 publications per category.
More strikingly, the average percentage shortfall is -296%, indicating that
these categories produce approximately three times fewer papers than expected
based on NIH funding. This significant under-representation may be anticipated
in areas such as biodefense, where the alignment with NIH funding priorities
is less direct (-2163%), or in emerging fields like coronavirus research where
biobank resources may not yet meet the research demand (-368%). Conversely,
other categories seemingly well-suited for biobank research, like rare
diseases (-98%), HIV/AIDS (-499%), stem cell research (-399%), and precision
medicine (-310%), also exhibit notable under-representation.

Altogether, these results indicate that the majority of scientific impact in
biobank research is concentrated in a small number of diseases and research
areas. This trend reflects a historical focus on certain disorders and their
genetic makeup [31]. However, the research diversity has seen notable
improvements in the last decade, aided significantly by new biobanks [32]. Our
findings suggest that funding aligns well with research output for most
diseases, effectively enhancing the representation of underrepresented areas
within the biobank community.

### Exploring the Multifaceted Impact of Biobanks

While the standard measure of impact is citation based, scientific impact is
multifaceted, hence cannot be fully captured by citation-based metrics alone
[33, 34]. This is especially true for biobanks, which often lack standardized
citation credit [35, 36, 27, 37]. To assess the broader impact of biobanks, we
analyzed metadata across 271,394 documents mentioning them. The research
impact of biobanks is evident in the 228,632 publications directly mentioning
them, which collectively received 10.3 million citations. An additional
194,872 publications were funded by grants where biobanks were mentioned,
revealing a different dimension of biobank research impact. These 16,204
biobank-related grants allowed researchers to secure $30.8 billion USD to
support research, personnel training, institutional collaboration, and
researcher career development, contributing to the career development of
scientists.

The indirect impact of biobanks is captured by their reach across grants,
patents, clinical trials, and policies, a metric based on the number of
documents referencing a publication mentioning a biobank. For example, a
biobank’s policy reach equals the number of public policies referencing one of
the articles impacted by the biobank, representing a second-degree measure of
policy impact. Reach is a good predictor of future impact [33], potentially
indicating biobanks’ future impact in each sector.

The indirect impact captured by reach is illustrated for the UK Biobank
(Figure 3A). The UK Biobank has been directly mentioned in 423 grants (direct
impact), a number eclipsed by the 9000 grants that cite one of its related
publications (indirect impact). Similarly, only 14 clinical trials refer to
the UK Biobank directly in their summary (direct impact), but 378 cite one of
its publications (indirect impact), a 26-fold increase in impacted trials. On
average, the indirect impact (reach) of the UK Biobank is 14.8 times larger
than its direct impact (mentions), showcasing its broad influence across
multiple health-related sectors.

More generally, we find reach to be significant across all biobanks, revealing
that at least one biobank-related publication is referenced in 72,817 grants,
23,576 patents, 16,752 policies, and 10,679 clinical trials (Figure 3B). The
reach of biobanks is considerably larger than their direct impact: 5.5 times
larger for patents, 14.2 for policy documents, 6.6 for clinical trials, and 33
in the case of grants. These results confirm the multidimensional aspect of
biobank impact, supporting the claim that a comprehensive metric of biobank
impact should also include health sectors outside of academic research.

### Biobank Impact Factor

Biobank mentions across various document types reveal their multidimensional
impact. Strong correlations exist between mentions in publications and grants
(Pearson correlation $r=0.8$) and between publications and clinical trials
($r=0.5$), highlighting the crucial role of biobanks in research. However,
correlations between publications and public policy ($r=0.1$) and publications
and patents ($r=0.4$) are weaker, suggesting that some biobanks significantly
impact specific domains despite lower publication impact.

To measure the true impact of a biobank, we introduce the Biobank Impact
Factor (BIF, Figure 3C), capturing biobank references in scientific research,
innovation, and public health. To minimize bias towards large biobanks and
cohort studies, we include a metric of disease impact based on existing
indicators of high-value bioresources [27, 38]. Specifically, we account for
the depth and scope of disease research that relies on a biobank, including
its use to advance knowledge of rare diseases (SI 7). BIF is a weighted
average of a biobank’s impact across publications, grants, patents, clinical
trials, public policy, and disease, normalized between -1 and 1 (SI 10). To
minimize bias due to the length of time a biobank has been available, we
normalize its BIF based on the year that the biobank was first mentioned. We
computed the BIF for all 1,326 biobanks in our dataset with at least 20
publications, a cutoff chosen to have enough disease data while retaining at
least half of the biobanks.

Among the biobanks with the highest BIF (Figure 3D), we find the Diabetes
Prevention Program (2nd, BIF $=0.48$), the Women’s Health Initiative (4th, BIF
$=0.35$), the Human Microbiome Project (5th, BIF $=0.32$), the Cancer Genome
Atlas Program (6th, BIF $=0.31$), the Framingham Heart Study (8th, BIF
$=0.28$), and the Genotype-Tissue Expression Project (9th, BIF $=0.25$), all
supported by the National Institutes of Health. The list also includes two UK-
based biobanks: The UK Biobank (1st, BIF $=0.61$) and the European Collection
of Authenticated Cell Cultures (7th, BIF $=0.29$), as well as two other US-
based studies, the National Health and Nutrition Examination Survey (3rd, BIF
$=0.44$), and the Health and Retirement Study (10th, BIF $=0.25$), completing
the top-10 list. To allow easy access to the collected data and metrics, we
developed an online tool to search, explore, and compare the scientific impact
of all biobanks, distinguish the biobank communities and diseases studied,
together with their measures of scientific impact, available as a dashboard at
http://biobanks.pythonanywhere.com/.

### Biobank Impact is National and Institutional

A current survey on biobank use concluded that researchers have a strong
preference for local and familiar sources [39], prompting us to measure the
extent to which biobanks have local versus global impact. We first identify
the host institution of each biobank (SI 5.2) and extract the share of
scientific impact, measured by mentions across publications, coming from the
host institution or the host country. We find that on average 73.5% of the
publication impact comes from researchers in the host country of the biobank,
and 29.4% have the same institutional affiliation (Figure 4 A). We compare
these results to a null model where we randomly rewire the citation network
while preserving the number of citations of each biobank, finding that the
local impact by country and affiliation are statistically highly significant
($p$-value $<10^{-10}$, SI 6).

To compare the impact levels of biobanks, we used the BIF of each biobank,
allowing us to assess the share of mentions coming from the same country and
affiliation for top and bottom biobanks (Figure 4 B). The percentage of same-
country mentions of bottom-20% biobanks, their national impact, is on average
$76.8\%\pm 3.2\%$, a value higher than the top-20% biobanks ($71.1\%\pm
3.1\%$). Even if the impact is mostly national for a biobank, the
international impact of bottom biobanks is 6% weaker than that of top biobanks
($p<0.02$). A finer assessment of local impact is the share of institutional
impact, i.e., the share of mentions coming from research teams from the same
institution of the biobank itself. We find that the average percentage of the
institutional impact of bottom-20% biobanks is $37\%\pm 27\%$, significantly
larger than top-20% biobanks ($20\%\pm 19\%$, $p<10^{-16}$), indicating that
the impact of top biobanks is at the same time less institutional and more
international.

### Access to Biobanks Driven by Co-authorship

Biobanks often restrict scientists’ access to their data, partly driven by
privacy considerations (personal data security is one of the main concerns for
controlling access), and less justifiably so, because maintaining and
supplying the data is costly [36]. Yet, the often lengthy application process
to obtain access to the data is often bypassed via co-authorship with the
biobank team. This practice has a profound effect on the authorship of the
147,656 articles mentioning a biobank for which we identified its supporting
team (SI 5.2). Indeed, we find that, on average, at least one member of the
biobank is a co-author on 59.6% of the articles mentioning the biobank,
usually being the PIs of the biobank (Figure 4 C). This is congruent with
survey data, concluding that co-authorship is a prime incentive for data
sharing from biobanks [40].

On average, members from biobanks with top-20% biobank BIF are listed as co-
authors in 44% of the papers mentioning the biobank, compared to 67% papers
for bottom-20% biobanks (Figure 4 D). This suggests that a lower share of co-
authorship of a biobank may be indicative of the demand for the data. On
average, non-PI members of top biobanks produce 24 external collaborations
without the biobank PIs, a significant percentage (24%, $p<0.036$) compared to
lower impact biobanks (12%, $p=0.06$), where non-PIs collaborate in only 3
papers without the biobank PIs. A significant majority of all the recorded
collaborations take place within the same country (39,496 out of 49,192, or
80%), which may explain the strong national impact of the bottom-20% biobanks,
whose impact is predominantly collaborative. In contrast, only 185 (0.3%) of
all collaborations happen with other PIs from the same institution, suggesting
that most institutional impact comes from the biobank team itself. In other
words, collaborative work increases the recognition and scientific impact of
the biobank creators, with the downside of setting geographical barriers to
their use.

### Citations Underestimate the True Scientific Impact of Biobanks

The reference paper of BioBank Japan [41] has gathered $470$ citations and the
article introducing the UK Biobank [42] has been cited $3,052$ times, raising
the question, does citation-count capture the true scientific impact of
biobanks? We find that not all papers that use biobank data cite the reference
papers published by the biobank. Therefore, to estimate the true scientific
impact of biobanks, we measured hidden citations, which are articles that
mention the biobank in their title, abstract, or acknowledgments sections
without citing the resource article of the biobank.

We find 14,995 articles that cite at least one of the 5 UK biobank reference
papers [42, 11, 43, 44, 45]. Yet, we find an additional 10,123 articles that
mention the UK Biobank after the first reference paper was published. Most
importantly, only 59% of those articles cite one of the biobank’s reference
papers, indicating that 41% of the users do not acknowledge through the
traditional method of impact, i.e. citations, their reliance on the biobank.
To assess the hidden impact of other biobanks, we identified the 962 reference
papers of 500 biobanks and computed their number of hidden citations across
96,745 mentioning articles (Figure 4E and SI 5). We find that, on average,
41.2% of the 203 articles mentioning a biobank do not cite any of their
reference papers, suggesting a general pattern of undercitation. Some of the
most undercited biobanks include the GTEx Project (Figure 4F, 967 hidden
citations, 77.1% of users), the South West Dementia Brain Bank (SWDBB, 85
hidden citations, 78.7%), and the Copenhagen Aging and Midlife Biobank (CAMB,
55 hidden citations, 77.4% of users). Biobanks with a lower number of hidden
citations include the Guangzhou Biobank (28, 14.5%), the China Kadoorie
Biobank (66, 19.5%), and the Generation R Study (108, 10.6%), indicating that,
traditional measures of impact, i.e. citations, highly underestimate the true
academic impact of biobanks.

To obtain a more accurate measure of the scientific reach of biobanks, we
considered all articles either citing their reference papers or the papers
mentioning biobanks, obtaining 120,551 unique papers for the UK Biobank and
3,465 for BioBank Japan, representing more than 7 times their current
detectable citations. On average, we find that the scientific reach of a
biobank is 13 times greater than the number of citations (Fig 4). Note that
these numbers still underestimate the true impact of these biobanks as we have
not scanned the full text of the scientific papers, which may reveal
additional papers that take advantage of biobank resources without citing
them.

### Biobank Features and their Relation to BIF

The reach and the scientific impact of a biobank are driven by multiple
variables, ranging from cohort size (number of individuals sampled) to type
(health-based or population-based), genetic data availability, and the
biobank’s PI level of recognition. Understanding which of these variables
plays a more defining role can help biobank creators identify and implement
the best strategies to increase their reach and scientific impact. To
differentiate the role of those variables, we built a biobank Inherent
Characteristics Model (ICM), based on a generalized linear model with a
Gaussian family designed to explain the BIF of a biobank ($Y$) given its set
of characteristics,

$\log(Y+1)\propto\beta_{0}+\beta_{1}\times X_{1}+\beta_{2}\times
X_{2}+\dots+\beta_{14}\times X_{14}+\epsilon.$ (1)

On the r.h.s we list the 14 features of biobanks that could affect their
impact (SI 9), namely large cohort sample (sample size in the top 10%), open
data index (2 levels, low, and high), the biobank’s PIs citation level at the
time it was created (2 levels, low and high), population-based (1 for
population-based biobanks, 0 for health-system based), available genetic data
(sub-divided in GWAS, Whole-genome sequencing, and gene-environment data),
registries, surveys & questionnaires, follow-up data, and medical records. The
error term $\epsilon$ follows a standard normal distribution.

The model was fitted using data from 468 biobanks ($R^{2}=0.41$), and the
$p$-values of the coefficients were Bonferroni corrected. The deviance of the
model was $0.686$, which, along with a Pearson Chi-square of $0.687$, suggests
a good fit to the data (SI 11). We find five statistically significant
coefficients, capturing the more important characteristics related to BIF
(Figure 5). The largest model coefficient is the one related to a high open-
data index ($\beta_{\text{oa}}=0.0345$, $p=0.002$), followed by whole genome
sequencing data ($\beta_{wg}=0.0286$, $p=2\times 10^{-6}$). It is important to
note that BIF is a normalized metric of impact with a maximum of $0.47$ for
the UK Biobank, so possessing a feature with a coefficient of $0.01$ sets a
biobank in the top half of the BIF distribution.

The open data index is proportional to the number of external requests for the
biobank data, and its significant correlation with BIF demonstrates the
importance of data applications not necessarily leading to co-authorship (SI
9.9). In other words, well-defined data policies, especially those open to new
researchers, can lead to a higher impact with the downside of renouncing
direct recognition to the biobank’s team through collaborations.

Relatedly, we find that having a highly cited founder is significantly
correlated with an increase in BIF ($\beta_{pi}=0.0198$, $p=0.002$). Besides
expertise, the strong reputation of the biobank’s PIs provides an extensive
network of collaborations, along with trustful credentials for potential
users, a factor shown by previous surveys to be a major reason for data
application [39]. On the other hand, we find that having weakly cited PIs at
the time of biobank formation is not significant to BIF ($p=0.18$).

The principal feature of a biobank, however, is the type of data it provides.
Besides whole genome sequencing, gene-environment interaction data, joining
genetic and environmental information, is a type of genetic data strongly
associated with biobank impact ($\beta_{ge}=0.0267$, $p=9\times 10^{-8}$).
Data from genetic markers alone is not significantly correlated to BIF
($p=0.96$), perhaps due to the fact that most biobanks in our list ($74\%$)
include genetic markers. Similarly, GWAS data is not significantly related to
BIF ($p=0.62$) and is included in most biobanks ($52\%$).

The last important feature related to BIF is the inclusion of the medical
records of the cohort sample ($\beta_{mr}=0.0187$, $p=2\times 10^{-5}$),
providing evidence to survey-based findings listing the availability of
medical records as the most cited reason to apply for biobank data [39]. We
find the other data features, to not be significantly related to BIF,
including follow-up data ($p=0.0037$, not significant after Bonferroni
correction), surveys ($p=0.5$), registries ($p=0.06$), and a large cohort
sample ($p=0.426$), suggesting that neither data types are a condition to
biobank impact.

## Conclusions and Discussions

A key asset of a successful biobank is the research community it engages. The
finding that high-impact biobanks have strong ties to their founders’
scientific community, reflected by the high co-authorship rates of the
articles citing the biobank, suggests that a biobank’s long-term impact relies
strongly on the scientific ties of their founding and operating teams to the
core group of scientists who benefit from the collected data. For a new
biobank to succeed, it is therefore paramount for its founding team to be well
integrated within the biomedical community it serves, best achieved by the
lead of an experienced and reputable scientist. Activating the collaboration
network of the founders appears to not only be essential to promote the
biobank but also to extract its potential scientific value by attracting
external collaborators.

The exceptional number of biobanks, and the limited visibility of most of
these valuable resources, raise the question, of how to increase awareness of
the value of the data offered by them. This could be enhanced by a global
biobank locator system, that helps identify the best biobank for a particular
research project. Although several biobank databases exist, they are limited
to particular geographical areas and lack the resource articles with the
biobank [3, 20].

To address this need, we are releasing as an online tool the biobank dataset
we collected together with all the impact metrics included here, allowing
researchers to assess the diseases studied by a biobank community, find
adjacent resources, and explore the scientific impact of each biobank. The
application could allow biobanks attract potential users and find
collaborative opportunities by identifying research teams that belong to the
same scientific community. A list of features includes a summary of the origin
and usage of biobanks, their collaborators, and team characteristics. A
ranking based on the biobank BIF is available, together with different
networks representing biobanks under different connectivity frameworks, e.g.,
based on co-mentions in patents, clinical trials, or grants.

High-throughput genetic data, coupled with statistical tools and increased
computing power, have dramatically increased the size and power of genetic-
based population studies [46, 47]. Not surprisingly, our work shows that whole
genome sequencing and gene-environment interaction data are an important
element of high-impact biobanks. Medical records data that could be used to
explore multiple diseases is another key characteristic. Mega-biobanks with a
large corpus of genetic data covering a wide diversity of diseases are
expected to play a significant role in the future of biobanks. These biobanks,
like the Million Veteran Program (MVP) in the United States [16], can store
genetic information from hundreds of thousands or even millions of
individuals. Having an open approach to data sharing is essential to reach
external researchers, as our results show. Making their data more accessible
to independent studies gives biobanks the potential to help unlock new
insights into the causes of diseases and pave the way for more effective
treatments and personalized medicine.

Indeed, our results indicate that a general-purpose biobank with genetic data
is an optimal resource for high-impact population studies. Note, however, that
scientific impact is not the only measure of success. For instance, a biobank
created to target a rare disease may not be seen as a high-impact biobank by
the traditional measures of academic impact but, in reality, could pave the
way towards finding critical cures, having a profound impact on both
participants and researchers. For this reason, it is important to include
these impact metrics to assess biobank impact. This motivated the proposed
metric of impact, the BIF, as a measure to capture the depth and scope of
disease research produced with biobanks, as well as the impact across areas
like innovation and public policy.

Although our analysis of the factors related to biobank impact considers
multiple data characteristics, it lacks information on processing times, data
quality, and access costs, important factors for potential users [39]. As
biobanks become a central resource for biomedicine and their data policies
become more widespread, these factors should be included in future assessments
of biobank impact.

### Acknowledgments

We thank the wonderful research community at the Center for Complex Network
Research, particularly those in the success and biology groups, for valuable
discussions and comments. We would also like to thank the community of biobank
developers and maintainers, without which this and many other studies would
not be possible.

### Funding

This work was funded by the United States Department of Veteran Affairs.

Table 1: Collected Features of Biobanks Type |  Features
---|---
Cohort |  Size, age group, origin, type (population/health-based)
Data |  Genetic (Whole-genome sequencing, GWAS, or environmental), socioeconomic, follow-up, medical records, surveys, registries
Papers |  MeSH identifiers, RCDC categories, HRCS classification
Patents |  Cooperative Patent Classification codes, assignee
Grants |  NIH activity codes, USD amount, funders
Clinical Trials |  Study type, RCDC categories
Public Policy |  Organization type, country
Figure 1: Biobank origin, use, and mentions. A) The origin of a biobank is based on the nationalities of the biobank’s cohort sample (SI 9). B) The countries using a biobank are based on the affiliation of authors mentioning a biobank in their publications. D) Year distribution of biobank mentions across papers, grants, patents, clinical trials, and public policy documents between 1985 and 2022. Figure 2: The Biobank Disease Universe. A) The biobank co-citation network whose nodes are biobanks that are connected by an edge if their mentioning publications are frequently cited by the same articles. The size of each node is proportional to its number of mentions, each biobank is colored by its principal disease category (SI LABEL:disease=impact). B) Number of biobanks by disease category, the colors are matched with the communities in the co-citation network. C) The main conditions studied by biobanks in each disease category, based on the Medical Subject Headers (MeSH) of the articles citing the biobank. We extracted the Research, Condition, and Disease Categorization (RCDC) classification of biobank publications, along with each RCDC category’s average annual funding by the NIH, to study: D) The relationship between the number of biobank publications and funding per RCDC category. E) Over- and under-represented RCDC categories in biobank publications based on their distance to a linear regression’s fit. Figure 3: Dimensions of the Biobank Impact Factor (BIF). We built a Biobank Impact Factor based on the number of mentions a biobank has across science, innovation, public policy, and the depth and scope of its disease impact, including rare diseases. The number of documents directly impacted (mentions) and indirectly impacted (reach) by A) the UK Biobank and B) all biobanks. C) The formula for obtaining the BIF based on impact, disease scope, and depth. D) The BIF of the top 20 biobanks, impact, disease depth & scope values. Table 2: Overview of grants mentioning biobanks | Total | Biobank Mean (STD)
---|---|---
Grants | 16204 | 14.3 (48.5)
Million USD | 30,863 | 28.3 (115)
Funding Organizations | 298 | 3.4 (5.6)
Funding Countries | 37 | 1.6 (1.5)
Research Organizations | 2,421 | 8.7 (21.3)
Research Countries | 74 | 2 (2.7)
Training grants | 1,359 | 1.3 (6.7)
Collaboration grants | 1106 | 1 (4.6)
Resulting publications | 193,884 | 238.6 (894.4)
Figure 4: Provenance of biobank research impact and hidden citations. A) The
joint distribution of national (same country, purple) and institutional (same
research affiliation, pink) impact of biobanks based on mentioning papers. B)
Mean percentage and 95% confidence intervals of mentions coming from papers in
the same country and institution for bottom-20% (orange) and top-20% biobanks
(blue) based on biobank impact factor. Error bars represent 95% confidence
intervals. C) Joint distribution of the percentage of mentioning papers
listing at least one principal investigator (PI, light yellow) or a team
member (light blue) of the biobank. D) Mean percentage and 95% confidence
intervals of mentions listing a PI or a biobank team member for bottom-20%
(orange) and top-20% (blue) biobanks. E) Distribution of papers mentioning a
biobank but not citing its reference papers (green), mentioning a biobank
(orange), citing its reference papers (purple), or citing its mentioning
papers (biobank reach, pink). Presumably, mentioning articles should include a
reference to one of the reference papers of each biobank so the number of
hidden citations should be small, as they account for papers for which the
biobank is central but fail to cite its main articles. Figure 5: Related
biobank features to biobank impact factor. We present a generalized linear
model to identify the key features explaining biobank impact. The coefficients
of different features are captured by model 1 considering different binary
characteristics of biobanks, including whether the cohort size is large (top
10%), sampled from a general population, data access is open to external
researchers, the average citation count of the biobank PIs (bottom and top-10%
with respect to all PIs), along with availability of genetic data (subdivided
into genetic markers or DNA, GWAS, whole-genome sequencing, or gene-
environment interactions), follow-up data, disease-specific data (based on
registries), surveys & questionnaires, and linked medical records. Each
feature’s coefficient is shown together with its 95% confidence interval.
Significant features after applying Bonferroni correction are indicated with a
star symbol and red color.

## References

  * [1] Alice Park  “10 Ideas Changing the World Right Now - TIME”  In _Time_ , 2009  URL: https://content.time.com/time/specials/packages/article/0,28804,1884779_1884782_1884766,00.html
  * [2] Judita Kinkorová  “Biobanks in the era of personalized medicine: objectives, challenges, and innovation”  In _EPMA Journal_ 7.1, 2016, pp. 4  DOI: 10.1186/s13167-016-0053-7
  * [3] H.-Erich Wichmann et al.  “Comprehensive catalog of European biobanks” Number: 9 Publisher: Nature Publishing Group  In _Nature Biotechnology_ 29.9, 2011, pp. 795–797  DOI: 10.1038/nbt.1958
  * [4] Smadar Shilo, Hagai Rossman and Eran Segal  “Axes of a revolution: challenges and promises of big data in healthcare” Number: 1 Publisher: Nature Publishing Group  In _Nature Medicine_ 26.1, 2020, pp. 29–38  DOI: 10.1038/s41591-019-0727-5
  * [5] Timothy Caulfield et al.  “A review of the key issues associated with the commercialization of biobanks”  In _Journal of Law and the Biosciences_ 1.1, 2014, pp. 94–110  DOI: 10.1093/jlb/lst004
  * [6] Judita Kinkorová and Ondřej Topolčan  “Biobanks in Horizon 2020: sustainability and attractive perspectives”  In _EPMA Journal_ 9.4, 2018, pp. 345–353  DOI: 10.1007/s13167-018-0153-7
  * [7] William B. Kannel et al.  “Factors of Risk in the Development of Coronary Heart Disease—Six-Year Follow-up Experience” Publisher: American College of Physicians  In _Annals of Internal Medicine_ 55.1, 1961, pp. 33–50  DOI: 10.7326/0003-4819-55-1-33
  * [8] Rachel Hajar  “Framingham Contribution to Cardiovascular Disease”  In _Heart Views : The Official Journal of the Gulf Heart Association_ 17.2, 2016, pp. 78–81  DOI: 10.4103/1995-705X.185130
  * [9] Donald M Lloyd-Jones et al.  “Framingham risk score and prediction of lifetime risk for coronary heart disease”  In _The American Journal of Cardiology_ 94.1, 2004, pp. 20–24  DOI: 10.1016/j.amjcard.2004.03.023
  * [10] Syed S Mahmood, Daniel Levy, Ramachandran S Vasan and Thomas J Wang  “The Framingham Heart Study and the epidemiology of cardiovascular disease: a historical perspective”  In _The Lancet_ 383.9921, 2014, pp. 999–1008  DOI: 10.1016/S0140-6736(13)61752-3
  * [11] Clare Bycroft et al.  “The UK Biobank resource with deep phenotyping and genomic data” Publisher: Nature Publishing Group  In _Nature_ 562.7726, 2018, pp. 203–209  DOI: 10.1038/s41586-018-0579-z
  * [12] Ernest Turro et al.  “Whole-genome sequencing of patients with rare diseases in a national health system”  In _Nature_ 583.7814, 2020, pp. 96–102  DOI: 10.1038/s41586-020-2434-2
  * [13] Nicole M. Warrington et al.  “Maternal and fetal genetic effects on birth weight and their relevance to cardio-metabolic risk factors”  In _Nature Genetics_ 51.5, 2019, pp. 804–814  DOI: 10.1038/s41588-019-0403-1
  * [14] Katherine S. Ruth et al.  “Using human genetics to understand the disease impacts of testosterone in men and women”  In _Nature Medicine_ 26.2, 2020, pp. 252–258  DOI: 10.1038/s41591-020-0751-5
  * [15] Gareth J. Griffith et al.  “Collider bias undermines our understanding of COVID-19 disease risk and severity”  In _Nature Communications_ 11.1, 2020, pp. 5749  DOI: 10.1038/s41467-020-19478-2
  * [16] John Michael Gaziano et al.  “Million Veteran Program: A mega-biobank to study genetic influences on health and disease”  In _Journal of Clinical Epidemiology_ 70, 2016, pp. 214–223  DOI: 10.1016/j.jclinepi.2015.09.016
  * [17] Zhengming Chen et al.  “China Kadoorie Biobank of 0.5 million people: survey methods, baseline characteristics and long-term follow-up”  In _International Journal of Epidemiology_ 40.6, 2011, pp. 1652–1666  DOI: 10.1093/ije/dyr120
  * [18] S Krokstad et al.  “Cohort Profile: The HUNT Study, Norway”  In _International Journal of Epidemiology_ 42.4, 2013, pp. 968–977  DOI: 10.1093/ije/dys095
  * [19] Ben M. Brumpton et al.  “The HUNT study: A population-based cohort for genetic research”  In _Cell Genomics_ 2.10, 2022, pp. 100193  DOI: 10.1016/j.xgen.2022.100193
  * [20] Sheila O’Donoghue, Simon Dee, Jennifer A. Byrne and Peter Hamilton Watson  “How Many Health Research Biobanks Are There?” Publisher: Mary Ann Liebert, Inc., publishers  In _Biopreservation and Biobanking_ , 2021  DOI: 10.1089/bio.2021.0063
  * [21] Santo Fortunato et al.  “Science of science” Publisher: American Association for the Advancement of Science  In _Science_ 359.6379, 2018, pp. eaao0185  DOI: 10.1126/science.aao0185
  * [22] Dashun Wang and Albert-László Barabási  “The Science of Science” Google-Books-ID: 3TIiEAAAQBAJ  Cambridge University Press, 2021
  * [23] Jacob G. Foster, Andrey Rzhetsky and James A. Evans  “Tradition and Innovation in Scientists’ Research Strategies” Publisher: SAGE Publications Inc  In _American Sociological Review_ 80.5, 2015, pp. 875–908  DOI: 10.1177/0003122415601618
  * [24] Santo Fortunato  “Community detection in graphs”  In _Physics Reports_ 486.3, 2010, pp. 75–174  DOI: 10.1016/j.physrep.2009.11.002
  * [25] Illumina Inc.  “Population matters: Biobanks accelerate geno–pheno discoveries”, 2020  URL: https://www.nature.com/articles/d42473-020-00238-1
  * [26] Morris Swertz et al.  “Towards an Interoperable Ecosystem of Research Cohort and Real-world Data Catalogues Enabling Multi-center Studies” Publisher: Georg Thieme Verlag KG  In _Yearbook of Medical Informatics_ 31.1, 2022, pp. 262–272  DOI: 10.1055/s-0042-1742522
  * [27] Laurence Mabile et al.  “Quantifying the use of bioresources for promoting their sharing in scientific research”  In _GigaScience_ 2.1, 2013  DOI: 10.1186/2047-217X-2-7
  * [28] Laura Annaratone et al.  “Basic principles of biobanking: from biological samples to precision medicine for patients”  In _Virchows Archiv_ 479.2, 2021, pp. 233–246  DOI: 10.1007/s00428-021-03151-0
  * [29] Margaret Otlowski, Dianne Nicol and Mark Stranger  “Biobanks Information Paper 2010” Publisher: University of Tasmania - Faculty of Law  In _Journal of Law, Information and Science_ 20.1, 2020, pp. 97–227  DOI: 10.3316/ielapa.618168436076064
  * [30] Daniel W. Hook, Simon J. Porter and Christian Herzog  “Dimensions: Building Context for Search and Evaluation”  In _Frontiers in Research Metrics and Analytics_ 3, 2018  URL: https://www.frontiersin.org/articles/10.3389/frma.2018.00023
  * [31] Alexander J. Gates, Deisy Morselli Gysi, Manolis Kellis and Albert-László Barabási  “A wealth of discovery built on the Human Genome Project — by the numbers”  In _Nature_ 590.7845, 2021, pp. 212–215  DOI: 10.1038/d41586-021-00314-6
  * [32] Melinda C. Mills and Charles Rahal  “A scientometric review of genome-wide association studies” Number: 1 Publisher: Nature Publishing Group  In _Communications Biology_ 2.1, 2019, pp. 1–11  DOI: 10.1038/s42003-018-0261-x
  * [33] James W. Weis and Joseph M. Jacobson  “Learning on knowledge graph dynamics provides an early warning of impactful research” Number: 10 Publisher: Nature Publishing Group  In _Nature Biotechnology_ 39.10, 2021, pp. 1300–1307  DOI: 10.1038/s41587-021-00907-6
  * [34] Dag W. Aksnes, Liv Langfeldt and Paul Wouters  “Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories” Publisher: SAGE Publications  In _SAGE Open_ 9.1, 2019, pp. 2158244019829575  DOI: 10.1177/2158244019829575
  * [35] Anne Cambon-Thomsen  “Assessing the impact of biobanks” Bandiera_abtest: a Cg_type: Nature Research Journals Number: 1 Primary_atype: Correspondence Publisher: Nature Publishing Group  In _Nature Genetics_ 34.1, 2003, pp. 25–26  DOI: 10.1038/ng0503-25b
  * [36] Elena Bravo et al.  “Developing a guideline to standardize the citation of bioresources in journal articles (CoBRA)”  In _BMC Medicine_ 13.1, 2015, pp. 33  DOI: 10.1186/s12916-015-0266-y
  * [37] Tushar Vora and Nirav Thacker  “Impacts of a biobank: Bridging the gap in translational cancer medicine”  In _Indian Journal of Medical and Paediatric Oncology_ 36.01, 2015, pp. 17–23  DOI: 10.4103/0971-5851.151773
  * [38] Amanda Rush et al.  “Improving Academic Biobank Value and Sustainability Through an Outputs Focus”  In _Value in Health_ 23.8, 2020, pp. 1072–1078  DOI: 10.1016/j.jval.2020.05.010
  * [39] Emma Lawrence et al.  “The Barriers and Motivators to Using Human Tissues for Research: The Views of UK-Based Biomedical Researchers” Publisher: Mary Ann Liebert, Inc., publishers  In _Biopreservation and Biobanking_ 18.4, 2020, pp. 266–273  DOI: 10.1089/bio.2019.0138
  * [40] Erika Kleiderman, Amy Pack, Pascal Borry and Ma’n Zawati  “The author who wasn’t there? Fairness and attribution in publications following access to population biobanks” Publisher: Public Library of Science  In _PLOS ONE_ 13.3, 2018, pp. e0194997  DOI: 10.1371/journal.pone.0194997
  * [41] Akiko Nagai et al.  “Overview of the BioBank Japan Project: Study design and profile”  In _Journal of Epidemiology_ 27.Supplement_III, 2017, pp. S2–S8  DOI: 10.1016/j.je.2016.12.005
  * [42] Cathie Sudlow et al.  “UK Biobank: An Open Access Resource for Identifying the Causes of a Wide Range of Complex Diseases of Middle and Old Age” Publisher: Public Library of Science  In _PLOS Medicine_ 12.3, 2015, pp. e1001779  DOI: 10.1371/journal.pmed.1001779
  * [43] Anna Fry et al.  “Comparison of Sociodemographic and Health-Related Characteristics of UK Biobank Participants With Those of the General Population”  In _American Journal of Epidemiology_ 186.9, 2017, pp. 1026–1034  DOI: 10.1093/aje/kwx246
  * [44] Thomas J. Littlejohns et al.  “The UK Biobank imaging enhancement of 100,000 participants: rationale, data collection, management and future directions” Publisher: Nature Publishing Group  In _Nature Communications_ 11.1, 2020, pp. 2624  DOI: 10.1038/s41467-020-15948-9
  * [45] Steffen E. Petersen et al.  “Imaging in population science: cardiovascular magnetic resonance in 100,000 participants of UK Biobank - rationale, challenges and approaches”  In _Journal of Cardiovascular Magnetic Resonance_ 15.1, 2013, pp. 46  DOI: 10.1186/1532-429X-15-46
  * [46] Daniel John Lawson et al.  “Is population structure in the genetic biobank era irrelevant, a challenge, or an opportunity?”  In _Human Genetics_ 139.1, 2020, pp. 23–41  DOI: 10.1007/s00439-019-02014-8
  * [47] Jean-Baptiste Pingault et al.  “Using genetic data to strengthen causal inference in observational research” Number: 9 Publisher: Nature Publishing Group  In _Nature Reviews Genetics_ 19.9, 2018, pp. 566–580  DOI: 10.1038/s41576-018-0020-3

## References

  * [48] Gert-Jan B. Ommen et al.  “BBMRI-ERIC as a resource for pharmaceutical and life science industries: the development of biobank-based Expert Centres” Publisher: Nature Publishing Group  In _European Journal of Human Genetics_ 23.7, 2015, pp. 893–900  DOI: 10.1038/ejhg.2014.235
  * [49] Carol A. Giffen et al.  “Providing researchers with online access to NHLBI biospecimen collections: The results of the first six years of the NHLBI BioLINCC program” Publisher: Public Library of Science  In _PLOS ONE_ 12.6, 2017, pp. e0178141  DOI: 10.1371/journal.pone.0178141
  * [50] Niranjan Bose, Anthony J. Brookes, Phil Scordis and Pieter Jelle Visser  “Data and sample sharing as an enabler for large-scale biomarker research and development: The EPND perspective” Publisher: Frontiers  In _Frontiers in Neurology_ 13, 2022  DOI: 10.3389/fneur.2022.1031091
  * [51] Charlene Liggins et al.  “International Alzheimer’s Disease Research Portfolio (IADRP) aims to capture global Alzheimer’s disease research funding” _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1016/j.jalz.2013.12.013  In _Alzheimer’s & Dementia_ 10.3, 2014, pp. 405–408  DOI: 10.1016/j.jalz.2013.12.013
  * [52] K Joeri Velde et al.  “MOLGENIS research: advanced bioinformatics data software for non-bioinformaticians”  In _Bioinformatics_ 35.6, 2019, pp. 1076–1078  DOI: 10.1093/bioinformatics/bty742
  * [53] Sylvie Ouellette and Anne Marie Tassé  “P3G — 10 years of toolbuilding: From the population biobank to the clinic”  In _Applied & Translational Genomics_ 3.2, 2014, pp. 36–40  DOI: 10.1016/j.atg.2014.04.004
  * [54] Zhaoming Wang et al.  “Improved imputation of common and uncommon SNPs with a new reference set” Publisher: Nature Publishing Group  In _Nature Genetics_ 44.1, 2012, pp. 6–7  DOI: 10.1038/ng.1044
  * [55] Sarah Bauermeister et al.  “The Dementias Platform UK (DPUK) Data Portal”  In _European Journal of Epidemiology_ 35.6, 2020, pp. 601–611  DOI: 10.1007/s10654-020-00633-4
  * [56] Martine Vrijheid et al.  “European Birth Cohorts for Environmental Health Research” Publisher: Environmental Health Perspectives  In _Environmental Health Perspectives_ 120.1, 2012, pp. 29–37  DOI: 10.1289/ehp.1103823
  * [57] Amy E. Kennedy et al.  “The Cancer Epidemiology Descriptive Cohort Database: A Tool to Support Population-Based Interdisciplinary Research”  In _Cancer Epidemiology, Biomarkers & Prevention_ 25.10, 2016, pp. 1392–1401  DOI: 10.1158/1055-9965.EPI-16-0412
  * [58] Stephanie A. Smith-Warner et al.  “Methods for Pooling Results of Epidemiologic Studies: The Pooling Project of Prospective Studies of Diet and Cancer”  In _American Journal of Epidemiology_ 163.11, 2006, pp. 1053–1064  DOI: 10.1093/aje/kwj127
  * [59] Anita Bandrowski  “A decade of GigaScience: What can be learned from half a million RRIDs in the scientific literature?”  In _GigaScience_ 11, 2022, pp. giac058  DOI: 10.1093/gigascience/giac058
  * [60] Julie Bergeron et al.  “Fostering population-based cohort data discovery: The Maelstrom Research cataloguing toolkit” Publisher: Public Library of Science  In _PLOS ONE_ 13.7, 2018, pp. e0200926  DOI: 10.1371/journal.pone.0200926
  * [61] Kimberly A. Tryka et al.  “NCBI’s Database of Genotypes and Phenotypes: dbGaP”  In _Nucleic Acids Research_ 42.D1, 2014, pp. D975–D979  DOI: 10.1093/nar/gkt1211
  * [62] Stefanie Lerche et al.  “Methods in Neuroepidemiology Characterization of European Longitudinal Cohort Studies in Parkinson’s Disease - Report of the JPND Working Group BioLoC-PD”  In _Neuroepidemiology_ 45.4, 2015, pp. 282–297  DOI: 10.1159/000439221
  * [63] Elliot Sollis et al.  “The NHGRI-EBI GWAS Catalog: knowledgebase and deposition resource”  In _Nucleic Acids Research_ 51.D1, 2023, pp. D977–D985  DOI: 10.1093/nar/gkac1010
  * [64] Cisca Wijmenga and Alexandra Zhernakova  “The importance of cohort studies in the post-GWAS era” Publisher: Nature Publishing Group  In _Nature Genetics_ 50.3, 2018, pp. 322–328  DOI: 10.1038/s41588-018-0066-3
  * [65] Xiangyi Meng, Onur Varol and Albert-László Barabási  “Hidden Citations Obscure True Impact in Science” arXiv:2310.16181 [physics]  arXiv, 2023  DOI: 10.48550/arXiv.2310.16181
  * [66] Monya Baker  “Building better biobanks” Bandiera_abtest: a Cg_type: Nature Research Journals Number: 7401 Primary_atype: Special Features Publisher: Nature Publishing Group Subject_term: Diseases;Molecular biology Subject_term_id: diseases;molecular-biology  In _Nature_ 486.7401, 2012, pp. 141–146  DOI: 10.1038/486141a

cabanac_2014_eponym marx2009informal mccain2012obliteration

Quantifying the Impact of Biobanks and Cohort Studies

The code presented here can be found at https://github.com/Barabasi-
Lab/quantifying_biobanks. The code and the accompanying data is also stored in
a Zenodo repository at 10.5281/zenodo.11671293. A dashboard is available at
http://biobanks.pythonanywhere.com/.

##  1 Biobank dataset

The process to build a corpus of biobank names was based on two steps. First,
we compiled a set of 16 biobank catalogs (SI 1.1), and second, we extracted
biobank names from the text of 3,310,320 biobank-related articles (SI 1.2).
After each step, we processed the names identified to find and remove
duplicates based on word similarity and co-appearance in articles of each pair
of names (SI 2.1). The resulting corpus of biobanks contains 2,663 unique
biobank names that were then used to gather biobank mentions from textual
documents (SI 8). Finally, the dataset was validated using an external
database of GWAS studies (SI 3).

###  1.1 Biobank Catalogs

Although there is yet no central biobank repository including all biobanks,
there have been several efforts to build catalogs for the scientific
community. The largest to date is the BBMRI-ERIC Directory of Biobanks and
their collections. The resource was established to “improve accessibility and
interoperability between academic and industrial parties to benefit
personalized medicine” [48]. As of 10/2023, the directory included a list of
617 European biobanks together with their 3,176 collections and related
metadata, providing a large set of biobank names.

To complement this corpus, we integrated the biobank names found in 15
prominent directories of population bioresources (the resulting corpus of
biobanks can be found in raw and processed formats in the data folders
‘cohorts’ and ‘raw_cohorts’, respectively). Specifically, this dataset
includes biobanks from BIOLINCC [49], EPND [50], IADRP [51], Molgenis [52],
P3G (Public Population Project in Genomics and Society) [53], DCEG [54], DPUK
[55], UKRI cohort directory, Birthcohorts [56], CEDCD [57], The Pooling
Project of Prospective Studies of Diet and Cancer [58], Wikipedia’s list of
biobanks, SciCrunch [59], Maelstrom [60], dbGAP [61], and JPND [62]. The
resulting list contained 1,576 unique names. Most of these resources were
available for download with the exception of Birthcohorts, IADRP, Pooling
Project of Prospective Studies of Diet and Cancer, and UKRI for which we
crawled the websites (code found at python/crawl).

Furthermore, we extracted the biobank names found in the titles of papers
published in the ‘Cohort Profile series’ of the International Journal of
Epidemiology, and BMJ Open Journal, both having strict guidelines to start the
title with ‘Cohort Profile:’, followed by the biobank name. We identified
these publications using regular expression on the full publication dataset of
Dimensions (files SQL/cohort_profile.sql and SQL/cohort_profile2.sql),
resulting in 488 publications containing 368 biobank names.

Finally, we used ChatGPT to generate a list of biobanks using the prompt
‘produce a list of biobank names along with their country of origin’,
resulting in 738 names. After processing to remove special characters,
trailing spaces, and parenthesis, the resulting list contained 2,207 unique
biobank names (file data/cohorts/final_cohorts.csv). The code to produce the
list is found at python/cohort_names/raw_cohorts.py.

###  1.2 Biobank names expansion

In order to expand our biobank list, we did a bibliography search based on a
set of 3,310,320 biobank-related publications. These publications were
obtained by first identifying 162,988 articles mentioning one of the 2,207
biobanks previously obtained, and then extracting their citing articles. The
resulting set of publications is then either mentioning directly a biobank or
citing a paper mentioning one. The corpus of mentioning publications was
obtained by searching in the title, abstract, and acknowledgments sections of
articles in Dimensions (SQL/mentions/publications.sql). The code to obtain the
set of biobank-related publications is SQL/expansion/papers_citations.sql.

We used regular expressions based on common keywords found in the names of
biobanks and similar repositories, including Biobank, Tissue Bank, Registry,
Biorepository, Project, and 12 other keywords. The complete list of regular
expressions as well as the code to obtain the potential biobank names can be
found at SQL/expansion/clean_expanded_cohorts.sql. The resulting list of names
yielded 23,435 potential biobank names. We filtered potential biobanks in this
corpus of names based on each biobank’s number of mentions in two different
corpora: the set of biobank-related papers (N=3,310,320) and the set of papers
mentioning a biobank from the catalogs (N=162,988). A potential biobank was
removed from consideration if less than 5% of the articles mentioning it
contained a mention to a biobank from the catalogs and less than 50% of its
mentions came from biobank-related publications. The code to obtain papers
mentioning the potential biobanks is in SQL/expansion/expanded_papers.sql (set
of papers mentioning a potential biobank) and
SQL/expansion/expanded_papers_internal.sql (set of mentions from biobank-
related papers). The code to obtain their number of co-mentions with biobanks
from the catalogs is at SQL/expansion/clean_expanded_cohorts.sql and the code
to calculate the percentages of mentions on each corpus is
SQL/expansion/cohorts_counts.sql.

This expansion process resulted in an additional 1,924 biobank names (file
data/expansion/selected_cohorts.csv), resulting in a total of 4,131 biobanks.
The code to clean, preprocess, and obtain this list of biobanks is found at
python/expansions/total_inner_ratio.py.

##  2 Biobank mentions

In order to assess the impact of biobanks we searched their mentions across
multiple textual documents, from the Dimensions academic database we include
scientific publications, clinical trials, grants, and public policy documents.
From Google Patents Public Data we searched for patents mentioning biobanks,
as they provide the full-text of each patent. The search for mentions was case
insensitive and included the name of the biobank preceded by the word ‘the’ to
reduce false positives. The code to search for mentions can be found in
multiple files named after the type of document they were searched on in the
folder SQL/mentions/ for 2,207 catalog biobanks, and SQL/expansion/mentions/
for the expanded list of 1,924 biobanks.

To focus on biobanks with traceable presence in biomedical science, we remove
those without at least one mention in either the title, abstract, or
acknowledgments section of a scientific publication. This included 1,163
biobanks from the catalogs and 1,882 biobanks from the expanded list, for a
total of 3,045 biobanks.

###  2.1 Removing Biobank Duplicates

A common issue with biobanks is the multiplicity of names a single biobank can
accumulate across publications. The reasons behind this issue are multiple:
from name alterations in follow-up studies (e.g., Framingham Heart Study,
Framingham Children’s Study, and Framingham Offspring Study), use of
abbreviations (e.g., ARIC instead of Atherosclerosis Risk in Communities), to
inconsistencies in their naming across publications (e.g., Northern Sweden
Medical Biobank and Medical Biobank of Northern Sweden or Leeds Biobank and
Leeds Multidisciplinary Research Tissue Bank). Additionally, biobanks and
their corresponding cohort studies can be used interchangeably, like in
Lifelines Biobank and Lifelines Cohort Study, similarly to biobanks that are
part of a consortium (e.g., African Neurobiobank for Precision Stroke Medicine
and H3Africa).

In order to minimize the number of name pairs referring to the same biobank,
we performed a deduplication process based on two factors: (i) string
similarity and (ii) co-mentions. The former allows to identify similar names
of a single biobank and the latter to identify duplicates lacking string
similarity such as abbreviations from their full name. String similarity of
each biobank pair $a$ and $b$ was calculated using the Indel distance
$I(a,b)$, measuring the minimal number of insertions and deletions needed to
transform one string into the other, normalized by the maximum number of
possible deletions and insertions ($I(a,b)=1$ implying that $a=b$). We used
two variants of these distance: the partial ratio $I_{pr}$, measuring the
Indel distance needed to match the smallest string into a sub-sequence of the
larger string (here $I_{pr}(a,b)=1$ does not necessarily imply that $a=b$);
and the token set ratio $I_{sr}$, based on the minimal number of word
deletions and insertions needed to match $a$ and $b$ (the word order is
irrelevant). Using Spacy, a natural language model, we ‘cleaned’ the names
prior to their analysis to remove coordinating conjunctions, auxiliaries,
punctuation, symbols, and numerals.

Next, we measured the co-mention similarity of each biobank pair across the
228,761 publications and 19,299 patents mentioning biobanks. The co-mention
similarity $S(a,b)$ between two biobanks $a$ and $b$ is equal to the
proportion of documents mentioning $a$ that also mention $b$ in their text. In
other words, $S(a,b)=1$ implies that $a$ is exclusively mentioned in
publications mentioning $b$, and $S(a,b)=0$ that no publication co-mentions
$a$ and $b$. Note that $S$ is not symmetric: for cases where mention counts of
biobanks $a$ and $b$ differ, we have that $S(a,b)\neq S(b,a)$.

Finally, we identified a pair of biobanks $a$ and $b$ as duplicates if at
least one of the following statements occurred: (i)
min$\\{S(a,b),S(b,a)\\}>0.2$ and $I_{pr}(a,b)>0.9$, (ii)
min$\\{S(a,b),S(b,a)\\}>0.05$ and $I_{pr}(a,b)=1$, (iii)
max$\\{S(a,b),S(b,a)\\}>0.3$ and $I_{pr}(a,b)>0.9$, (iv)
max$\\{S(a,b),S(b,a)\\}>0.1$ and $I_{sr}(a,b)=1$, where co-mentions where
extracted from publications. For patent-based similarity, we only considered
$a$ and $b$ to be duplicates if all three inequalities were true:
max$\\{S(a,b),S(b,a)\\}>0.4$, min$\\{S(a,b),S(b,a)\\}>0.3$, and
$I_{pr}(a,b)=100$, where stricter similarity conditions accounted for a higher
probability of co-mentions in full-text documents.

The deduplication process resulted in 337 duplicate pairs, often forming
duplicated groups where a biobank had 3 or more alternative names (An example
being the Framingham Heart Study or FHS, with 10 variations). To obtain all
duplicate groups, we constructed the duplicate network in which nodes
represent biobank names and edges duplicated pairs, and each connected
component represents name variations of the same biobank. We completed the
deduplication process by selecting the biobank name with the greatest number
of article mentions in each duplicate group to be the name variation to
replace the others.

After deduplication, the final biobank corpus contained 2,263 names mentioned
across 228,761 articles, 16,210 grants, 15,469 patents, 1,769 clinical trials,
and 9,468 public policy documents. The code containing the deduplication
process is found at python/cohort_names/deduplicate.py.

###  2.2 Scientific articles

In order to look for mentions across Dimensions’ publications (from October
2023), we first processed the text from the title, abstract, and
acknowledgments sections to remove Unicode characters, double spaces, commas,
and periods. The code is found at SQL/clean_text/publications.sql. Once the
articles’ text is pre-processed, we look for textual mentions of biobanks
within the title, abstract, and acknowledgment sections of 141,219,539
articles resulting in 250,857 biobank-article mentions on 228,761 articles
(data/expansion/cohort_patents.csv).

###  2.3 Patents

We use the publicly available Google Patents Open Data to search for biobank
mentions on the full text of 153,696,878 patents. The search was done using
Google BigQuery on the table patents-public-data.patents.publications,
resulting in 18,183 biobank-patent mentions on 15,469 unique patents
(data/expansion/cohort_patents.csv. The patents found were then merged to the
patent data from Dimensions using the grant application number id
(SQL/join_dim_google_patents.sql).

###  2.4 Grants

We used grant data from Dimensions, data composed of 5,040,039 grants. We
found 18,251 biobank-grant mentions on 6,210 unique grants
(data/expansion/cohort_grants.csv).

###  2.5 Clinical trials

We used clinical trials from Dimensions, data composed of 801,708 clinical
trials. We retrieved 1,881 biobank-clinical-trial mentions from 1,769 unique
clinical trials (data/expansions/cohort_clinical_trials.csv).

###  2.6 Public Policy Documents

We used the public policy documents from Dimensions, a dataset composed of
1,783,533 public policy documents. We identified 10,285 biobank-public-policy
mentions on 9,468 unique public policy documents
(data/expansions/cohort_public_policy.csv).

##  3 Data Validation

To assess the comprehensiveness of our database of biobanks and related
documents, we used the NHGRI-EBI Catalog of Genome-Wide Association Studies
(GWAS) as a reference to measure the scope of our data (the catalog was
accessed June 2024), compiling more than 100,000 genetic associations to
disease from 6,899 research publications [63]. Indeed, biobanks and population
studies play a major role in the discovery of biomarkers by providing samples
and genetic data used in GWAS. They are not the only contributors, however,
cross-sectional and case-control studies—largely found in clinical trials and
not really considered in our dataset—remain the most used resources for
biomarker discovery [64]. Another potential caveat to consider is our lack of
full-text publications for our search, a limitation not present in the
reference GWAS catalog, directly receiving its data from the authors of the
articles.

To be able to compare the overlap between the two datasets, we used Dimensions
to search the PubMed Ids of our 228,761 biobank-related publications,
successfully doing so for 76.6%, but dropping 1/4 of the biobank-related
papers in the process. We start the overlap assessment with a naive analysis
by directly considering all 6,899 publications from the NHGRI-EBI catalog,
finding that 2,602 GWAS publications, or 38% of the total, are also included
in our publication data. In our dataset, these publications collectively
mention 486 biobanks, the most mentioned by far being the UK Biobank with 748
publications, followed by FHS (169), Wellcome Trust Case Control Consortium
(166), and the Rotterdam Study (156).

To measure our dataset’s breadth in the reported mentions by individual
biobank, we compared them to the reported numbers found in the NHGRI-EBI
Catalog. Specifically, we extracted the number of publications that each
biobank in the catalog is referenced as a data source and compared it to the
number of mentions we find for that biobank across the catalog’s publications.
We calculate these numbers for 89 biobanks having the same name in both
datasets, finding that our dataset identifies, on average, 10.3 more
publication-biobank associations than NHGRI-EBI. The UK Biobank is found in
748 publications in our dataset, but only in 485 publications of the reference
catalog, a difference showing a much higher breadth in our dataset for this
Biobank. The 263 publications mentioning the UK Biobank that do not list this
biobank in the reference catalog have ‘NR’ (230) missing values for the column
‘cohort’ (33), suggesting that authors of the GWAS studies did not include the
biobank source in the NHGRI-EBI Catalog. On average, the 72 biobanks better or
equally covered in our dataset have 14.4 missing publications in the reference
catalog. On the other hand, the 17 biobanks that are better covered in the
NHGRI-EBI Catalog have 7.1 missing publications in our dataset.

Removing publications without a biobank assigned (cohort values ’NR’, ’other’,
or ’NA’), our dataset covers 60% of the 727 remaining publications in the GWAS
catalog. These results show a good coverage of our dataset not without its
limitations: while not all of the GWAS publications not covered (40%) may have
a population biobank as a data source, we miss mentions in the full-text of
the paper. To have an approximate of what the percentage of missing mentions
due to our lack of access to the full-text of a publication, we consider the
485 publications in the GWAS catalog identifying the UK biobank as their
source. From these, 129 are not covered in our dataset, suggesting that 26% of
mentions of the UK Biobank may only be found in the body of the manuscript
(excluding title, abstract, and acknowledgements). The data including the
number of covered and missing publications is data/database/meta/coverage.csv
and the code for the coverage analysis at
python/cohort_metadata/gwas_papers.py.

##  4 Co-citation network

The biobank co-citation network of a biobank is built on top of the
publications mentioning biobanks. The nodes of the network correspond to
biobanks, and two biobanks are connected by an edge if their corresponding
publications are often cited together. We operationalize the citations of a
biobank as the number of articles citing the biobank’s articles where it is
mentioned. Mathematically, let $C_{i}=i_{1},i_{2},\dots i_{n}$ be the citing
articles of biobank $i$, and $C_{j}=j_{1},j_{2},\dots j_{m}$ the citing
articles of biobank $j$, then the union $C_{i}\cap C_{j}$ is composed by the
articles citing both biobanks $i$ and $j$. For each biobank $i$, we only
consider its most similar biobank by adding a single edge to the biobank $j$
such that $|C_{i}\cap C_{j}|=\max_{x}{|C_{i}\cap C_{x}|}$, for each biobank
$x\neq i$. In other words, each biobank $i$ is connected by an edge to the
biobank $j$ that is more often cited together with $i$.

Figure 6: Share of edges between node categories.

##  5 Hidden Citations

Hidden citations are a measure of the hidden impact of a biobank based on its
reference papers [65], and refer to unambiguous allusions made to a biobank
without a corresponding citation. The idea is rooted in eponyms
[cabanac_2014_eponym], informal citations [marx2009informal], and
“obliteration by incorperation” [mccain2012obliteration]. In the context of
biobanks, these allusions appear as implicit citations based on contextual
mentions of a biobank in the text of a scientific paper. A hidden citation to
a biobank is defined as the difference between the explicit citations from
papers mentioning the biobank and its total number of mentions.

###  5.1 Reference papers

In order to compute hidden citations, we first identify the foundational or
reference papers of each biobank from the pool of papers that mention it. We
identify a paper $x$ a reference paper of a biobank if $x$ mentions the
biobank on its title or abstract and either (1) $x$ is the first publication
mentioning the biobank; (2) a paper $y$ references $x$ with probability
$P(x|y=m)>0.2+P(x)$, where $P(x|y=m)$ is the probability that $y$ references
$x$ given that $y$ mentions the biobank and $P(x)$ is the probability that $x$
is cited by any paper; or (3) the title of $x$ contains the words ‘design’,
‘baseline characteristics’, or ‘rationale’. We identify at least one reference
paper for 500 biobanks. The code to compute the missing citations is found at
python/iumpact/recognition/foundational.py.

###  5.2 Biobank Teams

The team of a biobank is composed of the union of all authors listed in one of
its reference papers (Section 5.1). The data can be found at
data/expansion/database/cohort_impact/reference_papers.csv.

##  6 Null model

In order to test the significance of the results showing that the impact of
biobanks is local in terms of affiliation, country, and co-authorship, we
create a null model in which we randomize the citing papers of each biobank
such that the biobank remains with the same number of citations but with a
random set of articles citing it. Once we have a randomized version of the
citation network, where citing papers are randomized, we compute the mean
share of in-house citations (coming from the same affiliation), the share of
same-country citations, and the share of citations containing a collaborator
of the biobank’s team, and the share of citing papers containing its lead-
author. We repeat this process 100 times and then use a z-test comparing the
real value of each measure with its respective sequence of 100 randomized
values to obtain a $p$-value.

##  7 Disease Impact

Based on MeSH terms, we link papers with diseases they have studied. For each
disease, we count the total number of related articles per year for the period
between 2000 and 2013. The number of articles in the sample is 10,486,605,
studying 4,982 diseases. To have an idea of the disease impact of each
biobank, we compute the ratio of publications relative to the total number of
articles published for that disease in each year when the biobank was active.
We limit the analysis to diseases with at least 10 publications per year in
the global dataset. The resulting dataset then contains the proportion of
articles related to each disease where the biobank is mentioned.

The last impact area of our proposed BIF is disease. This metric has three
impact components of disease based on biobank publications: disease scope,
disease depth, and rare disease (SI 7). The disease scope of the biobank is
based on the number of different conditions studied from impacted papers
relative to the total studies published for those diseases since the inception
of the biobank, the UK Biobank leads this metric with 731 conditions studied,
followed by NHANES (617), WHI, and the Telethon Network of Genetic Biobanks
(TNGB, 199). The disease depth is based on the number of studies on a
condition mentioning the biobank, relative to all publications on that
condition, here the Avon Longitudinal Study of Parents and Children (ALSPAC)
leads for environmental illness (14% of all publications in 2020, followed by
the Human Microbiome Project (HMP) for neoplastic processes (9% of all
publications in 2017), and the China National Genebank database (CNGBdb) for
agricultural workers’ diseases (7% of publications in 2021). Finally, rare
disease impact is a measure of the number of biobank publications on rare
diseases, relative to all biobank publications, and is led by the St. Jude
Lifetime Cohort Study (St. Jude LIFE) with 101 papers out of 104 on rare
diseases, followed by the Swiss Childhood Cancer Survivor Study (SCCSS, 96%),
the NCI Childhood Cancer Survivor Study (CSS, 94%), and the Ovarian Cancer
Association Consortium database (OCAC, 88%).

###  7.1 MeSH diseases

We use the definition of diseases from MeSH and operationalize a MeSH term as
a disease if it is under the ’C’ category of the MeSH tree. To compute the
disease impact, we only consider articles published after the year 2000
mentioning a biobank, resulting in a total of 156,330 papers, we identify
75,269 as papers related to a condition (48% of the total of papers mentioning
985 biobanks in the given time period).

###  7.2 UK Clinical Research Collaboration: UKCRC Diseases

The HRCS system is used by a large number of health research funders in the UK
and is subdivided into the Research Activity Classifications (RAC) and Health
Categories (HC).

###  7.3 Research, Condition, and Disease Categorization (RCDC)

This categorization system is used by the NIH to report to Congress and is a
biomedical system consisting of 237 categories, some of which are very
specific in the topic (e.g. “ataxia-telangiectasia”), and others more general
(e.g. “neuroscience”).

###  7.4 Broad disease impact

For each biobank, we define its broad disease impact as the sum across
diseases studied with the biobank data of the proportion of papers mentioning
one of the 985 biobanks each year.

###  7.5 Specific disease impact

To account for the impact of biobanks focusing on a single disease, we define
the specific disease impact as the highest proportion of articles published
mentioning the biobank for a given disease relative to the total number of
articles published for that disease during the active years of the biobank.
Here, we limit our study to diseases in the second classification level of
MeSH with at least 100 publications to account only for diseases with an
established interest from the scientific community, resulting in 180 specific
diseases related to 632 biobanks.

##  8 Mentions data

For each document mentioning a biobank, we collect the relevant metadata to
implement each of the impact measures developed.

###  8.1 Grants data

For each grant, we extract its NIH activity code, the funder, funder amount,
and disease categories (based on MeSH, RCDC, and HRCS). The NIH activity codes
are then used to identify training, collaboration, and prestigious grants.

###  8.2 Patents data

For each patent, we identify its Cooperative Patent Classification (CPC) and
its disease categories.

###  8.3 Clinical trials data

From each clinical trial, we extract its disease categories, as well as its
assignee, and its institutional type.

###  8.4 Public policies data

For each public policy document, we extract its country of publication based
on the affiliation of the institution implementing the policy, and the type of
its implementing institution.

##  9 Biobank data

Biobanks differ in the scope and kind of data they offer. A Biobank may focus
on a particular physical sample, e.g., brain tissue, while another may offer
multiple sample types and questionnaire information from its participants.
Genetic biomarkers, those gene variants related to higher risk or diagnosis of
a disease, are among the most coveted goals of cohort studies and require DNA
data from biobanks. The kind of genetic data, however, can define the
methodology used by the study, however, and is therefore highly relevant.
Similarly, medical records can be used to trace risk factors to multiple
diseases and conditions suffered by the biobank’s participants, complementing
its data. Here, we assess the type of data available from the biobank across
several factors including, genetic data type, environmental data, follow-up
data, medical health records, and disease-specific registry data. In order to
identify the type of data available from the biobank, we looked at the MeSH
classification terms of the articles mentioning the biobank. If multiple
biobank papers are Genome-Wide Association Studies (GWAS), it is highly likely
the biobank contains GWAS data available. Here we present the different MeSH
terms we used to associate each data type. The complete set of MeSH terms and
tree numbers used can be found at data/mesh_and_meta/mesh_terms.md.

###  9.1 Genetic data

In order to assess whether a biobank has genetic data available, we looked for
the following MeSH terms in the articles mentioning it: Genome-wide
Association Study, Whole Genome Sequencing, Genetic Association Studies,
Genetic Techniques, as well as any MeSH term under the tree classification of
Genetic Predisposition to Disease (MeSH tree number C23.550.291.906). We then
sub-classified the DNA data on GWAS data and Whole Genome Sequencing. The code
can be found at python/expansion/data/dna.py.

###  9.2 Environment data

We used the sub-branches of each of the following MeSH terms to identify
papers with environmental data: Gene-environment Interaction, Environmental
Exposure, Environmental Biomarkers, Environmental Indicators, Environment, and
Environment Design. The code can be found at
python/expansion/data/environment.py.

###  9.3 Follow-up data

Often, a biobank’s cohort is followed through time, and samples and data are
collected on each follow-up. To identify follow-up studies of a biobank, we
used the MeSH term ‘Follow-Up Studies’ (D005500). The code can be found at
python/expansion /data/follow_up.py.

###  9.4 Medical records

We used the MeSH terms under the tree classification of ‘Medical Records’
(MeSH tree ID E05.318.308.940.968). The code can be found at python/expansion
/data/medical_records.py.

###  9.5 Registries and disease-specific data

In order to look for registries and other types of disease-specific data such
as the one coming from panels, we used the following MeSH terms: Registries
(E05.318.308.970), National Program of Cancer Registries
(I01.409.418.750.600.650.200.760), and Mass Screening (E01.370.500). The code
can be found at python/expansion /data/registries _disease_specific.py.

###  9.6 Surveys and Questionnaires

Some studies rely on data from surveys and questionnaires to build a social
profile of the participants of the biobank. To identify those biobanks with
survey and questionnaire data we identify the papers under the mesh term
Surveys and Questionnaires (E05.318.308.980). The code can be found at
surveys_and_questionnaires.py.

###  9.7 Age groups

Most biobank cohorts are designed with an age group in mind, usually defined
by the first wave of data collected from its participants. In order to
identify the age group of a biobank, we identified all the age groups under
the MeSH classification of Age Groups (M01.060) and classified it into 7
categories depending on the tree number of the age group, namely: Middle Aged
(M01.060.116.630), aged (M01.060.116.100), young adult (M01.060.116.815),
adolescent (M01.060.057), infant (M01.060.703), child (M01.060.406), and birth
(M01.060.261). The code can be found at persons.py.

###  9.8 Country of origin

We collected the country of origin of the cohort from each biobank by
identifying the MeSH terms related to countries in their publications. This
includes every term under the MeSH tree classification of Geographic Locations
(Z01), except for cities (Z01.433). We also consider the countries where the
authors of the papers are affiliated based on their institution on each paper.
The code can be found at countries.py.

###  9.9 Open data index

In order to assess the level of _data openness_ of a biobank, we consider
three factors related to the papers mentioning the biobank: the percentage of
papers led by the top corresponding author, the percentage of papers co-
authored by the top 10 authors, and the percentage of papers with at least one
author affiliated to the top affiliation. We then define the ‘open data index’
of a biobank as 100 minus the mean of these percentages.

###  9.10 Cohort size

The cohort size of a biobank, i.e., the number of participants or individuals
included for sampling, surveys, and/or follow-up, may be of interest to
researchers looking to access its data. Although Biobanks usually publish the
size of their cohort on their design or foundational papers, it may be subject
to change in subsequent updates of the biobank, or it may be a target number.
Oftentimes, moreover, researchers may be interested in a sub-sample of the
cohort depending on multiple factors such as the disease of interest, the
analytical method of their study, or the type of data they need. In order to
have an idea of the cohort size that is used across the research papers that
make use of a biobank’s data, we look for the self-disclosed sample sizes
appearing in the abstracts of such studies. This approach makes use of papers
mentioning a biobank, resulting in a distribution of sample sizes used across
time and topics. In order to identify sample sizes we use a regular expression
matching numbers followed by a keyword (code available at
SQL/expansion/cohort_size.sql), and to filter out those papers that may be
using more than one biobank, we only consider articles mentioning one biobank
in our sample. For each biobank, we then consider the 90th percentile as its
cohort size to consider the larger sub-samples of the biobank and to avoid
outliers that may represent noisy values. The code to create the table with
the sample size of each biobank is found at python/data/cohort_size.py.

###  9.11 Population vs. health based biobanks

Biobank data is retrieved from a cohort of participants that may be recruited
from a population defined by a geographical location, a line of work, or any
other common factor, or it can be based on patients (dead or alive) coming
from a hospital or clinic and usually sharing a disease [66]. The border
between population and patients, however, is not always clear. This is the
case when biobanks recruit individuals from a general population that are
patients are local clinics, like in the UK Biobank. Indeed, often participants
are invited to participate in the Biobank’s cohort from a clinic or a
hospital, not because they are patients there, but because it is strategic or
convenient (e.g., to link their medical records to their samples).

To assess whether a biobank’s design had a population or a group of patients
in mind we identified several keywords from the abstracts of the papers
mentioning the biobank, including ‘population-based’, ‘patients’, ‘hospital’,
‘health-based’, and ‘clinic’. To classify the cohort type of the biobank, we
then computed the ratio of papers with each keyword and assigned the type with
the highest proportion of papers. Under this approach, the UK Biobank, even if
around 10% of its papers contain the word ‘patients’, is classified as a
population-based biobank as 29% of its articles contain the keyword
‘population’. The code can be found at
python/expansion/database/cohort_type.py.

##  10 Biobank Impact Factor

We considered two factors to calculate the Biobank Impact Factor (BIF):
mentions and disease impact. To calculate the mention impact of a biobank, we
standardize its number of mentions for each document type separately (i.e.,
publications, grants, patents, clinical trials, and public policy documents).
For each document type, standardization is done by taking the number of
mentions $m$ of a biobank and returning $(m-\mu)/\sigma$, where $\mu$ and
$\sigma$ is the mean and standard deviation over all biobanks. Once every
value is standardized, we sum the values of each biobank across all document
types, this is the mention impact. To consider the advantage in mentions of
older biobanks, where the number of mentions may be related to the time period
they have been available, we normalize mention impact by the number of years
the biobank have been mentioned in scientific publications.

This mention impact takes into account the overall presence of biobanks across
science, innovation, and policy, as well as their lack thereof. Because
mentions are standardized by document type, a biobank can have a negative
value that results in a ‘penalty’ for its impact. The BIF scores higher
balanced biobanks, those having an impact, even if modest, across most
sectors. The code to compute mention impact is found in
python/impact/cohort_impact_factor/impact_factor.py.

Another factor that we consider for BIF is the scope and depth of research
produced by each biobank (SI 7). This metric considers the number of
publications each biobank helps produce for a disease, compared to the total
number of publications about the disease (depth); as well as the total number
of diseases studied on publications mentioning the biobank (scope).
Additionally, disease impact considers the impact the biobank has on rare
diseases, measured by the number of mentioning publications on a rare disease,
relative to the all other biobanks. This ensures that biobanks contributing to
a single disease, and those contributing to rare diseases, are considered in
the BIF. The code to compute disease impact is found in
python/impact/cohort_impact_factor/disease_impact.py.

The code to compute the BIF is found in
python/impact/cohort_impact_factor/target.py.

##  11 Generalized Linear Model

We used a Generalized Linear Model (GLM) to measure the relation between
biobank data characteristics and BIF. The dependent variable is then BIF and
the 14 independent variables were all binary values depending on the data
characteristics of each biobank (Table LABEL:SItable:model). Specifically, we
used a Gaussian family GLM with identity function linked to the data, using
the statsmodels library in Python. The model was applied to a set of 468
biobanks for which we could identify each feature (SI 9). For each dependent
feature we obtained its coefficient and associated $p$-value, labeling it as
significant if $p$-value $0.05/n$, where $n$ is the number of features in the
model (Bonferroni correction for each independent feature). The model presents
a an $R$-square metric of 0.4 and log-likelihood of 862.65. The code to fit
the data to the GLM and to obtain the significant variables is at
python/stat_models/glm.py.

Dep. Variable: | target | No. Observations: | 468
---|---|---|---
Model: | GLM | Df Residuals: | 453
Model Family: | Gaussian | Df Model: | 14
Link Function: | Identity | Scale: | 0.0015157
Method: | IRLS | Log-Likelihood: | 862.65
Date: | Mon, 15 Apr 2024 | Deviance: | 0.68663
Time: | 10:02:51 | Pearson chi2: | 0.687
No. Iterations: | 3 | Pseudo R-squ. (CS): | 0.4097
Covariance Type: | nonrobust |  |
| coef | std err | z | P$>|$z$|$ | [0.025 | 0.975]
---|---|---|---|---|---|---
const | -0.0186 | 0.009 | -2.102 | 0.036 | -0.036 | -0.001
dna | 0.0002 | 0.005 | 0.039 | 0.969 | -0.010 | 0.011
gwas | 0.0024 | 0.005 | 0.489 | 0.625 | -0.007 | 0.012
whole_genome | 0.0286 | 0.006 | 4.752 | 0.000 | 0.017 | 0.040
gene_env | 0.0267 | 0.005 | 5.346 | 0.000 | 0.017 | 0.037
follow_up | 0.0144 | 0.005 | 2.900 | 0.004 | 0.005 | 0.024
medical_records | 0.0187 | 0.004 | 4.210 | 0.000 | 0.010 | 0.027
surveys | -0.0049 | 0.008 | -0.608 | 0.543 | -0.021 | 0.011
registries | 0.0077 | 0.004 | 1.857 | 0.063 | -0.000 | 0.016
population | -0.0010 | 0.005 | -0.221 | 0.825 | -0.010 | 0.008
large_cohort | 0.0048 | 0.006 | 0.796 | 0.426 | -0.007 | 0.017
open_data_high | 0.0345 | 0.007 | 5.196 | 0.000 | 0.021 | 0.048
open_data_low | -0.0051 | 0.008 | -0.618 | 0.537 | -0.021 | 0.011
PI_high_impact | 0.0198 | 0.006 | 3.079 | 0.002 | 0.007 | 0.032
PI_low_impact | -0.0092 | 0.007 | -1.336 | 0.182 | -0.023 | 0.004