# Assessing Your Observatory’s Impact:
Best Practices in Establishing and Maintaining Observatory Bibliographies

Raffaele D’Abrusco Chandra Data Archive, Chandra X-ray Center (CXC) / Center
for Astrophysics, Harvard & Smithsonian, 60 Garden Street, Cambridge, MA
02138, USA  Monique Gomez Instituto de Astrofísica de Canarias (IAC), Vía
Láctea, s/n E-38205, La Laguna - Tenerife, Spain  Uta Grothkopf European
Southern Observatory (ESO), Karl-Schwarzschild-Straße 2, 85748 Garching bei
München, Germany  Sharon Hunt NSF’s NOIRLab, 950 N Cherry Ave, Tucson, AZ
85719, USA  Ruth Kneale  National Solar Observatory (NSO)/DKIST, retired, 3665
Discovery Drive, Boulder, CO 80303, USA  Mika Konuma  National Astronomical
Observatory of Japan (NAOJ), 2-chōme-21-1 Ōsawa, Mitaka, Tokyo 181-8588, Japan
Jenny Novacescu Space Telescope Science Institute (STScI), 3700 San Martin
Dr., Baltimore, MD 21218, USA  Luisa Rebull Infrared Science Archive (IRSA),
IPAC, MS 100-22, Caltech, 1200 E. California Blvd, Pasadena, CA 91125, USA
Elena Scire IPAC, Mail Code 314-6, Caltech, 1200 E. California Blvd. Pasadena,
CA 91125, USA  Erin Scott Chandra Data Archive, Chandra X-ray Center (CXC) /
Center for Astrophysics, Harvard & Smithsonian, 60 Garden Street, Cambridge,
MA 02138, USA  Donna Thompson ADS / Center for Astrophysics, Harvard &
Smithsonian, 60 Garden Street, Cambridge, MA 02138, USA  Lance Utley  National
Radio Astronomy Observatory (NRAO), 520 Edgemont Road Charlottesville, VA
22903-2475, USA  Christopher Wilkinson  Space Telescope Science Institute
(STScI), 3700 San Martin Dr., Baltimore, MD 21218, USA  Sherry Winkelman
Chandra Data Archive, Chandra X-ray Center (CXC) / Center for Astrophysics,
Harvard & Smithsonian, 60 Garden Street, Cambridge, MA 02138, USA Sharon Hunt
<EMAIL_ADDRESS>
###### Abstract

Observatories need to measure and evaluate the scientific output and overall
impact of their facilities. An observatory bibliography consists of the papers
published using that observatory’s data, typically gathered by searching the
major journals for relevant keywords. Recently, the volume of literature and
methods by which the publications pool is evaluated have increased. Efficient
and standardized procedures are necessary to assign meaningful metadata,
enable user-friendly retrieval, and provide the opportunity to derive reports,
statistics, and visualizations to impart a deeper understanding of the
research output.

In 2021, a group of observatory bibliographers from around the world convened
online to continue the discussions presented in Lagerstrom (2015). We worked
to extract general guidelines from our experiences, techniques, and lessons
learnt. This paper explores the development, application, and current status
of telescope bibliographies and future trends. The paper briefly describes the
methodologies employed in constructing the databases, along with the various
bibliometric techniques used to analyze and interpret them. We explain reasons
for non-standardization and why it is essential for each observatory to
identify metadata and metrics that are meaningful for them; caution the
(over-)use of comparisons among various facilities that are, ultimately, not
comparable through bibliometrics; and highlight the benefits of telescope
bibliographies, both for researchers within the astronomical community and for
stakeholders beyond the specific observatories. There is tremendous diversity
in the ways bibliographers track publications and maintain databases, due to
parameters such as resources (personnel, time, budget, IT capabilities), type
of observatory, historical practices, and reporting requirements to funders
and outside agencies. However, there are also common sets of Best Practices.
This paper describes some of our results from our collaborative discussions.

Astronomy Databases (83) — Astronomical reference materials (90) —
Observatories (1147) — Telescopes (1689)

##  1 Introduction

It has become increasingly important for astronomical observatories to measure
and evaluate the scientific output, overall impact, and general success of
their facilities. Many methods exist to do so. A commonly used one is to
create databases with descriptions of the scientific papers that have been
published using data from the observatory. The resulting compilations are
typically referred to as telescope or observatory bibliographies and the
technique of interpreting and analyzing them as bibliometrics. In most
observatories, librarians and archive specialists manage these information
resources. They also play a defining role in the further development of the
bibliographies to ensure their continued value for the specific observatory
including its management, governing bodies, and project and instrument
scientists, as well as the wider astronomy community.

Observatory bibliographies may include telescope data publications, data
products publications, staff publications, mission bibliographies, archival
products bibliographies, etc. They are typically compiled by scanning the
major journals for scientific papers that use and analyze data generated by
the respective observatories. During recent years, there has been a constant
increase in not only the volume of astronomical literature (Chen et al.,
2022a) but also the number and methods by which the pool of publications is
evaluated. Efficient and standardized procedures are necessary to assign rich
and meaningful metadata, enable user-friendly retrieval, and provide the
opportunity to derive reports, statistics, and visualizations that enable a
deeper understanding of the research output from various perspectives.

Astronomy has a long-standing history of sharing and exchanging papers, data,
and software. Despite (or possibly because of) the large geographical
distances between observatories, collaboration has always been a vital element
to stay informed about recent developments and be able to build on the results
of peer researchers. This is equally true for astronomy librarians with their
series of meetings, such as the Library and Information Services in Astronomy
(LISA) conferences111http://www.eso.org/sci/libraries/lisa.html, and other
formal and informal ways of exchanging best practices in a fair, transparent,
and collaborative way. It is therefore not surprising that astronomy
librarians, data scientists, and archive specialists have been discussing
observatory bibliographies for many years. Many papers describing individual
observatories’ bibliographies have been published (see Appendix B), along with
articles about Best Practices (Lagerstrom, 2015; Chen et al., 2022b).

In 2021, a group of observatory bibliographers from around the world convened
online to continue the discussions and efforts presented in Lagerstrom (2015).
Many of the institutions represented in that paper are the same, although
individuals have changed, and representatives from additional institutions
were added to broaden the astronomy disciplines managing observatory
bibliographies. We formed the Observatory Bibliographers Collaboration (OBC),
with the goals of sharing transparent and fair bibliography methods and
standards and creating shared documentation and processes. The group has
worked to extract generally applicable guidelines from the experiences,
techniques, and lessons learnt of the various observatories.

Over the course of our discussions, a few key actions emerged that shape the
objectives of this paper:

  * •

cover all aspects of observatory bibliographies to identify commonalities and
to present the creative ways in which observatory bibliographers have met
their challenges despite limited resources

  * •

provide some lessons learnt with a focus on the biggest and most serious
errors over time and how they were addressed

  * •

provide updated standards/recommendations, recognizing that as science
changes, so must observatory bibliographies

The best practices that we outline in this paper are intended to serve as
guidelines to both new and experienced bibliographers. We hope that we can
provide guidance to those needing to create new observatory bibliographies
because, as we discovered, a quick-and-dirty approach can lead to incomplete
or wrong conclusions and may ultimately be worse than no bibliography at all.
For those individuals tasked with maintaining observatory bibliographies who
are not credentialed librarians, the descriptions and recommendations
contained in this paper will fill gaps in their knowledge of library
practices. In addition, those individuals who use and assess the metrics
generated from observatory bibliographies can gain an understanding of how
these bibliographies are constructed and their constraints.

In this paper, we explore the development, application, and current status of
observatory bibliographies and the direction in which they are moving. We
briefly describe the methodologies employed in constructing the
bibliographies, along with the various bibliometric techniques used to analyze
and interpret the papers collected within these bibliographies. We explain the
reasons for non-standardization, i.e., why it is essential for each
observatory to identify the metadata and metrics that are meaningful for them;
caution on the (over-)use of comparisons among various facilities that are,
ultimately, not comparable through bibliometrics; and highlight the benefits
of observatory bibliographies, both for researchers within the astronomical
community and for stakeholders beyond the specific observatories. The paper
concludes with a look at essential skills for observatory bibliographers, as
well as a glimpse towards future trends and technologies that will influence
the creation, maintenance, and further development of observatory
bibliographies.

##  2 Identifying Candidate Publications

Most bibliographers rely on automated and/or semi-manual keyword matching
against the full-text files of publications provided by publishers to begin
the bibliography process. Bibliographers then scan the literature visually
and/or aided by text-mining software to identify candidate publications for
inclusion in their observatory bibliography.

###  2.1 SAO/NASA Astrophysics Data System (ADS)

The SAO/NASA Astrophysics Data System (ADS) is a digital index operated by the
Smithsonian Astrophysical Observatory (SAO) under NASA Cooperative Agreement
80NSSC21M0056 (Eichhorn, 2004; Chyla et al., 2015). ADS currently contains
more than 15 million records that cover the NASA Science Mission Directorate
(SMD) disciplines of astrophysics, planetary science, heliophysics, and earth
science. ADS will soon become the NASA Science Explorer (SciX), and any new
bibliographic projects should be started on that platform when released. NASA-
funded research in biological and physical sciences will also be added to
SciX. Currently, the records are grouped among three primary collections:
astronomy and astrophysics, physics, and general science. Abstracts and full
text of major astronomy and physics publications are indexed and searchable
within ADS, and arXiv e-prints are ingested daily as well. The ADS website
offers help pages as well as a blog and news updates.

Observatory bibliographers use the ADS database and its functionalities
extensively to identify and track observatory publications. They can query the
ADS database using either the ADS web submission search forms or the system’s
application programming interface (API) to identify publications to be
assessed for inclusion in observatory bibliographies. Users can search in a
variety of fields, including full text, author, affiliation, keyword,
publication name or abbreviation, and date. It is also possible to string
together multiple search terms to develop a query. Users can create libraries
of ADS records for specific telescopes, data products, missions, and other
subjects. These libraries can be kept private, shared with specific
collaborators, or made public for viewing by any ADS user. With assistance
from ADS curators, many institutions have created publicly shared ADS
libraries which can be set as a unique bibliographic group in ADS such as
bibgroup:chandra and bibgroup:GALEX. Updates to these libraries can be set
automatically or done manually on a quarterly, annual, or other regular
cadence.

###  2.2 CrossRef

CrossRef222https://www.crossref.org is a non-profit organization and
aggregator for publisher information. More than 6,000 societies and publishers
deposit metadata about their journals, ebooks, conference proceedings, and
other publications through CrossRef. With the appropriate licenses and a no-
cost CrossRef agreement in place, an observatory can mine the full-text files
for a set of journals, similar to full-text field searching in ADS. CrossRef
offers its own RestAPI333https://www.crossref.org/documentation/retrieve-
metadata/rest-apiandextensivedocumentation.

###  2.3 Keyword Searching

To comprehensively search through thousands of articles per year,
bibliographers must consider all common forms of keywords, telescope names,
instruments, high-profile programs, etc., and search acronyms, former names,
accepted colloquial forms, and community-accepted spelling and punctuation
variations in addition to the official name form.

Comprehensive search and discovery using keywords are complicated by simple
typos or misspellings not caught during the editorial process and differences
in hyphenation and punctuation. While ADS does an excellent job of mapping
synonymous terms and name forms in the background, they cannot be held
responsible for identifying every variation, including newly created ones. As
an example, the Two Micron All Sky Survey (2MASS) can also be found written as
Two-Micron All Sky Survey, Two Micron All-Sky Survey, 2MASS, 2-MASS, and 2
MASS. Another example is the National Solar Observatory’s (NSO) Daniel K.
Inouye Solar Telescope observatory, DKIST. It was previously called the
Advanced Technology Solar Telescope (ATST), and in its beginning, the acronym
didn’t include the first T, so was just AST. It’s also referred to as Inouye,
Inouye Solar Telescope, and DKI Solar Telescope. There are published papers
using every one of these variations.

For the Chandra X-ray Observatory, the keyword set includes Chandra, CXO, and
AXAF and their variants. The European Southern Observatory (ESO) searches over
200 keywords; many of them are acronyms as well as long versions of facility
and telescope names. Often acronyms reveal unrelated homonyms. An overview of
the experience gained through more than 20 years of the ESO telescope
bibliography are described in Grothkopf et al. (2018).

Common acronyms are difficult to manage and add to the time needed to assess
papers (Scire et al., 2022). In the case of HST and JWST respectively, the
acronyms for the Cosmic Origins Spectrograph (COS) and the Near Infrared
Spectrograph (NIRSpec) were removed entirely from bibliography search strings
due to the number of false hits. NIRSPEC (typically in capitalized form) is
also used to identify an instrument on the Very Large Telescope (VLT) managed
by ESO and one of the telescopes at the W. M. Keck Observatory. Some
mitigation strategies to deal with common keywords and acronyms in the
literature include keyword pairs, such acronym 1 AND (keyword 1 OR keyword 2),
e.g., NIRSpec AND (“James Webb Space Telescope” OR JWST), written
parenthetically in ADS. Some acronyms can also be limited to certain journals.
For example, a search can be created to ignore instances of the acronym ACS in
SPIE Proceedings, where ACS can refer to multiple concepts across sciences
rather than its intended meaning – Advanced Camera for Surveys – more commonly
found in astronomy journals.

NSF’s NOIRLab keywords include instruments, telescopes, surveys, the archive,
and data products such as the Astro Data Lab. Searching for the Blanco and
Mayall telescopes is complex, since individuals use a variety of names to
refer to these long-lived telescopes. For example, the Victor R. Blanco
Telescope is referred to as the Blanco, the CTIO 4m, the 4m telescope at CTIO,
the Cerro Tololo 4-meter/4m/4 m/4-m. In addition, the name Blanco appears
often in reference lists and in star names. Similarly, the Legacy programs
SIMPLE, GOALS, Taurus, Cygnus-X, which use large volumes of Spitzer Space
Observatory data, were removed from the search terms (Scire et al., 2010) due
to the common use of these words in the literature.

For Spitzer, the keyword set included the short name of the observatory
(Spitzer) and acronyms for the instruments (IRAC, MIPS, IRS) and also acronyms
for the Legacy, Exploration Science and Frontier Legacy programs. Spitzer has
since been handed over to the Infrared Science Archive (IRSA), which searches
on a subset of these terms. Searching by instrument names and acronyms in
addition to telescope and observatory names is necessary because some papers
may only cite the instrument without mentioning the observatory itself. For
IRSA, it is not possible to track all papers using all datasets that have
varying names and acronyms. Because of limited resources, only papers for
certain data sets are tracked, typically the most popular such as WISE,
Planck, 2MASS, and IRAS. Accounting for all of the papers that emerge from
diverse data sets can be challenging, particularly when the publication does
not specify the name of the observatory or even the instruments used to obtain
the data.

###  2.4 Alternatives to Keyword Searching and Limitations

One might think that an alternative and more accurate search could be
accomplished by looking for citations of key publications. The 2MASS project
requests that people cite Skrutskie et al. (2006), the journal article
describing 2MASS, but many people cite instead the delivery document, Cutri et
al. (2003). For Spitzer, only 62% of papers that use data from the observatory
cited one of the key publications about the observatory or used the
acknowledgment statement correctly (see Scire et al., 2022, fig. 2).

Searching by standard acknowledgement phrases doesn’t always work either, even
when authors remember to include one. For example, the 2MASS project has been
around for nearly two decades and has been fundamental to so many papers; the
words in the requested acknowledgement seem to be “carried along” in the
template LaTeX for many authors. There are many examples of papers that
acknowledge 2MASS but do not appear to actually use 2MASS data, even for
calibration, in the analysis presented therein. If those papers weren’t
manually excluded by checking each one, they would artificially inflate
publication statistics. While it may also be helpful to search the
Acknowledgements section for mentions of grants, the existence of a grants
statement does not guarantee that new, original analysis was done and should
not be relied on as a primary method to identify science papers either.

##  3 Evaluation of Publications for Inclusion in Observatory Bibliographies

After a candidate paper for inclusion is identified, the bibliographer
analyzes the paper for inclusion in the observatory bibliography. Papers
should be examined “by eye”, following the guidelines set out in this article.
Most papers still are tagged and categorized in a semi-manual fashion by
bibliographers, requiring human review to determine whether publications that
mention a telescope, observatory, instrument, data product, or survey in the
full text do in fact qualify as reportable “science”, “data”, or
“observational” papers.

###  3.1 Science Papers

A critical component to enabling fair comparisons and benchmarking of
observatory publications is a common definition for “science”, “data”, or
“observational” papers. When a paper is classified as one of these, it is
typically included in the official list of publications attributed to an
observatory and tabulated in internal and external reporting metrics.
Classification as “non-science” is discussed later in this section.

The OBC agrees on the following general criteria for tagging papers we roughly
categorize as “science”, otherwise called “data” or “observational” papers at
some observatories. Each observatory may interpret these guidelines in
slightly different ways, but the spirit of what defines a science publication
is consistent across our observatories.

  1. 1.

To qualify as a science paper, it must be apparent that data or data
product(s) from the observatory were used and that the data or data product(s)
formed the basis for reaching a new scientific conclusion.

Data and/or data products referenced in a publication can be from a single
telescope or many observatories. Borrowing from the NOAO Inclusion Criteria
document: “The amount of … data analyzed does not have a bearing upon
inclusion; if a small amount of … data in relation to the total amount of
datasets in the paper is used in the conclusions, the paper is included.” In
another example, if the 2MASS data were being used to calibrate the astrometry
and/or the flux for data obtained by authors at another observatory, and that
other secondary observatory’s data were used to reach a new scientific
conclusion, the paper may still be included in the 2MASS/IRSA bibliography,
even if the 2MASS data were not the basis for the new conclusion.

  2. 2.

“Data” can be defined in a number of ways – it may be original data taken by
the author(s) as part of their proposal(s) or archival data reduced or
analyzed anew by the author(s).

Like the 2MASS example, the data might not be observational data, but position
or flux calibration for some observatories or data sets. Science papers may
also rely on enhanced data products such as catalogs (including source
catalogs) or spectra that combine processed telescope data in a meaningful
format for the greater astronomical community, so long as it is apparent these
refined data products were a component of the author’s analysis. Spitzer
tagged all papers that used enhanced data products, which is useful in
measuring the popularity of the advanced data products.

Bibliographers generally do not count publications that cite or acknowledge an
observatory without actually having used its data. For many observatories, it
is not uncommon to find a paper in which the author(s) acknowledge an
observatory or one of its data products, e.g., 2MASS, HST grant number, or ESO
project number, without actually using their data in the paper.

  3. 3.

To qualify as a science paper, it does not matter if the observations have
been published before, so long as the way the results are presented constitute
new analysis or use.

A simplified summary or quote mentioning how others used or analyzed data in
an earlier publication generally does not constitute new use. A statement such
as “Smith & Doe found the same using the ELT” would not qualify, whereas a
statement such as “see Bain et al. (2009), Table 3, for a list of Chandra
observations used in our analysis” would constitute reanalysis.

  4. 4.

Further refinement of science papers is possible.

The Chandra X-ray Center (CXC) tags their science papers as either “direct” or
“indirect”. In CXC’s definition, a Chandra Science Paper (CSP) is one in which
Chandra data make a significant contribution to the scientific conclusions of
the paper. “CSP-direct” papers start with data products that can be retrieved
from the Chandra archive. “CSP-indirect” papers present new analysis of
Chandra data starting from results in a referenced paper, as opposed to
firsthand data retrieval from the archive. Indirect papers are predominantly
papers based on published catalogs of Chandra data and the Chandra Source
Catalog. Both CSP-direct and CSP-indirect are included in the annual science
paper statistics and qualify as science papers from the point of view of other
observatories, even though other bibliographies may not separate “direct” and
“indirect” science in the same way.

Inversely, the Space Telescope Science Institute (STScI) and Hubble Mission
Office decided to track the use of non-reprocessed HST data, in particular
from catalogs, calibration studies, etc., in the literature and created a new
paper classification called data influenced in 2018. These papers are not
included in the annual science paper metrics since they do not present a
(re-)analysis of HST data. They are significant to the Mission Office,
however, because they present another use case for how the telescope enables
astrophysical research, even if not counted among the official statistics.

As the nature of astrophysical research changes, the policies and guidelines
for inclusion of papers need to be updated regularly and adapted. As in the
Chandra and HST examples, rules for the use of catalogs may have to be
established to properly distinguish between papers that merely cite
measurements or results from the literature and those that actually relied on
downloaded (archival) data or (re-)analyzed them to achieve new scientific
results.

  5. 5.

Each observatory makes the decision on whether or not refereed and unrefereed
publications are counted as official science papers.

Some observatories include non-refereed publications in their official science
paper count while others do not. The NSO includes SPIE unrefereed publications
in their official count; the rest of the publications in their bibliography
are refereed. The STScI generally does not include unrefereed publications in
its official science paper count, with the exception of some major conference
proceedings (IAU, ASP, AIP, SPIE), constituting less than 4% of HST science
papers through 2021. The CXC bibliography ingests all data and catalog papers
it finds in the worldwide literature, refereed or not. Chandra also includes
some circulars (e.g., the Astronomer’s Telegram) that supply the full content
of the article as an abstract to the ADS. Those publications are considered
full articles for the internal bibliography, but are not tabulated in external
metric reports. Some proceedings publish only the abstract as the complete
contents. Those publications are not considered to be full articles and are
not entered into the CXC internal bibliography (Rots et al., 2012). NOIRLab
excludes all non-refereed publications in its official science paper count, as
does Spitzer.

Dissertations are another variable in science paper bibliographies. Some
observatory bibliographers may include them in reportable metrics, while
others might not. In some cases, dissertations may be their own reportable
metric, separate from the official observatory bibliography.

###  3.2 Non-science Papers

The following list describes some common examples of attributes of papers that
may be tagged as non-science publications, referred to as a “mention” by
various observatories. This list is not exhaustive, but represents the most
common scenarios when bibliographers may choose not to classify a paper as
pure “science” or “data”.

  * •

Lack of new analysis: Compares previously published results and scientific
conclusions to results in current paper without new analysis. Uses only values
extracted from observational data without new processing or analysis of the
data. This includes instances where a paper uses measurements or results from
the existing literature.

  * •

Background Content: Cites observatories or missions in the history,
background, or introduction of a research paper to summarize other people’s
research or the entire body of literature.

  * •

Fields and Locations: Uses a defined field first observed by the telescope to
select where to point another telescope, without further discussion or proof
that data from original observations in that field were analyzed beyond
processed results in an earlier publication. Uses finalized results from an
earlier publication to determine source selection and location of observations
presented in the paper. Uses data to calibrate positions or fluxes for data
from other telescopes, e.g., 2MASS.

  * •

Instrumentation and/or Software: Describes instrumentation or software. Some
observatories, such as NSO, include these papers in their official science
paper count. Spitzer tracked these papers, but unless they used observational
data from the observatory, they were not included in the official science
paper counts. NOIRLab tracks instrumentation papers, but does not include them
in their official science paper count, nor does STScI.

  * •

Archival Tools: Describes tools or protocols at archives for astronomers to
use. Some institutions, such as IRSA, may choose to include these papers in
their official science paper count.

  * •

Future Observations: Proposes future observations with telescopes or
instruments by creating models or simulations, using data merely as examples
or to validate a proposed model’s output. These may be tabulated in a
different way that is more significant than a passing “mention”, but are
generally not counted as official “science” papers.

  * •

Images as References: Shows images as a visual reference or as an overlay with
other telescopes’ visuals without further discussion. This is typically done
to layer observations from other parts of the spectrum or to indicate regions
of the sky considered in a paper. Publications that use visuals to achieve
scientific results or that provide additional details about the visual and its
underlying data may qualify as a science publication.

  * •

Nested Catalogs: For most observatories, only papers citing the original
catalog and demonstrating use of the original catalog are included in the
science papers. Papers citing a catalog which absorbed processed data from the
original catalog are typically not included. Nested catalogs are common in
extragalactic deep fields. To illustrate the subtleties, IRSA will count
papers if they use data from sources that were downloaded from IRSA (or could
have been downloaded from IRSA), including catalogs, but not products derived
from catalogs (e.g., extinction maps, unless also served by IRSA). The Spitzer
bibliography counted papers that explicitly stated they used the catalogs
produced by the large survey Legacy or Exploration Science and Frontier Legacy
programs, as those programs were supported by the observatory’s data analysis
funding for the specific purpose of creating the catalogs.

  * •

Review Articles: These generally summarize the literature rather than present
new science results. Some may qualify as new research, but this is rare.

  * •

Advance Access Articles: Advanced electronic versions of papers that later
appear as a final publication are not included. This includes, for example,
arXiv, MNRAS.tmp, and Natur.tmp bibcodes from ADS. These pre-prints may be
tracked internally, but are not reported in external metrics as official
science papers unless they later appear in refereed form.

  * •

Standard Stars: Published standard stars for calibration purposes are
generally not considered new science.

  * •

Enhanced Data Products: Catalogs and other enhanced data products created by
the community using data from programs not specifically funded to produce
these high-level products. For example, on Spitzer, some programs resulted in
the creation of enhanced data products, but are not official
Legacy444https://irsa.ipac.caltech.edu/data/SPITZER/docs/spitzermission/observingprograms/legacy/
or Exploration
Science555https://irsa.ipac.caltech.edu/data/SPITZER/docs/spitzermission/observingprograms/es/
programs. The Spitzer bibliographer did not count papers using these products
because Spitzer did not necessarily host the enhanced products, and producing
the enhanced products was not part of the funding that was explicitly
allocated. However, IRSA does count these for some programs. The HST and JWST
bibliographies tend to include papers that cite catalogs and other High-Level
Science Products666https://archive.stsci.edu/hlsp hosted by the Mikulski
Archive for Space Telescopes (MAST) because part of MAST’s mission is to
collect and curate community contributed products based on MAST data in order
to make them available to the wider community for reuse.

##  4 Journal Coverage

Journal coverage is an important consideration in any discussion of
bibliometric best practices. The set of journals searched can impact end
counts for the number of science papers included in an observatory
bibliography. Different institutions search different sets of journals to find
science papers. Important factors in choosing the list of journals to cover
include the historic record, the science areas covered by instruments or
missions, and the resources available at an institution.

We tallied the representation of each ADS-indexed journal in our institutional
bibliographies to gauge the historic record, reporting on the number of papers
found in specific journals as a percentage of our total bibliographic dataset.
The overwhelming majority of papers in every institution’s historic
bibliography came from one of five core journals, and fewer than a dozen other
titles are significantly represented. Among all the possible titles available
in ADS, only a small subset is statistically significant to the various
bibliographies maintained by the OBC members. The list of these most common
journals across institutions is provided in Appendix A. The historic record
makes a compelling argument for where to focus journal coverage in new or
future bibliographies when staff and labor limitations dictate a curated
approach from the start.

An important consideration in choosing journals to search is subject coverage.
Certain fields in astronomy are inherently different from one another; for
example, extragalactic versus solar astronomy, or planetary science versus
cosmology, and may rely on different subdisciplines (Henneken & Kurtz, 2019).
The journals that are relevant to these fields will vary. For example, Icarus
is better suited to planetary missions and their resulting publication, as are
some American Geophysical Union (AGU) titles. Astronomy & Astrophysics is a
steady source of new research on extragalactic and stellar topics. If an
institution’s instruments or missions are particularly focused on one area of
practice within astronomy, this should inform the journals a bibliographer
searches for science papers. It is incumbent on a bibliographer to determine
which journals an observatory’s data is published in.

Bibliographers need to consider the resources available for their efforts -
software, tools, staff time - when deciding which journals to search. The more
journals included in a search and the more terms, keywords, or acronyms
searched, the more results there will be for candidate papers. Tools,
technology, and a considered approach to query strings will reduce the time
required to review results.

There are three main strategies a bibliographer can employ when choosing what
journals to search. The first is the broadest use of ADS where all available
journals are searched, either using the API or manually running searches on
the ADS site. The second strategy is to make a focused search on only a select
list of journals, either through ADS or through the individual journals. The
third strategy is a full-text search of specific journals using text mining or
other technologies, which may rely on CrossRef. A combination of strategies
may be best depending on your specific circumstances. At times, there are
benefits to casting a broad net to determine if data and discoveries by an
observatory are impacting other disciplines in astronomy. Many observatory
bibliographers use a curated approach throughout the year, then run a wider
search against negative results at year’s end to identify relevant papers that
may not have been found using the more precise criteria. One example is
combining designated JWST keywords with NOT bibgroup:jwst in ADS, scouring
through additional refereed journals after known JWST papers have been tagged
from the pre-defined journal set for the year.

The choice of journal coverage at an institution is necessarily specific to
that institution’s current needs, historic record, subject areas within
astronomy, and available resources. Just as no two observatories or telescopes
are identical, journal coverage that is the basis of an institution’s
bibliography will be different. It is fundamental that a bibliographer
understand not all institutions are considering the same pools of journal
coverage. If an institution beginning a new bibliography is unsure which
journals to begin focusing on, we recommend considering the core list of
journals highlighted in Appendix A. Other best practices in journal coverage
we recommend are to use only refereed journals to build your count of science
papers and to strive to attain a 90% capture rate of those papers before
considering non-refereed publications in ADS.

##  5 Metadata

Once a publication is chosen for inclusion in an institution’s bibliography,
the bibliographer must assign metadata relevant to their observatory to that
publication. In this section we present guidelines and considerations for
developing and applying metadata tags to publications in an observatory
bibliography.

###  5.1 Core Metadata Set

Below is a list of the core set of fields the OBC recommends that a new
observatory considers collecting for local analysis. An existing observatory
may consider using the recommendations to backfill their metadata for analysis
and reporting, in order to develop a deeper understanding of science trends
over time. This set is based on our shared experience of metadata that we 1)
have found useful from the start and continue to find useful, 2) wish we had
collected in the past and continued collecting, and 3) consider worth
backfilling in our observatory bibliographies. Each observatory should make
its own determination as to which of the core metadata fields are relevant for
its purposes.

Bibliographers should regularly review this set of core metadata fields to
keep up with publication practices and changes in the way the astronomical
community uses data produced by the observatory. Expanding or refocusing the
scope of bibliographic metadata collection is an unavoidable step when a new
category of papers (covering, for example, new types of data products or
software produced by an observatory) is being collected and classified.

Many of the recommended metadata fields are available in ADS, so documentation
of them in an internal observatory bibliography may not be necessary. It is
possible to analyze and retrieve these data from ADS without storing the
metadata locally. Fields available in ADS are marked with an asterisk*.

Identifiers

  * •

ADS bibcode*

  * •

proposal/program ID(s) or observation ID(s)

  * •

associated principal investigators (PIs) and co-investigators (Co-Is)

  * •

data product, dataset(s), or dataset(s) DOI(s), exactly how the data is
bundled or cited will vary according to the data archive

  * •

software products used in paper analysis

Bibliographic Information

  * •

paper title*

  * •

publication/journal title*

  * •

publication date*

  * •

publication volume, issue, and pages*

  * •

published article DOI (version of record)*

  * •

optional: preprint DOI or identifier*

Author Information

  * •

first author name*

  * •

first author affiliation(s)*

  * •

first author country/ies*

  * •

additional author name(s)*

  * •

additional author(s) affiliation(s)*

  * •

additional author(s) country/ies*

  * •

observatory staff publication (if tracked by your institution)

  * •

author(s) category (e.g., postdoc, visitor, student)

Publication Type

  * •

refereed / non-refereed* (if including both types in bibliography)

  * •

thesis/dissertation (often, but not always in ADS)

  * •

paper classification: science, non-science/mention, data influenced,
instrument, engineering, etc.

Telescope and Instrumentation

  * •

telescope/observatory (for bibliographies that capture more than one
telescope’s data)

  * •

instrument(s), mode(s), channel(s), etc.

  * •

filters(s) and wavelength(s)

Data Products and Services

  * •

archival flag (PI/observer vs. archival use)

  * •

observatory data services or high-level products used (for example, NOIRLab’s
Astro Data Lab, or the Chandra Source Catalog)

Surveys/Missions

  * •

survey observatory or pointed / object-specific observatory

###  5.2 Additional Metadata for Inclusion

Each observatory has unique characteristics, needs, and reporting
requirements, which may necessitate the inclusion of additional metadata
fields. As examples, the following metadata fields may be considered for
inclusion:

  * •

Page charges paid by observatory

  * •

Data analysis funding/grant information

  * •

Length of time to publish since data was released

  * •

Names of other observatories whose data was used (for multi-mission papers)

##  6 ADS Bibgroups

ADS offers bibliographers the option to create bibliographic groups (called
bibgroups in the ADS web submission form and API) to collate publications that
used their observatory’s data. Bibgroups also make it easier to identify
multimission/multiwavelength papers that use data from multiple telescopes or
observatories. The Union, Intersection, and Difference functions for ADS
Libraries777https://ui.adsabs.harvard.edu/help/libraries/ can augment
discovery and arrangement of new paper sets focused on two or more
observatories. Some real-life examples include searching for papers on a
particular scientific topic, e.g., proper motions and astrometry for stellar
clusters using HST data only, Gaia data only, or combined HST and Gaia data. A
researcher planning a new proposal focused on AGNs, quasars, or neutron stars
might also use the bibgroup:chandra in combination with bibgroup:NOIRLab to
understand what past research has been done on their intended target(s).

For new bibgroups, ADS recommends that you create an ADS account for your
group to manage the bibliography. While logged into this account, execute your
search. Then proceed to add your results to a private library and set it to
public. These libraries can be shared with other team members, and they will
always contain the canonical bibcodes for the publications contained in them.
Bibliographers can maintain multiple libraries and use powerful tools to
generate one master library from the individual libraries or provide the ADS
team with all individual libraries to be combined into one bibgroup. After you
have created the library or libraries for your bibliography, send the URL(s)
to<EMAIL_ADDRESS>ADS staff will formally create the bibgroup in
ADS. From this point on, you can manage your bibliography directly by adding
or removing items to the related library(ies), and ADS users will have access
to the most recent updates.

For the ADS libraries and bibgroups to be useful, each observatory must
maintain the contents of its own bibgroup(s) and provide ADS with the
parameters of its bibliography when first setting up a new bibgroup. Moreover,
maintaining one bibgroup per telescope rather than per organization makes
combined searches easier to execute. Ideally, bibgroups for active
observatories are maintained through the most recently completed calendar
year, or the one prior, so that more complex searches are possible.

##  7 Reports and Metrics

Publication lists and metrics are valuable data that are often included in
reports to funding agencies, governing bodies, and the astronomical community
to demonstrate the scientific output and value of the observatory. An
observatory’s annual report typically includes this information; in some
cases, partial information is included in monthly and/or quarterly reports as
well.

In addition, observatory bibliographers are often asked to provide publication
counts and lists on-demand for a specific topic or subset of programs, as
required by research staff, funders, and other stakeholders. A recent example
from STScI is an internal assessment of the number and types of papers using
ULLYSES legacy program888https://archive.stsci.edu/hlsp/ullyses data, gathered
by trained bibliographers as opposed to publications loosely associated with
the single keyword ULLYSES in an ADS full-text search.

Beyond ADS bibgroups, graphical representations of publication metrics are
useful additions to observatory websites. For example, NRAO maintains online
graphs that show refereed paper totals by instrument and year and citation
totals by year for the most recent 10 years. Chandra maintains online graphs
showing the total number of observational papers per year, both in total and
broken down by metrics such as quarter, category of target, grant type, etc.
NOIRLab has a dynamic publications dashboard that shows publication counts for
each of their telescopes for the past 10 years which links to its ADS public
libraries for individual telescopes and data products. The ADS Citation
Metrics Report functionality provides an overview of citations, usage, and
derived indicators for a set of ADS records. The public interface of ESO’s
telescope bibliography provides a large variety of options to export and
visualize search results. ESO librarians combine the most frequently requested
reports and publish them on the web999https://www.eso.org/libraries/pubstats.
STScI provides visual representations as well as public-facing numeric counts
through the previous calendar year for both HST and JWST.

There are risks attached to reporting on metrics, especially when trying to
interpret those maintained by another observatory. See the the below section 8
below for a discussion on why comparing publication numbers among
observatories is problematic. It is vital that each bibliographer makes the
criteria for inclusion evident so that the metrics can be evaluated
appropriately.

##  8 Caveats of Comparing Bibliographies

###  8.1 Intra-observatory Comparison

An intra-observatory comparison is any comparative assessment for the period
of time during which the observatory has been operating, for example, looking
at the number and types of science publications using Spitzer data before and
after the start of its warm mission101010https://www.jpl.nasa.gov/news/nasas-
spitzer-begins-warm-mission.

Often technical events can visibly modify the bibliographic landscape for a
period of time. As an example, one of the Chandra instruments – the High
Resolution Camera (HRC) – stopped functioning in February 2022. The
engineering team managed to bring it back to life in April
2023111111https://groups.google.com/a/cfa.harvard.edu/g/chandra-
announce/c/Z_ggkSlEV2w. Without this background information, someone could
interpret the reduction in number of papers using HRC data during and
following the HRC hiatus as a decrease in the scientific utility or interest
in the detector, due to the amount of time it takes from data collection to
publication. In reality, the dip is caused by the fact that no HRC
observations were being taken for approximately 14 months. The CXC
bibliographers are anticipating a statistically significant increase in the
number of papers using HRC sometime in the future (2024-2025), as a result of
the larger number of HRC observations (relative to normal scheduling) that
have been taken by Chandra since resumption of HRC observations. Without
context and an understanding of observatory operations, the dip and peak could
be misinterpreted as being real ”bibliographic” phenomena when instead these
extremes often reflect technical issues experienced by a facility.

Similarly, the HST saw declines in the total number of data publications in
the periods of time leading up to and sometimes following the various
servicing
missions121212https://science.nasa.gov/mission/hubble/observatory/missions-to-
hubble/. There was a notable decline from 1998-1999 (Servicing Mission 3A) and
again in 2008-2009 (Servicing Mission 4), shown clearly in the annual HST
Publication
Statistics131313https://archive.stsci.edu/hst/bibliography/pubstat.html. The
number of papers that rely on data obtained using a particular instrument(s)
are closely tied to each servicing mission and the suite of instruments
installed or decommissioned during each mission. This in turn impacts the type
of archival observations the community has access to as instruments are placed
online and taken offline. In short, observatory bibliographies are never
independent of what is happening on the ground or in space, and are closely
linked to the operational status of the facility.

Tracking the use of purely or partially archival data is another common use of
intra-observatory comparisons. By tracking the gradual increase in archival
data use, it is possible to advocate for ongoing support and funding for the
data archive while the community continues to access and rely on that data to
make new scientific discoveries, often in combination with fresh guest
observer data or other observatories’ data.

###  8.2 Inter-observatory Comparison

It is difficult to compare bibliographies between observatories, even ones
that seem like they might be comparable. As discussed above in sections 2.3
(Keywords) and 4 (Journals), the search criteria applied by each observatory
can also have an outsized impact on the bibliography, making it difficult to
conduct true comparisons between different facilities. Below we identify other
issues the OBC have encountered when asked to complete inter-observatory
comparisons, and our collected experiences trying to fairly compare
observatories when required to do so.

Some journals do not update all of their article metadata or full-text files
for up to three months, which makes searching for candidate papers difficult
until the following quarter. Therefore, one cannot compare the yield of
articles in 2023 until April 2024 at the earliest, for example, if a
comprehensive annual bibliography is expected. It is important to ask: “Is the
information I have, especially from outside my observatory, up-to-date for the
investigation I have in mind?”. While it is possible to search and limit to a
specific observatory’s bibliography using the ADS bibgroups, and then limit by
refereed/unrefereed status as needed, generating equitable comparisons can
prove problematic. All too often well-meaning staff use partial or incomplete
data made available through an ADS bibgroup or public-facing page for a
different institution and falsely assume it is “current” through the year or
month for which the assessment is being done. This has resulted in uninformed,
inaccurate, and misleading comparisons across institutions and observatories.
Due to staffing and the manual nature of bibliographic work at present, many
bibliographies run six months to multiple years behind in assessment.

For this reason, the OBC recommends contacting the bibliography staff at peer
institutions whenever stakeholders or funders request an inter-observatory
comparison. By understanding the limitations of your own publications database
and those of peer institutions, you can more clearly state the caveats
attached to the comparative metrics you provide, and do your best to show a
like-for-like comparison.

In its overview of basic publication statistics, ESO provides a number of
prepared statistics and graphs about their facilities’ science papers. One of
them (“ESO and Other Observatories”) puts ESO’s research output in context by
showing publication numbers from other observatories. However, ESO explicitly
points to the caveats arising from such a comparison. Comparing the numbers of
publications is the most simplistic way of comparing facilities, and it favors
large institutions with many telescopes and instruments over smaller ones. A
more meaningful investigation would normalize the numbers in some way, e.g.,
by number of observing hours, by actual share of data used in the papers
(since many scientific articles use data from more than one observatory), or
by budget. The “ESO and Other Observatories” section also states that when
comparing publication statistics among different observatories, it is
essential to assess the selection criteria applied by each observatory, and
that comparison data and graphs have to be used with utmost care.

An example of when cross-observatory metrics can be useful, even in broad
stroke comparisons, is with regards to larger societal or economic trends. In
2020 and 2022, the Basic Publication Statistics maintained in ESO’s telbib
database showed a marked decline in both 2020 and 2022 across multiple
observatories, which correlates with the initial shutdown of facilities in
early 2020 and the transition back to on-site operations in 2022 during the
COVID-19 pandemic. While such metrics can rarely demonstrate causation, they
do give stakeholders pause when trying to understand observatory operations in
the wider context beyond the astrophysics community.

####  8.2.1 Survey vs. pointed missions

When proposing surveys that require many hours of observations, astronomers
need to make a strong case that the survey data will be useful enough to
warrant the allocation. This tends to result in programs that a broader range
of astronomers support and use, and data that is used for other purposes long
past when it was taken.

For example, for Spitzer, the Legacy programs were large, coherent
investigations proposed prior to launch and carried out early in the mission.
The teams were funded prior to launch specifically to enable both rapid data
reduction and delivery of high-level data products back to
IRSA141414https://irsa.ipac.caltech.edu. The surveys that were done in the
first 2.5 years of the mission produced a much higher “papers per hour of
data” rate (0.37) than the non-survey General Observer observations (0.17)
taken at the same time (see Scire et al., 2022, fig. 9). As time passes, the
survey programs from that observatory continue to provide most of the papers
as observations from those survey programs are republished (Scire et al.,
2022, fig. 6a and 10). The Legacy programs were recognized to be so successful
that the community continued to propose large investigations that promised
deliveries back to IRSA. These ready-to-use data products continue to generate
substantial use of Spitzer data and citations.

All-sky surveys such as Gaia or 2MASS or (NEO)WISE can be used for nearly any
investigation, provided the targets are bright enough. Observatories that are
primarily focused on pointed observations (such as Chandra, Hubble, Spitzer,
or JWST) will not necessarily have data at any given location on the sky. By
their very nature, then, all-sky surveys have the potential to generate more
publications and citations than pointed observatories.

####  8.2.2 Large collaboration papers vs. small research groups

Missions that started with large international collaborations will often
result in more papers referencing original collaborative publications and may
accrue more citations compared to observatories that traditionally have
smaller teams observing and using data.

Similarly, Guaranteed Time Observations (GTO) programs often result in
proportionally more papers, since the teams of researchers are funded over a
longer time span, often lasting for two or more years/cycles. The influence of
accumulated volumes of data on observatory bibliographies is discussed further
in Blecksmith et al. (2005) for Chandra and shown in fig. 9 of Scire et al.
(2022) with regards to Spitzer.

####  8.2.3 Science topics and community size

Observations focused on topics that are less widely studied or have smaller
research communities, such as masers, or specialized methods or topics, like
IR spectroscopy of Galilean satellites, will inherently have smaller
bibliographic counts. Observations of targets that are more popular in the
public sphere or have been observed for a longer period of time, e.g., M31,
can result in a greater number of publications and more citations. Searching
for the object M31 in the HST bibgroup using ADS yields over 1,700 results,
accounting for more than 12% of the entire HST bibliography to date.

Even just within the Spitzer archive, which should be similar in terms of
people interested in the wavelength regime and consequent astrophysics, the
spectra from IRS have a lower publication per hour rate than the two imaging
instruments, IRAC and MIPS (Scire et al., 2022). Some of that likely arises
from the fact that an image can serendipitously capture objects whereas all
the spectroscopy must be deliberately pointed, but it is also likely that more
specialized knowledge is required to analyze the spectroscopy, and therefore
intrinsically fewer people attempt it.

Spitzer also provides a ready-made lesson on the change in publication rate
per observing hour when it comes to the nature of the observations. Early in
the mission, many of the large time allocations were given to projects
covering relatively large areas of sky. Those projects’ data continue to
accumulate myriad citations as scientists use data from those surveys in
conjunction with current observations. Later in the mission, after the cryogen
was exhausted and only the shortest two channels of IRAC were still working, a
much higher fraction of the observing time was devoted to projects that were
monitoring targets for time variability – that is, staring at one place on the
sky – and once a paper is written presenting the light curve (brightness as a
function of time), future investigators cite that paper, rather than reusing
the original data. This results in far lower publications per hour of
observing time, given the strict definition for science papers; see further
discussion in Scire et al. (2022).

##  9 Technology Considerations & Standard Tools

An automated tool for searching journals or text mining can substantially
reduce the number of staff hours required to review a list of journals for
candidate papers. Questions to ask about opportunities to apply technologies
at your institution should include the following:

  * •

What expertise is available to code and curate tools to ensure they remain
effective for paper discovery?

  * •

Is it better to a) search journal tables of contents, b) search within full-
text papers using the ADS GUI, or c) use an API such as those from ADS or
CrossRef to search journals’ full-text content?

  * •

Are all the necessary publisher licenses in place to allow access to papers in
full for detailed evaluation, whether using ADS, CrossRef, or publisher web
forms?

  * •

How should my entry point(s) for full-text searching inform the design of my
automation and tools, including the program or database in which I track
verified science papers?

  * •

How will I retain a list of papers I consciously ignored or keep track of
which journals and volumes I’ve already evaluated?

Beyond the search tools and search methodology, you will need to ask yourself
the following:

  * •

What additional metadata, beyond what’s available in ADS, does my observatory
wish to maintain alongside each publication record?

  * •

Do I need to include publications that fall outside ADS, and if so, how will I
ingest metadata from those publications, and is the metadata comparable to
what’s found in the bulk of the literature in ADS?

  * •

How will paper categorization affect design?, i.e., if tracking non-science
papers in addition to pure science papers, should those non-science papers be
split between historic references and forward-facing simulations?

  * •

How often will my stakeholders want reports on publication metrics and how
might that influence design and processes? Which metadata fields and values
will be necessary to track to satisfy reporting needs? See section 5,
Metadata, for further discussion.

  * •

Am I (or the appropriate software development team) expected to build a
public-facing component in addition to an ADS library or bibgroup. If so, how
will it be updated from the internal bibliography tool(s)?

  * •

Am I (or the appropriate software development team) prepared to maintain the
database and continually develop it in response to outside systems like ADS?

  * •

Am I (or the appropriate software development team) prepared to maintain
ongoing documentation and versioning of the observatory bibliography software?

###  9.1 Standard Tools: Low to High Investment

Although many commercial products exist for managing bibliographies, no known
off-the-shelf software exists for building or maintaining an observatory
bibliography, and adopting another institution’s application frequently
requires a lot of modification and technical expertise to fit your
observatory’s needs. To manage a sustainable bibliography, you (or your
institution) may need to create in-house tools to get the job done. Here we
present multiple options, from basic, low-investment spreadsheets and ADS
libraries to complex databases integrated with an institution’s data archive.

Some observatories may find they need to rely on a basic approach because of
restrictions on resources and IT support. Varying levels of IT support,
limited access to software developers within your organization, time, and the
bibliographer’s skills and training may all impact design decisions when
creating a new bibliography. It’s important to bear in mind that without
dedicated and ongoing support, observatories cannot easily add complexity,
adjust their metadata model and paper categorization, or make significant
changes to the software underpinning the database.

A perfectly usable, low-investment option is a basic MS Excel workbook or
spreadsheets, tracking the ADS bibcodes or other unique identifier of papers
that qualify as science and non-science papers (if also tracked) that are
associated with your observatory. A related ADS library and bibgroup can be
established and updated in tandem or at regular intervals. This is the
approach used by NOIRLab, and is how most observatory bibliographies begin. In
addition, NOIRLab built a web interface for bibliography metrics, which pulls
directly from ADS.

The DKIST Observatory bibliography at NSO is maintained in an MS Access
database. Other observatories, such as the National Radio Astronomy
Observatory (NRAO), use a slightly more complex, homegrown multi-table
database. NRAO uses a browser-based middleware application to interface with
the data stored in a MySQL database, which reduces the need for library staff
to have expertise in SQL and performs simultaneous transactions more
efficiently than running multi-table queries. MySQL is used most often for
queries to gather metrics requested by staff. NRAO also operates a faceted
search tool running on Blacklight and SOLR that the public can use to search
for papers in the bibliography. Two dynamic graphs produced by a separate API
show paper totals and historic citation totals.

Spitzer made an upfront investment in its bibliography early in the mission’s
lifetime. Until it was decommissioned in 2020 and the data transferred to
IRSA, Spitzer used a MySQL database with Shell and Perl scripts to interact
with the database, with three tables for pending, valid, and rejected papers.
For more information see Scire et al. (2010). The Spitzer database did not
include an internal general user interface (GUI) like some other observatory
bibliography databases, but did allow the bibliographer to associate program
IDs with papers. Tables were used to populate the Spitzer Biographical
Database web search interface which was public facing.

Other observatories have invested more time or software development resources
in recent years and manage observatory bibliography software that is highly
complex, with multiple relationships and dependencies between ADS and their
institution’s data archive. Examples of more complex bibliography tools can be
found at the Chandra X-ray Center, ESO, and STScI, all of which offer a UI for
bibliographers who are not adept at software engineering in their day-to-day
work. All three link the associated data from their respective archives to
publication records in ADS.

ESO’s telescope bibliography (telbib) started in 1995. Its predecessor was a
simple list of observatory papers compiled for the Annual Report. Since then,
telbib has evolved into a sophisticated system with many features and facets.
For the current system, ESO has developed a web application called FUSE
(Fulltext Search tool, using PHP/MySQL) that queries ADS via their API in
order to identify relevant papers. The telbib software is managed through
another web interface (PHP/Sybase). It allows rich tagging with metadata, a
large variety of export options, and reports in various formats. In addition
to these internal applications, there is a public interface (PHP/Solr) that
allows searching, exporting, and visualizing data via a web interface or an
API.

The Chandra bibliography is managed through a graphical user interface written
in Perl and SQL and is accessible through a web interface. Due to the level of
customization, especially for managing metadata and connections between CXC
systems, it may be difficult to revise the bibliography software for other
missions in future. In hindsight, the CXC bibliographer team stated it would
have designed a database with less technical complexity so it is more
adaptable for other Center for Astrophysics (CfA) needs.

The STScI and MAST bibliographies are managed through a front-end graphical
user interface used daily by archive scientists and bibliographers, with
underlying code rendered in Python and SQL. The local tool is called
PaperTrack, and it was rebuilt in 2018-2019 to take full advantage of the ADS
API released alongside the ADS 2.0 Bumblebee platform.

PaperTrack is used for three main purposes. PaperTrack stores bibliographic
information about publications found in ADS that use mission data (Science),
are related to mission engineering or instruments (Engineering/Instrument), or
reference MAST missions/data already published (Mention/Supermention/Data-
Influenced). Secondarily, the tool provides institutional benchmarking numbers
to STScI; its mission offices; NASA; its parent organization, AURA; and
external bodies. Annual counts of refereed science publications are reported
as one measure of scientific output. Lastly, STScI commits to an annual update
for its mission metrics in late spring of each year in order to provide other
observatories and science centers a record of publications that use or
reference MAST mission data for their own benchmarking measures. This last use
is discussed in more detail in sections 7, Reports & Metrics, and 8, Caveats
of Comparing Bibliographies.

To identify papers, STScI relies on the ADS API to look for relevant keywords
in the full-text files housed in ADS. Not every publisher deposits its full-
text files at a regular cadence with ADS, which limits the utility of the ADS
API, through no fault of the ADS team. Candidate papers are manually assessed
to determine if they only mention one of its three flagship missions in
passing, or clearly used data to reach a new scientific conclusion. The HST,
JWST, and the Nancy Grace Roman Space Telescope (RST; NGRST) comprise the
flagship mission searches. The same assessment occurs for over 15 MAST
missions including TESS, GALEX, Kepler, IUE, and many others. Searching the
literature for candidate papers using MAST missions’ data is managed by the
MAST archive scientists. MAST offers a public facing, external database, but
for reasons expressed in section 8, Caveats of Comparing Bibliographies, it is
always advisable to contact the STScI bibliographers for assistance gathering
the metrics you need. The database can be accessed at
https://archive.stsci.edu/publishing/bibliography#/. It allows external users
to view information about mission publications for previous years, find papers
using their HST or JWST program data, and gather basic publication metric
information.

Regardless of the tool your institution develops, the OBC advises new
bibliographers to manage the flow of new candidate papers and the steps
involved in paper assessment on a rolling basis. Doing so will ensure
connections between your bibliography software, ADS, your institution’s data
archive, and public systems are functioning as expected at regular intervals.
When designing bibliography software or choosing to implement a low-tech
option, consider to what degree you are willing to rely on other technology.
Scraping full-text html, for example, even if permitted by the publisher will
require intervention each time a publisher’s journal platform changes. If the
ADS API undergoes a major re-haul, it may break existing scripts that allow
you to search full-text files and ingest candidate paper metadata into your
database. Consider reducing unnecessary dependencies on other systems whenever
possible to ensure long-term maintenance. It’s critical that those involved in
building new bibliography software or enhancing existing software communicate
proactively with the groups who manage related systems, such as your
institution’s data archive, ADS, and publishers.

##  10 Staffing

An observatory bibliographer works with a diverse group of technical,
scientific, and non-technical personnel both within the observatory and at
outside research institutions, funding agencies, and observatories. Strong
communication skills, a knowledge of the scientific literature, and
proficiency in database searching are all important competencies for
observatory bibliographers. Many bibliographers are either the sole staff or
part of a small team performing these tasks, so a strong commitment to
professional networking is a valuable asset. An observatory bibliographer’s
roles and responsibilities typically include the following:

  * •

Searching the literature to identify papers that use data from an
observatory’s telescopes, or that were published by observatory staff

  * •

Reviewing the literature for inclusion in the observatory bibliography
according to the observatory’s publications Criteria for Inclusion

  * •

Tagging relevant publications with the appropriate metadata in the database(s)

  * •

Investigating and solving problems related to metadata, linked archival data,
and ADS index records

  * •

Updating and improving data citation and metadata standards on a continuous
basis

  * •

Educating users on how to use and interpret bibliography tools and reports

  * •

Explaining the publication tracking process to bibliography stakeholders and
observatory staff

  * •

Interpreting publication metrics and preparing reports that present numeric
and visual data for both recurring and ad hoc needs.

##  11 Future Trends & Technologies

Efforts are underway to implement an automated paper classification system at
STScI/MAST to identify science papers within a set of candidate papers;
however, even if this product comes to fruition in the 2020s, it is expected
that human intervention will be needed to extract additional information about
the paper to complete certain metadata fields such as program ID(s),
instrument, and observing mode (for JWST). Additional work to understand how
this system will handle false hits on papers that contain keywords or phrases
of interest, but have nothing to do with MAST or flagship missions, is
underway. A summary of the pilot, its accuracy and completeness, and remaining
challenges is expected to be released in 2024 (Pegues et al., in progress).
The first alpha release for the automated classifier is anticipated for fall
2024. It is expected that only a portion of the papers can be confidently
tagged as science papers. A sizable portion may still require human review to
accurately assign a paper classification such as science, mention,
engineering, etc. Moreover, not all publishers are willing to provide access
to entire full-text files, even for non-profit, internal, non-generative
purposes; the automation tool might only be possible for use with content from
certain publishers until licensing agreements can be reached to allow for use
of additional content in machine learning tools. Licensing restrictions
generally apply to full-text obtained via the publisher’s online platform in
html or pdf form, as well as the full-text JSON files available through
CrossRef.

Similar efforts are also taking place at other observatories. For instance, an
ML approach to identify data in scientific papers for inclusion into NED (the
NASA/IPAC Extragalactic Database) has recently been described by Chen et al.
(2022a). Similar to STScI and MAST’s experience, staff at NED expect that
their ML tool will provide a partial solution but are still seeing a return on
investment: “This ML application on both subject and content classifications
has been in operations since January 2021, and has gained the team about 0.4
FTE, i.e., the equivalent of 4 person-years of labor over a decade of NED
operations”. At ESO, an AI interest group is using the observatory
bibliography as an example to study how algorithms can enhance the process of
properly classifying research articles.

While all of these efforts hold promise for more efficient, complete, and
timely paper classification and metadata extraction, it will be essential to
keep a balanced view of the role of new and emerging technologies in
observatory bibliographies vis a vis human control. Per Chen et al. (2022a):
“It is worth noting that this ML approach does not completely remove human
involvement in the process. Human expertise and learning are needed for
marginal cases that are not resolved by existing capabilities. Further, the
trained model is applied to a domain-shifted data set for predictions. Newly-
emerged data content, variations in formats and representations can all
contribute to deteriorating performance over time. Hence, there needs to be a
continuing education program for retraining and updating the classifier models
with new literature, which will require new labels identified by human experts
periodically”.

##  12 Lessons Learnt

Over the course of our discussions, we have formulated some lessons learnt and
guidelines to help others understand the work that goes into observatory
bibliographies and their purpose. We also anticipate this article will help
those starting new bibliographies, as well as those interested in revising
current ones. Bibliographies provide value to their institutions, individual
researchers, and data archives, as well as oversight committees and funding
agencies.

  * •

To Institutions, they demonstrate scientific output, provide data on
telescope/instrument use, and act as templates for either new or revised
observatory bibliographies.

  * •

To Researchers, they show their research teams’ impact and reach; aid in
understanding how their work fits within the astronomical community; help
identify areas of interest and potential collaborations for ongoing work; lead
to suggestions for new facilities and missions; and support current projects
and upcoming proposals.

  * •

To Archives, they demonstrate connections between publications and data usage
over time, especially for archival data use. Many bibliographies enrich the
ADS records with links to datasets or DOIs, making it easier for the community
to locate data used in papers, leading to reuse, validation of research
findings, and new discoveries.

  * •

To Oversight Committees and Funding Agencies, they demonstrate utility and
level of use of facilities and observatories by the astronomical community;
show return on investments; inform support priorities; illustrate how
observatories are advancing science; and guide decisions for future facilities
and archives development, especially through reports to Visiting Committees,
User Groups, and Decadal Survey efforts.

Some common insights and shared considerations across observatory
bibliographies are as follows:

  * •

ADS: ADS (soon to be SciX) is an important tool for searching the scientific
literature, deriving metrics, and maintaining public bibliographies.

  * •

Criteria for Inclusion: Each institution should document their Criteria for
Inclusion and make these publicly available to guide bibliographic work and
inform users of the bibliography of its parameters. Institutions and
bibliographers need to evaluate and revise their bibliographic policies and
procedures regularly to account for changes in areas such as reporting
requirements, missions, goals, keywords, and technology. There is tremendous
diversity among bibliographies related to the unique characteristics, goals,
and subject area focus of each observatory.

  * •

Science Publications: Most bibliographies focus on refereed “science” papers.
A ”science” paper is one where 1) data or data product(s) from the observatory
were used, and 2) the data or data product(s) formed the basis for reaching a
new scientific conclusion. It does not matter if the observations have been
published before, so long as the way the results are presented constitute new
analysis or use.

  * •

Journal Coverage: There is a core subset of journals that most observatories
search to compile their bibliographies. Differences in journals searched
beyond this subset relate to the unique characteristics, goals, and subject
areas of the observatory.

  * •

Metadata: Bibliographers need to regularly assess their metadata tags as these
will change over time as the science evolves and the institution transforms.
Having a bibliography from the start will allow the observatory to identify
which papers it may need to reclassify or augment through a more robust
classification scheme in the future. Observatories should consider what data
points to collect from science publications from first light or launch to
demonstrate to funding agencies that the observatory is making a difference in
the research field(s) and is enabling new findings.

  * •

Comparing Bibliographies: Direct comparison of the number of science papers
between observatories and even projects within a single observatory may not be
the most effective or equitable way to analyze productivity. We do not advise
assessing total output over time between facilities or even within a single
observatory. If such comparisons are required to satisfy oversight committees
or funding agencies, observatory bibliography metrics should be provided with
proper context. Caveats or limitations of the data and inherent differences in
the datasets being compared should be provided along with the metrics
report(s).

  * •

Technology: There are exciting technological advances in this field, including
the use of machine learning to classify papers. However, it is important to
emphasize that automation is never a substitute for human evaluation.
Automated classification will need to be periodically inspected and validated
by staff, and language models will need to be revised over time and then re-
validated.

  * •

Reports and Metrics: Bibliographies are regularly included in reports to
funding agencies and to observatory leadership. It is vital that there is
clear documentation about how the bibliographies were compiled and what the
metrics may or may not show.

  * •

Staffing: Many bibliographers are either solo or part of a small staff.
Sharing of information, policies, and procedures through networking among
bibliographers is vital to maintain the high standards, usability, and
credibility of observatory bibliographies.

  * •

New and Revised Bibliographies: If a bibliography is not initiated at first
light, it is difficult to begin one after years of observations without
investing significant time to catch up. The current observatory bibliographies
and our guidelines and experiences can act as templates when devising or
revising bibliographies.

##  13 Conclusion

Observatory bibliographers play a vital role in demonstrating the scientific
impact and value of an observatory. Our work is aligned with the goals and
mission of our respective institutions, and there is tremendous diversity in
the ways we track publications and maintain publications databases. This
diversity is linked to parameters such as observatory resources (personnel,
time, money, IT capabilities), type of observatory, historical practices at an
observatory, and reporting requirements to outside agencies. Despite this
diversity, there is a common set of best practices that guide our work. In
this article, we provided examples and guidelines to construct and maintain
robust and valuable observatory bibliographies. We also emphasized the
importance of documenting, evaluating, continually refining, and sharing our
bibliography practices with other bibliographers and the entire astronomical
community.

This research has made use of NASA’s Astrophysics Data System Bibliographic
Services.

## References

  * Blecksmith et al. (2005) Blecksmith, S., Bright, J., Rots, A. H., et al. 2005, in Astronomical Society of the Pacific Conference Series, Vol. 347, Astronomical Data Analysis Software and Systems XIV, ed. P. Shopbell, M. Britton, & R. Ebert, 380
  * Chen et al. (2022a) Chen, T. X., Ebert, R., Mazzarella, J. M., et al. 2022a, PASP, 134, 014501, doi: 10.1088/1538-3873/ac3c36
  * Chen et al. (2022b) Chen, T. X., Schmitz, M., Mazzarella, J. M., et al. 2022b, ApJS, 260, 5, doi: 10.3847/1538-4365/ac6268
  * Chyla et al. (2015) Chyla, R., Accomazzi, A., Holachek, A., et al. 2015, in Astronomical Society of the Pacific Conference Series, Vol. 495, Astronomical Data Analysis Software an Systems XXIV (ADASS XXIV), ed. A. R. Taylor & E. Rosolowsky, 401, doi: 10.48550/arXiv.1503.05881
  * Cutri et al. (2003) Cutri, R. M., Skrutskie, M. F., van Dyk, S., et al. 2003, 2MASS All Sky Catalog of point sources.
  * Eichhorn (2004) Eichhorn, G. 2004, Astronomy and Geophysics, 45, 3.07, doi: 10.1046/j.1468-4004.2003.45307.x
  * Grothkopf et al. (2018) Grothkopf, U., Meakins, S., & Bordelon, D. 2018, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 10704, Observatory Operations: Strategies, Processes, and Systems VII, 107040R, doi: 10.1117/12.2311667
  * Henneken & Kurtz (2019) Henneken, E., & Kurtz, M. J. 2019, in American Astronomical Society Meeting Abstracts, Vol. 233, American Astronomical Society Meeting Abstracts #233, 453.01
  * Lagerstrom (2015) Lagerstrom, J. 2015, in Astronomical Society of the Pacific Conference Series, Vol. 492, Open Science at the Frontiers of Librarianship, ed. A. Holl, S. Lesteven, D. Dietrich, & A. Gasperini, 99
  * Rots et al. (2012) Rots, A. H., Winkelman, S. L., & Becker, G. E. 2012, PASP, 124, 391, doi: 10.1086/665581
  * Scire et al. (2010) Scire, E., Chan, B. H. P., Silbermann, N., & Shields, A. 2010, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 7737, Observatory Operations: Strategies, Processes, and Systems III, ed. D. R. Silva, A. B. Peck, & B. T. Soifer, 77371V, doi: 10.1117/12.857735
  * Scire et al. (2022) Scire, E., Rebull, L., & Laine, S. 2022, PASP, 134, 055001, doi: 10.1088/1538-3873/ac4959
  * Skrutskie et al. (2006) Skrutskie, M. F., Cutri, R. M., Stiening, R., et al. 2006, AJ, 131, 1163, doi: 10.1086/498708

##  Appendix A Most Commonly Represented Journals for Astronomy Bibliographies

HST(1991-2021) | ESO(1996-present) | NRAO(1957-present) | Spitzer(2003-2020) Mid and far IR | NSF’s NOIRLab
---|---|---|---|---
ApJ 43% | A&A 46% | ApJ 42.13% | ApJ 43.4% | MNRAS 35%
MNRAS 19% | MNRAS 23% | MNRAS 17.68% | MNRAS 19.5% | ApJ 26%
A&A 14% | ApJ 16% | A&A 17.67% | A&A 19.0% | AJ 14%
AJ 10% | ApJL 5% | AJ 8.62% | AJ 6.4% | A&A 10%
ApJS 2% | AJ 4% | ApJS 2.65% | ApJS 4.1% | ApJL 4%
Icar 2% | A&ASuppl 1% | Nature 1.6% | PASJ 1.1% | ApJS 4%
PASP 1% | ApJS 1% | PASJ 0.94% | PASP 0.8% | PhRvD 1%
Nature 1% | Nature 1% | Science 0.68% | Nature and NatAs 1.1% | PSJ 1%
| Icar 1% |  | Icar 0.6% |
| AN 1% |  | AN, Ap&SS, Science 0.2% each |

The share of Spitzer papers published in the ApJ dropped from 56.1% in 2009 to
32.2% in 2020, while the share published in MNRAS rose from 10.4% to 28.0%.
This was due to changes in the funding profiles for observers granted time on
the observatory. See (Scire et al., 2022) for more information.

##  Appendix B Further Reading on Observatory Bibliographies

  1. 1.

Abt, H. A. 2005, Information Obtainable from Bibliometric Studies, coas.conf,
2, 2005coas.conf….2A

  2. 2.

Accomazzi, A., Eichhorn, G. 2004, Publishing Links to Astronomical Data On-
line, ASPC, 314, 181, 2004ASPC..314..181A

  3. 3.

Accomazzi, A., et al. 2012, Telescope bibliographies: an essential component
of archival data management and operations, SPIE, 8448, 84480K,
2012SPIE.8448E..0KA

  4. 4.

Alonso-Valdivielso, M. Á., Antonio, E. G. 2010, Why Include Bibliometric
Analysis in the Activities of a Library Specialized in Astronomy? — Notes From
the Libraries of INTA, ASPC, 433, 95, 2010ASPC..433…95A

  5. 5.

Bordelon, D., et al. 2016, Trends and developments in VLT data papers as seen
through telbib, SPIE, 9910, 99102B, 2016SPIE.9910E..2BB

  6. 6.

Cortes, R., Depoortere, D., Malaver, L. 2018, Astronomy in Chile: Assessment
of scientific productivity through a bibliometric analysis, EPJWC, 186, 05002,
2018EPJWC.18605002C

  7. 7.

Crabtree, D. 2019, Canada’s astronomy performance based on bibliometrics,
clrp, 2020, 14, 2019clrp.2020…14C

  8. 8.

Erdmann, C., Grothkopf, U. 2010, Next Generation Bibliometrics and the
Evolution of the ESO Telescope Bibliography, ASPC, 433, 81, 2010ASPC..433…81E

  9. 9.

Grothkopf, U., Meakins, S. 2012, ESO Telescope Bibliography: New Public
Interface, Msngr, 147, 41, 2012Msngr.147…41G

  10. 10.

Grothkopf, U., Meakins, S. 2015, ESO telbib: Linking In and Reaching Out,
ASPC, 492, 63, 2015ASPC..492…63G

  11. 11.

Grothkopf, U., et al. 2004, The ESO Telescope Bibliography on the Web: Linking
Publications and Observations, AAS, 205, 182.06, 2004AAS…20518206G

  12. 12.

Grothkopf, U. 2011, Astronomy Libraries - your Gateway to Information, EAS,
49, 107, 2011EAS….49..107G

  13. 13.

Grothkopf, U., Meakins, S. 2012, The ESO Telescope bibliography at your
fingertips, SPIE, 8448, 844821,  2012SPIE.8448E..21G

  14. 14.

Grothkopf, U., Novacescu, J. 2022, The Role of Astronomy Librarians in FAIR
Bibliography Curation and Metric Analyses, astr.conf, 3, 2022astr.confE…3G

  15. 15.

Grothkopf, U., Meakins, S., Bordelon, D. 2015, If we build it, will they come?
Curation and use of the ESO telescope bibliography, scop.conf, 1,
2015scop.confE..26G

  16. 16.

Grothkopf, U., Lagerstrom, J. 2011, Telescope Bibliometrics 101, ASSP, 1, 109,
2011ASSP…24..109G

  17. 17.

Grothkopf, U., Meakins, S., Bordelon, D. 2018, Use Cases of the ESO Telescope
Bibliography, EPJWC, 186, 06001, 2018EPJWC.18606001G

  18. 18.

Henneken, E. A., Kurtz, M. J. 2019, Usage Bibliometrics as a Tool to Measure
Research Activity, hsti.book, 819, 2019hsti.book..819H

  19. 19.

Hourclé, J. 2014, Data Citation in Astronomy, lisa.conf, 21,
2014lisa.confP..21H

  20. 20.

Kitt, S., Grothkopf, U. 2010, Telescope Bibliography Cookbook: Creating a
Database of Scientific Papers that Use Observational Data, ASPC, 433, 109,
2010ASPC..433..109K

  21. 21.

Lagerstrom, J., et al. 2012, Observatory bibliographies: current practices,
SPIE, 8448, 84481X, 2012SPIE.8448E..1XL

  22. 22.

Madrid, J. P., Macchetto, F. D. 2007, A Method to Measure the Scientific
Output of the Hubble Space Telescope, ASPC, 377, 79, 2007ASPC..377…79M

  23. 23.

Meakins, S., Grothkopf, U. 2012, Linking Publications and Observations: The
ESO Telescope Bibliography, ASPC, 461, 767, 2012ASPC..461..767M

  24. 24.

Meakins, S., et al. 2014, Two years of ALMA bibliography: lessons learned,
SPIE, 9149, 914926, 2014SPIE.9149E..26M

  25. 25.

National Research Council. 2012. For Attribution: Developing Data Attribution
and Citation Practices and Standards: Summary of an International Workshop.
Washington, DC: The National Academies Press DOI:10.17226/13564

  26. 26.

Perret, E., et al. 2018, Shared nomenclature and identifiers for telescopes
and instruments, EPJWC, 186, 04002, 2018EPJWC.18604002P

  27. 27.

Rots, A. H., Winkelman, S. L., Becker, G. E. 2012, Meaningful metrics for
observatory publication statistics, SPIE, 8448, 84480J, 2012SPIE.8448E..0JR

  28. 28.

Savaglio, S., Grothkopf, U. 2014, Swift Publication Statistics and the
Comparison with Other Major Observatories, lisa.conf, 4, 2014lisa.confP…4S

  29. 29.

Schmitz, M., et al. 1995, A Uniform Bibliographic Code, VA, 39, 272,
1995VA…..39R.272S

  30. 30.

Sterzik, M., et al. 2016, The impact of science operations on science return
at the Very Large Telescope, SPIE, 9910, 991003, 2016SPIE.9910E..03S

  31. 31.

Stevens-Rayburn, S., Grothkopf, U. 2007, Creating Telescope Bibliographies
Electronically – Are We There Yet?, ASPC, 377, 53, 2007ASPC..377…53S

  32. 32.

Winkelman, S., Rots, A. 2012, Observatory bibliographies: not just for
statistics anymore, SPIE, 8448, 844829, 2012SPIE.8448E..29W

  33. 33.

Wynholds, L. 2011, Linking to Scientific Data: Identity Problems of Unruly and
Poorly Bounded Digital Objects, International Journal of Digital Curation, 6,
214 DOI:10.2218/ijdc.v6i1.183

##  Appendix C Abbreviations & Acronyms

2MASS: Two Micron All Sky Survey
ADS: Astrophysics Data System
AI: artificial intelligence
AIP: American Institute of Physics
AJ: Astronomical Journal
Ap&SS: Astrophysics and Space Sciences
API: application program interface
ApJ: The Astrophysical Journal
ApJL: The Astrophysical Journal Letters
ApJS: The Astrophysical Journal Supplement
ASP: Astronomical Society of the Pacific
ATST: Advanced Technology Solar Telescope
AUI: Associated Universities Inc.
AURA: Association of Universities for Research Astronomy, Inc.
AXAF: Advanced X-ray Astrophysics Facility, previous name of Chandra
bibcode: bibliographic code [ADS]
bibgroup: Bibliographic group [ADS]
CfA: Center for Astrophysics — Harvard & Smithsonian
CLASH: HST Legacy Program
COS: Cosmic Origins Spectrograph [HST]
COSMOS: Cosmological Evolution Survey; HST Legacy Program
CSP: Chandra Science Paper
CTIO: Cerro Tololo Inter-American Observatory
CXC: Chandra X-ray Center
DKIST: Daniel K. Inouye Solar Telescope
DMSO: Data Science Mission Office [STScI]
DOI: digital object identifier
ELT: Extremely Large Telescope
ESO: European Southern Observatory
FTE: full-time equivalent
GALEX: Galaxy Evolution Explorer
GLASS: HST Legacy Program
GOALS: Legacy Spitzer Space Telescope program
GTO: Guaranteed Time Observations
GUI: Graphical User Interface
HST: Hubble Space Telescope
IAC: Instituto de Astrofísica de Canarias
IAU: International Astronomical Union
ID: identifier
IRAC: Infrared Array Camera; Spitzer Space Telescope instrument
IRAS: Infrared Astronomical Satellite
IRS: Infrared Spectrograph; Spitzer Space Telescope instrument
IRSA: Infrared Science Archive
IT: Information technology
IUE: International Ultraviolet Explorer
JSON: JavaScript Object Notation
JWST: James Webb Space Telescope, formerly Next Generation Space Telescope
(NGST)
MAST: Mikulski Archive for Space Telescopes [HST]
MIPS: Multiband Imaging Photometer; Spitzer Space Telescope instrument
ML: machine learning
MNRAS: Monthly Notices of the Royal Astrophysical Society
NAOJ: National Astronomical Institute of Japan
NASA: National Aeronautics and Space Administration
NatAs: Nature Astronomy
NED: NASA/IPAC Extragalactic Database
NEOWISE: NASA mission
NGRST: Nancy Grace Roman Space Telescope, formerly Wide Field Infrared
Telescope (WFIRST)
NGST: Next Generation Space Telescope, former name of the James Webb Space
Telescope (JWST)
NIRSpec: Near Infrared Spectrograph [JWST]
NOAO: National Optical Astronomy Observatory
NSF’s NOIRLab / NOIRLab: NSF’s National Optical-Infrared Astronomy Research
Laboratory
NRAO: National Radio Astronomy Observatory
OBC: Observatory Bibliographers Collaboration
PASJ: Publications of the Astronomical Society of Japan [journal]
PASP: Publications of the Astronomical Society of the Pacific
PI: principal investigator
postdoc: postdoctoral researcher
RTS: Roman Space Telescope, abbreviated acronym for the Nancy Grace Roman
Space Telescope, formerly the Wide Field Infrared Survey Telescope (WFIRST)
SAO: Smithsonian Astrophysical Observatory
SciX: NASA Science Explorer
SIMPLE: Spitzer IRAC/MUSYC Public Legacy Survey in the Extended Chandra Deep
Field South; legacy Spitzer Space Telescope program
SMD: Science Mission Directorate
SPIE: Society of Photographic Instrumentation Engineers
STScI: Space Telescope Science Institute
TESS: Transiting Exoplanet Survey Satellite
UI: user interface
VLT: Very Large Telescope [ESO]
WFIRST: Wide Field Infrared Survey Telescope (former name of Nancy Grace Roman
Space Telescope, NGRST or RST)
WISE: Wide-field Infrared Survey Explorer