# An Alternative Issue Tracking Dataset of Public Jira Repositories

Lloyd Montgomery<EMAIL_ADDRESS>0000-0002-8249-1418
University of HamburgHamburgGermany ,  Clara Lüders  clara.marie.lueders@uni-
hamburg.de 0000-0001-7743-4067 University of HamburgHamburgGermany and  Walid
Maalej<EMAIL_ADDRESS>0000-0002-6899-4393 University of
HamburgHamburgGermany

###### Abstract.

Organisations use issue tracking systems (ITSs) to track and document their
projects’ work in units called issues. This style of documentation encourages
evolutionary refinement, as each issue can be independently improved,
commented on, linked to other issues, and progressed through the
organisational workflow. Commonly studied ITSs so far include GitHub, GitLab,
and Bugzilla, while Jira, one of the most popular ITS in practice with a
wealth of additional information, has yet to receive such attention.
Unfortunately, diverse public Jira datasets are rare, likely due to the
difficulty in finding and accessing these repositories. With this paper, we
release a dataset of 16 public Jiras with 1822 projects, spanning 2.7 million
issues with a combined total of 32 million changes, 9 million comments, and 1
million issue links. We believe this Jira dataset will lead to many fruitful
research projects investigating issue evolution, issue linking, cross-project
analysis, as well as cross-tool analysis when combined with existing well-
studied ITS datasets.

††conference: MSR ’22: Proceedings of the 19th International Conference on
Mining Software Repositories; May 23–24, 2022; Pittsburgh, PA, USA

##  1\. Introduction

Table 1. Dataset consisting of 16 public Jira repositories Column names:
Documented Issue Types (DIT); Used Issue Types (UIT); Documented Link Types
(DLT); Used Link Types (ULT); Changes per Issue (Ch/I); Comments per Issue
(Co/I); Unique Projects (UP)
---
Jira repo | Born | Issues | DIT | UIT | Links | DLT | ULT | Ch/I | Co/I | UP
Apache | 2000 | 1,014,926 | 48 | 49 | 264,108 | 20 | 20 | 10 | 5 | 657
Hyperledger | 2016 | 28,146 | 9 | 9 | 16,846 | 6 | 6 | 12 | 2 | 36
IntelDAOS | 2016 | 9,474 | 4 | 11 | 2,667 | 12 | 12 | 15 | 3 | 7
JFrog | 2006 | 15,535 | 30 | 22 | 3,303 | 17 | 10 | 9 | 1 | 35
JIRA* | 2002 | 274,545 | 52 | 38 | 110,507 | 19 | 18 | 21 | 3 | 123
JiraEcosystem | 2004 | 41,866 | 122 | 40 | 12,439 | 19 | 18 | 15 | 2 | 153
MariaDB | 2009 | 31,229 | 11 | 10 | 14,950 | N/A | 6 | 12 | N/A | 24
Mindville | 2015 | 2,134 | 2 | 2 | 46 | N/A | 4 | 3 | N/A | 10
Mojang | 2012 | 420,819 | 1 | 3 | 215,821 | 6 | 5 | 8 | 2 | 16
MongoDB | 2009 | 137,172 | 31 | 37 | 92,368 | 15 | 13 | 17 | 3 | 95
Qt | 2005 | 148,579 | 19 | 14 | 41,426 | 10 | 10 | 12 | 3 | 38
RedHat | 2001 | 353,000 | 74 | 55 | 163,085 | 23 | 19 | 13 | 2 | 472
Sakai | 2004 | 50,550 | 43 | 15 | 20,292 | 7 | 7 | 10 | 4 | 55
SecondLife | 2007 | 1,867 | 10 | 12 | 674 | 9 | 5 | 30 | 8 | 3
Sonatype | 2008 | 87,284 | 16 | 21 | 4,975 | 10 | 9 | 7 | 4 | 17
Spring | 2003 | 69,156 | 13 | 14 | 14,716 | 7 | 7 | 8 | 3 | 81
Sum |  | 2,686,282 | 485 | 352 | 978,223 | 180 | 169 |  |  | 1822
Median |  | 59,853 | 18 | 14 | 15,898 | 10 | 10 | 12 | 3 | 37
Std Dev |  | 251,973 | 31 | 16 | 81,744 | 6 | 5 | 6 | 2 | 178
* Note that our data source tool name is “Jira” while the organisation “JIRA” is written in all capitals to help distinguish the two throughout the article.

Software projects use Issue Tracking Systems (ITSs) to collect and track
issues that need to be addressed such as bug reports and feature requests.
Issues are units of information usually including a summary (title), a
description, and a number of properties like status, priority, and fix
version. A key focus of ITSs is the evolutionary refinement of the issues
(Ernst and Murphy, 2012) (also known as iterative improvement), which means
that information is gained and refined over time, while developers and
stakeholders collaborate to address the issues.

Over the last two decades, software engineering research has intensively
studied issues and issue trackers, often based on
Bugzilla111https://www.bugzilla.org and GitHub222https://github.com. The
primary research focus has been on the specific issue type of bug reports: the
understanding and improvement of information quality therein (Bettenburg et
al., 2008; Zimmermann et al., 2010) and the prediction of bug properties such
as severity (Lamkanfi et al., 2010, 2011), assignee (Jeong et al., 2009), and
duplicate reports (Wang et al., 2008; Deshmukh et al., 2017; He et al., 2020)
for supporting software evolution and maintenance.

Another prominent but rather understudied issue tracker is Jira.
Jira333https://www.atlassian.com/software/jira is an agile planning platform
that offers features such as scrum boards, kanban boards, and roadmap
management, beyond issue tracking. Jira’s ticket-centric design mimics that of
GitHub444https://docs.github.com/en/issues,
GitLab555https://docs.gitlab.com/ee/user/project/issues/, and Bugzilla.
Benefits of Jira over other ITSs includes a history of issue changes, complex
issue linking networks, and a diverse set of custom field configurations
across organisations. According to slintel666www.slintel.com/tech/bug-and-
issue-tracking/atlassian-jira-market-share,
Datalyze777www.datanyze.com/market-share/project-management--217/jira-market-
share, and Enlyft888https://enlyft.com/tech/products/atlassian-jira, Jira is
by far the most popular tool in the issue tracking and agile project
management markets. Yet it is rather under-represented in software engineering
research.999Ortu et al. released a dataset of four Jira repos in 2015(Ortu et
al., 2015).

We collected data from 16 pubic Jira repositories (repos) containing 1822
projects and 2.7 million issues. Our dataset includes historical records of 32
million changes, 9 million comments, and 1 million issue links that connect
the issues in multiple ways. Table 1 shows an overview of the data. The table
lists for each Jira repo the year it first was used, the number of issues,
number of documented and used issue types, number of documented and used link
types, number of changes per issue, number of comments per issue, and number
of unique projects.101010Documented issue types may be removed over time, but
their use is still recorded in the data, leading to a lower number of
documented issue types than used issue types. Apache is the largest repo with
1 million issues and 10.5 million changes. Mindville and SecondLife are the
smallest repos with $\scriptstyle\mathtt{\sim}$2,000 issues each. The number
of unique projects is particularly important because each Jira repo contains
documentation for multiple projects, e.g. 657 projects for Apache. While our
dataset has 16 different repositories, it represents 18220 actual projects
that can be studied. Our data and code is publicly available with an open
source license.111111https://doi.org/10.5281/zenodo.5882881

Each Jira repo has a number of documented issue types, but only a subset of
them are actually used in the projects. As shown in Table 1, 485 types are
defined while only 352 are used. Notably, the median number of used issue
types is 14, which is significantly higher than expected from research
discussing ITSs (bug reports (Bettenburg et al., 2008; Zimmermann et al.,
2010), feature requests (Fitzgerald et al., [n. d.]; Merten et al., 2016),
tasks (Ernst and Murphy, 2012), or technical debt (Bavota and Russo, 2016;
Xavier et al., 2020)). The projects also vary in terms of the number of
documented and used link types, totally 186 and 169, respectively. The median
number of used link types is 10 which is considerably more than the mainly
well-researched “duplicate”, or “depends/relates” types.

Figure 1. Jira MongoDB database scheme

##  2\. Dataset Description

The Jira data is stored in MongoDB as document data. Figure 1 describes the
data as an entity relationship diagram (ERD). Given the nature of document-
oriented databases, generating an ERD is not straightforward and indeed Fig. 1
is a simplification of the real data structure. We have diagrammed the key
objects in the data, simplified structures where the complexity is self-
explanatory, and extrapolated the data types to give an idea of what to
expect. The primary documents to expect in our dataset are “issues”, each unit
of which represents a single issue within Jira. The two primary documents
nested within each issue are “fields” and “changelog”. The fields document
contains the current attributes of the issue, and is described in Table 2. The
issue field descriptions are primarily from the Jira official documentation
121212https://confluence.atlassian.com/adminjiraserver/issue-fields-and-
statuses-938847116.html. The “changelog” stores the changes that have occurred
to this issue. When an issue is changed (saved), a new “history” is saved,
where each changed attribute is stored as an “item”. “Issuelinks” connect
issues to each other, and are stored on both linked issues. “Comments” are
made by the community and form a discussion around the issue.

Table 2. Issue fields Issue Field |  Description
---|---
|  Content
summary |  A brief one-line summary of the issue.
description |  A detailed description of the issue.
comments |  Community discussion on each issue.
|  Context & Links
issuetype |  The issue purpose within the organisation.
project |  The parent project to which the issue belongs.
components |  Project component(s) to which this issue relates.
labels |  Labels to which this issue relates.
isuelinks |  A list of links to related issues.
subtasks |  Sub-issues to this issue; can only be one level deep.
|  Workflow
fixVersions |  Project version in which the issue was (or will be) fixed.
status |  The stage the issue is currently at in its lifecycle.
priority |  The issue importance in relation to other issues.
resolution |  A record of the issue’s resolution, once resolved or closed.
timeestimate |  Estimated amount of time required to resolve the issue.
timespent |  Amount of time spent working on this issue.
|  People & Community
creator |  The person who created the issue.
reporter |  The person who found/reported the issue. Defaults to the creator unless otherwise assigned.
assignee |  The person responsible to resolve the issue.
votes |  Number of people who want this issue addressed.
watches |  Number of people watching this issue.
|  Timestamps
created |  Time and date this issue was entered into Jira.
updated |  Time and date this issue was last edited.
resolutionedate |  Time and date this issue was resolved.
duedate |  Time and date this issue is scheduled to be completed.

The dataset is available on Zenodo as a MongoDB dump file. The raw data
includes a README file, the scripts used to download the data, the scripts
used to produce the tables and figures in this paper, and a license (CC BY
4.0131313https://creativecommons.org/licenses/by/4.0/). In principle, using
this dataset is as easy as importing the MongoDB dump file and modifying the
included JupyterLab notebook to explore the data. The data can also be queried
using standard MongoDB queries. Figure 1 can be used to guide the initial
exploration; however, the figure is an intentional simplification of the data
for explanation purposes. The full details of the data may add complexity to
how it is used.141414For example, issuelinks do not have an actual database
link to the corresponding linked issue, but rather a stripped-down summary of
the linked issue.

The data was downloaded using a Python script utilising the Jira API exposed
by public Jira ITSs. We obtained the list of repos through a manual search for
public Jira repos on the internet. This search involved reviewing
GoogleScholar results for “Jira”, as well as Google search results discussing
a public Jira repo. With the public Jira URLs, the data was downloaded using
the Jira REST API
V2151515https://developer.atlassian.com/cloud/jira/platform/rest/v2 and a
Python script contained within a JupyterLab notebook. In total,
$\scriptstyle\mathtt{\sim}$15GB of data was downloaded and stored in the
MongoDB database. The initial data download was performed in May 2021 and the
data was updated in January 2022. Unfortunately, MariaDB and Mindville are no
longer valid data sources, so their data is from the initial download in May
2021.

##  3\. Enhancing the Dataset

To enrich the dataset and support the comparison of the repositories, we
enhanced the dataset through the addition of manual qualitative labelling of
the issue types and link types.

###  3.1. Issue Type Mapping

Jira is a highly customisable tool, where each organisations defines specific
aspects such the issue types they use. The issue type indicates the general
organisational category the issue belongs to, and likely has impact on the
organisational workflow and subsequent research analysis. Given the 352 unique
issue types (see Table 1), it is unclear which issue types are offered by each
Jira, how to group them, and then how to unify them across the Jiras for more
general analyses. To address this, we mapped the unique issue types to
software life-cycle activities they represent.

We conducted a thematic analysis (Braun and Clarke, 2006; Cruzes and Dyba,
2011) of all issue types across all 16 Jira repos to provide a unified mapping
between the issue types.161616The thematic analysis was performed using the
May 2021 data. We followed the recommendations by Braun and Clarke (Braun and
Clarke, 2006) and Cruzes and Dyba (Cruzes and Dyba, 2011), including 5 main
analysis steps. First, we familiarised ourselves with the data by looking at
the issue type descriptions in the repos and 50 example issues for each used
type. Second, we merged issue types with similar names and meaning, and
identified initial phrases or codes describing the types. Third, we iterated
through the codes and example issues and identified an initial set of themes.
Fourth, we reviewed and reduced the themes during two peer-discussion
sessions. Finally, we finalised our themes by forming definitions for each.

Our thematic analysis produced a mapping between each issue type in each Jira,
and a unified set of themes and codes.171717The codes are available in our
replication package and will not be discussed in this paper. The themes (and
issue proportions) are Requirements (30%), Development (18%), Maintenance
(51%), User Support (1%), and Other (¡1%). This mapping (stored as a JSON)
allows one to programmatically ask questions such as “give me the
‘Requirements’ issues for Apache”.

Here we describe the five themes in more detail. Requirements activities are
documented in ITSs through the use lightweight representations (Ernst and
Murphy, 2012) such as epics, stories, and feature requests. Both top-down as
well as bottom-up requirements are captured in the ITSs. We found top-down
epics and stories, as well as bottom-up feature requests and change requests.
This shows the breadth of requirements knowledge maintained within ITSs.
Development issue types track development activities in the ITSs such as what
needs to be done, who is doing it, and when the work is completed. Maintenance
issue types correspond to maintenance activities documented within ITSs
through both bottom-up (Bug Reports) as well as top-down (Technical Debt,
Documentation) work. This includes quality assurance issues, legacy upgrades,
and continuous integration activities. User Support issues directly assist
users of software systems with respect to how to use the product. While users
can submit Bug Reports to the Maintenance theme above, Bug Reports focus on
issues with the code or product, while issues submitted to the User Support
theme are focused on issues with the user and their use or interpretation of
the code or product. Other issue types are not representative across the Jira
repos, or could not be categorised at all. Only two Jira repos contained
organisational process issues for managing their Jira, website, and other
organisational activities. Additionally, many issue types were lacking enough
examples to understand what they were documenting.

Table 3. Overview of cleaned link types, their description, number of Jiras with that link type, and their mapped categories. Link Type |  Description | Jiras
---|---|---
Relate |  Two issues are related to each other in a general way or stakeholders are unsure about the nature of the connection. | 16
Duplicate |  An issue is already in the system and someone added the same issue by accident. | 15
Subtask |  An issue is a part of a larger set of issues. | 14
Clone |  An issue was duplicated intentionally by the users, e.g., when an old bug resurfaces in a newer version. | 13
Depend |  An issue cannot be resolved until another issue is resolved. | 10
Epic |  A link that consists of higher-level issue which has multiple low-level issues attached to it, similar to Parent-Child or Incorporate. | 9
Block |  An issue cannot be resolved until another issue is resolved. | 9
Incorporate |  An issue is included or contained in another issue. E.g., an issue is used to collect multiple other issues, similar to Parent-Child. | 8
Split |  An issue is split into multiple issues to reduce the size or complexity. | 8
Cause |  Issue A describes a problem that is at least a partial root cause for Issue B. | 7
Bonfire Testing |  An issue was discovered while testing another issue. | 7
finish-finish |  Two issues must be finished at the same time or patch. | 4
Supercede |  An issue was refined based on existing issues. | 4

###  3.2. Link Type Mapping

One of the main strengths of Jira consists of its ability to link issues and
form complex issue graphs. An “issue link” is configured as a relationship
between two issues, where the relationship has a name and a description.
Examples of issue links include “duplicate”, “clone”, and “depend.” Issue
links are an important part of Jira and issue trackers in general. More
research is needed to understand how issue linking works, what effect links
have on the issue tracking ecosystems, and how they can be leveraged to create
more productive and efficient development environments. However, only the
“duplicate” issue links have been studied in great detail, while our dataset
reveals that there are obviously many more to be explored. One potential
problem with exploring these issue types, is the sheer number of link types
that exist, and the lack of uniformity across different Jira repos. To address
this issue, we performed a qualitative labelling process similarly to the
issue types to unify the 75 different link types across the 16 Jira repos.

To create a unified set of link types, we first grouped the link types
together based on types with similar word stems (e.g. “finish-finish [gantt]”,
“gantt: finish-finish” or “Depend”, “Dependency”, “Dependent”, “Depends”).
Then, we compared the link types by their meaning. We looked up random
examples in the respective ITS and reviewed the texts and properties of the
involved issues as well as the language used to describe the link. For
instance, we saw that 9 projects had a different link type name for
“Incorporate”, but all 8 used “includes” or synonyms such as “contains” or
“incorporates”. After this step, we arrived at 30 distinct link types. We list
the link types that appeared in at least 25% of the repos in Table 3 with
brief descriptions, and the count of repos where they are used.

##  4\. Relevance of the Dataset

Studying developers behaviour, organisational processes, and user-developer
interactions are common research areas, with work published at large venues
such as Mining Software Repositories (MSR), the International Conference on
Software Engineering (ICSE), and the International Conference on Requirements
Engineering (RE), as well as journals such as Transactions on Software
Engineering (TSE), Empirical Software Engineering (EMSE) and Requirements
Engineering (REJ). The breadth and depth of studies that are conducted rely on
access to real developers, real organisations, and real data. Our dataset
contributes to this research line of studying real organisational data:
including a diversity of issues, issue types, and link types across multiple
organisations, developers, and projects.

#### Triangulate results from studies on Bugzilla and GitHub Issues.

There are many studies investigating issues in Bugzilla and GitHub, but very
little work done to verify these findings across organisations and platforms.
Our dataset can act as an additional research source to triangulate the
findings of those past studies, adding further validity to issue tracking
research, such as “what makes a good bug report” (Bettenburg et al., 2008) and
“good first issues for newcomers” (Steinmacher et al., 2018). Our dataset is
not only fairly large and heterogeneous, but also brings an additional ITS
which is popular among practitioners.

#### Issue and link types comparative analyses.

Issue tracking systems contain much more than just the heavily studied bug
reports (Bettenburg et al., 2008; Zimmermann et al., 2010) and feature
requests (Heck and Zaidman, 2013). By our preliminary analysis, there are at
least four major categories: requirements, maintenance, development, and user
support. As a starting place, trying to reliably separate these issue types
(beyond just the stated issue type) is a much needed area of research. Herzig
et al. investigated issues labelled as bug reports and found that roughly
1/3rd are misclassified, leading to the conclusion that data needs to be
better classified (Herzig et al., 2013). Additionally, studying the
differences between these issue types, across organisations and tools, is
likely to reveal critical workflow differences key to running successful
software projects. Similarly, the link type “duplicate” has been studied in
great detail, while the full ecosystem of other link types largely remains
unexplored.

#### Issue evolution studies and support.

A key focus of ITSs (over traditional requirements) is the ongoing
evolutionary refinement of the issues (Ernst and Murphy, 2012), where
information is gained and refined over time as developers and stakeholders
collaborate to address the issues. This evolution has not been studied
directly, possibly because no such dataset existed.181818Heck and Zaidman
(Heck and Zaidman, 2013) investigated the requirements evolution in ITSs, but
to the best of our knowledge did not investigate the historical changes. This
Jira dataset provides the historical changes for each issue, allowing for a
direct investigation of issue evolution.

#### User support analysis.

Understanding and supporting users is an important area of software
engineering research. While bug reports have been thoroughly explored, and
feature requests are now receiving much more attention, the concept of
customer relationship management is a much broader topic. Customer
relationship management involves integrating artefacts, tools, and workflows
to successfully initiate, maintain, and (if necessary) terminate customer
relationships (Reinartz et al., 2004). Jira stores a number of user support
issue types, all of which combine together to tell a story about the users of
these software products. Possible areas of investigation include support
ticket escalation prediction (Montgomery and Damian, 2017), investigating
sentiment of customer and developer communication (Ortu et al., 2016), and
linking customer support issues to created issues in other parts of the
system.

##  5\. Challenges and Limitations

There were a number of challenges associated with creating this Jira dataset.
1. Unlike GitHub, GitLab, and Bugzilla, Jira does not have a centrally-hosted
public list of Jira repos. The software is designed to be hosted by individual
organisations, who largely opt for closed-access. For those who do opt for
public, there is no central place to find all of these repos, making them very
difficult to find. 2. While GitHub, GitLab, and Bugzilla are customisable
tools, Jira offers far more options for the customisation of issue types, link
types, workflows, automated scripts, custom fields, linking across Jiras and
to other issue trackers, etc. For this reason, the data is richer, but also
more complex and hard to unify. We have made a first attempt at unifying the
major components by starting with the issue and link types. 3. Due to the
complexity and diversity of Jira repos, mitigating construct validity while
generalising across Jiras involves deep investigations of individual Jiras and
their documentation. We have included such analysis in this work, offering a
starting place for future researchers. 4. Downloading this data took weeks,
and updating the data presented inconsistencies such as Jira repos no longer
existing. The MariaDB and Mindville data is from May 2021 and thus is missing
the comments entirely.

Future improvements to the data can certainly be made. This includes
additional qualitative labelling, cleaning based on use cases, and finding
more Jira API hooks worth pulling data from to add to the breadth of data
available for each repo.

##  6\. Summary

We provide a dataset of public Jira repos with the scripts to re-download the
data, the data itself, scripts for an initial investigation, and qualitative
labelling to enhance the research potential. In total, there are 16 Jira
repos, with 1822 projects, consisting of 2.7 million issues, with 32 million
changes, 9 million comments, and 1 million issue links. We believe this data,
in combination with our qualitative labelling enhancements, serves as an
alternative data source for investigating issue tracking systems in software
engineering practice.

## References

  * (1)
  * Bavota and Russo (2016) Gabriele Bavota and Barbara Russo. 2016.  A large-scale empirical study on self-admitted technical debt. In _Proceedings of the 13th international conference on mining software repositories_. 315–326.
  * Bettenburg et al. (2008) Nicolas Bettenburg, Sascha Just, Adrian Schröter, Cathrin Weiss, Rahul Premraj, and Thomas Zimmermann. 2008.  What makes a good bug report?. In _Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering_. 308–318.
  * Braun and Clarke (2006) Virginia Braun and Victoria Clarke. 2006.  Using thematic analysis in psychology.  _Qualitative research in psychology_ 3, 2 (2006), 77–101.
  * Cruzes and Dyba (2011) D. S. Cruzes and T. Dyba. 2011.  Recommended Steps for Thematic Synthesis in Software Engineering. In _2011 International Symposium on Empirical Software Engineering and Measurement_. 275–284.  https://doi.org/10.1109/ESEM.2011.36
  * Deshmukh et al. (2017) J. Deshmukh, K. M. Annervaz, S. Podder, S. Sengupta, and N. Dubash. 2017.  Towards Accurate Duplicate Bug Retrieval Using Deep Learning Techniques. In _2017 IEEE International Conference on Software Maintenance and Evolution (ICSME)_. 115–124.
  * Ernst and Murphy (2012) N. A. Ernst and G. C. Murphy. 2012.  Case Studies in Just-in-Time Requirements Analysis. In _2012 Second IEEE International Workshop on Empirical Requirements Engineering (EmpiRE)_. IEEE, Chicago, IL, USA, 25–32.  https://doi.org/10.1109/EmpiRE.2012.6347678
  * Fitzgerald et al. ([n. d.]) Camilo Fitzgerald, Emmanuel Letier, and Anthony Finkelstein. [n. d.].  Early failure prediction in feature request management systems. In _2011 IEEE 19th International Requirements Engineering Conference_. IEEE, 229–238.
  * He et al. (2020) Jianjun He, Ling Xu, Meng Yan, Xin Xia, and Yan Lei. 2020.  Duplicate Bug Report Detection Using Dual-Channel Convolutional Neural Networks. In _Proceedings of the 28th International Conference on Program Comprehension_ (Seoul, Republic of Korea) _(ICPC ’20)_. Association for Computing Machinery, New York, NY, USA, 117–127.  https://doi.org/10.1145/3387904.3389263
  * Heck and Zaidman (2013) Petra Heck and Andy Zaidman. 2013.  An Analysis of Requirements Evolution in Open Source Projects: Recommendations for Issue Trackers. In _Proceedings of the 2013 International Workshop on Principles of Software Evolution_ _(IWPSE 2013)_. Association for Computing Machinery, New York, NY, USA, 43–52.  https://doi.org/10.1145/2501543.2501550
  * Herzig et al. (2013) Kim Herzig, Sascha Just, and Andreas Zeller. 2013.  It’s not a bug, it’s a feature: How misclassification impacts bug prediction. In _2013 35th International Conference on Software Engineering (ICSE)_. 392–401.  https://doi.org/10.1109/ICSE.2013.6606585
  * Jeong et al. (2009) Gaeul Jeong, Sunghun Kim, and Thomas Zimmermann. 2009.  Improving Bug Triage with Bug Tossing Graphs _(ESEC/FSE ’09)_. Association for Computing Machinery, New York, NY, USA, 111–120.  https://doi.org/10.1145/1595696.1595715
  * Lamkanfi et al. (2010) Ahmed Lamkanfi, Serge Demeyer, Emanuel Giger, and Bart Goethals. 2010.  Predicting the severity of a reported bug. In _2010 7th IEEE Working Conference on Mining Software Repositories (MSR 2010)_. 1–10.  https://doi.org/10.1109/MSR.2010.5463284
  * Lamkanfi et al. (2011) Ahmed Lamkanfi, Serge Demeyer, Quinten David Soetens, and Tim Verdonck. 2011.  Comparing mining algorithms for predicting the severity of a reported bug. In _2011 15th European Conference on Software Maintenance and Reengineering_. IEEE, 249–258.
  * Merten et al. (2016) Thorsten Merten, Matúš Falis, Paul Hübner, Thomas Quirchmayr, Simone Bürsner, and Barbara Paech. 2016\.  Software Feature Request Detection in Issue Tracking Systems. In _2016 IEEE 24th International Requirements Engineering Conference (RE)_. 166–175.  https://doi.org/10.1109/RE.2016.8
  * Montgomery and Damian (2017) Lloyd Montgomery and Daniela Damian. 2017.  What do support analysts know about their customers? on the study and prediction of support ticket escalations in large software organizations. In _2017 IEEE 25th international requirements engineering conference (RE)_. IEEE, 362–371.
  * Ortu et al. (2015) Marco Ortu, Giuseppe Destefanis, Bram Adams, Alessandro Murgia, Michele Marchesi, and Roberto Tonelli. 2015.  The jira repository dataset: Understanding social aspects of software development. In _Proceedings of the 11th international conference on predictive models and data analytics in software engineering_. 1–4.
  * Ortu et al. (2016) Marco Ortu, Alessandro Murgia, Giuseppe Destefanis, Parastou Tourani, Roberto Tonelli, Michele Marchesi, and Bram Adams. 2016\.  The emotional side of software developers in JIRA. In _2016 IEEE/ACM 13th Working Conference on Mining Software Repositories (MSR)_. IEEE, 480–483.
  * Reinartz et al. (2004) Werner Reinartz, Manfred Krafft, and Wayne D Hoyer. 2004\.  The customer relationship management process: Its measurement and impact on performance.  _Journal of marketing research_ 41, 3 (2004), 293–305.
  * Steinmacher et al. (2018) Igor Steinmacher, Christoph Treude, and Marco Aurelio Gerosa. 2018.  Let me in: Guidelines for the successful onboarding of newcomers to open source projects.  _IEEE Software_ 36, 4 (2018), 41–49.
  * Wang et al. (2008) Xiaoyin Wang, Lu Zhang, Tao Xie, John Anvik, and Jiasu Sun. 2008.  An Approach to Detecting Duplicate Bug Reports Using Natural Language and Execution Information. In _Proceedings of the 30th International Conference on Software Engineering_ (Leipzig, Germany) _(ICSE ’08)_. Association for Computing Machinery, New York, NY, USA, 461–470.  https://doi.org/10.1145/1368088.1368151
  * Xavier et al. (2020) Laerte Xavier, Fabio Ferreira, Rodrigo Brito, and Marco Tulio Valente. 2020.  Beyond the code: Mining self-admitted technical debt in issue tracker systems. In _Proceedings of the 17th International Conference on Mining Software Repositories_. 137–146.
  * Zimmermann et al. (2010) Thomas Zimmermann, Rahul Premraj, Nicolas Bettenburg, Sascha Just, Adrian Schröter, and Cathrin Weiss. 2010.  What Makes a Good Bug Report?  _IEEE Transactions on Software Engineering_ 36, 5 (2010), 618–643.  https://doi.org/10.1109/TSE.2010.63


* [21] R. Douc, A. Durmus, A. Enfroy, J. Olsson, Boost your favorite Markov Chain Monte Carlo sampler using Kac’s theorem: The Kick-Kac teleportation algorithm, arXiv:2201.05002 [cs, math, stat] (2022) 1–35arXiv:2201.05002.
  * [22] G. O. Roberts, R. L. Tweedie, Geometric convergence and central limit theorems for multidimensional hastings and metropolis algorithms, Biometrika 83 (1) (1996) 95–110.
  * [23] C. Andrieu, G. O. Roberts, The pseudo-marginal approach for efficient Monte Carlo computations, The Annals of Statistics 37 (2) (2009) 697–725.  doi:10.1214/07-AOS574.
  * [24] C. Andrieu, M. Vihola, Convergence properties of pseudo-marginal Markov chain Monte Carlo algorithms, The Annals of Applied Probability 25 (2) (2015) 1030–1077.  doi:10.1214/14-AAP1022.
  * [25] M. D. Hoffman, A. Gelman, The No-U-Turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo., J. Mach. Learn. Res. 15 (1) (2014) 1593–1623.
  * [26] C. Durkan, A. Bekasov, I. Murray, G. Papamakarios, Neural Spline Flows, in: Advances in Neural Information Processing Systems, Vol. 32, Curran Associates, Inc., 2019.
  * [27] M. Gabrié, G. M. Rotskoff, E. Vanden-Eijnden, Adaptive Monte Carlo augmented with normalizing flows, Proceedings of the National Academy of Sciences 119 (10) (2022) e2109420119.
  * [28] K. W. Wong, M. Gabrié, D. Foreman-Mackey, flowMC: Normalizing-flow enhanced sampling package for probabilistic inference in Jax, arXiv preprint arXiv:2211.06397 (2022) 1–3arXiv:2211.06397.
  * [29] L. Tierney, Markov chains for exploring posterior distributions, the Annals of Statistics (1994) 1701–1728doi:10.1214/aos/1176325750.
  * [30] R. Douc, E. Moulines, Limit theorems for weighted samples with applications to sequential Monte Carlo methods, The Annals of Statistics 36 (5) (2008) 101–107.  doi:10.1214/07-AOS514.


# Simple End-to-end Deep Learning Model for CDR-H3 Loop Structure Prediction

Natalia Zenkova
JetBrains Research
<EMAIL_ADDRESS>
&Ekaterina Sedykh
University of Tartu
<EMAIL_ADDRESS>
Tatiana Shugaeva
BIOCAD
<EMAIL_ADDRESS>
&Vladislav Strashko
BIOCAD
<EMAIL_ADDRESS>
&Timofei Ermak
BIOCAD
<EMAIL_ADDRESS>
&Aleksei Shpilman
HSE University
JetBrains Research
<EMAIL_ADDRESS>

###### Abstract

Predicting a structure of an antibody from its sequence is important since it
allows for a better design process of synthetic antibodies that play a vital
role in the health industry. Most of the structure of an antibody is
conservative. The most variable and hard-to-predict part is the third
complementarity-determining region of the antibody heavy chain (CDR H3).
Lately, deep learning has been employed to solve the task of CDR H3
prediction. However, current state-of-the-art methods are not end-to-end, but
rather they output inter-residue distances and orientations to the
RosettaAntibody package that uses this additional information alongside
statistical and physics-based methods to predict the 3D structure. This does
not allow a fast screening process and, therefore, inhibits the development of
targeted synthetic antibodies. In this work, we present an end-to-end model to
predict CDR H3 loop structure, that performs on par with state-of-the-art
methods in terms of accuracy but an order of magnitude faster. We also raise
an issue with a commonly used RosettaAntibody benchmark that leads to data
leaks, i.e., the presence of identical sequences in the train and test
datasets.

##  1 Introduction

Antibodies play a key role in adaptive immunity. Their ability to specifically
bind target molecules makes antibodies effective fighters against pathogens
[18] and one of the major classes of therapeutic molecules in the field of
anticancer therapy, autoimmune diseases, and more [5] [13]. Experimental
determination of the tertiary structure is expensive, therefore, computational
approaches are widely used nowadays [18]. Antibodies are composed of two types
of protein chains: heavy and light, presented in two copies. The light chain
contains variable (VL) and constant domains (CL), the heavy chain contains one
variable (VH) and three constant domains (CH1, CH2, CH3). The VH-VL pair forms
a variable antigen-binding Fv fragment, which in turn is subdivided into
conserved framework regions (FRs) and hypervariable complementarity-
determining regions (CDRs). CDRs are the most important regions for antigen
binding; they form six variable loops on the surface of the Fv fragment. Three
of them are located on the light chain and are named L1, L2, L3, respectively,
three others are located on the surface of the heavy chain and are designated
as H1, H2, and H3 [10].

Loops L1-L3 and H1-H2 typically fall into a limited set of conformations,
therefore, it is relatively easy to determine their 3D configuration [2]. In
contrast, the third loop of the heavy chain is the most variable and cannot be
described using the limited set of structural clusters [11]. Its high
variability does not allow the application of modern protein folding
techniques based on multiple sequence alignment (MSA) of homologous sequences,
such as AlphaFold2 [9] and RoseTTAFold [1]. Thus, CDR H3 structure prediction
requires special approaches.

Recently, in addition to statistical and physics-based [16] prediction
methods, deep learning models have been employed for the task of H3 structure
prediction. Current state-of-the-art solution DeepH3 uses deep learning to
calculate the distribution of inter-residue distances and orientations, which
are then passed to the RosettaAntibody package that employs both physics-based
and statistical methods. Because of that second step and the causing time
constraints, this method can not be feasibly used to perform a wide in silico
screening of possible antibody structures.

In this paper, we present a simple end-to-end deep learning method based on
ELMo embedding and forward and backward LSTM passes that directly output
coordinates of backbone atoms of amino acids in the H3 loop. This method, that
we named SimpleDH3, performs on on par with DeepH3 in terms of root mean
square deviation (RMSD) but is at least an order of magnitude faster.

##  2 Model

Our SimpleDH3 model pipeline consists of the following steps:

  1. 1.

Convert each amino acid from the input sequence into a 1024-dimensional
embedding representation using a context-dependent model ELMo (Embeddings from
Language Models) [12] pretrained on the UniRef50 dataset [17] from [6];

  2. 2.

Independent forward and backward LSTM [7] pass. LSTM cell with 256 neurons in
the hidden and the output layer receives an amino acid embedding from ELMo
model concatenated with the prediction coordinates for the previous amino
acid.

  3. 3.

Independent feedforward neural networks with 9-dimensional output after LSTM
cell. 9 dimensions correspond to the coordinates of $C$, $C_{\alpha}$ and $N$
atoms in a single amino acid that completely determine its position and
orientation.

  4. 4.

Averaging the output from forward and backward passes.

Figure 1: Architecture of our simple model (SimpleDH3). It takes the sequence
for CDR H3 and 2 additional amino acids: one on the each end. Each amino acid
from a sequence is transformed with pretrained ELMo model from [6].
Afterwards, the embeddings are concatenated with the prediction coordinates
for the previous amino acid are passed to the backward LSTM cell pass combined
with linear layer. Then independently forward LSTM cell pass combined with its
own linear layer. As a result, the model outputs backbone coordinates, i.e.,
for $C$, $C_{\alpha}$, and $N$ atoms, averaged from both passes.

It is important to note that SimpleDH3 model is trained only on CDR H3 loops
with two amino acids at the ends. In contrast to DeepH3 [14] and other methods
[15] which take Fv fragment, our model use only loop sequences.

The model was trained with the following loss function:

$\mathcal{L}=\lambda_{1}\mathcal{L}_{coords}+\lambda_{2}\mathcal{L}_{dist}+\lambda_{3}\mathcal{L}_{torsion},$

where

$\displaystyle\mathcal{L}_{coords}=MSE_{coords},\ \
\mathcal{L}_{dist}=MSE_{dist},$
$\displaystyle\mathcal{L}_{torsion}=MSE_{torsion},\ \ \lambda_{1}=0.2,\ \
\lambda_{2}=3.0,\ \ \lambda_{3}=10.0.$

$\mathcal{L}_{coords}$ is a mean-square deviation of $C$, $C_{\alpha}$ and $N$
atoms coordinates; $\mathcal{L}_{dist}$ is a mean-square deviation of distance
between two consecutive atoms, such as $(C;C_{\alpha})$, $(C_{\alpha};N)$ and
$(N;C)$; $\mathcal{L}_{torsion}$ is similar to other components of the loss
function, but considers torsion angles cosines instead.

##  3 Experiments and Results

###  3.1 Data collection and preprocessing

The training and testing datasets were retrieved from the SAbDab database [4].
Only the structures with resolution better than 3.0Å threshold are selected
and all sequences are annotated by Chothia numbering scheme [3]. Then Fv
fragments are extracted using the annotation information. To make the dataset
non-redundant, agglomerative clustering is applied [8] (with the complete
linkage rule) with $1-sequence\,identity$ as a distance metric (for two chain
antibodies, the distance metric is the average mean of $1-sequence\,identity$
for heavy and light chains). We break down two-chain (VH-VL) and single-chain
(VHH) structures into different bins and clusterize them separately. Next, we
determine 100 farthest clusters, i.e., the ones with the maximal sum of the
distances to all other clusters, from the two-chain and 50 farthest ones from
the single-chain bin with a greedy approach and marked them as the test
dataset. The rest of the clusters are the training set. Then, from each
cluster we take only the center, which is determined by RMSD among the cluster
members, considering the center as the most representative structure of the
cluster. Based on the annotation, CDR H3 loop structures are extracted and
then only those structures are used for training and testing. Afterwards, we
filter out CDR H3 structures containing gaps.

We take +1 additional amino acids before the loop and +1 after. We then shift
the coordinates so that the first atom’s coordinates are $(0,0,0)$ and the
last one is placed on the Y-axis. It is important to note that DeepH3 is
trained on a full Fv fragment and our algorithm, in that way, requires less
data to train.

###  3.2 Train and test split

The model was trained on 1970 structures from data described in subsection
3.1.

To compare our results with other methods, we have tested the network on the
commonly used RosettaAntibody benchmark [19], containing 49 structures with
CDR H3 loop lengths ranging from 7 to 17 residues. First, we have deleted
structures with these 49 IDs from our dataset. However, we have quickly
identified identical CDR H3 loop sequences with different IDs in our training
data. The structures are different if compared by the whole Fv-fragment
sequences, but if we only measure RMSD on the CDR H3 loop, RMSD on the test
dataset will be wrongfully better due to a problem, known as data leaks.
Therefore, we eliminated them so there are only unique sequences in all train
and test sets. After this step, the size of our train dataset has decreased to
1945 structures. To make a fair comparison of our method to others we do it on
the original RosettaAntibody benchmark. However, to make a fair evaluation, we
also present the result on the data without duplicate sequences.

###  3.3 Training setup

SimpleDH3 model was trained using NVIDIA Tesla K80 GPU for 500 epochs ( 2
hours) until convergence on the antibody dataset, described in Section 3.1
with batch size of 64 and a learning rate of 0.001.

###  3.4 Results

Table 1 shows the results of our model in terms of mean RMSD and mean
inference time per sequence in seconds.

As we can see from the results, our simple model performs on par with current
state-of-the-art DeepH3 model, but dramatically outperforms it in terms of
inference time. Moreover, the time results for DeepH3 [14] only show the time
it takes to run the network part of their method, but, as Rosetta package is
needed to get final results, the speed benefit will only increase.

Methods | Mean RMSD | Mean time per sequence
---|---|---
DeepH3 [14] | 2.2 | 6.65
SimpleDH3 (ours) | 2.04 | 0.59
SimpleDH3 (ours, unique) | 2.77 | 0.59
Table 1: Performance of CDR H3 loops prediction on RosettaAntibody benchmark
for both non-unique (original) and unique data.

We also show the dependence of RMSD on the sequence length for both non-unique
(original) and unique data (Figure 2. RMSD is measured only on target H3 loop
sequence, not taking into account additional amino acids used in training and
inference.

Figure 2: SimpleDH3 RMSD performance across various H3 loop sequence lengths.

##  4 Conclusion

In this paper, we present a novel end-to-end deep learning method for the
prediction of CDR H3 loop structure. By utilizing pretrained ELMo model and
independent forward and backward LSTM passes, we can directly output the
coordinates of atoms in the backbone of the amino acid sequence by only
training on loop sequences and not whole Fv regions.

We demonstrate that SimpleDH3 approach performs on par with state-of-the-art
methods in terms of RMSD, but is at least an order of magnitude faster. This
brings us closer to the possibility of large-scale in silico screening for a
specific antibody for a targeted therapeutic target.

We also point out a flaw in the current benchmark for this task regarding
identical sequences with different PDB-IDs and present a method for creating a
better benchmark.

Our code is available at https://github.com/jbr-ai-labs/SimpleDH3.

## Acknowledgments

The publication was supported by the grant for research centers in the field
of AI provided by the Analytical Center for the Government of the Russian
Federation (ACRF) in accordance with the agreement on the provision of
subsidies (identifier of the agreement 000000D730321P5Q0002) and the agreement
with HSE University No. 70-2021-00139.

## References

  * Baek et al. [2021] M. Baek, F. DiMaio, I. Anishchenko, J. Dauparas, S. Ovchinnikov, G. R. Lee, J. Wang, Q. Cong, L. N. Kinch, R. D. Schaeffer, et al.  Accurate prediction of protein structures and interactions using a three-track neural network.  _Science_ , 373(6557):871–876, 2021.
  * Chothia and Lesk [1987] C. Chothia and A. M. Lesk.  Canonical structures for the hypervariable regions of immunoglobulins.  _Journal of molecular biology_ , 196(4):901–917, 1987.
  * Chothia et al. [1989] C. Chothia, A. M. Lesk, A. Tramontano, M. Levitt, S. J. Smith-Gill, G. Air, S. Sheriff, E. A. Padlan, D. Davies, W. R. Tulip, et al.  Conformations of immunoglobulin hypervariable regions.  _Nature_ , 342(6252):877–883, 1989.
  * Dunbar et al. [2013] J. Dunbar, K. Krawczyk, J. Leem, T. Baker, A. Fuchs, G. Georges, J. Shi, and C. M. Deane.  SAbDab: the structural antibody database.  _Nucleic Acids Research_ , 42(D1):D1140–D1146, 11 2013.  ISSN 0305-1048.  doi: 10.1093/nar/gkt1043.  URL https://doi.org/10.1093/nar/gkt1043.
  * Ecker et al. [2015] D. M. Ecker, S. D. Jones, and H. L. Levine.  The therapeutic monoclonal antibody market.  In _MAbs_ , volume 7, pages 9–14. Taylor & Francis, 2015.
  * Heinzinger et al. [2019] M. Heinzinger, A. Elnaggar, Y. Wang, C. Dallago, D. Nechaev, F. Matthes, and B. Rost.  Modeling aspects of the language of life through transfer-learning protein sequences.  _BMC bioinformatics_ , 20(1):1–17, 2019.
  * Hochreiter and Schmidhuber [1997] S. Hochreiter and J. Schmidhuber.  Long short-term memory.  _Neural computation_ , 9(8):1735–1780, 1997.
  * Jain and Dubes [1988] A. K. Jain and R. C. Dubes.  _Algorithms for clustering data_.  Prentice-Hall, Inc., 1988.
  * Jumper et al. [2021] J. Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov, O. Ronneberger, K. Tunyasuvunakool, R. Bates, A. Žídek, A. Potapenko, et al.  Highly accurate protein structure prediction with alphafold.  _Nature_ , 596(7873):583–589, 2021.
  * Marks and Deane [2017] C. Marks and C. Deane.  Antibody h3 structure prediction.  _Computational and Structural Biotechnology Journal_ , 15:222–231, 2017.
  * North et al. [2011] B. North, A. Lehmann, and R. L. Dunbrack Jr.  A new clustering of antibody cdr loop conformations.  _Journal of molecular biology_ , 406(2):228–256, 2011.
  * Peters et al. [2018] M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer.  Deep contextualized word representations.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 2227–2237, New Orleans, Louisiana, June 2018. Association for Computational Linguistics.  doi: 10.18653/v1/N18-1202.  URL https://aclanthology.org/N18-1202.
  * Reichert [2017] J. M. Reichert.  Antibodies to watch in 2017.  In _MAbs_ , volume 9, pages 167–181. Taylor & Francis, 2017.
  * Ruffolo et al. [2020] J. A. Ruffolo, C. Guerra, S. P. Mahajan, J. Sulam, and J. J. Gray.  Geometric potentials from deep learning improve prediction of cdr h3 loop structures.  _Bioinformatics_ , 36(Supplement_1):i268–i275, 2020.
  * Ruffolo et al. [2021] J. A. Ruffolo, J. Sulam, and J. J. Gray.  Antibody structure prediction using interpretable deep learning.  _bioRxiv_ , 2021.
  * Sivasubramanian et al. [2009] A. Sivasubramanian, A. Sircar, S. Chaudhury, and J. J. Gray.  Toward high-resolution homology modeling of antibody fv regions and application to antibody–antigen docking.  _Proteins: Structure, Function, and Bioinformatics_ , 74(2):497–514, 2009.
  * Suzek et al. [2015] B. E. Suzek, Y. Wang, H. Huang, P. B. McGarvey, C. H. Wu, and U. Consortium.  Uniref clusters: a comprehensive and scalable alternative for improving sequence similarity searches.  _Bioinformatics_ , 31(6):926–932, 2015.
  * Tiller and Tessier [2015] K. E. Tiller and P. M. Tessier.  Advances in antibody design.  _Annual review of biomedical engineering_ , 17:191–216, 2015.
  * Weitzner et al. [2014] B. D. Weitzner, D. Kuroda, N. Marze, J. Xu, and J. J. Gray.  Blind prediction performance of rosettaantibody 3.0: grafting, relaxation, kinematic loop modeling, and full cdr optimization.  _Proteins: Structure, Function, and Bioinformatics_ , 82(8):1611–1623, 2014.


# Snowmass 2021 White Paper: The Windchime Project

The Windchime Collaboration: Alaina Attanasio  Purdue University, West
Lafayette, IN, USA  Sunil A. Bhave  Purdue University, West Lafayette, IN, USA
Carlos Blanco  Princeton University, Princeton, NJ, USA  Stockholm University,
Stockholm, Sweden  Daniel Carney  Lawrence Berkeley National Laboratory,
Berkeley, CA, USA  Marcel Demarteau  Oak Ridge National Laboratory, Oak Ridge,
TN, USA  Bahaa Elshimy  Purdue University, West Lafayette, IN, USA  Michael
Febbraro  Oak Ridge National Laboratory, Oak Ridge, TN, USA  Matthew A.
Feldman  Oak Ridge National Laboratory, Oak Ridge, TN, USA  Sohitri Ghosh
Joint Quantum Institute/Joint Center for Quantum Information and Computer
Science, University of Maryland, College Park/National Institute of Standards
and Technology, Gaithersburg, MD, USA  Abby Hickin  Purdue University, West
Lafayette, IN, USA  Seongjin Hong  Oak Ridge National Laboratory, Oak Ridge,
TN, USA  Rafael F. Lang  Purdue University, West Lafayette, IN, USA  Benjamin
Lawrie  Oak Ridge National Laboratory, Oak Ridge, TN, USA  Shengchao Li
Purdue University, West Lafayette, IN, USA  Zhen Liu  School of Physics and
Astronomy, University of Minnesota, Minneapolis, MN, USA  Juan P. A. Maldonado
Purdue University, West Lafayette, IN, USA  Universidad Nacional de Colombia,
Bogotá, Colombia  Claire Marvinney  Oak Ridge National Laboratory, Oak Ridge,
TN, USA  Hein Zay Yar Oo  Purdue University, West Lafayette, IN, USA  Yun-Yi
Pai  Oak Ridge National Laboratory, Oak Ridge, TN, USA  Raphael Pooser  Oak
Ridge National Laboratory, Oak Ridge, TN, USA  Juehang Qin  Purdue University,
West Lafayette, IN, USA  Tobias J. Sparmann  Purdue University, West
Lafayette, IN, USA  Jacob M. Taylor  Joint Quantum Institute/Joint Center for
Quantum Information and Computer Science, University of Maryland, College
Park/National Institute of Standards and Technology, Gaithersburg, MD, USA
Hao Tian  Purdue University, West Lafayette, IN, USA  Christopher Tunnell
Rice University, Houston, TX, USA

###### Abstract

The absence of clear signals from particle dark matter in direct detection
experiments motivates new approaches in disparate regions of viable parameter
space. In this Snowmass white paper, we outline the Windchime project, a
program to build a large array of quantum-enhanced mechanical sensors. The
ultimate aim is to build a detector capable of searching for Planck mass-scale
dark matter purely through its gravitational coupling to ordinary matter. In
the shorter term, we aim to search for a number of other physics targets,
especially some ultralight dark matter candidates. Here, we discuss the basic
design, open R&D challenges and opportunities, current experimental efforts,
and both short- and long-term physics targets of the Windchime project.

Submitted to the Proceedings of the US Community Study

on the Future of Particle Physics (Snowmass 2021)

###### Contents

  1. I Introduction
  2. II Detector concept
  3. III Physics targets
  4. IV Key R&D needs and current efforts
    1. IV.1 Environmental isolation
    2. IV.2 Quantum noise reduction
    3. IV.3 Large-scale arrays
    4. IV.4 Computational techniques
  5. V Outlook

##  I Introduction

All evidence to date for the existence of dark matter (DM) is through its
gravitational coupling to visible matter. On the other hand, all direct
detection searches for dark matter to-date have necessarily had to assume some
additional coupling to the Standard Model, for example a Weak nuclear coupling
in the case of WIMPs, or gluonic/photonic couplings in the case of axions. A
clearly desirable target would be to search for particle DM directly through
its gravitational coupling alone.

Recently, it has been suggested that a purely gravitational direct detection
strategy could be possible, albeit very challenging, with a terrestrial
experiment Hall  _et al._ (2018); Kawasaki (2019); Carney  _et al._ (2020).
This idea leverages the incredible and rapid progress in quantum readout and
control of mechanical sensing devices with optical or microwave light
Aspelmeyer  _et al._ (2014); Blencowe (2004); Snowmass 2021 Instrumentation
Frontier: Quantum Sensing for HEP Science — Interferometers, Mechanics,
Clocks, and Traps . These devices have been demonstrated as a promising
platform for a searches for number of dark matter candidates Carney  _et al._
, spanning the ultralight Graham  _et al._ (2016); Carney  _et al._ (2019);
Manley  _et al._ (2020); Snowmass 2021 CF2 Whitepaper: New Horizons: Scalar
and Vector Ultralight Dark Matter , light Afek  _et al._ (2022), and up to
WIMP-scale and heavier mass ranges Monteiro  _et al._ (2020). In particular,
Ref. Carney  _et al._ (2020) suggested that a large array consisting of at
least $10^{6}$ mechanical sensors, each with mass around the gram scale, could
be sensitive to the gravitational signatures of dark matter with masses around
the Planck scale $m_{\rm Pl}\approx 2\times 10^{18}~{}{\rm GeV}\approx
4~{}{\mu g}$. See the Snowmass 2021 community whitepaper Snowmass 2021 Cosmic
Frontier White Paper: Ultraheavy dark matter  for an overview of these
ultraheavy dark matter candidates.

In this Snowmass white paper, we outline a nascent experimental effort, which
we dub the Windchime project, to develop such dark matter detectors. The core
program is to construct and operate arrays of many quantum-limited mechanical
accelerometers in parallel. Such a system would be uniquely capable of
searching for a number of interesting signals, with gravitational dark matter
detection a very long-term goal. Many technical developments are needed,
spanning four key axes: thermal isolation, quantum measurement noise below the
standard quantum limit, scaling in the number of sensors and their readout,
and data handling and analysis techniques for continuous data streams from
many detectors. Many shorter-term physics opportunities will be enabled during
the development of these technologies, and the R&D program will have a vast
number of applications beyond the search for dark matter. We outline the
technical challenges, physics opportunities, our current efforts, and pathways
to the fulfillment of the long-term program.

##  II Detector concept

Figure 1: Schematic illustration of the Windchime detector concept. Left: an
array of mechanical sensors, here depicted as suspended pendula, with a
potential DM track signal. Center: cross-section, emphasizing the “track”
signal. Right: single-sensor depiction of a gravitational DM event. Here, for
conceptual illustration, the sensor is depicted in a small optical cavity with
laser readout. In practice, readout through either fibers or microwave
transmission lines will be more convenient for a densely packed array.

The central concept of the Windchime detector is to use an array of mechanical
sensors, operated as a single detector, see Fig. 1 (and Ref. Carney  _et al._
(2020) for more details). This setup should be uniquely sensitive to signals
which are coherent over one, many, or even all of the mechanical devices.
Examples include both gravitationally-coupled, localized DM, which acts
coherently on sensors near its location (Fig. 1, center), as well as
ultralight DM, which can act coherently on many or all of the sensors (Fig. 1,
right). In this section, we provide a brief, high-level description of both
the individual mechanical sensing elements, as well as the operation of a
large array of these. More details about physics signals of interest are then
given in Sec. III, followed by details about possible physical implementations
of the detector concepts in Sec. IV.

The individual mechanical sensors of the array are
optomechanical/electromechanical systems. The essential architecture is a
mechanical element (e.g., suspended reflective pendula Abramovici  _et al._
(1992); Cataño-Lopez  _et al._ (2020), optically levitated dielectric spheres
Delić  _et al._ (2020); Monteiro  _et al._ (2020), or magnetically levitated
superconducting masses Moody  _et al._ (2002); Romero-Isart  _et al._ (2012)),
coupled to a readout and control system consisting of an optical or microwave
field. The readout field encodes the position or momentum of the mechanical
element continuously in time, so the data stream per sensor is a time series
$\mathbf{x}(t)$ or $\mathbf{p}(t)$. Using linear response, this can be
converted in software to a continuous measurement of an external force
$\mathbf{F}(t)$. The core idea is to use this readout to look for the small
impulse $\Delta\mathbf{p}=\int\mathbf{F}(t)dt$ imparted to the mechanical
system while a DM candidate acts on it. Sensors may be designed with readout
on multiple axes, enabling detection of the full _vector_ $\Delta\mathbf{p}$,
i.e., enabling perfect directional sensitivity.

Placing many of these sensors in an array offers a variety of important
advantages. For signals coherent over $N$ sensors, each of which has
independent noise, the signal-to-noise (SNR) can be improved by standard
$\sqrt{N}$ statistics. Crucially, in the case of ultraheavy DM detected
through gravity, the signal is coherent over a “track” of sensors, as shown in
Fig. 1. This enables both $\sqrt{N}$ SNR enhancement and, even more
importantly, exquisite background rejection. Almost all traditional DM
detection backgrounds like cosmic rays or neutrinos will act only on one or a
few individual sensors, or be at faster speeds than those expected from DM.
Therefore, the correlated track-like signature of a passing heavy DM particle
makes a unique signal. This way, not only the direction of the DM itself, but
also the speed of a passing DM object, can be determined.

##  III Physics targets

A full realization of the Windchime detector would have exquisite sensitivity
in a number of channels. Any signal that generates a small acceleration or
force coherent over scales comparable to the detector will be uniquely
accessible. Here, we outline a few targets already identified as interesting
examples. Further theory input for potential possibly interesting signals
would be of substantial value.

The central, long-term target of this project is Planck-scale DM detection
through gravity alone. This target sets the basic design parameters of the
experiment. Because the flux of DM is expected to be approximately

$\Phi_{\chi}=\frac{\rho_{\chi}}{m_{\chi}}v\approx\frac{1}{\rm
yr~{}m^{2}}\times\frac{m_{\rm Pl}}{m_{\chi}},$ (1)

we need a total detector cross-section of at least around a square meter
Bramante  _et al._ (2018). Moreover, the sensitivity of an accelerometer
scales with its mass like $\sqrt{m}$, while the gravitational signal scales
like $1/b$, where $b$ is the typical impact parameter. Numerically, this leads
one to look at gram-scale sensors, spaced at approximately centimeter
distances, in an array roughly $1000$ sensors on a side. We discuss the
substantial technical challenges to reaching this detection target in detail
in Section IV.

Prior to this long-term gravitational detection target, a number of other DM
candidates can be probed. In terms of localized, particle-like excitations
Snowmass 2021 Cosmic Frontier White Paper: Ultraheavy dark matter , this
includes composite DM and solitons coupled to the Standard Model (SM) through
some new non-gravitational force. As long as this force has a range of order
the size of a sensor, the detection problem is essentially similar to the
gravitational interaction. As a concrete example, in Monteiro  _et al._
(2020), a search for TeV-scale composite DM charged under a gauged $B-L$
symmetry was performed, leading to novel constraints with just a single
$\mu$g-scale optomechanical sensor and a few days of observation time.
Generically, we can parametrize the interaction potential in terms of the
charge-to-mass ratio, $\lambda_{SM(DM)}$, of the SM (DM) and the new long-
range coupling $g_{\phi}$. The impulse imparted on each sensor during a
transit, and therefore the sensitivity of the detector, scales as
$F\sim(g_{\phi}^{2}\lambda_{SM}\lambda_{DM})/b^{2}$. Fifth-force searches
generically constrain the new coupling to be weaker than $g_{\phi}\lesssim
10^{-21}$ Schlamminger  _et al._ (2008); Wagner  _et al._ (2012); Abbott  _et
al._ (2021). Therefore, the charge-to-mass ratio of the dark matter must be
rather large, $\lambda_{DM}\gtrsim 1200/\text{GeV}$, in order for the DM to
leave visible tracks through the array Blanco  _et al._ (2021). This is
particularly clear when the DM is confined to a scale smaller than the typical
sensor separation ($\lesssim{\rm mm-cm}$); detection prospects with less-
confined, blob-like objects presents another interesting possibility Grabowska
_et al._ (2018).

The theory space in which the DM appears as point-like and couples through a
new long-range force can be spanned by two complementary model classes. In the
first, the charge confinement is realized via the formation of bound states of
constituent fundamental particles. These could either belong to the SM Bodmer
(1971); Witten (1984) or to a dark sector Bai  _et al._ (2019); Hardy  _et
al._ (2015a, b); Gresham  _et al._ (2017); Redi and Tesi (2019); Detmold  _et
al._ (2014a, b); Gresham  _et al._ (2018a); Krnjaic and Sigurdson (2015);
Gresham  _et al._ (2018b). In order to be visible, these composite models must
generically be strongly confining in order to evade BBN constraints. On the
other hand, the dark charge can be confined in a non-fundamental particle
solution to the field equations such as in non-topological solitons, i.e.
Q-balls Coleman (1985); Friedberg  _et al._ (1976); Kusenko and Shaposhnikov
(1998); Kusenko (1997, 2006); Hong  _et al._ (2016, 2020); Holdom (1987a);
Lohiya (1994); Stojkovic (2003); Macpherson and Campbell (1995); Holdom
(1987b); Zhitnitsky (2003); Ogure  _et al._ (2003); Frieman  _et al._ (1988);
Affleck and Dine (1985); Dine and Kusenko (2003); Krylov  _et al._ (2013).
Thick-walled Q-balls are an especially promising class of models that could be
probed significantly with Windchime for charges between $10^{20}$ to $10^{30}$
and for masses above $10^{16}\;\text{GeV}$ Blanco  _et al._ (2021). In the
case where the DM couples only gravitationally, the parameter space is broadly
split into two regions: the sub-Planckian and super-Planckian regimes. Below
the Planck scale fundamental particles can be produced gravitationally either
as WIMPZILLAs around the inflationary epoch Chung  _et al._ (2001, 1998a,
1998b) or by the evaporation of primordial black holes (PBHs) following an era
of PBH domination Hooper  _et al._ (2019). Above the Planck scale, DM can be
produced in the form of Planck-scale relics of extremal gravitational
singularities Bernal and Zapata (2021); Lehmann  _et al._ (2019); Bai and
Orlofsky (2020); Aydemir  _et al._ (2020); Abbott  _et al._ (2021); MacGibbon
(1987); Salvio and Veermäe (2020).

Figure 2: Left: Detection reach for accelerometer searches of ultralight dark
matter, taking a vector $B-L$ boson as an example. We assume one day of
integration time, and the use of a pair of accelerometers with differential
neutron-to-nucleon ratio $\Delta=N_{1}/A_{1}-N_{2}/A_{2}=0.05$. For more
details, see discussion in the text and in Refs. Graham  _et al._ (2016);
Carney  _et al._ (2019, ). Right: Schematic illustration of the detection
signal. Here the ultralight DM field acts as a wave with coherence length
larger than the linear scale of at least one sensor (a single grey blob), and
possibly many sensors.

Another quite different target that has been identified is ultralight bosonic,
or “fuzzy”, dark matter Snowmass 2021 CF2 Whitepaper: New Horizons: Scalar and
Vector Ultralight Dark Matter . In an ultralight scenario, the DM is comprised
mainly of fields with mass $m_{\chi}\lesssim 0.1~{}{\rm eV}$, which
necessarily behaves more like a classical wave than single particle
excitations due to the large number density. If this DM couples to matter, it
will produce a weak, persistent, and nearly monochromatically time-varying
force on each sensor. See Fig. 2, right panel. For example, if this ultralight
field is a vector field coupling to $B-L$ charge, it will produce a force
$F(t)=g_{BL}N_{n}F_{0}\sin(m_{\chi}t)$ on each sensor, where $N_{n}$ is the
number of neutrons in the sensor, $g_{BL}\lesssim 10^{-24}$ is a coupling
already well-constrained by fifth force experiments, and
$F_{0}=\sqrt{\rho_{\chi}}\approx 10^{-15}~{}{\rm N}$ is set by the background
density of DM quanta. For $m_{\chi}\lesssim 0.1~{}{\rm\mu eV}$ it will act
coherently over _all_ the sensors in the array, leading to substantial signal-
to-noise enhancements. Crucially, this force scales with the mass of the
sensor in a material-dependent way: it produces an acceleration sensitive to
the ratio of neutrons to total nucleons of the sensor. Thus two sensors, or
one sensor referenced to a fixed target, will feel a differential acceleration
Graham  _et al._ (2016); Carney  _et al._ (2019, )

$a_{\rm signal}(t)=g_{BL}\Delta a_{0}\sin(m_{\chi}t)$ (2)

where $\Delta=A_{1}/Z_{1}-A_{2}/Z_{2}$ is the difference between the neutron-
to-nucleon ratios of the two materials, typically a quantity of order
$0.01-0.1$. Here $a_{0}=F_{0}/m_{\rm neutron}\approx 7\times 10^{12}~{}{\rm
m/s^{2}}$. This means that we are looking for accelerations at the
$10^{-12}~{}{\rm m/s^{2}}$ level. This is an ideal task for a network of
quantum-limited (or enhanced) mechanical accelerometers! In Figure 2, we
sketch the detection reach for such ultralight DM for sensors with
acceleration sensitivities in this range. This signal can be integrated
coherently over many sensors, not just a track, leading to substantial
possible detection reach.

##  IV Key R&D needs and current efforts

Reaching the ultimate goal of gravitational direct detection requires
substantial technical development in four central directions. These are
isolation from thermal environments, reduction of quantum measurement-added
noise, construction and operation of a large-scale array of optomechanical
sensors, and computational techniques for handling the massive data stream
generated by such an array. In this section, we briefly discuss all of these
issues in some detail, and some current solutions being pursued by the
Windchime collaboration and others.

###  IV.1 Environmental isolation

The sensors must be well isolated from thermal backgrounds in order to bring
these below the weak gravitational signals of interest. With physically
suspended devices, very high mechanical quality factors are of crucial
importance. For an example from the current state of the art, the Matsumoto
group has produced mg-scale optomechanical pendula with $Q\gtrsim 2\times
10^{6}$ Cataño-Lopez  _et al._ (2020). Moreover, the bath temperature needs to
be as low as possible. Meter-scale experiments at dilution refrigeration
temperatures $\sim 10~{}{\rm mK}$ are already demonstrated (e.g. the SuperCDMS
experiment Agnese  _et al._ (2014)). Similar cooling methods should be
adaptable to optomechanical arrays. Additionally, free-space and fiber coupled
optical readout of materials and devices within dilution refrigerators at mK
temperatures are also already in operation Lawrie  _et al._ (2021).

Another option is to go beyond suspended systems, removing the phononic
environment by levitating or even dropping the sensors. In this case, the
background thermal noise comes primarily from collisions with the ambient gas
molecules. Here the primary technical requirement becomes achieving a
sufficiently good vacuum. This also requires cryogenic cooling, although only
at the 4 K level. Pressures similar to those achieved in the LHC tunnel
Gröbner (2001) appear to be sufficiently low that one could see the
gravitational signal Carney  _et al._ (2020). More detailed study of gas
damping noise spectra in the mechanical systems at these kinds of pressure,
and when looking for impulsive signals, will be of crucial importance.

###  IV.2 Quantum noise reduction

With control over technical noise sources such as stray background fields and
thermal noise, mechanical sensors become limited by noise due to the quantum
mechanical uncertainty of the readout system. A typical benchmark for the
quantum measurement-added noise is given by the so-called Standard Quantum
Limit (SQL),

$\Delta p_{\rm SQL}=\sqrt{\frac{\hbar m_{s}}{\tau}}\approx 600~{}{\rm
GeV/c}\times\left(\frac{m_{s}}{1~{}{\rm
g}}\right)^{1/2}\left(\frac{1~{}{\rm\mu s}}{\tau}\right)^{1/2},$ (3)

where $m_{s}$ is the mass of the mechanical element and $\tau$ is the
integration time. This is roughly set by the scale of vacuum fluctuations of
the mechanical element Caves (1981). Noise levels around the SQL are now
routinely achieved in many systems, including the impulsive optomechanical
detector of Monteiro  _et al._ (2020). However, SQL-level noise is orders of
magnitude larger than the gravitational impulse signal Carney  _et al._
(2020). Thus, methods to get below the SQL level are of crucial importance to
this program. Current techniques to go below the SQL are aleady in use in,
e.g., advanced LIGO Aasi  _et al._ (2013), with world records at $\sim 15{\rm
dB}$ below the SQL Vahlbruch  _et al._ (2016a). We are also developing new
techniques specialized to our particular detection problems, which should have
wide utility beyond the Windchime project.

A key advantage here over more typical quantum sensing problems is that the
impulses are expected to be delivered relatively quickly (sub $\mu$s for
typical expectations of dark matter wind velocities). Techniques for going
well below the SQL can be used more readily in this case than in low frequency
detection systems. Two concurrent paths have promise, and can be combined for
maximum effectiveness. The first is to take advantage of squeezing of the
light (or microwaves) used to measure effects of the impulse on the sensor.
The role of squeezing is to trade off lower imprecision – analogous to less
shot noise – at the price of higher backaction, in which intensity
fluctuations are imprinted on the sensor and limit ultimate performance.
Overall, this allows an improvement in the limiting impulse that can be
measured, but only at the square root of the squeezing improvement.

The other technique is to directly monitor the velocity or momentum of the
sensor, rather than infer its momentum via measurement of the position over
time Caves  _et al._ (1980); Braginsky and Khalili (1990); Ghosh  _et al._
(2019). This enables a so-called back action evading measurement in the free
mass limit – when the detection frequency is well above the mechanical
resonance of the sensor mass. In its simplest incarnation, a back action
evading sensor has a velocity- or rotation rate-dependent phase shift of an
optical or microwave resonance, as occurs, e.g., in a waveguide optical gyro.
Improvements in phase estimation enabled by squeezing can thus be directly
combined with back action evasion. To date, experimental progress in back
action evasion remains nascent, with most results in the low mass ($\mu$g or
smaller) regime, and thus substantial work remains to bring this to the right
mass domain. Parallel to our current experimental efforts on this front, we
are also actively pursuing new theoretical proposals for backaction evasion
measurements tailored to impulse sensing Ghosh  _et al._ (2019).

As a first step, we are developing a squeezed optical readout technique that
will be used to measure the displacement of sensors with potential for greater
than 9 dB of quantum noise reduction below the shot noise limit Glorieux  _et
al._ (2010). Improved future squeezed light sources aim to reach and succeed
the record in squeezing of 15 dB Vahlbruch  _et al._ (2016b). The two-mode
squeezed light source used here is based on four-wave-mixing (4WM) in 85Rb,
and an image of the apparatus and an example of the frequency dependent
squeezing in the system is shown in Fig. 3. Because the 4WM process amplifies
multiple spatial modes, this approach will allow for proof-of-principle
scaling of the squeezed readout field across an array of sensors Lawrie  _et
al._ (2019). In this case, the sensor displacement or velocity will be read-
out using a truncated nonlinear interferometer, which leverages the continuous
variable entanglement in two-mode squeezed light sources to access amplitude
difference or phase sum squeezing Anderson  _et al._ (2017). We have already
used this detection scheme to characterize the displacement of an atomic force
microscope microcantilever with quantum enhanced sensitivity Pooser  _et al._
(2020); Pooser and Lawrie (2015), and we are now extending it to the domain of
larger mechanical devices.

Figure 3: Left: Two-mode squeezed light source apparatus, viewed from the
probe arm before going through the vapor cell. Right: Noise power measurement
of the two-mode squeezed light source (blue) in comparison to a shot-noise-
limited classical light source (green) of comparable power, with over 5.6 dB
of squeezing observed at 180 kHz.

In an improved detection scenario, the sensor response may be transduced onto
both channels of the two mode squeezed light source, allowing for quantum
noise reduction together with a doubling of the readout signal. Because the
shot noise level is defined by the total power at the detector, it is possible
to improve the shot-noise-limited sensitivity by utilizing high power local
oscillators with squeezed vacuum readout fields, leading to substantially
reduced heating compared to schemes that rely on single mode squeezing Pai
_et al._ (2022). This benefit is particularly critical for the ultralow
temperature implementations of the sensing scheme described above. In the long
term, arrays of large numbers of these sensors will be used to measure vector
information, and squeezing will be used at each sensor. To accomplish this in
a scalable way, waveguided sources of squeezed and entangled light will be
required. We are currently developing integrated, telecom wavelength sources
for this purpose, with entangled light spread over a range of frequencies that
are resonant with each sensor. Including linear-optical operations in line
with the resonators and sources will also allow for complex state preparation
schemes in which multiparty entangled states can be shared between sensing
nodes.

###  IV.3 Large-scale arrays

Operating an array of many optomechanical sensors provides a unique and
exciting challenge. Issues range from an appropriate choice of single-sensor
architecture and readout, multiplexing, and large-scale environmental
isolation. There are also substantial issues with real-time data processing,
which are discussed in the next subsection.

As a first step, we are constructing a small array of milligram-scale
accelerometers named _Protochime_. This prototype is an array of 15 commercial
MEMS (microelectronic mechanical systems) accelerometers, set up to take data
continuously and in parallel for 16 channels with sampling rates in the order
of low MHz. The sensors are mechanically mounted on one or more solid material
blocks with sizes in the cm-m range. This array is able to measure correlated
vibrations produced by external driving forces along the sensitive axis if
they are above a certain noise level determined by the accelerometer noise
level, the electronics and the bit-precision in the digitization process of
the data. Successful measurements with this prototype include a proof-of-
principle constraint (not competitive with pre-existing fifth force bounds) on
ultralight DM models, following the general methodology proposed in Graham
_et al._ (2016); Carney  _et al._ (2019). Protochime serves as the first
experimental realization capable of producing real data to test the novel
approach of Windchime in DM detection. It is useful to explore current
hardware and software scope and limitations.

Laser shot-noise limited optomechanical accelerometers have been demonstrated
by coupling optical free-space readout of suspended proof-masses Zhou  _et
al._ (2021); Guzmán Cervantes  _et al._ (2014). However, Windchime requires at
least $10^{6}$ such accelerometers. A frequency multiplexed and optically
coherent readout scheme is thus necessary. One potential solution is to
interface the extremely sensitive MEMS Hutchison and Bhave (2012) with
photonic integrated circuits (PIC) and take advantage of the frequency tuning
Tian  _et al._ (2020), multiplexing, combs Liu  _et al._ (2020) and coherent
readout technologies demonstrated on PIC platforms. The next generation array
that we are working on will use $\sim 10^{2}$ dedicated accelerometers based
on integrated Si3N4 photonic circuits, achieving a noise density in the order
of ${\rm\mu g}/\sqrt{\rm Hz}$ at room temperature and pressure. These sensors
are based around an optical microring resonator, and would be read out using a
fibre-coupled inteferometric setup Krause  _et al._ (2012). The optical
resonance frequency is adjustable on the order of GHz by integrating
piezoelectric actuators Liu  _et al._ (2020), allowing for frequency domain
multiplexing. A single frequency laser can be electro-optically modulated for
producing a frequency comb Zhang  _et al._ (2019), where each comb line can be
aligned with each sensor with low cross-talk. Such a setup will serve as a
test-bed for the technological developments needed to detect dark matter with
an optomechanical array.

###  IV.4 Computational techniques

Windchime presents interesting R&D opportunities as we develop the
computational techniques needed for large arrays of quantum-enabled mechanical
sensors. The array produces streaming spatiotemporal data, where each sensor
is producing its own time series data. The novelty of Windchime is in using a
large number of sensors to take advantage of incoherent noise. Beyond the
challenges of having many sensors, the data analysis presents its own
challenges. However, there is a substantial body of previous work on related
problems; as a benchmark, the CMS silicon tracker has 10 million readout
channels being sampled at a rate of 40 MHz, then read out at up to 100 kHz
Foudas  _et al._ (2005).

There are two different scientific objectives that provide competing
requirements for how signals could be detected in such an array. Heavy dark
matter will produce a fast transient signal in the array with signals
localized in a few sensors, which requires a faster digitization frequency
thus high date rate per sensor. However, ultralight dark matter is a coherent
effect across the entire array, where information from all sensors is needed
to search for a signal, thereby scaling the high sensor rate by a large number
of sensors. Therefore, different techniques are required for both of these
extremes.

For heavy dark matter, we are performing track finding in an optomechanical
sensor array to identify sensors nearby the track. Track finding is a mature
field in particle physics, as this is a core part of the analysis of most
collider physics detectors. The main computational challenges related to this
task are areas of active research in computational science. First, how does
one perform triggering, which is also known as object detection, to only
record interesting data? This requires having streaming algorithms to identify
where they may be a track. Second, how does one perform the analysis or
optimization to determine where a track might be? An interesting possibility
is to use signal processing techniques to preprocess this data. Third, what
opportunities are there to quantify uncertainties on the track parameters,
such as is done for sky localization in multimessenger astronomy?

Because gravity is a long range interaction, even sensors that are not
directly impacted by the track can contribute to a signal. This means that
parameter inference could be more sophisticated than basic track finding, for
example using Bayesian methods to infer the posterior distribution of the
track from data. This has been successfully trialed using a simulated array of
$8^{3}$ sensors and the ultranest package Buchner (2021). Such methods,
however, would be highly computationally intensive, necessitating the use of a
trigger as mentioned above. In addition, triggering might be needed for data
storage reasons. For example, an array of $10^{6}$ sensors sampled at
$10\,\mathrm{MHz}$ at 12 bits would result in a data rate of about
$15\,\mathrm{TB/s}$. Only storing triggered data will be crucial to reduce the
data storage requirements.

For ultralight dark matter, the noise level can be suppressed by up to
$\sqrt{N_{det}}$, as discussed above. The main challenge is designing a data
acquisition system that can probe coherent effects across the entire array.
The ultralight dark matter signal has a finite coherence time, inversely
proportional to its bandwidth, $T_{coh}\sim 10^{6}/\omega_{DM}$. Therefore,
for the experiment time scale that is shorter than the dark matter coherence
time (e.g., $T_{coh}>$1yr for $m_{DM}<10^{-13}$eV), the signal behaves as a
monochromatic wave with a single frequency across all sensors. In that case, a
time-domain Wiener optimal filter can be used (even at the FPGA level). For
larger DM mass, thus smaller coherence time, the DM produces an incoherent
power signal. This may require offline processing of the time series,
potentially with a method similar to a Dicke radiometer, whose analysis
framework was presented in Chaudhuri  _et al._ (2018), while for smaller
timescales analysis procedures similar to the one presented in Salemi  _et
al._ (2021) can be used.

In all cases above, an exciting opportunity is performing machine-learning-
enabled experimental design. As we are operating a cubic lattice, simulations
and forward models of the processes involved lend themselves to optimization
techniques that operate on large matrices. Existing routines for analyzing
data on a grid, such as convolutional neural networks, mean that there is an
exciting opportunity for being able to perform global detector optimization in
a way that is not as tractable in other particle physics experiments.

##  V Outlook

The direct detection of dark matter through gravity alone was long thought to
be impossible. Thanks to advances made in the control of optomechanical
systems, we have identified a possible avenue to achieve this goal in the
foreseeable future. While certainly a long-term endeavour, the Windchime
project is charting a path towards achieving sensitivity to well-motivated
Planck-mass dark matter through gravitational interaction alone. On the way to
such sensitivity to this well-motivated parameter space, Windchime is also
probing ultralight DM candidates, and possibly a number of other targets of
interest. A number of applications of this technology to other areas—including
acceleration sensing, impulse metrology, and multiplexing of large quantum
sensing arrays—can be expected in the process. Taken together, the Windchime
project constitutes a unique and well-motivated approach to enhance our
sensitivity to particle dark matter.

## References

  * Hall  _et al._ (2018) E. D. Hall, R. X. Adhikari, V. V. Frolov, H. Müller, M. Pospelov, and R. X. Adhikari, Phys. Rev. D 98, 083019 (2018), arXiv:1605.01103 [gr-qc] .
  * Kawasaki (2019) A. Kawasaki, Phys. Rev. D 99, 023005 (2019), arXiv:1809.00968 [physics.ins-det] .
  * Carney  _et al._ (2020) D. Carney, S. Ghosh, G. Krnjaic, and J. M. Taylor, Phys. Rev. D 102, 072003 (2020), arXiv:1903.00492 [hep-ph] .
  * Aspelmeyer  _et al._ (2014) M. Aspelmeyer, T. J. Kippenberg, and F. Marquardt, Reviews of Modern Physics 86, 1391 (2014).
  * Blencowe (2004) M. Blencowe, Physics Reports 395, 159 (2004).
  * (6) Snowmass 2021 Instrumentation Frontier: Quantum Sensing for HEP Science — Interferometers, Mechanics, Clocks, and Traps, arXiv:2203.xxxxx .
  * (7) D. Carney, G. Krnjaic, D. C. Moore, C. A. Regal, _et al._ , arXiv:2008.06074 [physics.ins-det] .
  * Graham  _et al._ (2016) P. W. Graham, D. E. Kaplan, J. Mardon, S. Rajendran, and W. A. Terrano, Physical Review D 93, 075029 (2016).
  * Carney  _et al._ (2019) D. Carney, A. Hook, Z. Liu, J. M. Taylor, and Y. Zhao, (2019), arXiv:1908.04797 [hep-ph] .
  * Manley  _et al._ (2020) J. Manley, M. D. Chowdhury, D. Grin, S. Singh, and D. J. Wilson, (2020), arXiv:2007.04899 [quant-ph] .
  * (11) Snowmass 2021 CF2 Whitepaper: New Horizons: Scalar and Vector Ultralight Dark Matter, arXiv:2203.xxxxx .
  * Afek  _et al._ (2022) G. Afek, D. Carney, and D. C. Moore, Phys. Rev. Lett. 128, 101301 (2022), arXiv:2111.03597 [physics.ins-det] .
  * Monteiro  _et al._ (2020) F. Monteiro, G. Afek, D. Carney, G. Krnjaic, J. Wang, and D. C. Moore, (2020), arXiv:2007.12067 [hep-ex] .
  * (14) Snowmass 2021 Cosmic Frontier White Paper: Ultraheavy dark matter, arXiv:2203.xxxxx .
  * Abramovici  _et al._ (1992) A. Abramovici, W. E. Althouse, R. W. Drever, Y. Gürsel, S. Kawamura, F. J. Raab, D. Shoemaker, L. Sievers, R. E. Spero, K. S. Thorne, _et al._ , Science 256, 325 (1992).
  * Cataño-Lopez  _et al._ (2020) S. B. Cataño-Lopez, J. G. Santiago-Condori, K. Edamatsu, and N. Matsumoto, Phys. Rev. Lett. 124, 221102 (2020), arXiv:1912.12567 [quant-ph] .
  * Delić  _et al._ (2020) U. Delić, M. Reisenbauer, K. Dare, D. Grass, V. Vuletić, N. Kiesel, and M. Aspelmeyer, Science 367, 892 (2020).
  * Moody  _et al._ (2002) M. V. Moody, H. J. Paik, and E. R. Canavan, Review of scientific instruments 73, 3957 (2002).
  * Romero-Isart  _et al._ (2012) O. Romero-Isart, L. Clemente, C. Navau, A. Sanchez, and J. Cirac, Physical review letters 109, 147205 (2012).
  * Bramante  _et al._ (2018) J. Bramante, B. Broerman, R. F. Lang, and N. Raj, Phys. Rev. D 98, 083516 (2018), arXiv:1803.08044 [hep-ph] .
  * Schlamminger  _et al._ (2008) S. Schlamminger, K. Y. Choi, T. A. Wagner, J. H. Gundlach, and E. G. Adelberger, Phys. Rev. Lett. 100, 041101 (2008), arXiv:0712.0607 [gr-qc] .
  * Wagner  _et al._ (2012) T. A. Wagner, S. Schlamminger, J. H. Gundlach, and E. G. Adelberger, Class. Quant. Grav. 29, 184002 (2012), arXiv:1207.2442 [gr-qc] .
  * Abbott  _et al._ (2021) R. Abbott _et al._ (LIGO Scientific, Virgo, KAGRA), (2021), arXiv:2105.13085 [astro-ph.CO] .
  * Blanco  _et al._ (2021) C. Blanco, B. Elshimy, R. F. Lang, and R. Orlando, (2021), arXiv:2112.14784 [hep-ph] .
  * Grabowska  _et al._ (2018) D. M. Grabowska, T. Melia, and S. Rajendran, Phys. Rev. D 98, 115020 (2018), arXiv:1807.03788 [hep-ph] .
  * Bodmer (1971) A. R. Bodmer, Phys. Rev. D 4, 1601 (1971).
  * Witten (1984) E. Witten, Phys. Rev. D 30, 272 (1984).
  * Bai  _et al._ (2019) Y. Bai, A. J. Long, and S. Lu, Phys. Rev. D 99, 055047 (2019), arXiv:1810.04360 [hep-ph] .
  * Hardy  _et al._ (2015a) E. Hardy, R. Lasenby, J. March-Russell, and S. M. West, JHEP 06, 011 (2015a), arXiv:1411.3739 [hep-ph] .
  * Hardy  _et al._ (2015b) E. Hardy, R. Lasenby, J. March-Russell, and S. M. West, JHEP 07, 133 (2015b), arXiv:1504.05419 [hep-ph] .
  * Gresham  _et al._ (2017) M. I. Gresham, H. K. Lou, and K. M. Zurek, Phys. Rev. D 96, 096012 (2017), arXiv:1707.02313 [hep-ph] .
  * Redi and Tesi (2019) M. Redi and A. Tesi, JHEP 04, 108 (2019), arXiv:1812.08784 [hep-ph] .
  * Detmold  _et al._ (2014a) W. Detmold, M. McCullough, and A. Pochinsky, Phys. Rev. D 90, 115013 (2014a), arXiv:1406.2276 [hep-ph] .
  * Detmold  _et al._ (2014b) W. Detmold, M. McCullough, and A. Pochinsky, Phys. Rev. D 90, 114506 (2014b), arXiv:1406.4116 [hep-lat] .
  * Gresham  _et al._ (2018a) M. I. Gresham, H. K. Lou, and K. M. Zurek, Phys. Rev. D 98, 096001 (2018a), arXiv:1805.04512 [hep-ph] .
  * Krnjaic and Sigurdson (2015) G. Krnjaic and K. Sigurdson, Phys. Lett. B 751, 464 (2015), arXiv:1406.1171 [hep-ph] .
  * Gresham  _et al._ (2018b) M. I. Gresham, H. K. Lou, and K. M. Zurek, Phys. Rev. D 97, 036003 (2018b), arXiv:1707.02316 [hep-ph] .
  * Coleman (1985) S. R. Coleman, Nucl. Phys. B 262, 263 (1985), [Erratum: Nucl.Phys.B 269, 744 (1986)].
  * Friedberg  _et al._ (1976) R. Friedberg, T. D. Lee, and A. Sirlin, Phys. Rev. D 13, 2739 (1976).
  * Kusenko and Shaposhnikov (1998) A. Kusenko and M. E. Shaposhnikov, Phys. Lett. B 418, 46 (1998), arXiv:hep-ph/9709492 .
  * Kusenko (1997) A. Kusenko, Phys. Lett. B 405, 108 (1997), arXiv:hep-ph/9704273 .
  * Kusenko (2006) A. Kusenko, in  _Workshop on Exotic Physics with Neutrino Telescopes_ (2006) arXiv:hep-ph/0612159 .
  * Hong  _et al._ (2016) J.-P. Hong, M. Kawasaki, and M. Yamada, JCAP 08, 053 (2016), arXiv:1604.04352 [hep-ph] .
  * Hong  _et al._ (2020) J.-P. Hong, S. Jung, and K.-P. Xie, Phys. Rev. D 102, 075028 (2020), arXiv:2008.04430 [hep-ph] .
  * Holdom (1987a) B. Holdom, Phys. Rev. D 36, 1000 (1987a).
  * Lohiya (1994) D. Lohiya, (1994), arXiv:gr-qc/9407014 .
  * Stojkovic (2003) D. Stojkovic, Phys. Rev. D 67, 045012 (2003), arXiv:hep-ph/0111061 .
  * Macpherson and Campbell (1995) A. L. Macpherson and B. A. Campbell, Phys. Lett. B 347, 205 (1995), arXiv:hep-ph/9408387 .
  * Holdom (1987b) B. Holdom, (1987b).
  * Zhitnitsky (2003) A. R. Zhitnitsky, JCAP 10, 010 (2003), arXiv:hep-ph/0202161 .
  * Ogure  _et al._ (2003) K. Ogure, T. Yoshida, and J. Arafune, Phys. Rev. D 67, 123518 (2003), arXiv:hep-ph/0212332 .
  * Frieman  _et al._ (1988) J. A. Frieman, G. B. Gelmini, M. Gleiser, and E. W. Kolb, Phys. Rev. Lett. 60, 2101 (1988).
  * Affleck and Dine (1985) I. Affleck and M. Dine, Nucl. Phys. B 249, 361 (1985).
  * Dine and Kusenko (2003) M. Dine and A. Kusenko, Rev. Mod. Phys. 76, 1 (2003), arXiv:hep-ph/0303065 .
  * Krylov  _et al._ (2013) E. Krylov, A. Levin, and V. Rubakov, Phys. Rev. D 87, 083528 (2013), arXiv:1301.0354 [hep-ph] .
  * Chung  _et al._ (2001) D. J. H. Chung, P. Crotty, E. W. Kolb, and A. Riotto, Physical Review D 64 (2001), 10.1103/physrevd.64.043503.
  * Chung  _et al._ (1998a) D. J. H. Chung, E. W. Kolb, and A. Riotto, Phys. Rev. D 59, 023501 (1998a), arXiv:hep-ph/9802238 .
  * Chung  _et al._ (1998b) D. J. H. Chung, E. W. Kolb, and A. Riotto, Phys. Rev. Lett. 81, 4048 (1998b), arXiv:hep-ph/9805473 .
  * Hooper  _et al._ (2019) D. Hooper, G. Krnjaic, and S. D. McDermott, JHEP 08, 001 (2019), arXiv:1905.01301 [hep-ph] .
  * Bernal and Zapata (2021) N. Bernal and O. Zapata, JCAP 03, 015 (2021), arXiv:2011.12306 [astro-ph.CO] .
  * Lehmann  _et al._ (2019) B. V. Lehmann, C. Johnson, S. Profumo, and T. Schwemberger, Journal of Cosmology and Astroparticle Physics 2019, 046–046 (2019).
  * Bai and Orlofsky (2020) Y. Bai and N. Orlofsky, Phys. Rev. D 101, 055006 (2020), arXiv:1906.04858 [hep-ph] .
  * Aydemir  _et al._ (2020) U. Aydemir, B. Holdom, and J. Ren, Phys. Rev. D 102, 024058 (2020), arXiv:2003.10682 [gr-qc] .
  * MacGibbon (1987) J. H. MacGibbon, Nature 329, 308 (1987).
  * Salvio and Veermäe (2020) A. Salvio and H. Veermäe, JCAP 02, 018 (2020), arXiv:1912.13333 [gr-qc] .
  * Agnese  _et al._ (2014) R. Agnese, A. J. Anderson, M. Asai, D. Balakishiyeva, R. B. Thakur, D. Bauer, J. Beaty, J. Billard, A. Borgland, M. Bowles, _et al._ , Physical review letters 112, 241302 (2014).
  * Lawrie  _et al._ (2021) B. J. Lawrie, M. Feldman, C. E. Marvinney, and Y.-Y. Pai, in  _Quantum Nanophotonic Materials, Devices, and Systems 2021_ , Vol. 11806 (International Society for Optics and Photonics, 2021) p. 1180604.
  * Gröbner (2001) O. Gröbner, Vacuum 60, 25 (2001).
  * Caves (1981) C. M. Caves, Physical Review D 23, 1693 (1981).
  * Aasi  _et al._ (2013) J. Aasi, J. Abadie, B. Abbott, R. Abbott, T. Abbott, M. Abernathy, C. Adams, T. Adams, P. Addesso, R. Adhikari, _et al._ , Nature Photonics 7, 613 (2013).
  * Vahlbruch  _et al._ (2016a) H. Vahlbruch, M. Mehmet, K. Danzmann, and R. Schnabel, Phys. Rev. Lett. 117, 110801 (2016a).
  * Caves  _et al._ (1980) C. M. Caves, K. S. Thorne, R. W. Drever, V. D. Sandberg, and M. Zimmermann, Reviews of Modern Physics 52, 341 (1980).
  * Braginsky and Khalili (1990) V. B. Braginsky and F. J. Khalili, Physics Letters A 147, 251 (1990).
  * Ghosh  _et al._ (2019) S. Ghosh, D. Carney, P. Shawhan, and J. M. Taylor, (2019), arXiv:1910.11892 [quant-ph] .
  * Glorieux  _et al._ (2010) Q. Glorieux, L. Guidoni, S. Guibal, J.-P. Likforman, and T. Coudreau, in  _Quantum Optics_ , Vol. 7727 (SPIE, 2010) pp. 9–16.
  * Vahlbruch  _et al._ (2016b) H. Vahlbruch, M. Mehmet, K. Danzmann, and R. Schnabel, Physical review letters 117, 110801 (2016b).
  * Lawrie  _et al._ (2019) B. J. Lawrie, P. D. Lett, A. M. Marino, and R. C. Pooser, ACS Photonics 6, 1307 (2019).
  * Anderson  _et al._ (2017) B. E. Anderson, P. Gupta, B. L. Schmittberger, T. Horrom, C. Hermann-Avigliano, K. M. Jones, and P. D. Lett, Optica 4, 752 (2017).
  * Pooser  _et al._ (2020) R. Pooser, N. Savino, E. Batson, J. Beckey, J. Garcia, and B. Lawrie, Physical Review Letters 124, 230504 (2020).
  * Pooser and Lawrie (2015) R. C. Pooser and B. Lawrie, Optica 2, 393 (2015).
  * Pai  _et al._ (2022) Y.-Y. Pai, C. E. Marvinney, C. Hua, R. C. Pooser, and B. J. Lawrie, Advanced Quantum Technologies 5, 2100107 (2022).
  * Zhou  _et al._ (2021) F. Zhou, Y. Bao, R. Madugani, D. A. Long, J. J. Gorman, and T. W. LeBrun, Optica 8, 350 (2021).
  * Guzmán Cervantes  _et al._ (2014) F. Guzmán Cervantes, L. Kumanchik, J. Pratt, and J. M. Taylor, Applied Physics Letters 104, 221111 (2014).
  * Hutchison and Bhave (2012) D. N. Hutchison and S. A. Bhave, in  _2012 IEEE 25th international conference on micro electro mechanical systems (MEMS)_ (IEEE, 2012) pp. 615–619.
  * Tian  _et al._ (2020) H. Tian, J. Liu, B. Dong, J. C. Skehan, M. Zervas, T. J. Kippenberg, and S. A. Bhave, Nature communications 11, 1 (2020).
  * Liu  _et al._ (2020) J. Liu, H. Tian, E. Lucas, A. S. Raja, G. Lihachev, R. N. Wang, J. He, T. Liu, M. H. Anderson, W. Weng, _et al._ , Nature 583, 385 (2020).
  * Krause  _et al._ (2012) A. G. Krause, M. Winger, T. D. Blasius, Q. Lin, and O. Painter, Nature Photonics 6, 768 (2012).
  * Zhang  _et al._ (2019) M. Zhang, B. Buscaino, C. Wang, A. Shams-Ansari, C. Reimer, R. Zhu, J. M. Kahn, and M. Lončar, Nature 568, 373 (2019).
  * Foudas  _et al._ (2005) C. Foudas _et al._ , IEEE Trans. Nucl. Sci. 52, 2836 (2005), arXiv:physics/0510229 .
  * Buchner (2021) J. Buchner, The Journal of Open Source Software 6, 3001 (2021), arXiv:2101.09604 [stat.CO] .
  * Chaudhuri  _et al._ (2018) S. Chaudhuri, K. Irwin, P. W. Graham, and J. Mardon, (2018), arXiv:1803.01627 [hep-ph] .
  * Salemi  _et al._ (2021) C. P. Salemi, J. W. Foster, J. L. Ouellet, A. Gavin, K. M. Pappas, _et al._ , Physical Review Letters 127 (2021).