# Forking Around: Correlation of forking practices with the success of a
project

Anurag Dhasmana
<EMAIL_ADDRESS>Arindaam Roy
<EMAIL_ADDRESS>Divjeet Singh Jas
<EMAIL_ADDRESS>Kiranpreet Kaur
<EMAIL_ADDRESS>Pinn Prugsanapan
<EMAIL_ADDRESS>
###### Abstract

Forking-based development has made it easier and straightforward for
developers to contribute to open-source software (OSS). Developers can fork an
existing project and add changes in their local version without interrupting
the development process in the main project. Despite the efficiency of OSS,
more than 80% of the projects are not sustainable [1]. Identifying the
elements related to OSS success can enlighten developers regarding the
sustainability of a project. In our study, we explore whether or not the
inefficiencies which arise due to forking-based development like redundant
development, fragmented communities, lack of modularity, etc. [2] have any
relation to the outcome of a project in terms of sustainability. We formulate
eight metrics to quantify attributes for projects in the ASFI dataset [3]. To
find the correlation between the metrics and the success of a project, we
built a logistic regression model to metrics with significant p-values and
performed backward stepwise regression analysis, using the stepAIC function in
R to cross-check our findings. The findings show that modularity, centralized
management index, and hard forks are consequential for the success of a
project. Developers can use the outcomes of our research to plan and structure
their projects to increase the probability of their success.

##  1 Introduction

OSS is freely available to users, who can modify, enhance or view the source
code implementation. The massive popularity of OSS is due to its collaborative
nature. Collaboration is crucial for developing software projects at scale.
Developers around the globe come together to build an application or add
functionality as a team and sharpen their skills. Collaboration helps in
engendering novel ideas from the community of diverse developers, produces
better outcomes, and envisions support for a project at a massive scale [4].

The birth of social coding sites such as GitHub has completely democratized
the concept of collaboration in OSS. Before the existence of git, developers
had limited options for collaborative development such as SVN. Developers can
nowadays fork a project and add their changes to their local copy without
needing approval from the project owners.

The idea of forking in OSS has evolved. Traditionally, forking implied cloning
a project and branching off new independent development. Forking was rare and
was often intended to compete with or supersede the original project [5, 6,
7]. In the modern view, forks in version control systems such as git are
public copies of repositories that developers can modify, potentially, but not
necessarily, to integrate those changes back into the original repo. Forking
has made the reuse of existing code of software projects more straightforward.
Instead of exchanging emails with software patches, developers can separately
modify and experiment with the original project to either start a new project
or contribute to an existing one [8].

With the rise of social coding and explicit support in version control
systems, forking of OSS repositories has become very popular. Consequently,
developers are creating open-source code at an unprecedented rate. GitHub has
more than 19 million users and 52 million repositories (without excluding
forks) [8].

Despite the popularity of OSS and the relative ease of development provided by
forking-based social coding, more than 80% of the OSS projects fail [1]. This
great failure rate makes us ponder from a social point of view: Why do most of
the OSS projects do not succeed? According to our definition, success means
self-sustaining in the longer run. Previous research has attempted to solve
the question mentioned above [8, 4]. However, to the best of our knowledge, no
work has been done to investigate a direct relationship between an OSS
project’s success or failure and forking practices.

We make a foray in this direction as developing projects with forks comes with
costs. Independent development means contributions are not always visible to
others unless there is an explicit merge-back attempt. Coordination overhead
rises as the number of forks increases [9]. Specifically, we notice the
following hurdles:

  1. 1.

Redundant Development: Contributors often independently implement similar
functionality in their local forks that wastes development time and resources
[9, 2].

  2. 2.

Fragmented Communities: Developers maintain many software versions in parallel
through their forks, resulting in fragmented communities. It makes it harder
for users to choose the project variant that best matches their needs or is
the most actively maintained [2]. Hard forks, sometimes known as "forks of
forks," push the project in different directions than intended by the core
team.

  3. 3.

Management Issues: Lack of modularity in the code structure and limited
communication between contributors and core team members impede the growth of
a project. Monolithic architecture is difficult to reason with and even worse
to change. Inadequate communication between contributors and the core team
leads to uncertainty about future growth.

  4. 4.

Scalability Concerns: As the project’s size grows, so does the number of
commits and the frequency of pull requests. When a significant number of
commits and pull requests in a short period, it becomes difficult for the
maintainers to assess all of the changes promptly.

Overall, we consider many forking-based development inefficiencies in
projects; specifically redundant development, fragmented communities, lack of
modularity, insufficient communication between developers and contributors,
pushing large. numbers of commits, and frequently pull requests (PRs) to the
main branch. We investigate how these inefficiencies are co-related to a
project’s success or failure. Comprehensively, we aim to establish a weak
causal relationship between efficient forking practices and the success of a
project. Specifically, Does adhering to the efficient forking practices by a
project correlate with its success? We explore further to find answers to the
following questions:

######  RQ 1:

Is there a difference in the number of duplicate pull requests (PRs) and hard
forks causing redundancy between successful and unsuccessful projects?

######  RQ 2:

Does the frequency of pull requests (PRs) raised to the main project have
anything to do with the success of a project?

######  RQ 3:

Is the size of the commits pushed to the main branch relevant to the success
of a project?

######  RQ 4:

Is the modularity of a project characteristic of its success?

######  RQ 5:

Is coordination between the developers and the core team when working on a
problem or a new feature important for a project’s success?

To collect our metrics for quantifying the concerns which engender due to the
forking-based inefficiencies and find their significance to the success and
failure of a project, we used the ASFI dataset [3]. The dataset contains a
categorized list of retired (failed) and graduated (successful) projects with
their corresponding GitHub URL. We collect the set of metrics representing
forking-based attributes using GitHub APIs and python library, PyDriller.
Next, we build a logistic regression model to evaluate their significance to a
project’s success. We found that the central management index, logic coupling
index, additive contribution index, and presence of hard forks are significant
metrics that correlate with a project’s success.

We summarize the contribution of our research as follows:

  1. 1.

Investigation of various inefficiencies emerging due to forking-based
development in OSS and defined those quantitatively through a set of metrics.

  2. 2.

Collection of data for eight metrics for further analysis using over 300
projects mentioned in the ASFI dataset.

  3. 3.

Analysis of metrics using a logistic regression model and AIC to find the
significant metrics representing the forking-based development practices
affecting the success of a project.

The code for the metric calculation and analysis is available on GitHub
111Data and scripts are available: https://GitHub.com/anurag-asu/se-project-
team-17.

##  2 Literature Review

In this section, we shed light on the current research and techniques for
analyzing projects that use code forking. We also discuss some recent project
sustainability work. We will present various methods and their shortcomings as
they correlate to our work.

Apache Software Foundation Incubator: Open and diverse communities tend to be
more robust and productive than closed ones [10]. Apache Software Foundation
Incubator (ASFI) is one of the open communities that gives guidelines on how
to accept new committers and enables more community development. They believe
that good software emerges if they foster and take good care of the community
[11].

One of the goals of ASFI is to help projects become self-sustainable and
eventually join the ASF community. ASF guidelines require all projects to make
all communication publicly available through their mailing lists and
committers are required to discuss first before making any changes. If it did
not happen on the mailing list, it did not happen [12]. One unique aspect of
ASF is that these projects are incubated during their “nascent stage of
development,” when there is continuous recruitment of new committers, and
detailed discussions are happening with project mentors [3]. This data is
collected until the project exits the incubator. Since the definition of
success is entirely subjective, categorizing projects as successful or not is
quite challenging [4]. However, ASF alleviates this challenge by employing
experienced personnel to characterize the projects in the incubator.

Yin et al. [3] parsed the Apache Software Foundation Incubator (ASFI) to form
a comprehensive dataset of 312 projects, classified as graduated, retired, and
in incubation by the experienced admins of the AFSI community. This
extrinsically labeled dataset forms our basis for categorizing successful and
failed projects, as rightly envisioned by the authors.

Success and Sustainability of OSS Projects: Recently, ample research has been
done on predicting the success of open source projects [13, 14, 15] and
finding the reasons for their failures [8]. From the research done, it seems
that many projects fail due to their "one-man" nature [4]. Since project
knowledge is limited to one or a few developers, projects will start to
dwindle or die as soon as those single developers quit working. The phrase
"Truck Factor" refers to the number of developers who must be hit by a truck
before the project begins to suffer [16]. Coelho et al. [8] presented a list
of nine reasons like the project was usurped by a competitor, the project
became functionally obsolete, lack of time of the main contributor, etc. as to
why open source projects fail. The authors also demonstrated the presence of a
significant gap in terms of optimal open-source maintenance, like the
availability of contribution guidelines and the use of continuous integration,
between unsuccessful projects and the most popular and active projects on
GitHub. Other works attempted to develop a model based on statistical
correlation [17, 18, 13], but they did not have any objective heuristics to
define the success of a project.

Identifying the elements related to OSS success can aid in the development of
solutions when a project has a setback. In the paper [11], the authors try to
forecast a project becoming sustainable using their novel interpretable model.
Also, based on these models, they offer an approach for real-time monitoring
and actionable suggestions that projects may utilize to rectify their
sustainability trajectories. While they help ongoing projects on the path of
sustainability, our research will help developers to employ efficient forking
and development practices from the inception of the project to increase its
probability of being sustainable.

Forking-based Development: With the increase in social coding, there is a
rapid increase in forking, which is splitting an independent development line
to foster more collaborative development. Prior research has also shown that
forking increases community interaction opportunities [19, 20, 21, 22]. For
example, forks account for more than half of the commits in the Marlin project
[23]. The number of forks a project received seems to be a good reflection of
its quality and worth as it indicates that a project had a lot of interest
from the community and was therefore likely to be good or interesting [19].
The increase in forks, however, spawned new challenges.

_Redundant Development:_ Many projects reject PRs because of redundant
development [2, 20, 21, 24, 25]. Working on a change only to learn that other
developers have already made similar changes or have done so in tandem can be
demotivating. To tackle this, many researchers have built software to detect
such duplicates in the past [9, 25, 26, 27]. In our research, we also develop
a method to detect duplicates and calculate the ratio of duplicate PRs across
different projects to investigate the effects of this inefficiency in a
project.

_Fragmented Communities:_ Many OSS projects attract independent contributors
through forking. These forks can further draw more developers, creating a
separate line of development. However, these hard forks can steer the project
away from its original direction [28, 23]. Such fragmentation might pose a
threat to the long-term viability of open-source projects when two distinct
communities form [2]. When such types of communities spawn, fragmentation-
related attributes can hinder the development of the main project. We
calculate the presence of hard forks across projects to explore if this
attribute leads to a project’s collapse.

_Modularity:_ In an OSS development environment, modularity is a desirable
attribute of projects [13, 29, 30]. MacCormack et al. [31] proposed that more
modular projects would be more appealing to potential contributors. Midha et
al. [13] dis-covered that the modularity of open-source projects is positively
related to technical success. However, modularity may not necessarily
correspond to how developers think [7]. Researchers have presented various
metrics for measuring project modularity in the past. For example, many
methodologies use program analysis to identify relationships between software
structures [32, 33]. In our research, we employ some of these metrics,
specifically, logic coupling index and additive contribution index, to assess
the modularity of projects.

_Coordination mechanisms:_ OSS initiatives involve a large number of
individuals (over hundreds), making coordination a very crucial aspect of a
project. Bird [34] tried to find out if OSS projects are haphazard or
organized. The author concluded that during the OSS life cycle, subgroups
form. The same group of developers ends up working on the same files. The
author, however, did not relate the coordination of developers to its
sustainability. Zhou et al. [2] formulated two different metrics to quantify
coordination among developers in a fork-based development. In our study, we
employ both these metrics, namely centralized management, and pre-
communication index, to find the measure of coordination.

##  3 Hypotheses

Our goal is to find the correlation of efficient forking with a project’s
success. Here we generate our hypotheses based on prior work and formulate
metrics that will help us answer our research questions 1, 2, 3, 4, 5.

Many secondary forks (also known as forks of forks) contribute to other forks
but not to the original repository, causing these forks, also known as hard
forks to wander away from the main development line [28, 35]. Hard forks are
seen as risky and antisocial to projects, as they can fragment a community and
lead to confusion for both maintainers and users [5, 36, 6, 37]. There aren’t
many examples of communities from the main project surviving a hard fork [2].
Redundant development, on the other hand, is one of the common reasons for
rejecting PRs that impede contribution [20, 24, 25]. Working carefully to
resolve an issue only to discover that other developers have made similar
improvements previously or in parallel can be demotivating. Zhou et al. [2]
consider projects that reduce accidental redundancies and have fewer hard
forks as more efficient. Thus, we hypothesize that a successful project should
have a relatively fewer number of duplicate PRs and even fewer hard forks.

######  H 1:

Ratio of duplicate pull requests are lower for a successful project

######  H 2:

Presence of hard forks are lower for a successful project

Many researchers and practitioners emphasize the importance of modularity in
open-source development [29, 13, 30]. Torvalds [30] states, “for without
[modularity], you cannot have people working in parallel.” MacCormack et al.
[38] suggest that more modular projects can be more attractive to potential
contributors. We, therefore, hypothesize that modularity is one of the factors
that contribute to the success of a project. Logic coupling index and additive
contribution index [2] are both predictors of modularity. A lower logic
coupling index, which measures the degree to which two or more files change
together, signals higher modularity. The additive contribution index, which
measures to what degree contributions are additive, indicates better
modularity for a higher value.

######  H 3:

Logic coupling index is lower for a successful project

######  H 4:

Additive contributing index is higher for a successful project

Researchers have exhaustively explored different degrees of coordination and
their tradeoffs in distributed collaboration. Brandts and Cooper [39] state
that central coordination makes it easier to plan and manage a product. We
hypothesize that project coordinating contributions upfront encourages more
focused development activities in a project that aligns with the maintainer’s
vision. The issue tracker in version control systems keeps track of forks that
contain the corresponding code changes for each issue, as other forks are of
little interest to maintainers and unsolicited PRs remain ignored for years
[2]. We also hypothesize that better pre-communication, i.e. evaluating
developers’ discussions on changes for resolving an issue with the core team
before submitting PRs, leads to better coordination and, as a result, a higher
project success rate.

######  H 5:

Central Management index is higher for a successful project

######  H 6:

Pre-communication index is higher for a successful project

A commit represents the snapshot of a project in a given state and has all the
information about the code changes compared to the previous version. Prior
work [40] has modeled the distribution of commit sizes in open source projects
to find its relation with project sizes. The findings reveal that when the
number of developers on a project grows, the size of commits reduces
insignificantly. To correlate the effects of commit size with the success of a
project, we use the dimensionality metric. It calculates the average number of
lines of code changed per commit in a project. We, therefore, hypothesize that
the size of the commits should not be a significant factor for a project’s
success.

######  H 7:

Dimensionality index does not have a correlation success of a project

OSS projects are developed globally by contributors, who collaborate through
the pull-based model that lowers the barrier to entry for OSS developers by
synthesizing, automating, and optimizing the contribution process [41]. Hu et
al. [42] have shown that more review-switching may bring longer resolution
latency of a pull-request. As the number of PRs within a time frame increases,
denoted by the frequency index, there will be longer resolution time and
correspondingly fewer contributions to the project. Therefore, we hypothesize
that the rate of contributions should have a relation with the project’s
capability to incorporate changes efficiently and be sustainable.

######  H 8:

Frequency index value correlates with the success of a project

##  4 Methodology

In this section, we explain the process to gather and calculate the metrics to
quantify the features of the projects.

###  4.1 Dataset

We calculate the metrics for the projects in the ASFI dataset [3]. The
projects are labeled as graduated or retired by the ASFI community. As a
virtue of this, we have an objective classification of the projects. The
dataset contains 312 data points consisting of 204 graduated, 60 retired, and
48 ‘in incubation‘ projects.

_Data Pre-processing:_ Out of these 312 projects, the GitHub URLs for 91
projects were incorrect or no longer functional as they returned HTTP 404
responses. We could only find 63 working URLs replacements for those 91
projects. A summary of the dataset used is given in the table [1] above.

Table 1: ASFI Dataset Summary | Originally | Working GitHub URLs
---|---|---
Incubating | 48 | 48
Graduated | 204 | 200
Retired | 60 | 36
Total | 312 | 284

###  4.2 Data Methods

To gather the metrics for the projects, we used PyDriller [43] and its’ built-
in functions to retrieve the commit data and data which is intrinsically
present in the codebase. PyDriller, however, does not contain information
about forks or pull requests. We used GitHub API [44] to extract relevant
information to calculate the metrics relying on the PRs, Issues, and comment
data such as the presence of hard forks, ratio of duplicate PRs, etc. We used
the pandas library of python to interact with the data and collect the
metrics.

Issues faced with GitHub: GitHub API is free to use, however, its rate is
limited to 5,000 authenticated requests per hour. As a result, certain metrics
consumed days to run. To alleviate this issue, we had to batch and distribute
the data gathering to multiple accounts. Some of the projects listed on the
ASFI website are not hosted on GitHub. Though a GitHub mirror is present for
some of these projects, these mirrors do not contain any kind of communication
level data that one might expect from a GitHub project. This makes it
ineffective to use such repositories as it contributes towards a threat to
internal validity.

Issues faced with PyDriller: PyDriller is a slow tool and, through our
experiments, we found it taking up to 24 hours to calculate metrics. This made
it harder to iterate through code changes.

###  4.3 Metrics

We calculate a set of eight metrics to find a correlation between efficient
forking practices and the success of a project. The metric collected has been
summarized and visualized in Fig.4.

Logic coupling Index: is used to assess the modularity of the project by
measuring the degree to which two or more files need to change together or co-
evolve, in the most recent commits by contributors. We estimate the ratio of
file pairs that were changed, i.e. added or modified, together in those
commits out of all file pairs in the projects using PyDriller. Next, by
computing the mean of recent commits, we aggregate this metric at the project
level. We assess the most recent 150 commits for each project to remove bias
from past but now altered practices. Lower logic coupling indexes suggest
higher modularity, as fewer files are altered at the same time. Logic coupling
Index =

$\textstyle\dfrac{\sum_{commit=1}^{commit=n}\textstyle\dfrac{\textrm{file
pairs changed in the commit}}{\textrm{total file pairs in the
project}}}{\textrm{n}}$

, where n = total number of commits

Additive contribution Index: is a second modularity measure that assesses the
extent to which contributions are additive, i.e contributions by adding files
rather than editing existing ones. We calculate the ratio of new files added
out of all files modified per commit, extracted through PyDriller. For a
project, it’s the average ratio of overall commits from contributors. The
higher additive contribution index shows that more modifications were
additive, implying stronger modularity from the viewpoint of contributors.

Additive contribution Index =

$\textstyle\dfrac{\sum_{commit=1}^{commit=n}\textstyle\dfrac{\textrm{number of
files added in the commit}}{\textrm{number of files changed in the
commit}}}{\textrm{n}}$

, where n = total number of commits

Ratio of Duplicate PRs: is a clear metric for redundant work. We measure the
fraction of the closed PRs rejected, by maintainers, stating as being
redundant. We use GitHub API to fetch the attributes of the PR requests. To
identify duplicate PRs in a project, we use regex matching and look for
comments in the PR such as ‘duplicated’, ‘superseded’, ‘replicated’, etc. in
PR comments.

Ratio of duplicate PRs =

$\textstyle\dfrac{\textrm{duplicated closed PRs identified}}{\textrm{total
PRs}}$

Figure 1: Flowchart of finding hard forks using GitHub API calls. We first
look at the number of forks for each repository. Then, we get the PRs for each
fork, and check if the PR has been merged or not using the GitHub API "/merge"
call for the PR. And, if 2 or more PRs have been merged successfully, we
consider that a hard fork.

Presence of hard forks: a fork that has at least 2 merged pull requests from
external developers is identified as a hard fork [2]. For calculating this
metric, as shown in Fig.1, we use the REST API by GitHub to get a list of
forks, and for each fork, we get a list of pull requests. Then, we see the
merge status of each pull request in that fork to categorize if there are any
merged PRs in a particular fork. If we find 2 or more merged PRs for a fork,
we assign a boolean value of TRUE/FALSE for that project categorizing if the
project has a hard fork or not.

Centralized management Index: This metric is used to quantify the
contributions which start with a centralized discussion before a PR. It
measures the fraction of PRs that link to issues out of all PRs from all
contributors. The list of PRs is fetched using GitHub API for a project and
regex is used to search for keywords like ‘issue #123’ or ‘#123’ in the title,
comments, and/or git messages. Some projects refer to issues not hosted on
GitHub. The reference, however, still follows the convention of starting with
’#’.

Centralized Management Index =

$\textstyle\dfrac{\textrm{PRs with issues referenced in the
title/description}}{\textrm{description total PRs}}$

Pre-communication index: This observes the coordination amongst the developers
and contributors. in this metric, we look for the public announcement made by
the contributors regarding the issue they are working on. We do this in a
hierarchical order, as shown in Fig.2. We first look for the announcement on
the PR, then we look for if the author is one of the assignees of the PR.
Next, we check if the PR is linked to an issue and parse these issues to check
if the author has been assigned the issue or if the author has raised the
issue. Finally, we check the comments in the linked issue for traces of
participation by the author of the PR.

Pre-communication Index=

$\textstyle\dfrac{\textrm{PRs with pre-communication}}{\textrm{Total PRs}}$

Figure 2: Flow chart of figuring out if a PR’s author has pre-communicated
about the issue/feature before starting to work on it. We first look for the
announcement of ‘WIP’, then we look for if the author is one of the assignees
of the PR. Next, we check if the PR is linked to an issue and parse the issue
to check if the author has been assigned the issue. Finally, we check the
comments in the linked issue for traces of participation by the author of the
PR.

Dimensionality Index: measures the size of commits, more precisely average
lines of code changes (added + deleted) per file per commit in a project. This
metric tries to capture the distribution of the size of commits between
successful and failed projects. We used PyDriller [43] and traverse all
commits pushed by developers to a project to get the lines of code changed per
commit. The dimensionality index value for a project is the average of lines
of code changed to the number of files changed across all commits.

Dimensionality Index =

$\textstyle\dfrac{\sum_{commit=1}^{commit=n}\textstyle\dfrac{\textrm{lines of
code changed in the commit}}{\textrm{number of files changed in the
commit}}}{\textrm{n}}$

, where n = total number of commits

Frequency Index: is a metric to take into account the rate of contributions
made by developers to a project and the degree to how difficult it is for the
core team to incorporate the changes. It’s measured as the average number of
PRs raised to the main branch in a particular time interval. We decided on an
initial time interval of 14 days for collecting this metric. Noticing a great
variability in the data collected we then decided to check the number of PRs
raised during a month. Since PyDriller does not have pull requests data we
rely on GitHub APIs [44] to collect them for a project. Using paginated pull
APIs we get all the PRs and count the number of PRs raised during 30 days. The
index value is then just the average of the counts of all the PRs raised.

Frequency Index =

$\textstyle\dfrac{\sum_{interval=1}^{interval=n}\textrm{total number of PRs
raised}}{\textrm{n}}$

, where n = total number of intervals

Figure 3: Boxplot data distribution of significant metrics before (on the
left) and after (on the right) removing outliers.
Label: Green - Graduated projects, Pink - Retired projects. Note: Since
presence of hard forks has binary data, boxplot for it cannot be formed.

###  4.4 Quantitative Analysis

After all, values are gathered and calculated, the metrics are cleaned and
analyzed using R. Using box plots, as shown Fig.3, we inspected the data
variability and removed the outliers that are above or below the 1.5
interquartile range. In Fig.4, we show the data distributions of the collected
metrics in our dataset after the cleaning process.

Next, we use logistic regression to assess the significance of each predictor
since logistic regression is more suitable for a value with a binomial
property like the success of a project. It will tell us whether the predictor
is significant or not. If a predictor is significant we can also use the
coefficient to assess the scale of the predictor whether it has a high
influence or not. Then we use backward stepwise regression, using the stepAIC
function in R, that seeks the model with the least Akaike Information
Criterion(AIC) possible to determine the predictor(s) among all metrics to
find the most relevant variable to the success of a project. Lastly, we
crosscheck how well the model fits using McFadden’s pseudo-R2 measure.

Figure 4: Histogram distribution of our metrics data. Note: presence of hard
forks has binary data. The red arrows point towards the hypothesized higher
efficiency for the metric. Note: Dimensionality of commits and Frequency of
PRs has no arrows as we do not hypothesize any direction of efficiency but
only a presence of correlation with the outcome of a project.

##  5 Results

In this section, we discuss the results from the quantitative data analysis.
Using logistic regression and backward stepwise regression, we examined the
outcomes corresponding to our research questions.

###  5.1 Statistical Analysis

We used logistic regression to test each metric for its significance on the
success or failure of a project based on the graduated and retired status in
ASFI dataset [3]. Table [2] shows the summary of the logistic regression
model, where each metric was separately considered for its significance. And,
table [3] shows the summary of McFadden’s pseudo-R2 measure for all of the
metrics.

Table 2: Logistic Regression Summary Coefficients | Estimate Std. | p-value
---|---|---
(Intercept) | 0.88917 | 0.000258***
Frequency Index | 0.03136 | 0.554512
(Intercept) | 0.41228 | 0.495
Dimensionality Index | 0.01685 | 0.283
(Intercept) | 2.6377 | 2.2e-05***
Additive Contribution Index | -17.3226 | 0.00768**
(Intercept) | 1.7177 | 1.26e-06***
Logic Coupling Index | -151.6851 | 0.0434*
(Intercept) | 0.5952 | 0.0157*
Central Management Index | 13.1342 | 0.0209*
(Intercept) | 1.5773 | 3.93e-16***
Pre-communication Index | 4.0689 | 0.227
(Intercept) | 1.6081 | 2e-16***
Ratio of Duplicate PRs | 324.1628 | 0.25
(Intercept) | 0.4447 | 0.08262
Hard forks | 1.3318 | 0.00132
Signif. codes: 0 ‘***’ 0.001 | ‘**’ 0.01 ‘*’ | 0.05 ‘.’ 0.1 ‘ ’ 1

RQ1: Is there a difference in the number of duplicate PRs and the existence of
hard forks between successful and unsuccessful projects? [H1] [H2]
The ratio of PRs raised having redundant development is not a significant
variable with a p-value of 0.244, as shown in the table [2] above. But the
presence of hard forks [H2] is indeed significant having a p-value of 0.00132.
We, therefore, conclude that the number of duplicate PRs having redundant
development does not correlate to the success of a project but the presence of
hard forks does. We did not find enough evidence to accept [H1]. For the [H2],
we found an interesting observation. The number of hard forks has a positive
coefficient in our regression model, indicating that the number of hard forks
in a project increases the probability of success of a project. But, according
to our hypotheses [H2], there should be relatively fewer hard forks for a
successful project.

RQ2: Does the frequency of pull requests (PRs) raised to the main project have
anything to do with the success of a project? [H8]
To test our hypothesis [H8], we built a logistic regression model keeping
frequency index as an independent variable. From table [2] we found that the
numbers of PR’s raised within a time frame have a p-value of 0.492. This is
greater than 0.05, which implies that it is not a significant indicator in
determining the success of a project. Thus, we reject our hypothesis [H8]
since we didn’t find enough evidence to accept it. In short, we found that the
frequency of pull requests does not correlate with the success of a project.

RQ3: Is the size of the commits pushed to the main branch relevant to the
success of a project? [H7]
The size of the commits is another one of the additional metrics that we
believed to have no relationship with the project success. The dimensionality
index [H7] in our regression model resulted in a p-value of 0.145 which is
greater than the expected value of 0.05. This indicates that it is not a
significant variable relevant to the success of a project. Therefore, we
accept the null hypothesis [H7] to answer our third research question, i.e.
the size of commits does not have any correlation with the success of a
project.

Table 3: R2 value Metrics | McFadden’s pseudo-R2
---|---
Frequency Index | 0.00243885
Dimensionality Index | 0.01388824
Additive Contribution Index | 0.04913989
Logic Coupling Index | 0.02475378
Central Management Index | 0.06018577
Pre-communication Index | 0.1994757
Ratio of Duplicate PRs | 0.01255469
Hard forks | 0.07834034

RQ4: Is modularity of a project characteristic of it’s success? [H3, H4]
We were looking for the correlation of modular architecture of a project with
its success. In our methodology, the modularity of a project is quantified
using two metrics, logic-coupling index, and additive contribution index. From
table [2], both additive contribution index and logic coupling index has a
p-value lower than 0.05, which implied that they are significant metrics to
determine the success of a project. The coefficient of the logic coupling
index is negative, conforming to our hypothesis [H3] that a successful project
should not have logic coupling in multiple files across the project. However,
in contrast to our hypothesis [H4], the coefficient for the additive
contribution index was negative. It implied that more files added while
developing software may increase the possibility of failure to the project.
McFadden’s pseudo-R2 for additive contribution index and logic coupling index
are 0.05 and 0.03 consecutively which indicated that the additive contribution
index is a better predictor compared to Logic coupling in terms of determining
the success of a project. Therefore, we can answer our research question that
modularity is significant to the success of a project. The less logic-coupling
and fewer added files in the project signify a better chance that the project
will be a success.

RQ5: Is coordination between the developers and the core team of a project for
working on an issue or new feature relevant to its success? [H5, H6]
The coordination between developers and the core team was measured using two
metrics, the pre-communication index, and the central management index. As
shown in the table [2], the p-value for the central management index is
0.0209, signifying a correlation to the success of a project. The positive
coefficient for this metric confirmed our hypothesis [H5] and provided us with
enough evidence to accept it. For the pre-communication index, the p-value
suggested that it is not a significant predictor. Hence, we reject the
hypothesis [H6]. Furthermore, McFadden’s pseudo-R2 for central management
index of 0.05 implied that central management is a better model to predict the
success of a project. To summarize, we infer from our results that the
interaction between the contributors of a project before a PR is important for
its success. Whereas, declaration of commitment to an issue is not necessarily
important for its success.

Table 4: stepAIC Summary Coefficients | Estimate Std. | p-value
---|---|---
(Intercept) | 1.009589 | 0.1071
Dimensionality | 0.007572 | 0.3596
Additive Index | -8.568041 | 0.0569
Hard forks | 1.083335 | 0.0144*
Signif. codes: 0 ‘***’ | 0.001 ‘**’ 0.01 ‘*’ | 0.05 ‘.’ 0.1 ‘ ’ 1

###  5.2 Backward Stepwise Regression

After we investigated the significance of the metrics individually to a
project’s success using a logistic regression model, we performed a backward
stepwise selection analysis, using stepAIC function in R, on all of our
metrics combined. The results, as shown in the table [4], confirms that two of
the metrics which are additive contribution index and presence of hard forks
are significant predictors for a project’s success. The central management
index and logic coupling index, even though correlated individually, are not
considered as significant predictors in the stepAIC analysis.

###  5.3 Conclusion

From our findings, we answer our main research question, that not all forking
practices we examined to lead to a project’s success. Every measure of
modularity (logic coupling or additive contribution), which is ardently
supported by the software community, does not increase the odds of the success
of a project. The same findings were observed in the case of the management of
a project. Understanding the trajectory of the project’s future goals by
contributors and maintainers (central management) is more crucial than
communication about who will be working on a particular task (pre-
communication). Furthermore, the presence of hard forks in a project, as well
as the rate at which contributors attempt to incorporate modifications to a
project, have no negative impact on its success. The size of the commits
(dimensionality) is not a determining factor as it does not vary significantly
between successful and retired projects. Since multiple duplicate pull
requests (PRs) are rare in projects in the ASFI dataset, their presence isn’t
a significant factor that is correlated to a project’s failure.

##  6 Discussion

In this section, we present the main findings derived from the results of our
research. We also mention the strengths and limitations of our study,
including threats to validity. Finally, we discuss future work that can be
done based on our findings.

###  6.1 Main Findings

Based on the results, we present the following findings based on our metrics
data.

Redundant development: In software engineering, redundant work is seen as a
negative attribute in project development. We hypothesized that the ratio of
duplicate PRs should be low [H1] for a successful project as it indicates that
people are doing a similar task, which wastes development time and resources
[45, 2]. Our model doesn’t suggest that there’s adequate evidence that the
ratio of duplicate PRs has a significant impact on project success. We suspect
this happens because duplicates are rare in most projects in ASFI [2]. This
idea is further supported by the fact that we get sparse data for this metric.
The number of projects with duplicate PRs was extremely low, and the quantity
of duplicate PRs was minimal when compared to the overall number of PRs
closed.

Hard Forks: We also hypothesized that the presence of hard forks is not a
desired quality for a successful project [H2] as it leads to fragmented
communities [28, 2]. However, we discovered that hard forks are not a
"negative" attribute that will cause a project to fail based on the results of
our statistical analysis. We infer this because the coefficient for this
metric was positive implying that the presence of hard forks increases the
likelihood of a project becoming successful. We further believe that since the
project has been successful, many developers wish to use it as their base and
contribute to it, or change it to fit their needs. In addition, we find that
regardless of the presence of hard forks, successful projects with solid
central management and a clear vision of the project’s trajectory draw
contributors to the main project and keep it alive.

Modularity: The importance of modularity in software engineering projects is
well understood as it facilitates software evolution and provides
collaboration [46, 45, 47]. Both of the modularity measures, i.e. logic
coupling index, and additive contribution index, have been found as
significant factors in determining success. The coefficient of additive
contribution index, however, in the logistic regression table [2] suggests
otherwise [H4]. From our interpretation, both measures of modularity do not
signify the direction of a project, i.e. whether it is going to be successful
or not. To be exact, more files added than being modified during the
development does not mean the project is going to be successful.

Coordination: Even though forking has given the freedom to the developers to
create a local copy without any intervention from the maintainers and
contribute back when the development is done [48], coordination is associated
with significant improvements to the efficiency of an open-source community.
In our research, the result supports the hypothesis on the importance of
central coordination. We have the evidence to accept [H5] which indicates that
the central management index is highly influential to the success of a
project. On the contrary, we do not have enough evidence to confirm the
significance of the pre-communication index. A possible explanation to why we
do not have enough evidence is that the contributors could have used a
communication channel other than GitHub Issue and PR section. Furthermore, we
only examined the recent 150 PRs and Issues discussions associated with the
PRs to reduce the exorbitant time of fetching the entire data over the GitHub
API. Nevertheless, from the open-source communication over GitHub, it is
implied from our analysis that pre-communication was not significant for the
success of the ASFI projects. From the result of our research, we can conclude
that it is more important to have central communication on "what" to do with
the project rather than "who" would do the part.

Dimensionality and frequency of contributions: We also formulated two of the
metrics which measure the frequency of the PRs raised and the size of the
commits pushed by the developers. Our model suggests that the size of the
commits does not correlate with the success of a project which aligns with our
[H7]. Kolassa et al. in their exploration [40] also mentions that indeed the
scale of a project or the number of developers making contributions have no
significant influence on the commit size, i.e. it more or less remains the
same. However, our AIC model results, as seen in table [4] indicates the
dimensionality metric as one of the significant predictors, which is
surprising as we didn’t find enough evidence from our model to reject the
[H7]. We need further research and analysis to check why the metric came out
significant in AIC. Our model also did not give us any significant evidence to
accept the [H8]. We assume this happens as the core team isn’t concerned about
the quantity of PRs but about the ones targeting a specific issue, tracked by
the issue trackers in the version control system. So even though the rate of
contribution across the project is high, good central coordination keeps the
focus of the maintainers on the PRs achieving the goals as envisioned by the
maintainers.

###  6.2 Strengths and Limitations

Prior research on the sustainability of projects did not have any objective
way to declare a project as a success or failure. In our study, we examined
the ASFI dataset, which includes specific labels for the status of each
project. As a result, we were able to link the project’s qualities to
objectively specified labels. Previous works [2, 7, 19, 27] have looked into
efficient forking practices to foster contributions, but they didn’t study
whether self-sustaining projects use those efficient forking practices. We
investigated this through our study, where we test our hypotheses and
formulate metrics for assessing the correlation between forking practices and
the sustainability of a project. Additionally, our project features robust
python scripts for mining GitHub data, which can be used to enhance the
project list.

Threats to validity: One of the main drawbacks of using the ASFI dataset is
generalizing the research implications beyond the ASF community. The list of
projects is of a relatively small scale, with hundreds of projects only. There
is also limited diversity as we see an inherent imbalance between graduated
vs. retired projects. There may also be a presence of systematic bias in our
data, meaning what we measure is not the likelihood of getting that metric
data. With automatic detection, false-positive errors are inevitable, which
means that some pull requests (PRs) reported as duplicate pull requests (PRs)
aren’t duplicates when their context is studied thoroughly. Due to limited
access issues in the GitHub API, we did not identify external contributors. As
a result, some of the forks may be incorrectly categorized as hard forks only
based on the number of PRs in that fork.

Furthermore, developers may have opted to communicate outside of GitHub Pull
Request and Issue sections. We might have introduced some threat to internal
validity as we only calculate our metrics based on the information available
on the project’s GitHub page and not external chats or emails. The pre-
communication index required more than 100,000 requests over the GitHub API
due to the hierarchical nature of the metric. To reduce the calculation time
of this metric, we only consider the first 5 pages or 150 recent Pull Requests
per project.

The assumptions mentioned for redundant development are somewhat contrary to
Apache Software community guidelines as communication is of utmost importance
in ASF projects to cause redundancy. However, it is not hard to imagine these
scenarios where that data may not reflect as we think it is.

###  6.3 Future Work

We performed backward stepwise regression using all of our metrics as a
starter model. As shown in the table [4], the metrics that are left are
dimensionality index, additive contribution index, and presence of hard forks.
The dimensionality index, however, was not a significant predictor when
considered individually and hence needs further research. A machine learning
model can also be created to forecast the probability of success of a project.
For some of our metrics, the quality of data collected can be improved by
incorporating data about external contributors and external communication
channels. Also, for metrics relying heavily on GitHub APIs, we capped the
number of pages fetched considering the rate limit policy of GitHub and the
time constraints. We can refine them by using alternate GitHub mining sources
such as GHTorrent. Finally, we can extend our research beyond the ASFI dataset
to a larger dataset with a variety of projects to generalize the findings of
our research to the broader OSS community.

##  7 Team member and Attestation

Anurag Dhasmana, Arindaam Roy, Divjeet Singh Jas, Kiranpreet Kaur, Pinn
Prugsanapan
_*All team members have read and contributed equally to this research._

## References

  * [1] Likang Yin, Zhunagzhi Chen, Qi Xuan, and Vladimir Filkov.  Sustainability forecasting for apache incubator projects, 2021.
  * [2] Shurui Zhou, Bogdan Vasilescu, and Christian Kästner.  What the fork: a study of inefficient and efficient forking practices in social coding.  ESEC/FSE 2019: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, page Pages 350–361, August 2019.
  * [3] Likang Yin, Zhiyuan Zhang, Qi Xuan, and Vladimir Filkov.  Apache software foundation incubator project sustainability dataset.  In 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR), pages 595–599, 2021.
  * [4] Junaid Maqsood, Iman Eshraghi, and Syed Ali.  Success or failure identification for github’s open source projects.  pages 145–150, 01 2017.
  * [5] Karl Fogel.  Producing open source software: How to run a successful free software project.  O’Reilly Media, Inc., 2005.
  * [6] Linus Nyman.  Hackers on forking.  In OpenSym ’14: Proceedings of The International Symposium on Open Collaboration, pages 1–10, 08 2014.
  * [7] Linus Nyman, Tommi Mikkonen, Juho Lindman, and Martin Fougère.  Perspectives on code forking and sustainability in open source software.  In 8th International Conference on Open Source Systems (OSS), Sep 2012, Hammamet, Tunisia. pp.274-279, pages 1–10, 09 2012.
  * [8] Jailton Coelho and Marco Valente.  Why modern open source projects fail.  pages 186–196, 08 2017.
  * [9] Hamzeh Eyal Salman.  Feature-based insight for forks in social coding platforms.  Information and Software Technology, 140:106679, 2021.
  * [10] The Apache Software Foundation guide :: Guide to successful community building.  https://incubator.apache.org/guides/community.html.  Accessed: 2021-12-05.
  * [11] Likang Yin, Zhuangzhi Chen, Qi Xuan, and Vladimir Filkov.  Sustainability forecasting for apache incubator projects.  ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages Pages 1056–1067, August 2021.
  * [12] Peter C. Rigby and Ahmed E. Hassan.  What can oss mailing lists tell us? a preliminary psychometric text analysis of the apache developer mailing list.  In Fourth International Workshop on Mining Software Repositories (MSR’07:ICSE Workshops 2007), pages 23–23, 2007.
  * [13] Vishal Midha and Prashant Palvia.  Factors affecting the success of open source software.  Journal of Systems and Software, 85(4):895–905, 2012.
  * [14] Judith Piggott.  Open source software attributes as success indicators.  2013\.
  * [15] Chandrasekar Subramaniam, Ravi Sen, and Matthew L. Nelson.  Determinants of open source software project success: A longitudinal study.  Decision Support Systems, 46(2):576–585, 2009.
  * [16] Guilherme Avelino, Marco Túlio Valente, and André C. Hora.  What is the truck factor of popular github applications? a first assessment.  PeerJ Prepr., 3:e1233, 2015.
  * [17] Stefano Comino, Fabio Manenti, and Maria Laura Parisi.  From planning to mature: on the determinants of open source take off.  Maria Laura Parisi, 02 2007.
  * [18] Sang-Yong Lee, Hee-Woong Kim, and Sumeet Gupta.  Measuring open source software success.  Omega, 37:426–438, 04 2009.
  * [19] Laura Dabbish, H. Stuart, Jason Tsay, and James Herbsleb.  Social coding in github: Transparency and collaboration in an open software repository.  pages 1277–1286, 02 2012.
  * [20] Georgios Gousios, Martin Pinzger, and Arie van Deursen.  An exploratory study of the pull-based software development model.  Proceedings of the 36th International Conference on Software Engineering, 2014.
  * [21] Georgios Gousios, Andy Zaidman, Margaret-Anne Storey, and Arie van Deursen.  Work practices and challenges in pull-based development: The integrator’s perspective.  In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, volume 1, pages 358–368, 2015.
  * [22] Ines Mergel.  Open collaboration in the public sector: The case of social coding on github.  Government Information Quarterly, 32(4):464–472, 2015.
  * [23] Ştefan Stănciulescu, Sandro Schulze, and Andrzej Wąsowski.  Forked and integrated variants in an open-source firmware project.  In 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME), pages 151–160, 2015.
  * [24] Igor Steinmacher, Gustavo Pinto, Igor Scaliante Wiese, and Marco Aurélio Gerosa.  Almost there: A study on quasi-contributors in open-source software projects.  In 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE), pages 256–266, 2018.
  * [25] Yue Yu, Zhixing Li, Gang Yin, Tao Wang, and Huaimin Wang.  A dataset of duplicate pull-requests in github.  pages 22–25, 05 2018.
  * [26] Zhixing Li, Gang Yin, Yue Yu, Tao Wang, and Huaimin Wang.  Detecting duplicate pull-requests in github.  pages 1–6, 09 2017.
  * [27] Luyao Ren, Shurui Zhou, Christian Kästner, and Andrzej Wąsowski.  Identifying redundancies in fork-based development.  In 2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER), pages 230–241, 2019.
  * [28] Kam Hay Fung, Aybüke Aurum, and David Tang.  Social forking in open source software: An empirical study.  In CAiSE Forum, 2012.
  * [29] Bruce Kogut and Anca Metiu.  Open-source software development and distributed innovation.  Oxford Review of Economic Policy, 17:248–264, 02 2001.
  * [30] Linus Torvalds.  The linux edge.  Commun. ACM, 42:38–39, 1999.
  * [31] Alan MacCormack, John Rusnak, and Carliss Baldwin.  Exploring the structure of complex software designs: An empirical study of open source and proprietary code.  Management Science, 52:1015–1030, 07 2006.
  * [32] James M. Bieman and Linda M. Ott.  Measuring functional cohesion.  IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, 20(8):644–657, 1994\.
  * [33] Thomas J. Emerson.  A discriminant metric for module cohesion.  In Proceedings of the 7th International Conference on Software Engineering, ICSE ’84, page 294–303. IEEE Press, 1984.
  * [34] Christian Bird.  Sociotechnical coordination and collaboration in open source software.  In 2011 27th IEEE International Conference on Software Maintenance (ICSM), pages 568–573. IEEE, 2011.
  * [35] Stefan Stanciulescu, Thorsten Berger, Eric Walkingshaw, and Andrzej Wasowski.  Concepts, operations, and feasibility of a projection-based variation control system.  2016 IEEE International Conference on Software Maintenance and Evolution (ICSME), pages 323–333, 2016.
  * [36] Andrew M St Laurent.  Understanding Open Source and Free Software Licens- ing: Guide to Navigating Licensing Issues in Existing & New Software.  O’Reilly Media, Inc., 2004.
  * [37] Eric S Raymond.  The Cathedral & the Bazaar: Musings on linux and open source by an accidental revolutionary.  O’Reilly Media, Inc., 2001.
  * [38] Alan MacCormack, John Rusnak, and Carliss Baldwin.  Exploring the structure of complex software designs: An empirical study of open source and proprietary code.  Management Science, 52:1015–1030, 07 2006.
  * [39] David J. Cooper.  An experimental study of communication and centralization.  2018\.
  * [40] Carsten Kolassa, Dirk Riehle, and Michel Salim.  A model of the commit size distribution of open source.  volume 7741, 08 2014.
  * [41] Zhixing Li, Yue Yu, Minghui Zhou, Tao Wang, Gang Yin, Long Lan, and Huaimin Wang.  Redundancy, context, and preference: An empirical study of duplicate pull requests in oss projects.  IEEE Transactions on Software Engineering, pages 1–1, 2020.
  * [42] Dongyang Hu, Yang Zhang, Junsheng Chang, Gang Yin, Yue Yu, and Tao Wang.  Multi-reviewing pull-requests: An exploratory study on github oss projects.  Information and Software Technology, 115:1–4, 11 2019.
  * [43] ishepard.  PyDriller.  https://github.com/ishepard/pydriller, 2021.  [Online; accessed 5-Dec-2021].
  * [44] Github.  GitHub REST API.  https://docs.github.com/en/rest, 2021.  [Online; accessed 5-Dec-2021].
  * [45] Design Rules: The Power of Modularity. Vol. 1.  Producing open source software: How to run a successful free software project.  MIT press, 2000.
  * [46] How do committees invent. Datamation, 1968.
  * [47] David Parnas.  On the criteria to be used in decomposing systems into modules.  Communications of the ACM, 15:1053–, 12 1972.
  * [48] Laura Dabbish, H. Stuart, Jason Tsay, and James Herbsleb.  Social coding in github: Transparency and collaboration in an open software repository.  pages 1277–1286, 02 2012.


# Domain wall interpretation of the PTA signal confronting black hole
overproduction

Yann Gouttenoire  0000-0003-2225-6704<EMAIL_ADDRESS>School of
Physics and Astronomy, Tel-Aviv University, Tel-Aviv 69978, Israel  Edoardo
Vitagliano  0000-0001-7847-1281 Racah Institute of Physics, Hebrew University
of Jerusalem, Jerusalem 91904, Israel

###### Abstract

Recently, NANOGrav has reported the observation of a stochastic gravitational
wave background (SGWB) at nano-Hertz frequencies. String-wall networks and
domain walls have been proposed as possible sources. To be cosmologically
viable, these topological defect networks must annihilate before they dominate
the energy budget of the universe, producing a SGWB. However, a part of the
network can copiously produce primordial black holes that exceed current
bounds. Performing a Bayesian analysis of pulsar timing residual datasets we
find that the SGWB detected in PTA data is therefore hardly compatible with
such an origin. This lends credibility to other interpretations, including
supermassive black hole mergers, first order phase transitions, Nambu-Goto
strings, and curvature-induced gravitational waves.

##  I Introduction

The North American Nanohertz Observatory for Gravitational Waves (NANOGrav), a
pulsar timing array (PTA) part of the International Pulsar Timing Array
(IPTA), comprising the European Pulsar Timing Array (EPTA), the Parkes Pulsar
Timing Array (PPTA) in Australia, and the Indian Pulsar Timing Array Project
(InPTA), has recently reported the observation of a stochastic gravitational
wave background (SGWB) with a strain of $2.7^{+0.7}_{-0.6}\times 10^{-15}$
(median with 90% credible interval) at the frequency $\rm yr^{-1}$,
corresponding to a total energy density in the sensitivity band of
$\Omega_{\rm GW}h^{2}=6.5^{+4.1}_{-2.8}\times 10^{-9}$ [1]. This result
confirms the hint to a SGWB observed in previous years [2], by EPTA [3], PPTA
[4] and IPTA [5]. Sources of nanoHz GWs could be a population of supermassive
black hole binaries [6, 7, 8] or could be related to early universe phenomena
[9], such as first order phase transitions [10], second-order gravitational
waves produced during the formation of primordial black holes [11, 12, 13,
14], and topological defects [15, 16, 17, 18, 19, 20]. The simplest mechanism
forming Domain walls (DW) is the spontaneous breaking of a discrete symmetry,
e.g. $\mathcal{Z}_{2}$ [21, 22, 23, 24]. DW dilute slower than radiation in
expanding cosmology [25, 26]. To be viable, there must exist an energy bias
between distinct vacua so that DW are pulled toward annihilating with each
other. Upon annihilation, the DW system can abundantly produce GWs [27, 28,
29, 30, 24] with a nanoHz peak frequency within PTA window if the system
annihilates at a temperature $T_{\rm ann}\sim 10\,\rm MeV$ [31]. A another DW
formation scenario is when a global $U(1)$ symmetry is first spontaneously
broken to form cosmic strings and then is later explicitly broken when the
Goldstone mode receives mass corrections [32]. A network of global strings
alone cannot be the source of the observed PTA signal, as the string tension
needed to source such a large amplitude would imply the abundant production of
Goldstone bosons [33, 34, 35] in conflict with the Big-Bang Nucleosynthesis
(BBN) bound on the effective number of neutrino species $N_{\rm eff}$. The
evolution of the string-DW network can be more complicated and depends on the
number $N$ of minima along the orbit of vacua. For $N=1$, DW gets bounded by
strings and rapidly annihilate. The presence of DW only weakly enhance the GW
signal from global strings [36] and we conclude that interpretation of PTA
signal in term of string-wall network are excluded by $N_{\rm eff}$ bounds
[33, 34, 35]. If $N>1$, a stable string-wall network is produced, and the
evolution is similar to the discrete symmetry breaking domain-wall system
described above. Such system, often considered in the context of the QCD axion
[37, 38, 39, 40, 41, 42, 43] and more recently in the context of axion-like
particles [44, 45] and high-quality QCD axion models [46, 47], has also been
considered as a source of a signal compatible with PTA observations (see e.g.
[48, 20]).

In this Letter, we perform a Bayesian analysis of PTA datasets NG12.5 [2] and
IPTADR2 [5] in presence of GW from DW annihilation. We find that the
interpretation of pure domain-wall and $N>1$ string-wall systems as a possible
source of the PTA signal are in tension with the B overproduction of
primordial black holes (PBHs). In both cases, the system can feature spherical
domains which collapse to PBHs when they shrink below their Schwarzschild
radius [31, 45, 47]. While this mechanism might potentially be related to the
production of PBH dark matter [47] or of supermassive black holes [45], we
show that the same mechanism would overproduce PBHs if the annihilation
temperature and energy stored in the system are tuned to produce the amplitude
and frequency of the SGWB observed by PTAs.

##  II GWs from DW annihilation

### Friction vs scaling regime.

Denoting by $v$ their typical DW velocity, we can estimate that DW have
typical curvature radius $R\simeq vt$. Initially, the work of their surface
tension $\sigma$ with equivalent pressure $\mathcal{P}_{T}=\sigma/R$ toward
straightening the DW is dampened by friction pressure
$\mathcal{P}_{V}\simeq\,\beta\,T^{4}$ where the dimensionless $\beta$ sets the
strength of DW-plasma interactions with the plasma. DW starts moving with
relativistic velocity $v\simeq\mathcal{O}(0.1)$ below the temperature

$T_{\rm rel}\simeq 0.8g_{\star}^{1/4}\sqrt{\frac{\sigma}{\beta M_{\rm
pl}}}\simeq\frac{530\leavevmode\nobreak\ \rm
MeV}{\sqrt{10v\beta}}g_{\star}^{\frac{1}{4}}\left(\frac{\sigma^{1/3}}{10^{5}\leavevmode\nobreak\
\rm GeV}\right)^{\\!\frac{3}{2}},$ (1)

where $M_{\rm pl}\simeq 2.44\times 10^{18}\leavevmode\nobreak\ \rm GeV$,
$g_{*}$ the number of relativistic degrees of freedom and where we used
Friedmann’s equation $T=1.2\sqrt{M_{\rm pl}/t}/g_{*}^{1/4}$. The size of the
friction coefficient $\beta$ is model dependent [49]. In the present work, we
set $\beta\ll 1$ and briefly discuss its implication at the end. Numerical
simulations have shown that the energy stored in friction-less DW reaches the
scaling regime as cosmic strings do, [26, 23]

$\rho_{\rm DW}=\frac{\sigma}{R},\qquad\textrm{with}\leavevmode\nobreak\
R\simeq t/\mathcal{A},$ (2)

where $\mathcal{A}\simeq 0.8\pm 0.1$ is fitted on numerical simulations [30].
This results in the DW energy density redshifting slower than the main
background fluid $\rho_{\rm bkg}\simeq M_{\rm pl}^{2}/t^{2}$ such that DW
rapidly dominate the energy density of the universe [32] below the temperature

$T_{\rm
dom}\simeq\frac{1.4}{g_{\star}^{1/4}}\sqrt{\frac{\mathcal{A}\,\sigma}{M_{\rm
pl}}}\simeq 30\leavevmode\nobreak\ \rm MeV\leavevmode\nobreak\
\mathcal{A}^{1/2}\,g_{\star}^{\\!1/4}\left(\frac{\sigma^{1/3}}{10^{5}\leavevmode\nobreak\
\rm GeV}\right)^{\\!3/2}.$ (3)

### Bias potential terms.

We assume the presence of high dimensional operators that explicitly break
$U(1)$ symmetry. This transforms the flat direction into a discrete collection
of vacua. The vacuum energy difference $\mathcal{P}V=V{\rm bias}$ between
these new minimum points acts as a source of pressure which make DW repel or
attract each other until their eventual annihilation. DW annihilate when the
vacuum pressure surpasses the pressure $\mathcal{P}_{T}=\sigma/R$ arising from
their surface tension $\sigma$, below the temperature

$T_{\rm ann}\simeq\frac{100\leavevmode\nobreak\ \rm
MeV}{\mathcal{A}^{1/2}g_{\star}^{\\!1/4}}\left(\frac{10^{5}\leavevmode\nobreak\
\rm GeV}{\sigma^{1/3}}\right)^{3/2}\left(\frac{V_{\rm
bias}^{1/4}}{40\leavevmode\nobreak\ \rm MeV}\right)^{2}.$ (4)

### GW signal.

During the annihilation process, DWs are driven to relativistic speed and
radiate GW [21, 22, 40, 27]. The GW power spectrum today produced by long-
lived DWs annihilating at $T_{\rm ann}$ can be expressed as [28, 29, 30]

$\displaystyle\Omega_{\rm GW}h^{2}=\Omega_{\rm peak}h^{2}S_{\rm DW}(f)$ (5)

where the peak amplitude today follows from the quadrupole formula [23]

$\Omega_{\rm peak}h^{2}\simeq 7.2\times 10^{-10}\tilde{\epsilon}_{\rm
gw}\mathcal{A}^{2}\left(\frac{10}{g_{*s}(T_{\rm ann})}\right)^{4/3}\\\
\times\left(\frac{\sigma^{1/3}}{100\leavevmode\nobreak\ \rm
TeV}\right)^{6}\left(\frac{100\leavevmode\nobreak\ \rm MeV}{T_{\rm
ann}}\right)^{4}.$ (6)

while $\tilde{\epsilon}_{\rm gw}\simeq 0.7\pm 0.4$ is fitted on lattice
simulations. [30] The peak frequency today is given by

$f_{\rm peak}=\frac{a(t_{\rm ann})}{a(t_{0})}H(t_{\rm ann})\simeq
1.1\leavevmode\nobreak\ {\rm nHz}\left(\frac{g_{\star}(T_{\rm
ann})}{10}\right)\\\ \times\left(\frac{10}{g_{*s}(T_{\rm
ann})}\right)^{1/3}\left(\frac{T_{\rm ann}}{10\leavevmode\nobreak\ \rm
MeV}\right).$ (7)

We model the spectral function

$S_{\rm DW}(f)=\frac{2}{\left(f/f_{\rm peak}\right)+\left(f_{\rm
peak}/f\right)^{3}},$ (8)

where the IR slope is $\Omega_{\rm GW}\propto f^{3}$ to respect causality [50,
51, 52, 53] and the UV slope is $\Omega_{\rm GW}\propto f^{-1}$ as suggested
by lattice simulations results [30].

Figure 1:  GW spectra from DW annihilation (solid) with surface tension
$\sigma$ and annihilation temperature $T_{\rm ann}$ set at the mean values of
the posterior distribution in Tab. 1. The mean posterior for the GW signal
from SMBH binaries (dotted) is shown with strain amplitude $A_{\rm SMBH}\simeq
2.1-3.7\times 10^{-15}$ at $1\leavevmode\nobreak\ \rm yr^{-1}$ for NG12.5 and
IPTADR2 respectively. The gray band is the $90\%$ CL for the SGWB calculated
from a Monte-Carlo generated binary population [54].

##  III Bayesian analysis of PTA data

We performed a comprehensive Bayesian analysis of the DW interpretation of PTA
signal. Waiting for NANOGrav $15\leavevmode\nobreak\ {\rm yr}$ [1] to release
their data publicly, we used the first $5$ frequency bins of NANOGrav
$12.5\leavevmode\nobreak\ {\rm yr}$ [2] and the first 13 frequency bins of
IPTA DR2 [5]. To extract the GW signal from the various source of noises, we
closely followed the methodologies employed by the NANOGrav [2] and IPTA [5]
research groups, with additional insights from other relevant literature [48,
55]. We modified the software tools known as ${\tt enterprise}$ [56] and ${\tt
enterprise\\_extensions}$ [57] to include the spectrum from DW annihilation.
The parallel-tempering Markov Chain Monte-Carlo sampler, ${\tt PTMCMC}$ [58],
was used to explore the posterior distribution and, see the mean values in
Tab. 1, and ${\tt GetDist}$ tool [59] was used to visualize it, see Fig. 3.

$\log_{10}\leavevmode\nobreak\ \rm[GeV]$ | Posterior mean |
---|---|---
NG12.5 | IPTA |
$\mathcal{\sigma}^{1/3}$ | $4.37^{+0.25}_{-0.29}$ |  $4.928\pm 0.086$
$T_{\rm ann}$ | $-2.21\pm 0.43$ |  $-1.44^{+0.14}_{-0.11}$
Table 1:  Mean parameter values of the posterior distribution for the domain
wall interpretation of the PTA common red process.

##  IV BBN constraints

Ref. [48] has shown that the DW interpretation of the PTA GW signal is in
slight tension with BBN if DW annihilate into a hidden sector. We now revisit
the argument and minoring slight numerical differences, we reach similar
conclusions. The energy density fraction in DWs at temperature $T_{\rm ann}$,
normalized to radiation background reads:

$\alpha_{\rm DW}\equiv\frac{\rho_{\rm DM}}{\rho_{\rm
rad}}=\sqrt{\frac{g_{\star}(T_{\rm
ann})}{10.75}}\left(\frac{\sigma^{1/3}}{100\leavevmode\nobreak\ \rm
TeV}\right)^{3}\left(\frac{14\leavevmode\nobreak\ \rm MeV}{T_{\rm
ann}}\right)^{2},$ (9)

where we approximated $g_{*,s}=g_{*}$. The presence of extra number $\Delta
N_{\rm eff}$ of relativistic degrees of freedom at BBN and CMB would change
the expansion rate of the universe and impact the CMB data or the abundance of
light elements [60, 61]. In App. A, we show that relics with energy fraction
$\alpha_{\rm DW}$ contribute to the number of relativistic degrees of freedom
by:

$\Delta N_{\rm eff}=7.4\,\alpha_{\rm DW}\leavevmode\nobreak\
\lesssim\leavevmode\nobreak\ 0.3,$ (10)

We must distinguish two scenarios. If DW annihilate into a secluded sector,
then Eq. (10) applies for all $T_{\rm ann}$. Instead if DW annihilate
dominantly into Standard Model (SM) degrees of freedom, then Eq. (10) relaxes
for $T_{\rm ann}\gtrsim 1\leavevmode\nobreak\ \rm MeV$ [62]. Another bounds
from the possibility for DWs to dominate the energy budget of the universe. In
this case the universe starts expanding as $a\propto t^{2}$ [26] and it is
uncertain if DWs can efficiently annihilate in such a rapidly expanding
universe. To be conservative, we impose that DW must disappear before
dominating the universe:

$T_{\rm ann}\leavevmode\nobreak\ \gtrsim\leavevmode\nobreak\ T_{\rm dom}.$
(11)

The corresponding $N_{\rm eff}$ and DW domination constraints are shown in
purple and gray in Figs. 2 and 3.

Figure 2:  In orange and blue, we show the $68\%$ and $90\%$ confidence levels
of the DW interpretation of the PTA GW signal after performing a Bayesian
analysis of NG12.5 and IPTADR2 datasets. The scenario with dark reheating is
in slight tension with the $N_{\rm eff}\lesssim 0.3$ constraints. However, it
is in strong tension with PBH overproduction bounds, from DM overclosure
green, microlensing [63] (purple), $\rm kHz$ GW interferometers [64, 65, 66,
67, 68] (blue) and CMB [69, 70, 71] (yellow).

Figure 3:  Same as Fig. 2 where PBH constraints combined in brown. Results
from numerical simulations [43] suggest the annihilation rate exponent to be
$9\lesssim\alpha\lesssim 28$ which is exclude the DW interpretation of the PTA
signal. In theory, only a DM network annihilating faster than $\rho^{2}_{\rm
GW}\propto 1/t^{50}$ could explain PTA signal, assuming that the GW production
does not get suppressed.

##  V PBH constraints

Both pure domain-wall and $N>1$ string-wall systems feature closed
configurations. During the scaling regime, DW have a size comparable to the
cosmic horizon $\simeq t$. Closed DW collapse into PBHs if they shrink below
their Schwarzschild radius which can happen if the ratio

$p(t)=\frac{R_{\rm Sch}(t)}{t}=\frac{2GM(t)}{t}$ (12)

becomes smaller than one [72]. Close to the start of the annihilation process,
the mass within a closed wall reads

$M(t)\simeq\frac{4}{3}\pi t^{3}V_{\rm bias}+4\pi t^{2}\sigma.$ (13)

DW annihilate at $t_{\rm ann}$ when the volume term in Eq. (13) dominates over
the surface term. Therefore, the ratio $p(t)$ increases with time as $t^{2}$.
The temperature at which PBH collapse is defined by $p(T_{\rm PBH})=1$, which
implies

$T_{\rm PBH}\simeq 120\leavevmode\nobreak\ {\rm MeV}\mathcal{A}^{1/4}\,g_{\rm
eff,1}^{1/8}\left(\frac{T_{\rm ann}}{1\leavevmode\nobreak\ \rm
GeV}\right)^{1/2}\left(\frac{\sigma^{1/3}}{10^{5}\leavevmode\nobreak\ \rm
MeV}\right)^{3/4}$ (14)

where $g_{\rm eff,1}\equiv g_{*}^{2}(T_{\rm ann})/g_{*}(T_{\rm PBH})$, and
corresponding time is $t_{\rm PBH}=1/2H(T_{\rm PBH})$. The PBH mass is
$M(t_{\rm PBH})\simeq\frac{4\pi}{3}V_{\rm bias}t_{\rm PBH}^{3}$:

$M_{\rm PBH}\simeq\frac{19\leavevmode\nobreak\ {\rm
M_{\odot}}}{\mathcal{A}^{1/2}g_{*}^{1/4}(T_{\rm
ann})}\,\left(\frac{1\leavevmode\nobreak\ \rm GeV}{T_{\rm
ann}}\right)\left(\frac{10^{5}\leavevmode\nobreak\ \rm
GeV}{\sigma^{1/3}}\right)^{3/2}.$ (15)

The PBH contribution to the DM abundance is:

$f_{\rm PBH}=\frac{\rho_{\rm DW}(T_{\rm PBH})}{\rho_{\rm DM}(T_{\rm
PBH})}=\frac{\rho_{\rm DW}(T_{\rm PBH})}{\rho_{\rm DM}(T_{\rm
ann})}\frac{\rho_{\rm DW}(T_{\rm ann})}{\rho_{\rm DM}(T_{\rm PBH})}.$ (16)

The first factor describes how fast the energy stored in the DW network
disappears as annihilation proceeds which according to lattice simulations
appears to follow a power-law [43]

$\frac{\rho_{\rm DW}(T)}{\rho_{\rm DW}(T_{\rm ann})}=\left(\frac{T}{T_{\rm
ann}}\right)^{\alpha}.$ (17)

Results collected in Tab. VI of [43] suggests that $\alpha$, which
parameterizes how fast the network annihilates, can take values between 9 and
and 28 [43, 47] (though smaller values like $\alpha=7$ have also been
considered in the literature [31]). The second factor in Eq. (16) can be
evaluated from evolving DW, DM and radiation energy densities until today and
we get

$f_{\rm PBH}\simeq g_{\rm eff,2}^{1/2}\leavevmode\nobreak\ \frac{T_{\rm
PBH}^{(\alpha-3)}\leavevmode\nobreak\ T_{\rm
dom}^{2}}{T_{0}\leavevmode\nobreak\ T_{\rm
ann}^{(\alpha-2)}}\left(\frac{\rho_{\rm rad}}{\rho_{\rm DM}}\right)_{0},$ (18)

where

$g_{\rm
eff,2}\equiv\left(\frac{g_{s\star}(T_{0})}{g_{\star}(T_{0})}\right)^{\\!2}\frac{g_{\star}(T_{\rm
ann})g_{\star}(T_{\rm dom})}{g_{s\star}^{2}(T_{\rm PBH})}.$ (19)

In Fig. 2, we show the constraints due to PBH overclosure $f_{\rm PBH}\gtrsim
1$, but also from distortion of the Cosmic Microwave Background [69, 70, 71],
LIGO-Virgo-Kagra (LVK) bounds [64, 65, 66, 67, 68] and microlensing limits
from Eros datasets [63]. We collect all the constraints in Fig.
LABEL:ig:DW_constraints and vary the exponent $\alpha$ which sets how fast the
DW network is annihilating $(\rho^{2}_{\rm DW}\propto 1/t^{\alpha}$). We
conclude that the DW interpretation of PTA signal is excluded by PBH
overproduction.

### Impact of friction

. We know briefly discuss the impact of friction which we have neglected in
our analysis. The PBH abundance might be strongly impacted in presence of
friction. However this does not relieve the DW interpretation of the PTA
signal. In fact, in presence of friction the GW signal gets suppressed [49] so
that the confidence levels (blue and orange ellipses) rendering the PTA
signals will move to the DW domination region in gray in Fig. 3, excluding the
DW interpretation of PTA signal without having to study the PBH abundance in
friction-dominated DW network.

##  VI Discussion and outlook

Several PTA have reported the observation of a SGWB with an energy fraction of
$5\times 10^{-9}$ at nano-Hertz frequencies. The annihilation of topological
defect systems has been listed among the possible sources. In this paper, we
have shown that pure domain-wall and $N>1$ string-wall systems is in tension
with the overproduction of primordial black holes (PBHs). Parameterizing the
annihilation of the DW network by a power-law $\rho_{\rm DW}^{2}\propto
1/t^{\alpha}$, values $\alpha\lesssim 50$ result in a tension between the
amplitude and frequency of the SGWB observed in the different PTA datasets and
the overproduction of PBHs. This has been missed by previous works [48, 46,
20, 73, 74, 9] claiming a DW interpretation of PTA signals. To further
strengthen these results, dedicated simulations of the late evolution of
domain-wall and string-wall networks should be realized. Hence, we add the DW
to the graveyard of early universe phenomena failing short at explaining PTA
GW signal, together with global strings (see introduction) and scalar-induced
GW [55] in the Gaussian limit [75, 76]. Recent works have shown that first-
order phase transition can produce PBHs abundantly in the supercooled limit
[77, 78, 79, 80, 81]. Further studied are needed to infer whether 1stOPT
interpretation of the PTA signal [82, 83, 20, 9] is in the PBH graveyard too.

Our conclusions suggest that the only viable topological-defect origin of the
PTA signal is one arising from Nambu-Goto strings.

## Acknowledgments

YG thanks Simone Blasi, Alberto Mariotti, Oriol Pujolàs and Fabrizio Rompineve
for useful discussions. YG is grateful to the Azrieli Foundation for the award
of an Azrieli Fellowship. EV acknowledges support by the European Research
Council (ERC) under the European Union’s Horizon Europe research and
innovation programme (grant agreement No. 101040019).

##  Appendix A BBN bound

Domain walls (DW) form a component of the total energy density of the
universe. As such, they contribute to increase the expansion rate of the
universe which makes neutron freeze-out earlier, increase the $n/p$ ratio
which in turn increases the Helium abundance [84]. The presence of DW can be
described in terms of an extra number of neutrino species

$N_{\rm eff}=\frac{8}{7}\left(\frac{\rho_{\rm
DW}}{\rho_{\gamma}}\right)\left(\frac{11}{4}\right)^{4/3},$ (20)

where $\rho_{\gamma}$ is the photon number density. We introduce the DW energy
fraction in unit of radiation energy density at annihilation temperature

$\alpha_{\rm DW}(T)=\frac{\rho_{\rm DW}(T)}{\frac{\pi^{2}}{30}g_{*}(T)T^{4}}.$
(21)

where $T$ is the SM photon temperature. From Eq. (20) and Eq. (21), the
maximal DW contribution to $N_{\rm eff}$ occurs at annihilation temperature

$\Delta N_{\rm eff}(T)=2.20g_{*}(T)\alpha_{\rm DW}(T),$ (22)

To apply the BBN bound $\Delta N_{\rm eff}\lesssim 0.3$ [60, 61], the
effective number of extra relativistic degrees of freedom must be evaluated
below the neutrino decoupling temperature where $g_{*}(T<T_{\rm dec})\equiv
2+(7/8)\cdot 6\cdot(4/11)^{4/3}\simeq 3.36$. Hence, we obtain

$\Delta N_{\rm eff}=7.4\leavevmode\nobreak\ \alpha_{\rm
GW}\leavevmode\nobreak\ \lesssim\leavevmode\nobreak\ 0.3,$ (23)

which is slightly different from [48]. As discussed in the main text, we must
distinguish the scenario in which DW reheates to dark radiation, in which case
Eq. 23 is the BBN constraints, from the scenario in which DW reheates to SM,
in which case Eq. 23 applies only if DW annihilate below the neutrino
decoupling temperature $T_{\rm ann}\lesssim 1\leavevmode\nobreak\ \rm MeV$.

## References

  * Agazie  _et al._ [2023a] G. Agazie _et al._ (NANOGrav), The NANOGrav 15-year Data Set: Evidence for a Gravitational-Wave Background, Astrophys. J. Lett. 951, 10.3847/2041-8213/acdac6 (2023a), arXiv:2306.16213 [astro-ph.HE] .
  * Pol  _et al._ [2021] N. S. Pol _et al._ (NANOGrav), Astrophysics Milestones for Pulsar Timing Array Gravitational-wave Detection, Astrophys. J. Lett. 911, L34 (2021), arXiv:2010.11950 [astro-ph.HE] .
  * Chen  _et al._ [2021] S. Chen _et al._ , Common-red-signal analysis with 24-yr high-precision timing of the European Pulsar Timing Array: Inferences in the stochastic gravitational-wave background search 10.1093/mnras/stab2833 (2021), arXiv:2110.13184 [astro-ph.HE] .
  * Goncharov  _et al._ [2021] B. Goncharov _et al._ , On the evidence for a common-spectrum process in the search for the nanohertz gravitational-wave background with the Parkes Pulsar Timing Array 10.3847/2041-8213/ac17f4 (2021), arXiv:2107.12112 [astro-ph.HE] .
  * Antoniadis  _et al._ [2022] J. Antoniadis _et al._ , The International Pulsar Timing Array second data release: Search for an isotropic gravitational wave background, Mon. Not. Roy. Astron. Soc. 510, 4873 (2022), arXiv:2201.03980 [astro-ph.HE] .
  * Sesana  _et al._ [2004] A. Sesana, F. Haardt, P. Madau, and M. Volonteri, Low - frequency gravitational radiation from coalescing massive black hole binaries in hierarchical cosmologies, Astrophys. J. 611, 623 (2004), arXiv:astro-ph/0401543 .
  * Burke-Spolaor  _et al._ [2019] S. Burke-Spolaor _et al._ , The Astrophysics of Nanohertz Gravitational Waves, Astron. Astrophys. Rev. 27, 5 (2019), arXiv:1811.08826 [astro-ph.HE] .
  * Agazie  _et al._ [2023b] G. Agazie _et al._ (NANOGrav), The NANOGrav 15-year Data Set: Constraints on Supermassive Black Hole Binaries from the Gravitational Wave Background, (2023b), arXiv:2306.16220 [astro-ph.HE] .
  * Afzal  _et al._ [2023] A. Afzal _et al._ (NANOGrav), The NANOGrav 15-year Data Set: Search for Signals from New Physics, Astrophys. J. Lett. 951, 10.3847/2041-8213/acdc91 (2023), arXiv:2306.16219 [astro-ph.HE] .
  * Caprini  _et al._ [2010] C. Caprini, R. Durrer, and X. Siemens, Detection of gravitational waves from the QCD phase transition with pulsar timing arrays, Phys. Rev. D 82, 063511 (2010), arXiv:1007.1218 [astro-ph.CO] .
  * De Luca  _et al._ [2021] V. De Luca, G. Franciolini, and A. Riotto, NANOGrav Data Hints at Primordial Black Holes as Dark Matter, Phys. Rev. Lett. 126, 041303 (2021), arXiv:2009.08268 [astro-ph.CO] .
  * Sugiyama  _et al._ [2021] S. Sugiyama, V. Takhistov, E. Vitagliano, A. Kusenko, M. Sasaki, and M. Takada, Testing Stochastic Gravitational Wave Signals from Primordial Black Holes with Optical Telescopes, Phys. Lett. B 814, 136097 (2021), arXiv:2010.02189 [astro-ph.CO] .
  * Vaskonen and Veermäe [2021] V. Vaskonen and H. Veermäe, Did NANOGrav see a signal from primordial black hole formation?, Phys. Rev. Lett. 126, 051303 (2021), arXiv:2009.07832 [astro-ph.CO] .
  * Kohri and Terada [2021] K. Kohri and T. Terada, Solar-Mass Primordial Black Holes Explain NANOGrav Hint of Gravitational Waves, Phys. Lett. B 813, 136040 (2021), arXiv:2009.11853 [astro-ph.CO] .
  * Ellis and Lewicki [2021] J. Ellis and M. Lewicki, Cosmic String Interpretation of NANOGrav Pulsar Timing Data, Phys. Rev. Lett. 126, 041304 (2021), arXiv:2009.06555 [astro-ph.CO] .
  * Blasi  _et al._ [2021] S. Blasi, V. Brdar, and K. Schmitz, Has NANOGrav found first evidence for cosmic strings?, Phys. Rev. Lett. 126, 041305 (2021), arXiv:2009.06607 [astro-ph.CO] .
  * Buchmuller  _et al._ [2020] W. Buchmuller, V. Domcke, and K. Schmitz, From NANOGrav to LIGO with metastable cosmic strings, Phys. Lett. B 811, 135914 (2020), arXiv:2009.10649 [astro-ph.CO] .
  * Bian  _et al._ [2021] L. Bian, R.-G. Cai, J. Liu, X.-Y. Yang, and R. Zhou, Evidence for different gravitational-wave sources in the NANOGrav dataset, Phys. Rev. D 103, L081301 (2021), arXiv:2009.13893 [astro-ph.CO] .
  * King  _et al._ [2021] S. F. King, S. Pascoli, J. Turner, and Y.-L. Zhou, Gravitational Waves and Proton Decay: Complementary Windows into Grand Unified Theories, Phys. Rev. Lett. 126, 021802 (2021), arXiv:2005.13549 [hep-ph] .
  * Madge  _et al._ [2023] E. Madge, E. Morgante, C. P. Ibáñez, N. Ramberg, and S. Schenk, Primordial gravitational waves in the nano-Hertz regime and PTA data – towards solving the GW inverse problem, (2023), arXiv:2306.14856 [hep-ph] .
  * Vilenkin [1981] A. Vilenkin, Gravitational Field of Vacuum Domain Walls and Strings, Phys. Rev. D 23, 852 (1981).
  * Preskill  _et al._ [1991] J. Preskill, S. P. Trivedi, F. Wilczek, and M. B. Wise, Cosmology and broken discrete symmetry, Nucl. Phys. B 363, 207 (1991).
  * Saikawa [2017] K. Saikawa, A review of gravitational waves from cosmic domain walls, Universe 3, 40 (2017), arXiv:1703.02576 [hep-ph] .
  * Gelmini  _et al._ [2021a] G. B. Gelmini, S. Pascoli, E. Vitagliano, and Y.-L. Zhou, Gravitational wave signatures from discrete flavor symmetries, JCAP 02, 032, arXiv:2009.01903 [hep-ph] .
  * Zeldovich  _et al._ [1974] Y. B. Zeldovich, I. Y. Kobzarev, and L. B. Okun, Cosmological Consequences of the Spontaneous Breakdown of Discrete Symmetry, Zh. Eksp. Teor. Fiz. 67, 3 (1974).
  * Vilenkin and Shellard [2000] A. Vilenkin and E. P. S. Shellard,  _Cosmic Strings and Other Topological Defects_ (Cambridge University Press, 2000).
  * Gleiser and Roberts [1998] M. Gleiser and R. Roberts, Gravitational waves from collapsing vacuum domains, Phys. Rev. Lett. 81, 5497 (1998), arXiv:astro-ph/9807260 .
  * Hiramatsu  _et al._ [2010] T. Hiramatsu, M. Kawasaki, and K. Saikawa, Gravitational Waves from Collapsing Domain Walls, JCAP 05, 032, arXiv:1002.1555 [astro-ph.CO] .
  * Kawasaki and Saikawa [2011] M. Kawasaki and K. Saikawa, Study of gravitational radiation from cosmic domain walls, JCAP 09, 008, arXiv:1102.5628 [astro-ph.CO] .
  * Hiramatsu  _et al._ [2014] T. Hiramatsu, M. Kawasaki, and K. Saikawa, On the estimation of gravitational wave spectrum from cosmic domain walls, JCAP 02, 031, arXiv:1309.5001 [astro-ph.CO] .
  * Ferrer  _et al._ [2019a] F. Ferrer, E. Masso, G. Panico, O. Pujolas, and F. Rompineve, Primordial Black Holes from the QCD axion, Phys. Rev. Lett. 122, 101301 (2019a), arXiv:1807.01707 [hep-ph] .
  * Sikivie [1982] P. Sikivie, Of Axions, Domain Walls and the Early Universe, Phys. Rev. Lett. 48, 1156 (1982).
  * Chang and Cui [2020] C.-F. Chang and Y. Cui, Stochastic Gravitational Wave Background from Global Cosmic Strings, Phys. Dark Univ. 29, 100604 (2020), arXiv:1910.04781 [hep-ph] .
  * Gorghetto  _et al._ [2021] M. Gorghetto, E. Hardy, and H. Nicolaescu, Observing invisible axions with gravitational waves, JCAP 06, 034, arXiv:2101.11007 [hep-ph] .
  * Dror  _et al._ [2021] J. A. Dror, H. Murayama, and N. L. Rodd, Cosmic axion background, Phys. Rev. D 103, 115004 (2021), [Erratum: Phys.Rev.D 106, 119902 (2022)], arXiv:2101.09287 [hep-ph] .
  * Dunsky  _et al._ [2022] D. I. Dunsky, A. Ghoshal, H. Murayama, Y. Sakakihara, and G. White, GUTs, hybrid topological defects, and gravitational waves, Phys. Rev. D 106, 075030 (2022), arXiv:2111.08750 [hep-ph] .
  * Peccei and Quinn [1977] R. D. Peccei and H. R. Quinn, CP Conservation in the Presence of Instantons, Phys. Rev. Lett. 38, 1440 (1977).
  * Weinberg [1978] S. Weinberg, A New Light Boson?, Phys. Rev. Lett. 40, 223 (1978).
  * Wilczek [1978] F. Wilczek, Problem of Strong $P$ and $T$ Invariance in the Presence of Instantons, Phys. Rev. Lett. 40, 279 (1978).
  * Chang  _et al._ [1999] S. Chang, C. Hagmann, and P. Sikivie, Studies of the motion and decay of axion walls bounded by strings, Phys. Rev. D 59, 023505 (1999), arXiv:hep-ph/9807374 .
  * Hiramatsu  _et al._ [2012] T. Hiramatsu, M. Kawasaki, K. Saikawa, and T. Sekiguchi, Production of dark matter axions from collapse of string-wall systems, Phys. Rev. D 85, 105020 (2012), [Erratum: Phys.Rev.D 86, 089902 (2012)], arXiv:1202.5851 [hep-ph] .
  * Hiramatsu  _et al._ [2013] T. Hiramatsu, M. Kawasaki, K. Saikawa, and T. Sekiguchi, Axion cosmology with long-lived domain walls, JCAP 01, 001, arXiv:1207.3166 [hep-ph] .
  * Kawasaki  _et al._ [2015] M. Kawasaki, K. Saikawa, and T. Sekiguchi, Axion dark matter from topological defects, Phys. Rev. D 91, 065014 (2015), arXiv:1412.0789 [hep-ph] .
  * Gelmini  _et al._ [2021b] G. B. Gelmini, A. Simpson, and E. Vitagliano, Gravitational waves from axionlike particle cosmic string-wall networks, Phys. Rev. D 104, 061301 (2021b), arXiv:2103.07625 [hep-ph] .
  * Gelmini  _et al._ [2023a] G. B. Gelmini, A. Simpson, and E. Vitagliano, Catastrogenesis: DM, GWs, and PBHs from ALP string-wall networks, JCAP 02, 031, arXiv:2207.07126 [hep-ph] .
  * Zambujal Ferreira  _et al._ [2022] R. Zambujal Ferreira, A. Notari, O. Pujolàs, and F. Rompineve, High Quality QCD Axion at Gravitational Wave Observatories, Phys. Rev. Lett. 128, 141101 (2022), arXiv:2107.07542 [hep-ph] .
  * Gelmini  _et al._ [2023b] G. B. Gelmini, J. Hyman, A. Simpson, and E. Vitagliano, Primordial black hole dark matter from catastrogenesis with unstable pseudo-Goldstone bosons, (2023b), arXiv:2303.14107 [hep-ph] .
  * Ferreira  _et al._ [2023] R. Z. Ferreira, A. Notari, O. Pujolas, and F. Rompineve, Gravitational waves from domain walls in Pulsar Timing Array datasets, JCAP 02, 001, arXiv:2204.04228 [astro-ph.CO] .
  * Blasi  _et al._ [2023] S. Blasi, A. Mariotti, A. Rase, A. Sevrin, and K. Turbang, Friction on ALP domain walls and gravitational waves, JCAP 04, 008, arXiv:2210.14246 [hep-ph] .
  * Durrer and Caprini [2003] R. Durrer and C. Caprini, Primordial Magnetic Fields and Causality, JCAP 11, 010, arXiv:astro-ph/0305059 .
  * Caprini  _et al._ [2009] C. Caprini, R. Durrer, T. Konstandin, and G. Servant, General Properties of the Gravitational Wave Spectrum from Phase Transitions, Phys. Rev. D 79, 083519 (2009), arXiv:0901.1661 [astro-ph.CO] .
  * Cai  _et al._ [2020] R.-G. Cai, S. Pi, and M. Sasaki, Universal Infrared Scaling of Gravitational Wave Background Spectra, Phys. Rev. D 102, 083528 (2020), arXiv:1909.13728 [astro-ph.CO] .
  * Hook  _et al._ [2021] A. Hook, G. Marques-Tavares, and D. Racco, Causal Gravitational Waves as a Probe of Free Streaming Particles and the Expansion of the Universe, JHEP 02, 117, arXiv:2010.03568 [hep-ph] .
  * Rosado  _et al._ [2015] P. A. Rosado, A. Sesana, and J. Gair, Expected properties of the first gravitational wave signal detected with pulsar timing arrays, Mon. Not. Roy. Astron. Soc. 451, 2417 (2015), arXiv:1503.04803 [astro-ph.HE] .
  * Dandoy  _et al._ [2023] V. Dandoy, V. Domcke, and F. Rompineve, Search for scalar induced gravitational waves in the International Pulsar Timing Array Data Release 2 and NANOgrav 12.5 years dataset, (2023), arXiv:2302.07901 [astro-ph.CO] .
  * Ellis  _et al._ [2020] J. A. Ellis, M. Vallisneri, S. R. Taylor, and P. T. Baker, Enterprise: Enhanced numerical toolbox enabling a robust pulsar inference suite, Zenodo (2020).
  * Taylor  _et al._ [2021] S. R. Taylor, P. T. Baker, J. S. Hazboun, J. Simon, and S. J. Vigeland, enterprise_extensions (2021), v2.3.3.
  * Ellis and van Haasteren [2017] J. Ellis and R. van Haasteren, jellis18/ptmcmcsampler: Official release (2017).
  * Lewis [2019] A. Lewis, GetDist: a Python package for analysing Monte Carlo samples,  (2019), arXiv:1910.13970 [astro-ph.IM] .
  * Pitrou  _et al._ [2018] C. Pitrou, A. Coc, J.-P. Uzan, and E. Vangioni, Precision big bang nucleosynthesis with improved Helium-4 predictions, Phys. Rept. 754, 1 (2018), arXiv:1801.08023 [astro-ph.CO] .
  * Dvorkin  _et al._ [2022] C. Dvorkin _et al._ , The Physics of Light Relics, in  _2022 Snowmass Summer Study_ (2022) arXiv:2203.07943 [hep-ph] .
  * Hasegawa  _et al._ [2019] T. Hasegawa, N. Hiroshima, K. Kohri, R. S. L. Hansen, T. Tram, and S. Hannestad, MeV-scale reheating temperature and thermalization of oscillating neutrinos by radiative and hadronic decays of massive particles, JCAP 12, 012, arXiv:1908.10189 [hep-ph] .
  * Tisserand  _et al._ [2007] P. Tisserand _et al._ (EROS-2), Limits on the Macho Content of the Galactic Halo from the EROS-2 Survey of the Magellanic Clouds, Astron. Astrophys. 469, 387 (2007), arXiv:astro-ph/0607207 .
  * Nakamura  _et al._ [1997] T. Nakamura, M. Sasaki, T. Tanaka, and K. S. Thorne, Gravitational waves from coalescing black hole MACHO binaries, Astrophys. J. Lett. 487, L139 (1997), arXiv:astro-ph/9708060 .
  * Raidal  _et al._ [2019] M. Raidal, C. Spethmann, V. Vaskonen, and H. Veermäe, Formation and Evolution of Primordial Black Hole Binaries in the Early Universe, JCAP 02, 018, arXiv:1812.01930 [astro-ph.CO] .
  * Kavanagh  _et al._ [2018] B. J. Kavanagh, D. Gaggero, and G. Bertone, Merger rate of a subdominant population of primordial black holes, Phys. Rev. D 98, 023536 (2018), arXiv:1805.09034 [astro-ph.CO] .
  * Abbott  _et al._ [2019] B. P. Abbott _et al._ (LIGO Scientific, Virgo), Search for Subsolar Mass Ultracompact Binaries in Advanced LIGO’s Second Observing Run, Phys. Rev. Lett. 123, 161102 (2019), arXiv:1904.08976 [astro-ph.CO] .
  * De Luca  _et al._ [2020] V. De Luca, G. Franciolini, P. Pani, and A. Riotto, Primordial Black Holes Confront LIGO/Virgo data: Current situation, JCAP 06, 044, arXiv:2005.05641 [astro-ph.CO] .
  * Ali-Haïmoud and Kamionkowski [2017] Y. Ali-Haïmoud and M. Kamionkowski, Cosmic microwave background limits on accreting primordial black holes, Phys. Rev. D 95, 043534 (2017), arXiv:1612.05644 [astro-ph.CO] .
  * Poulin  _et al._ [2017] V. Poulin, P. D. Serpico, F. Calore, S. Clesse, and K. Kohri, Cmb Bounds on Disk-Accreting Massive Primordial Black Holes, Phys. Rev. D 96, 083524 (2017), arXiv:1707.04206 [astro-ph.CO] .
  * Serpico  _et al._ [2020] P. D. Serpico, V. Poulin, D. Inman, and K. Kohri, Cosmic microwave background bounds on primordial black holes including dark matter halo accretion, Phys. Rev. Res. 2, 023204 (2020), arXiv:2002.10771 [astro-ph.CO] .
  * Ferrer  _et al._ [2019b] F. Ferrer, E. Masso, G. Panico, O. Pujolas, and F. Rompineve, Primordial Black Holes from the QCD Axion, Phys. Rev. Lett. 122, 10301 (2019b), arXiv:1807.01707 [hep-ph] .
  * Bai  _et al._ [2023] Y. Bai, T.-K. Chen, and M. Korwar, QCD-Collapsed Domain Walls: QCD Phase Transition and Gravitational Wave Spectroscopy, (2023), arXiv:2306.17160 [hep-ph] .
  * Kitajima  _et al._ [2023] N. Kitajima, J. Lee, K. Murai, F. Takahashi, and W. Yin, Nanohertz Gravitational Waves from Axion Domain Walls Coupled to QCD, (2023), arXiv:2306.17146 [hep-ph] .
  * Franciolini  _et al._ [2023] G. Franciolini, A. Iovino, Junior., V. Vaskonen, and H. Veermae, The recent gravitational wave observation by pulsar timing arrays and primordial black holes: the importance of non-gaussianities, (2023), arXiv:2306.17149 [astro-ph.CO] .
  * Ferrante  _et al._ [2023] G. Ferrante, G. Franciolini, A. Iovino, Junior., and A. Urbano, Primordial black holes in the curvaton model: possible connections to pulsar timing arrays and dark matter, JCAP 06, 057, arXiv:2305.13382 [astro-ph.CO] .
  * Gouttenoire and Volansky [2023] Y. Gouttenoire and T. Volansky, Primordial Black Holes from Supercooled Phase Transitions, (2023), arXiv:2305.04942 [hep-ph] .
  * Lewicki  _et al._ [2023] M. Lewicki, P. Toczek, and V. Vaskonen, Primordial black holes from strong first-order phase transitions, (2023), arXiv:2305.04924 [astro-ph.CO] .
  * Kawana  _et al._ [2022] K. Kawana, T. Kim, and P. Lu, PBH Formation from Overdensities in Delayed Vacuum Transitions, (2022), arXiv:2212.14037 [astro-ph.CO] .
  * Kodama  _et al._ [1982] H. Kodama, M. Sasaki, and K. Sato, Abundance of Primordial Holes Produced by Cosmological First Order Phase Transition, Prog. Theor. Phys. 68, 1979 (1982).
  * Liu  _et al._ [2022] J. Liu, L. Bian, R.-G. Cai, Z.-K. Guo, and S.-J. Wang, Primordial black hole production during first-order phase transitions, Phys. Rev. D 105, L021303 (2022), arXiv:2106.05637 [astro-ph.CO] .
  * Arzoumanian  _et al._ [2021] Z. Arzoumanian _et al._ (NANOGrav), Searching for Gravitational Waves from Cosmological Phase Transitions with the NANOGrav 12.5-Year Dataset, Phys. Rev. Lett. 127, 251302 (2021), arXiv:2104.13930 [astro-ph.CO] .
  * Bringmann  _et al._ [2023] T. Bringmann, P. F. Depta, T. Konstandin, K. Schmidt-Hoberg, and C. Tasillo, Does NANOGrav observe a dark sector phase transition?, (2023), arXiv:2306.09411 [astro-ph.CO] .
  * Baumann [2022] D. Baumann,  _Cosmology_ (Cambridge University Press, 2022).