# On the Importance and Shortcomings of Code Readability Metrics: A Case Study
on Reactive Programming

Gustaf Holst, Felix Dobslaw  Mid Sweden University
Dept. of Computer Sciences
Östersund, Sweden
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Well structured and readable source code is a pre-requisite for maintainable
software and successful collaboration among developers. Static analysis
enables the automated extraction of code complexity and readability metrics
which can be leveraged to highlight potential improvements in code to both
attain software of high quality and reinforce good practices for developers as
an educational tool. This assumes reliable readability metrics which are not
trivial to obtain since code readability is somewhat subjective. Recent
research has resulted in increasingly sophisticated models for predicting
readability as perceived by humans primarily with a procedural and object
oriented focus, while functional and declarative languages and language
extensions advance as they often are said to lead to more concise and readable
code. In this paper, we investigate whether the existing complexity and
readability metrics reflect that wisdom or whether the notion of readability
and its constituents requires overhaul in the light of programming language
changes. We therefore compare traditional object oriented and reactive
programming in terms of code complexity and readability in a case study.
Reactive programming is claimed to increase code quality but few studies have
substantiated these claims empirically. We refactored an object oriented open
source project into a reactive candidate and compare readability with the
original using cyclomatic complexity and two state-of-the-art readability
metrics. More elaborate investigations are required, but our findings suggest
that both cyclomatic complexity and readability decrease significantly at the
same time in the reactive candidate, which seems counter-intuitive. We
exemplify and substantiate why readability metrics may require adjustment to
better suit popular programming styles other than imperative and object-
oriented to better match human expectations.

##  I Introduction

Code readability and complexity are both considered to be important aspects in
software development as they directly link to code maintainability [1, 2, 3].
Code maintainability is one of the critical code quality characteristics
defined by the standard model for assessing software quality in ISO/IEC
25010:2011 [4]. With the appearance of every new programming style or
paradigm, software developers and researchers are interested in finding
empirical evidence for its actual attributes, and benefits in particular.
Chidamber and Kemerer [5] as well as Abreu and Carapuca [6] have contributed
by establishing metrics for object-oriented programming (OOP) incorporating
characteristics such as coupling and cohesion. As pointed out in [6], a new
programming paradigm will only gain widespread practical acceptance when its
management aspects are carefully addressed. Developers can leverage that
knowledge to make informed technological choices, while researchers are
supported in their design decisions for the creation of the programming
languages of the future.

One of the claims of the reactive programming (RP) paradigm is that it
promotes readability. RP extensions to existing high-level languages, such as
Reactive Extensions111http://reactivex.io/. The Java version, RxJava, has as
of now over 40k stars on GitHub which places it in the top 10 of Java
projects., are increasingly used due to their consistent use across language
barriers and simplified multi-threading, amongst others. The work of
Salvaneschi et al. indicates that RP has an advantage over the traditional OOP
approach when it comes to program comprehension [7]. Their study used human
subjects rating two versions, one imperative and one reactive, of the same
code excerpts. High adaptation therefore highlights the need to understand
whether existing static analysis readability metrics are capable of capturing
high readability. Or put another way, does RP really improve readability and
if so, can human-noticable differences be detected with current readability
metrics?

Buse and Weimer in [1] investigated the relationship between code readability
and software quality. To this end, they first collected data using 120 human
subjects rating short code snippets and then developed a mathematical model to
estimate the readability of code, which we here refer to as the B&W model. The
B&W model has been expanded upon by [2, 8, 9], and Scalabrino et al. in [10]
propose a combined model, the Scalabrino Comprehensive (SC) model, that they
show to be more accurate at assessing code readability as perceived by humans.
Both these models are created using machine learning techniques and trained on
imperative code excerpts. However, the features used are generic with, for
example, low level features such as number of periods and parentheses or
visual and textual features that affect readability and are in no way
specifically designed for either imperative or object oriented programming.
The assessment of code-readability has nothing to do with the choice of
programming language and everything to do with the comprehension of programs
in general. If a metric is designed biased towards a certain programming
language/paradigm, it will empirically fail to capture high readability, which
is what we argue here is undesirable.

While [7] studied comprehension effects of RP involving human subjects, we
seek to know to what extent changes in readability between OOP and RP can be
detected, using common static code analysis. To the best of our knowledge, the
work in [7] is the only attempt at studying the difference in
comprehensibility between OO and RP and the authors call for more research in
the field. It is important here to point out the distinction we make between
_comprehensibility_ and _readability_. In lack of a commonly accepted
definition, we consider software comprehensibility to be a higher-level
property of a software system than code readability. Comprehensibility may
relate to how the structural relationship in software can be understood while
we consider readability to be assessed on a lower level, a level at which code
is processed by the cognitive apparatus of a human, i.e. methods or functions.
Both interlink and correlate, but are not the same thing.

Our study is focused on comparing two versions of the same code excerpts,
namely the original imperative version and a declarative (functional) version
leveraging abstractions of a reactive framework. The two tools we will use for
comparing readability, the implementations of B&W and SC, has limitations that
affect our methodology. The B&W tool seems limited excerpts of around 7 lines,
longer excerpts are likely to yield a readability score of 0.0. The SC tool on
the other hand can provide a readability score on class level but it is then
calculated as a mean of the readability scores for the individual methods.
Also, SC requires excerpts to be well-formed (compilable). We will refactor
and compare at method-level allowing us to, one, use the mentioned tools, and
two, compare excerpts performing the same tasks. A true reactive system would
be designed differently than our refactored version but we are interested in
the _readability_ of the reactive code and more specifically if its
declarative style, perhaps unjustly, penalizes readability.

The paper is structured as follows. The problem and research questions are
detailed in Section II. In section III we acknowledge and discuss the
limitations of the study. The Background section in IV prepares the reader
with a basic understanding of reactive programming, as well as readability and
code complexity measures, for which we summarized the related work in Section
V. In Section VI, we detail the conduct of the case study, for which we
present the results in Section VII. The research questions are answered in
Section VIII, Discussion. A summary of the work with concluding remarks can be
found in Section IX, Conclusions.

##  II Problem Statement

The importance of code readability and code complexity reflects from its
strong influence on other quality metrics, and is considered a key element in
software development [11]. In particular, increased readability eases
maintenance, which is often considered the most time consuming activity of a
software’s life-cycle [3]. Salvaneschi et al. show empirically how RP may
improve comprehensibility but stress the need for further research [7]. To the
best of our knowledge, there is still little to no research concerning the
effect of readability and complexity when comparing RP to OOP. Comments from
the human subjects in [7] suggest that RP improves comprehension due to
reduced boilerplate code and increased readability.

The goal of this study is to to better understand whether current state-of-
the-art metrics are applicable to RP or tailored RP metrics need to be
developed. To this end we use a complementary research method to [7], where
instead of using human subjects to provide a gold standard we simply rely on
the existing state-of-the-art models for measuring readability to tell us
whether a method is readable or not. We then search for evidence that supports
the hypothesis that RP is more readable or more comprehensible compared to
traditional OO source-code, and discuss whether the results justify its
marketing as such or not. We chose RP for reference as it has greatly grown in
popularity and combines both features of declarative and functional
programming.

### Research Questions

RP is claimed to improve code quality in various aspects. In this work, we
investigate how RP affects the readability of source code when compared to OOP
using existing readability metrics, leading to the first research question:

_RQ1. Using state-of-the-art readability metrics, to what degree can we
measure a difference in readability between OOP and RP?_

The second research question examines if the same metrics are applicable for
both RP and OOP, or whether the metrics are biased or need further
examination. Furthermore, code complexity has shown to be closely related to,
and traditionally often used as, a substitute index for code readability. We
are interested in what characteristics, if any, OOP or RP offer respectively
rendering them advantageous over the other when it comes to readability:

_RQ2. What features, if any, in current code readability metrics must be
modified or excluded in order to make them applicable to RP?_

With RP being a fairly new paradigm, no metrics have yet been developed aimed
towards it. Thus, metrics used for measuring source code readability might not
be well suited for RP. If so, it may be necessary to change weighting of
features or exclude them in order to more accurately assess both readability
and code complexity of RP code bases.

_RQ3. How does RP, with its declarative nature, affect code complexity
compared to OOP?_

When measuring source code readability and complexity, we hypothesize that RP
yields higher readability and lower complexity scores. However, some features
of RP might affect the score negatively. For instance, one characteristic of
RP (as well as functional programming in general) is its use of so called
chaining where several function calls are chained resulting in very long
expressions at times. While this feature might influence the score of some
metrics negatively its declarative style is generally considered a way to
increase readability and flexibility.

##  III Limitations

In this case study, we refactor an open source project, Kryonet, written in a
traditional imperative style, to a version using reactive pipelines. The
chosen project was based on the criteria mentioned in Section VI, Research
Methodology. The focus during the translation process has been to map the
original code as closely as possible into a reactive version in order to keep
the two versions comparable. Furthermore, efforts were made to keep the
original structure of the code. In other words, no functional decomposition
was performed and the interface of the refactored library was kept intact.
Thus, the refactor cannot be said to constitute a paradigm shift from
imperative to reactive as far as design goes and the true benefits of RP might
not be fully utilized in the same way as in a project developed with
reactivity in mind already during the design phase. However, with our
definition of readability, and its relation to comprehensibility as discussed
in Section I, we argue that we can draw conclusions on method-level
readability from the comparisons. Also, a method-level refactor might be
considered the first step in a paradigm shift from imperative to reactive,
though, for reasons given in VI the subsequent steps were not carried out in
this study. The RP library used in this study is RxJava due to its popularity,
thus limiting us to the features of its API.

Kryonet goes under the Revised BSD License (BSD 3) which permits unlimited
redistribution for any purpose as long as its copyright notices and the
license’s disclaimers of warranty are maintained.

##  IV Background

###  IV-A Reactive Programming

Reactive programming is a declarative programming paradigm that has been
proposed to aid programmers develop event-based applications. A major issue in
such applications is the inherent unpredictability in the timing of external
events. Traditionally this has been solved by using so called callbacks, often
leading to what is known as callback hell as projects grow [12]. RP
programming languages and libraries provide abstractions over time management
to release the burden of the programmer. Any change of states within the
reactive circuit is automatically propagated to all dependencies which keep
all the affected variables and objects up-to-date [12]. A well-known example
of the reactive principle is a common spreadsheet where a change in one cell
automatically causes updates in other cells that are linked to it. RP builds
around events and streams that are chained together using various sets of
abstractions to perform operations and computations. When operating in an
asynchronous environment, RP is seen as the better alternative compared to
traditional object oriented use with thread pools due to its declarative and
concise syntax for composing streams. Multiple languages support reactive
programming and numerous library extensions have been developed such as
Reactor, Akka and Reactive Extensions (Rx) [13].

###  IV-B RxJava

ReactiveX (Reactive Extensions) is a library for reactive programming with
implementations in several languages, RxJava being the Java implementation. It
is built around the observer design pattern and provides operators with which
the programmer can compose reactive streams in a declarative style. Precarious
elements of asynchronous programming such as thread-management, thread-safety
and synchronization are handled by the library through abstractions in the
form of designated operators.

RxJava promotes a functional programming style, as opposed to an imperative
one (using callbacks). The main difference from other functional APIs is that
where for example the Java Streams API is synchronous and pull based RxJava is
asynchronous and push based, meaning data is pushed to any observers whenever
available. The core type in RxJava is aptly named _Observable_. An observable
stream can emit three kinds of events, namely, a successful emission of data,
an error or a completion event signalling that no more emissions are to be
expected.

###  IV-C Readability

The term readability stands out among the general software metrics as it
refers to a non-physical characteristic, i.e. the subjective human judgement
of how easy a code is to read [14, 2]. Plenty of metrics for measuring the
readability of general text exist and the Flesch-Kincaid metric from 1948 is
only one but possibly the oldest example [15]. Until recently, with the lack
of scientifically substantiated readability metrics specifically aimed at
code, various complexity metrics have been used to assess software
maintainability [16, 17]. Complexity is closely coupled to the problem at hand
which is problematic since a more complex problem will require more complex
code. Also, code size is commonly used as a metric for complexity but it
cannot be taken for granted that more lines of code influence readability
negatively at all times. As Posnett et al. point out in [2]: ”if two functions
are equally readable then why should their concatenation be less readable?“.

Already in 1976, McCabe introduced his influential work on cyclomatic
complexity (CYC) as a measurement of code complexity [16]. It uses a simple
formula which in essence counts the number of possible paths through the code:

$\displaystyle CYC=E-N+2P,$ (1)

with

E

being the number of edges in the flow graph,

N

being the number of nodes in the flow graph, and

P

being the number of nodes that have exit points.

CYC has widely been criticized [18, 19, 20]. Shepperd [18] argues that it is
”based upon poor theoretical foundations and an inadequate model of software
development“. However, as of today, it remains a much cited and utilized index
for code complexity.

According to [21], source code complexity is defined by ”the degree to which a
system’s design or code is difficult to understand because of numerous
components or relationships among components“. Code complexity is said to
influence a large proportion of other quality factors, e.g. extensibility,
maintainability, readability, consistency and understandability. Complexity is
therefore considered a key characteristic when speaking of code quality.
Multiple metrics concerning code complexity have been proposed over the years,
with cyclomatic complexity, lines of code, and Halstead Complexity being the
most common ones.

##  V Related work

###  V-A The positive effects on comprehensibility of reactive programming

Salvaneschi et al. performed the first empirical study addressing the claim
that RP ”makes a wide class of otherwise cumbersome applications more
comprehensible“ [7]. They set up an experiment with ten, what they call
comprehension tasks; short programs written in Scala using the reactive
library REScala. Human subjects were first presented with these tasks and
later asked to fill out a questionnaire aimed at determining the level of
comprehension acquired for each task. A task consisted of two versions of the
same reactive application, a traditional OOP version using the Observer
pattern and an RP one using reactive abstractions such as signals and events.
Three categories of applications were developed:

  * •

_Synthetic applications_ where functional dependencies are defined between
variables and changes to those variables are propagated as updates occur,

  * •

_Graphical animations_ in which shapes are displayed and moved around on a
canvas in certain patterns, and

  * •

_Interactive applications_ requiring the user to perform common GUI related
actions such as clicking a button or inserting text into a form.

For the conduct of the study, a web application was developed which was used
to present the subjects with the comprehension tasks and, importantly, the
time needed for each task was recorded. The selected subjects were all
computer science students with similar backgrounds. They had been taught
Scala, Java programming, the observer pattern, as well as the basics of RP.
From their results, the authors concluded that RP has an advantage over OOP
and the observer pattern when it comes to comprehensibility. They mainly
contributed this difference to two characteristics: reduced boilerplate code
and better readability.

###  V-B Developing code readability metrics using human subjects

In 2010 Buse and Weimer [1] presented the first readability metric for source
code. They collected data by letting 120 human subjects rate the readability
of 100 short code fragments. The resulting data set was used to create a
predictive model for readability based on 25 code characteristics and their
results indicate that the average number of identifiers, line length (avg and
max), avg use of brackets, parentheses and punctuation, have the greatest
negative impact on readability. Finally, they tested for correlation between a
snippets readability score calculated by their model and the number of bugs
found by a static analysis tool, and found that their readability metric
correlated negatively to the number of defects found, i.e. the lower the score
the more likely a defect is found. They concluded that readability was not
closely related to the traditional notion of complexity while their notion of
readability was ”not orthogonal to complexity, it is in large part modeling a
distinct phenomena“. Posnett et al. [2] suggested a simpler formula using only
three features, lines of code, entropy, and Halstead’s Volume metric that they
found corresponds more accurately to the data collected in [1]. Dorn [8] later
pointed out that the work in [1] only used very small code snippets (avg. 7
lines) and that it was not clear whether their findings hold true for longer
code excerpts. Dorn concluded that visual elements are missing in the B&W
model and proposed a readability model with added features such as structural
patterns, syntax highlighting and operator alignment. He also incorporates the
simple linguistic feature of counting the number of English dictionary words
used in identifiers. The model proposed by Dorn [8] was shown to outperform
the basic B&W model in terms of code readability assessment.

Scalabrino et al. in [10] assess that the existing readability metrics (B&W,
Posnett, Dorn) rely mostly on structural features, suggesting the exploration
of linguistic aspects. Their proposed model considers textual features such as
identifier consistency, comment readability, the number of meanings and
textual coherence. The authors proposed a model combining all of the
individual models above which they showed to outperform each.

###  V-C Deriving a readability score

The readability metrics from [1] and [10] both rely on logistic regression
models to calculate probabilistic readability scores. The data collection was
done in a similar fashion in both studies, described as follows. Code snippets
were rated by humans on a 5-point scale where 1 denotes lowest, and 5 highest
readability. Binary readability classifiers were build based on the obtained
mean readability score of snippets as cut-off points ($\geq{3.14}$ in the case
of B&W and $\geq{3.6}$ for SC), thus leading to a binary separation of all
code snippets into readable or non-readable. For a given data-point, i.e. code
snippet, the model then returns how readable it is on a $0$-$1$ normalized
scale.

###  V-D Detecting changes using static analysis

Pantiuchina et al. [21] investigate to what degree state-of-the-art static
analysis tools capture source code improvements. The metrics they considered
were cohesion, coupling, complexity and readability. They applied repository
mining to extract commits from GitHub where the committer explicitly states
the purpose of the change to address one of these characteristics, and checked
through static analysis whether the change was noticeable. They concluded that
neither CYC, nor B&W or SC readability, could adequately detect any
improvements. These results has been confirmed in a similar study by Fakhoury
et al. [22] in which they even observed cases where readability as measured by
state-of-the-art readability metrics decreased after a readability commit
which indicates that there might be a discrepancy between readability as
defined by the metrics and as defined by programmers.

Scalabrino et al. investigated in [23] the possibilities of measuring code
comprehensibility using static analysis. A number of proxies for
understandability were established including reader’s own perception of
comprehension, actual comprehension and time time needed for perceived
comprehension. They tried to correlate 121 code-related, documentation-related
and developer-related metrics with these proxies for understandability but
without success, i.e. not even readability was found to correlate directly
with understandability. Among the evaluated metrics were CYC and SC as well as
individual features of B&W.

###  V-E Correlating readability with cognitive load and design quality

The possibility of using brain imaging techniques and eye tracking devices to
measure the cognitive load of readers of code has been studied by Fakhoury et
al. [24]. Human subjects were asked to fix bugs in code excerpts altered in
such a way that either lexical inconsistencies, poor structural
characteristics and code features that negatively affects readability were
introduced. Table I lists the features used to artificially manipulate the
readability of code excerpts.

Feature | Correlation
---|---
CYC | $---$
number of arguments | $---$
LOC | $---$
max nesting depth | $--$
number of loops | $-$
number of expressions | $-$
number of statements | $-$
variable declarations | $-$
number of comments | $++$
number of comment lines | $++$
number of spaces | $+$
TABLE I: Metrics used by Fakhoury et al. to alter the readability of code
snippets and their correlation levels and directions.

The results indicate that lexical inconsistencies (linguistic anti-patterns)
increase the cognitive load but its is not evident that poor structure (non
standard formatting) or poor readability has the same effect. However, the
combination of lexical inconsistencies and/or poor structure/readability had a
significant effect on comprehension.

Mannan et al. investigated readability in OSS projects. The study presented in
[25] focus on how readability changes during the lifetime of a project and
also how it impacts design quality. They found that readability, measured
using the Posnett model, does not correlate with design quality, measured by
amount of code smell using static analysis.

###  V-F Impact of lambda expressions on readability and complexity

Lucas et al. [26] conducted an extensive study on lambda expressions and the
impact they have on readability and complexity. Both a qualitative study using
human subjects rating, and commenting on, the readability of code excerpts and
a quantitative study measuring readability using two state-of-the-art
readability models were carried out. The readability models used were B&W and
the simplified model proposed by Posnett and the results (see Figure II for
summary) indicate that neither model is able to capture readability as
perceived by humans on code excerpts containing lambda expressions.

Evaluator | Increased | Decreased | Unchanged
---|---|---|---
B&W | 23 | 32 | 3
Posnett et al. | 11 | 43 | 4
Humans subjects | 51 | 3 | 4
TABLE II: Number of code excerpts that had an increase, decrease in
readability or were left unchanged after replacing anonymous inner classes
with lambda expressions by refactoring tools.

###  V-G Contributions of this paper

To the best of our knowledge, the work of Salvaneschi et al. in [7] is the
only study to date that addresses the positive effects of RP. Their study
involves human subjects to determine whether a program written using RP
principles yields more comprehensible code than a version of the same program
written using OOP principles. We intend to tie together the work of
Salvaneschi et al. and that of B&W and SC to assess differences in
readability. For that sake, an existing open source project was refactored
using Reactive Extensions (Rx). Besides using the readability metrics, we
further obtain source code complexity readings through CYC to better
understand whether it may be suitable for RP.

The work of Lucas et al. [26] is focusing on readability of code excerpts
containing lambda expressions. Reactive programming makes heavy use of lambda
expressions which is why understanding their impact on readability and
comprehensibility is important in this scope. Furthermore, while Lucas et al.
measured readability with B&W and the Posnett model, our study leverages SC
and B&W, with the Posnett model being incorporated into SC.

##  VI Research methodology

In this section, we first provide a brief outline of the applied methodology
and then go on to detail each step of the study.

The reactive library of choice is RxJava from Reactive Extensions, i.e.,
specifically its Java implementation in version 3.0.2. RxJava is among the
most popular Java projects on GitHub at the time of this writing, and the most
popular of Reactive Extensions considering GitHub stars.

Firstly, a suitable open source project for the study had to be selected. Once
a project was shortlisted which met the criteria explained in the upcoming
section, it was forked and refactored according to the RP paradigm, also
further explained below.

The original code was refactored on a per-method basis and once a complete
reactive version passed all tests in the test suite, both candidates got
processed through static analysis gathering CYC complexity and readability
according to the B&W and SC models. The results were extracted and compiled in
combination with those features of highest predictive power according to the
B&W model for analysis, namely the average number of identifiers, the average
line length, the average number of parentheses and brackets, as well as the
maximum line-length and average stops count. The refactored source code and
the measurements can be viewed in the replication package
222https://doi.org/10.5281/zenodo.4277872.

###  VI-A Project choice motivation

The criteria that guided the search for the case project were the following:

  * •

Java project (to enable the use of RxJava)

  * •

Associated test suite (for ensuring valid refactor)

  * •

Headless (to facilitate testing)

RP is said to facilitate the construction of event-based applications. With
that in mind we decided to search for a network library in which incoming and
outgoing events are handled. The choice of project fell on Kryonet (see Table
III), a TCP/UDP client/server library written in Java 7. Functional features
such as Lambdas got introduced in Java 8, which is why Kryonet did not contain
any of the functional elements, which rendered it a suitable candidate.
Kryonet further has a test suite to ensure valid refactoring and has about
1.5k stars on GitHub at the time of this writing.

| num classes | num methods | loc
---|---|---|---
Kryonet | $54$ | $283$ | $3664$
TABLE III: Size metrics for the original system.

###  VI-B Refactor strategy

Refactoring was conducted manually, i.e., not using any automated refactoring
tool, since no such tool was found to be adequate for the purposes of this
study. We chose to refactor the original code method per method, leaving the
public interface of the library intact. An alternative to this approach was a
complete overhaul of the code using RxObservables through the API. Three main
reasons justify our choice to refactor per method instead. Firstly, it allowed
the use of the test suite as is. Changing the public interface requires the
creation of new tests which implies a risk of introducing faults. Secondly,
the code sections that are compared are those accomplishing the same task
which substantially simplifies the comparison. Lastly, both B&W and SC
readability models were trained on smaller code snippets (average line length
of 7), and likely do not generalize well to larger constructs, such as
classes.

A number of methods were excluded from refactoring because of one or more of
the following reasons.

  * •

Number of Lines $>$ 50 (cutoff used in [10]).

  * •

No iteration over a Collection present.

  * •

No error handling present.

  * •

Methods containing a single statement.

The intention during refactoring was to place all statements in a method in a
reactive stream, i.e., into a chain of operators. For operations that need to
be performed once at the beginning or at the end of the code block, two side-
effect RxJava operators were utilized, doOnSubscribe and doFinally. Other
statements performed in loops were placed in doOnNext operators with each
statement being placed in a separate operator of the chain for reasons of
flexibility. For example, multithreading may be introduced anywhere in the
chain by simply adding an extra operator at the desired point. A single long
chain further allows us to get rid of try/catch blocks altogether in favor of
RxJava’s error handling channel for error delegation downstream. In a pure
migration project, many code segments would have been extracted and placed in
separate methods (functional decomposition) in order to bring out the
declarative qualities of RxJava. Such refactoring would clearly have benefited
the readability of the reactive stream but made for a misaligned comparison to
the original code. Overall, many original code segments simply had to be
placed in doOnNext operators. For example, the original code base contains
many log-level guarded logging statements. We placed these statements in
doOnNext operators in a way that is arguably not considered preferred lambda
syntax [27], i.e., using the shortest possible definition for the sake of
readability.

The convention used for naming lambda parameters was to use single characters.
This choice was inspired by Bob Martin in [28] who argues that the length and
descriptiveness of a variable should be in correlation to the size of its
scope.

The refactoring process resulted in 42 refactored methods from five different
classes and the success of the refactoring was confirmed by running and
passing the test suite.

###  VI-C Data Collection

We performed our study as a case study in which we manually refactored an
existing OOP code base to RP. One alternative to this approach would have been
to find a large number of open source projects of two categories, purely OOP
and purely RP, and measure and compare general readability between the two
categories. However, OOP projects are plentiful on GitHub, but finding pure RP
projects is difficult. Also, pinpointing the actual cause of any differences
in readability is hard as multiple factors, such as complexity of the problem
being solved or general coding style (naming conventions etc.) of the project,
could influence the result.

In [29], Köhler et al. propose a tool for automatic refactoring of OO code to
RP which could have been used in order to create a data set where the original
code is directly mapped to an RP version. That way, code doing the same thing
could have been compared, i.e., the complexity of the problem being solved and
the coding style would be the same. At the time of this investigation, the
Eclipse plugin called 2rx333https://github.com/stg-tud/rxrefactoring-code only
refactors a certain set of asynchronous constructs such as Future and
SwingWorker. The generated code wraps the original code into an instance of
RxObserver extended with the interface of the construct it wraps. As a
consequence, the refactored version will always be more complex and is
unlikely to yield a fair comparison between OO and RP from e.g. a readability
stand point. The circumstances outlined above led us to the decision to
perform a case study translating an existing open source project giving us two
versions of the same program. It must be pointed out that this method too has
potential flaws which we discuss in Section VIII-B, Threats to Validity.

###  VI-D Data Analysis

The refactored methods were analysed using static analysis tools. CYC was
measured using PMD444https://pmd.github.io/, a popular open source tool, while
readability was assessed utilizing the tools provided by the
authors555http://www.arrestedcomputing.com/readability/, and
https://dibt.unimol.it/report/readability/, respectively.. All measurements
were done per method and mean values over all refactored methods for each
version of the system were calculated in order to compare both versions. The
values calculated for readability were correlated with the five features of
highest predictive power in the B&W model. Similarly, the values calculated
for CYC were correlated with the number of control flow statements.

##  VII Results

In this section, we present a summary of all the data derived from the static
analysis. The measurements were performed on a per method basis which resulted
in a total of 42 analyzed methods in 5 classes. The diagrams below illustrate
a comparison between the refactored and the original versions. The staples in
the diagrams represent means over all methods for the individual measures
while the black lines describe the standard deviation. Furthermore, the five
most influential features of B&W metric model and the control flow statements
which CYC uses are extracted from each method and presented below.

###  VII-A Readability Score

Figure 1: Readability scores given as a mean value for all refactored methods
calculated by the B&W metric and the SC metric on the original code and the
refactored version respectively. SC predicts a higher readability overall and
a lower decrease in readability for the RP version compared to the original.

In Figure 1, readability scores are given between 0 (worst readability) and 1
(perfect readability). The SC metric suggest a reduction of the average
readability of the refactored methods by 39%, whilst the B&W metric suggests a
47% reduction. Although both metrics agree on reduced readability in the RP
version, the SC metric indicates overall higher readability for both OOP and
RP than the B&W metric.

###  VII-B Cyclomatic Complexity

Figure 2: Average cyclomatic complexity value for all refactored methods on
the original code and the refactored version respectively. Refactoring to a
reactive style has decreased complexity significantly. Figure 3: Average
numbers of occurrences per method of the features used to calculate cyclomatic
complexity in the original code and the refactored version respectively. The
features are logical and/or operators, catch blocks, if-statements, iterations
(loops), ternary operators and throw statements.

The complexity of each version was calculated using CYC. As can be seen in
Figure 2, an average reduction of $75\%$ in method-level complexity was
measured on the refactored version compared to the original. The average
numbers of loops (for, while), if-statements, catch-blocks, binary logical
operators, ternary operators and throw-statements per method were counted from
the project files (see Figure 3). These are all control flow statements used
to calculate CYC.

###  VII-C Features

The extracted feature readings of greatest predictive power in the B&W model
are summarized in Table IV and discussed here. The numbers of stops,
parentheses and brackets were counted per method and averaged per line. The
refactored version shows an increase by 150% of stops and an increase of 53%
of parentheses and brackets compared to the original version (see Figure 4).
In Java, identifiers are names of variables, methods, classes, packages and
interfaces that are contained within a class. The average number of
identifiers per line were calculated for each method. Figure 4 shows that the
number of identifiers increases in average by 22% compared to the original
version.

Measure |  OO ($\mu\pm\sigma$) |  RP ($\mu\pm\sigma$) | diff. (%)
---|---|---|---
CYC | $5.74\pm 4.41$ | $1.45\pm 1.47$ | $-75$
B&W score | 0.08 $\pm$ 0.16 | 0.04 $\pm$ 0.16 | $-47$
SC score | 0.63 $\pm$ 0.17 | 0.38 $\pm$ 0.23 | $-40$
number of ids. | 2.53 $\pm$ 0.56 | 3.10 $\pm$ 0.61 | $+22$
avg line length | 30.56 $\pm$ 6.10 | 34.44 $\pm$ 6.11 | $+13$
max line length | 76.43 $\pm$ 18.74 | 80.55 $\pm$ 23.98 | $+5.39$
br. and par. count | 0.95 $\pm$ 0.19 | 1.45 $\pm$ 0.26 | $+53$
stop count | 0.45 $\pm$ 0.16 | 1.13 $\pm$ 0.29 | $+150$
TABLE IV: A summary of the measured quantities for code complexity and
readability for both versions, object oriented and reactive including the
difference in percent. Figure 4: Average number of identifiers, stops and
brackets/parentheses per line in the original code and the refactored version.
The refactored code contains, on average, more identifiers due to the addition
of RxJava operators. The higher number of stops and parentheses is caused by
wrapping original code in chained RxJava operators, each such operator adding
one stop and and a pair of parentheses Figure 5: Max line length and average
line length in the original code and the refactored version, respectively. No
significant difference has been detected.

The longest line and the average length per line were calculated with barely
any notable difference (see Figure 5).

##  VIII Discussion

In this section, we examine the results to address our research questions.
Furthermore, we consider these answers when relating our findings to existing
studies. Finally, we discuss related threats to validity.

_RQ1. Using state-of-the-art readability metrics, to what degree can we
measure a difference in readability between OOP and RP?_

To answer this question, we considered the B&W and SC metrics. We saw a big
difference in the final scores between the two versions, where RP was shown to
yield a lower readability score using both metrics. The SC metric indicates a
smaller difference in readability for the OOP version than the B&W metric.
Furthermore, SC yields a significantly higher readability score on both RP and
OOP.

_RQ2. What features, if any, in current code readability metrics must be
modified or excluded in order to make them applicable to RP?_

The B&W metric consists of 25 features that the tool considers when evaluating
source code. Two out of the five features of highest predictive power in the
model are heavily used in RP where a stream consists of a chain of method
calls. This naturally gives a disadvantage to RP compared to the OO version
with respect to these features, and in extension to the B&W metric. For
instance, regular for and while loops are features that have only a small
effect on the score, whereas a corresponding loop in RP such as the
takeWhile() operator is an identifier and contains both a stop and a pair of
parentheses and therefore has a much stronger negative influence on the score.
Consider, for example, the sendToTCP method from the original code in Figure
6(a). Readability scores are significantly lower for the refactored version
(see 6(b)), while, at least to us, it looks more readable (a claim that would
have to be substantiated in future studies with humans).

⬇

1public void sendToTCP (int connectionID, Object object)

2{

3 Connection[] connections = this.connections;

4 for (int i = 0, n = connections.length; i < n; i++) {

5 Connection connection = connections[i];

6 if (connection.id == connectionID) {

7 connection.sendTCP(object);

8 break;

9 }

10 }

11}

(a) Original (object oriented)

⬇

1public void sendToTCP (int connectionID, Object object)

2{

3 Observable.fromArray(this.connections)

4 .filter(c -> c.id == connectionID)

5 .firstElement()

6 .subscribe(c -> c.sendTCP(object));

7}

(b) Refactored (reactive)

Figure 6: The original sendToTCP method (left) compared to the refactored
version (right) using the reactive code-style. The refactored version has a
much lower readability (0.054% as measured by B&W and 53.00% as measured by
SC) score than the original one (2.89% as measured by B&W and 77.39% as
measured by SC). The reduction of code-lines does not carry a positive effect
on readability. However, extra identifiers, stops, brackets and parentheses
have a strong negative impact on readability in our findings.

The reduction in the number of lines has little impact on the readability in
the metrics used, while the extra identifiers (RxJava operators), stops,
brackets and parentheses have a strong impact. Both metrics count these
characters as average occurrences per line. This means, for example, that the
original methods have an average of 0.4 stops (periods) per line while the RP
version has an average of 1.2 stops per line. This is an increase of 200%. We
argue that this does not reflect the difference in readability and that the
conciseness and the descriptive (declarative) nature of the reactive version
is not taken into account. Consequentially, we consider the B&W metrics
inapplicable for RP.

The SC metric was developed by combining existing known readability metrics,
i.e. the ones presented by B&W, Posnett, Dorn and Scalabrino’s first model
(using only textual features). It was found by the authors to achieve higher
predictive accuracy for readability with its combination of structural and
textual features. Considering the work of Salvaneschi et al., where RP was
found to be more comprehensible than OOP, partly due to better readability,
our results confirm that SC performs better than B&W since its readability
deviations are less pronounced. However, the structural features of the B&W
metric are an integral part of SC, which renders even SC less applicable for
RP than OOP. Regarding the slightly better results in favour of RP readability
with SC, we have been unable to trace the specific reasons for this and can
only speculate that it has to do with the spatial and visual features of the
Dorn model that may favour the declarative style of RP.

_RQ3. How does RP, with its declarative nature, affect code complexity
compared to OOP?_

Examining the score of CYC, RP has shown to decrease the code complexity
significantly compared to the original version. CYC calculates its score
simply by counting the number of potential paths through the code where the
number of flow statements determine the final score. Studying the flow
statement diagram from the previous section, we can detect important
differences in the two versions. The RP version has replaced the loops with
method calls (operators) from the RxJava API and by that minimized the use of
loops. The number of if-statements has been reduced to some extent by filters.
In addition, utilizing RxJava’s error channel, several try/catch-blocks have
been removed. The above-mentioned are all considered as flow statements which
affect the final complexity score.

###  VIII-A Contrasting Results with Related Work

Our study can be contrasted with the findings from the empirical study in [7]
where the authors claimed that RP increases source code comprehension when
directly compared to OOP. Moreover, they pointed out that their human subjects
found RP to be more comprehensive due to increased readability. The findings
from our case study suggest the contrary but do raise some doubts. By
answering the second research question, we can conclude that the current
metrics used to measure code readability are not applicable to RP, thus making
a fair comparison between OOP and RP hard to impossible. Furthermore, the
study conducted by Pantiuchina et al. shows that the B&W metric and
Scalabrino’s readability model are unable to recognize any code improvements,
which questions the legitimacy of the metrics.

To the best of our knowledge, no studies comparing code complexity between OOP
and RP have been conducted. CYC can be calculated on each version and still
make up for a fair comparison between the projects. It acts independently and
does not value any unique features of certain paradigms differently. Referring
back to the previous section, we conclude that RP seems to lower source code
complexity.

###  VIII-B Validity Threats

It has to be acknowledged that the refactored RP version of the code used in
this study may not be written in a way that makes it representative of all
reactive programming nor can it be claimed to utilize the full potential of
the RxJava API. As a case study the work cannot be expected to cover the
entire state of reactive programming, but serve as an indicator and be used as
a motivation for future research.

The manual refactoring conducted by the authors could be considered
susceptible to research bias, i.e. that special efforts could have been made
to favour readability. This threat was mitigated by, firstly, at the time of
the refactoring not having any detailed information about which features were
actually used in the two readability models, secondly, by refactoring each
method as a ’black box’, meaning only its inner structure has been changed and
no functionality has been neither added nor removed. For example, no unwieldy
logging statements were extracted from the methods and placed elsewhere, which
would clearly have improved the methods readability score, but not provided a
fair comparison between the two versions.

##  IX Conclusions

The purpose of this study was to gain a better understanding of whether
current complexity and readability metrics are suitable for non-traditional
coding-styles and paradigms that are growing in use. We therefore compared a
popular complexity metric [16] and two state-of-the-art readability metrics
from [1, 10] for an open source object-oriented project and a fork that we
refactored using a popular library for reactive programming as an example of
that category. Notable differences in both complexity and readability were
found. We found that the cyclomatic complexity in the reactive version is
significantly lower, which coincides with the ambitions around reactive
programming. The metric was suggested over 40 years ago but may extend well to
the reactive paradigm. However, the two readability metrics from the
literature indicate a significant reduction in readability which may seem
counter-intuitive, and which we challenged, also by the example given above,
as bad assessment. A potential source of error here is the large emphasis on
structural source-code features in the existing readability metrics which may
render them unsuitable for reactive code with it’s chaining character and use
of lambdas. The B&W metric from [1] solely relies on structural features while
the one from [10] combines structural and textual features resulting in a
still notable but less severe negative readability impact in reactive
programming. In essence, none of the metrics captures readability in a desired
way.

In order to better understand the impact of programming paradigms on code
readability, broader studies on both reactive but also other functional or
declarative coding styles are required. Multiple-dispatch should be mentioned
here too as a relevant object of investigation that gains traction, for
instance through the Julia programming language. A more in-depth analysis of
the existing metrics and their characteristics may further lead to their
improvement for code segments of various styles. In addition, languages such
as Java and C# support the arbitrary combination of styles within the same
project which may require work on detecting and differentiating code-styles
within a single project. Meaningful and objectively correct readability
characteristics, through e.g. linting, could prove to be invaluable in a
development environment. How to address the question of assessing source-code
readability objectively is a so far unanswered question, which is why we see
much research in that realm being called for.

## References

  * [1] R. P. Buse and W. R. Weimer, “Learning a Metric for Code Readability,” _IEEE Transactions on Software Engineering_ , vol. 36, no. 4, pp. 546–558, Jul. 2010, conference Name: IEEE Transactions on Software Engineering.
  * [2] D. Posnett, A. Hindle, and P. Devanbu, “A simpler model of software readability,” in _Proceedings of the 8th Working Conference on Mining Software Repositories_ , ser. MSR ’11. Waikiki, Honolulu, HI, USA: Association for Computing Machinery, May 2011, pp. 73–82.
  * [3] D. Alawad, M. Panta, M. Zibran, and M. R. Islam, “An Empirical Study of the Relationships between Code Readability and Software Complexity,” _arXiv:1909.01760 [cs]_ , vol. abs/1909.01760, Aug. 2019, arXiv: 1909.01760. [Online]. Available: http://arxiv.org/abs/1909.01760
  * [4] International Organization for Standardization, “ISO 25010,” 2017. [Online]. Available: https://iso25000.com/index.php/en/iso-25000-standards/iso-25010
  * [5] S. Chidamber and C. Kemerer, “A metrics suite for object oriented design,” _IEEE Transactions on Software Engineering_ , vol. 20, no. 6, pp. 476–493, Jun. 1994, conference Name: IEEE Transactions on Software Engineering.
  * [6] F. B. Abreu and R. Carapuça, “Object-oriented software engineering: Measuring and controlling the development process,” in _Proceedings of the 4th international conference on software quality_ , vol. 186, MLean, VA, USA, 1994.
  * [7] G. Salvaneschi, S. Proksch, S. Amann, S. Nadi, and M. Mezini, “On the Positive Effect of Reactive Programming on Software Comprehension: An Empirical Study,” _IEEE Transactions on Software Engineering_ , vol. 43, no. 12, pp. 1125–1143, Dec. 2017, conference Name: IEEE Transactions on Software Engineering.
  * [8] J. Dorn, “A general software readability model,” _MCS Thesis available from (http://www. cs. virginia. edu/weimer/students/dorn-mcs-paper. pdf)_ , vol. 5, pp. 11–14, 2012.
  * [9] S. Scalabrino, M. Linares-Vasquez, D. Poshyvanyk, and R. Oliveto, “Improving code readability models with textual features,” in _2016 IEEE 24th International Conference on Program Comprehension (ICPC)_. Austin, TX, USA: IEEE, May 2016, pp. 1–10.
  * [10] S. Scalabrino, M. Linares‐Vásquez, R. Oliveto, and D. Poshyvanyk, “A comprehensive model for code readability,” _Journal of Software: Evolution and Process_ , vol. 30, no. 6, 2018.
  * [11] M.D.University, A. Pahal, and R. S. Chillar, “Code Readability: A Review of Metrics for Software Quality,” _International Journal of Computer Trends and Technology_ , vol. 46, no. 1, pp. 1–4, Apr. 2017.
  * [12] E. Bainomugisha, A. L. Carreton, T. v. Cutsem, S. Mostinckx, and W. d. Meuter, “A survey on reactive programming,” Aug. 2013.
  * [13] H. Banken, E. Meijer, and G. Gousios, “Debugging Data Flows in Reactive Programs,” in _2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE)_. New York, NY, USA: Association for Computing Machinery, May 2018, pp. 752–763.
  * [14] Y. Tashtoush, Z. Odat, I. Alsmadi, and M. Yatim, “Impact of Programming Features on Code Readability,” _Computer Science Faculty Publications_ , vol. 7, no. 6, Jan. 2013.
  * [15] R. Flesch, “A new readability yardstick,” _Journal of Applied Psychology_ , vol. 32, no. 3, pp. 221–233, 1948, place: US Publisher: American Psychological Association.
  * [16] T. McCabe, “A Complexity Measure,” _IEEE Transactions on Software Engineering_ , vol. SE-2, no. 4, pp. 308–320, Dec. 1976, conference Name: IEEE Transactions on Software Engineering.
  * [17] M. H. Halstead _et al._ , _Elements of software science_. Elsevier New York, 1977, vol. 7.
  * [18] M. Shepperd, “A critique of cyclomatic complexity as a software metric,” _Software Engineering Journal_ , vol. 3, no. 2, pp. 30–36, Mar. 1988, publisher: IET Digital Library. [Online]. Available: https://digital-library.theiet.org/content/journals/10.1049/sej.1988.0003
  * [19] M. Pizka and F. Deissenböck, “How to effectively define and measure maintainability,” _SMEF 2007_ , p. 93, 2007.
  * [20] M. M. S. Sarwar, S. Shahzad, and I. Ahmad, “Cyclomatic complexity: The nesting problem,” in _Eighth International Conference on Digital Information Management (ICDIM 2013)_. IEEE, 2013, pp. 274–279.
  * [21] J. Pantiuchina, M. Lanza, and G. Bavota, “Improving Code: The (Mis) Perception of Quality Metrics,” in _2018 IEEE International Conference on Software Maintenance and Evolution (ICSME)_ , Sep. 2018, pp. 80–91.
  * [22] S. Fakhoury, D. Roy, A. Hassan, and V. Arnaoudova, “Improving Source Code Readability: Theory and Practice,” in _2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC)_. Montreal, QC, Canada: IEEE, May 2019, pp. 2–12. [Online]. Available: https://ieeexplore.ieee.org/document/8813254/
  * [23] S. Scalabrino, G. Bavota, C. Vendome, M. Linares-Vásquez, D. Poshyvanyk, and R. Oliveto, “Automatically assessing code understandability,” _IEEE Transactions on Software Engineering_ , pp. 1–1, 2019, conference Name: IEEE Transactions on Software Engineering.
  * [24] S. Fakhoury, Y. Ma, V. Arnaoudova, and O. Adesope, “The effect of poor source code lexicon and readability on developers’ cognitive load,” in _Proceedings of the 26th Conference on Program Comprehension_ , ser. ICPC ’18. New York, NY, USA: Association for Computing Machinery, 2018, p. 286–296. [Online]. Available: https://doi-org.proxybib.miun.se/10.1145/3196321.3196347
  * [25] U. A. Mannan, I. Ahmed, and A. Sarma, “Towards understanding code readability and its impact on design quality,” in _Proceedings of the 4th ACM SIGSOFT International Workshop on NLP for Software Engineering_ , ser. NL4SE 2018. Association for Computing Machinery, 2018, pp. 18–21. [Online]. Available: http://doi.org/10.1145/3283812.3283820
  * [26] W. L. M. Mendonça, R. B. de Almeida, J. Fortes, F. V. Lopes, D. Marcílio, E. D. Canedo, F. Lima, and J. Saraiva, “Understanding the impact of introducing lambda expressions in java programs,” _Journal of Software Engineering Research and Development_ , vol. 8, pp. 8–1, 2020.
  * [27] M. Vanags and R. Cevere, “The Perfect Lambda Syntax,” _Baltic Journal of Modern Computing_ , vol. 6, no. 1, 2018.
  * [28] R. C. Martin, _Clean Code: A Handbook of Agile Software Craftsmanship_ , 1st ed. USA: Prentice Hall PTR, 2008.
  * [29] M. Köhler and G. Salvaneschi, “Automated Refactoring to Reactive Programming,” in _2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE)_ , Nov. 2019, pp. 835–846.


&&&&&&\frac{-V_{\text{DD}}^{\text{(11,61,33,27)}}}{112}&\frac{-g_{3}\sqrt{\frac{3}{35}}V_{\text{DD}}^{\text{(2,13,6,3)}}}{16g_{2}}&\frac{-g_{3}V_{\text{DD}}^{\text{(2,13,6,3)}}}{16\sqrt{35}g_{2}}&\frac{-g_{3}\sqrt{\frac{5}{2}}V_{\text{DD}}^{\text{(2,13,6,3)}}}{224g_{2}}\\\
&&&&&&&0&0&0\\\ &&&&&&&&0&0\\\ &&&&&&&&&0\\\ \end{array}\right).}$

##  Appendix B Results of the fits

This study was performed neglecting isospin symmetry breaking effects. The
masses of particles used in the calculation are Zyla  _et al._ (2020)

$\displaystyle M_{\Lambda_{b}^{0}}$ $\displaystyle=5.6196~{}\text{GeV},\quad
M_{K^{-}}$ $\displaystyle=0.4937~{}\text{GeV},$ $\displaystyle M_{J/\psi}$
$\displaystyle=3.0969~{}\text{GeV},\quad M_{p}$
$\displaystyle=0.9383~{}\text{GeV},$ $\displaystyle M_{\eta_{c}}$
$\displaystyle=2.9839~{}\text{GeV},\quad M_{\pi}$
$\displaystyle=0.1380~{}\text{GeV},$ $\displaystyle M_{\Sigma_{c}}$
$\displaystyle=2.4535~{}\text{GeV},\quad M_{\Sigma_{c}^{*}}$
$\displaystyle=2.5181~{}\text{GeV},$ $\displaystyle M_{D}$
$\displaystyle=1.8680~{}\text{GeV},\quad M_{D^{*}}$
$\displaystyle=2.0086~{}\text{GeV},$ $\displaystyle M_{\Lambda_{c}}$
$\displaystyle=2.2865~{}\text{GeV}.\qquad\quad$ (99)

The fitted parameters to the three data of schemes I and II shown in Figs. 4
and 9 are collected in Tables 4 and 5, respectively. We do not show the unit
of $\mathcal{F}_{n}^{J}$ because an arbitrary overall normalization factor is
inseparable. In addition, we define a new parameter
$g_{D}^{\prime}=g_{D}k_{0}^{2}$, with
$k_{0}=\sqrt{\lambda(m^{2}_{0},m_{\psi}^{2},m_{p}^{2})}/(2m_{0})$ the c.m.
momentum of the proton at the energy
$m_{0}=(m_{\Sigma_{c}}+m_{\Sigma_{c}^{*}}+m_{D}+m_{D^{*}})/2$, to make
$g_{D}^{\prime}$ have the same unit as $g_{S}$.

Table 4: The best fit parameters of scheme I shown in Fig. 4. Here only the statistical uncertainties are presented. Parameter |  $\cos\theta_{P_{c}}$-weighted |  |  $m_{Kp}>1.9$ GeV |  |  $m_{Kp}$ all |
---|---|---|---|---|---|---
solution | $A$ | $B$ | $A$ | $B$ | $A$ | $B$
$C_{\frac{1}{2}}$ $[\text{GeV}^{-2}]$ | $-14.72(26)$ | $-9.86(32)$ | $-14.89(18)$ | $-9.06(49)$ | $-14.71(17)$ | $-9.66(34)$
$C_{\frac{3}{2}}$ $[\text{GeV}^{-2}]$ | $-10.33(25)$ | $-13.28(21)$ | $-10.16(20)$ | $-13.87(16)$ | $-10.54(18)$ | $-13.76(19)$
$g_{S}$ $[\text{GeV}^{-2}]$ | $1.17(201)$ | $2.23(132)$ | $1.39(90)$ | $4.51(94)$ | $2.42(89)$ | $3.63(80)$
$g_{D}^{\prime}$ $[\text{GeV}^{-2}]$ | $3.88(64)$ | $3.61(48)$ | $3.78(42)$ | $1.32(118)$ | $2.58(67)$ | $2.50(72)$
$\mathcal{F}^{\frac{1}{2}}_{1}$ | $27(249)$ | $716(1424)$ | $-139(219)$ | $362(1109)$ | $606(327)$ | $-3388(875)$
$\mathcal{F}^{\frac{1}{2}}_{2}$ | $883(257)$ | $1782(908)$ | $471(110)$ | $1631(743)$ | $208(247)$ | $4165(711)$
$\mathcal{F}^{\frac{1}{2}}_{3}$ | $-3509(453)$ | $-3333(572)$ | $-1879(120)$ | $-2866(766)$ | $-3365(574)$ | $-1676(334)$
$\mathcal{F}^{\frac{3}{2}}_{1}$ | $-3768(2140)$ | $-1948(1101)$ | $-2785(755)$ | $-497(335)$ | $1255(1087)$ | $2385(985)$
$\mathcal{F}^{\frac{3}{2}}_{2}$ | $-1743(1670)$ | $32(1255)$ | $-1459(541)$ | $32(-463)$ | $-305(735)$ | $-1495(1139)$
$\mathcal{F}^{\frac{3}{2}}_{3}$ | $-4662(648)$ | $-3400(351)$ | $-2396(186)$ | $-1530(242)$ | $-2901(306)$ | $934(458)$
$\mathcal{F}^{\frac{5}{2}}_{1}$ | $721(1593)$ | $-992(205)$ | $0(825)$ | $421(130)$ | $1953(292)$ | $-1284(226)$
Table 5: The best fit parameters of scheme II shown in Fig. 9. Here only the statistical uncertainties are presented.  Parameter |  $\cos\theta_{P_{c}}$-weighted |  |  $m_{Kp}>1.9$ GeV |  |  $m_{Kp}$ all |
---|---|---|---|---|---|---
solution | $A$ | $B$ | $A$ | $B$ | $A$ | $B$
$C_{\frac{1}{2}}~{}[\text{GeV}^{-2}]$ | $3.53(28)$ | $7.30(22)$ | $3.40(22)$ | $7.65(58)$ | $3.66(23)$ | $7.17(25)$
$C_{\frac{3}{2}}~{}[\text{GeV}^{-2}]$ | $-16.09(6)$ | $-16.89(21)$ | $-16.06(7)$ | $-17.36(24)$ | $-16.07(13)$ | $-17.02(22)$
$D_{b}^{SD}~{}[\text{GeV}^{-4}]$ | $-2.22(11)$ | $-1.30(12)$ | $-2.21(12)$ | $-1.51(21)$ | $-2.31(10)$ | $-1.33(12)$
$g_{S}~{}[\text{GeV}^{-2}]$ | $3.34(51)$ | $3.98(58)$ | $3.68(38)$ | $4.91(90)$ | $3.55(44)$ | $3.37(57)$
$g_{D}^{\prime}~{}[\text{GeV}^{-2}]$ | $1.17(64)$ | $2.39(29)$ | $1.32(35)$ | $1.78(78)$ | $2.07(36)$ | $2.10(44)$
$\mathcal{F}_{1}^{\frac{1}{2}}$ | $760(542)$ | $2169(761)$ | $-191(575)$ | $1299(889)$ | $-2543(395)$ | $-2401(468)$
$\mathcal{F}_{2}^{\frac{1}{2}}$ | $-276(310)$ | $1425(365)$ | $135(39)$ | $1343(604)$ | $1567(335)$ | $3332(453)$
$\mathcal{F}_{3}^{\frac{1}{2}}$ | $-2554(491)$ | $-2837(383)$ | $-1540(400)$ | $-1731(367)$ | $-833(128)$ | $-903(167)$
$\mathcal{F}_{1}^{\frac{3}{2}}$ | $-918(351)$ | $-981(493)$ | $-521(250)$ | $-252(339)$ | $-1074(1150)$ | $-596(598)$
$\mathcal{F}_{2}^{\frac{3}{2}}$ | $1565(776)$ | $-266(708)$ | $-381(210)$ | $-188(382)$ | $1556(1214)$ | $-170(969)$
$\mathcal{F}_{3}^{\frac{3}{2}}$ | $-752(435)$ | $-2264(234)$ | $-1085(189)$ | $-929(366)$ | $-1122(371)$ | $-1524(311)$
$\mathcal{F}_{1}^{\frac{5}{2}}$ | $-550(284)$ | $642(191)$ | $0(247)$ | $227(132)$ | $1270(299)$ | $839(168)$
Table 6: The best fit parameters of scheme III (solution $B$) shown in Fig. 12. Here only the statistical uncertainties are presented.  Parameter | background-1 | background-2 | background-3
---|---|---|---
solution | $B$ | $B$ | $B$
$C_{\frac{1}{2}}~{}[\text{GeV}^{-2}]$ | $7.41(47)$ | $9.34(82)$ | $8.23(39)$
$C_{\frac{3}{2}}~{}[\text{GeV}^{-2}]$ | $-17.65(48)$ | $-15.47(80)$ | $-16.69(40)$
$C_{\frac{1}{2}}^{\prime}~{}[\text{GeV}^{-2}]$ | $15.39(300)$ | $19.94(144)$ | $18.68(57)$
$D_{b}^{SD}~{}[\text{GeV}^{-4}]$ | $-1.45(43)$ | $-0.71(27)$ | $-0.90(16)$
${D}_{c}^{SD}~{}[\text{GeV}^{-4}]$ | $-2.23(90)$ | $-1.99(33)$ | $-2.83(11)$
$g_{S}~{}[\text{GeV}^{-2}]$ | $3.98(78)$ | $0.53(19)$ | $3.61(49)$
$g_{D}^{\prime}~{}[\text{GeV}^{-2}]$ | $1.51(37)$ | $0.24(10)$ | $2.50(29)$
$\mathcal{F}_{1}^{\frac{1}{2}}$ | $1118(882)$ | $12931(7254)$ | $1199(1096)$
$\mathcal{F}_{2}^{\frac{1}{2}}$ | $913(346)$ | $6493(6612)$ | $1160(480)$
$\mathcal{F}_{3}^{\frac{1}{2}}$ | $-1404(250)$ | $-23057(7843)$ | $-1404(261)$
$\mathcal{F}_{1}^{\frac{3}{2}}$ | $-390(195)$ | $-3599(2659)$ | $-1938(381)$
$\mathcal{F}_{2}^{\frac{3}{2}}$ | $60(255)$ | $-1422(1943)$ | $638(304)$
$\mathcal{F}_{3}^{\frac{3}{2}}$ | $-720(207)$ | $-7984(3574)$ | $-675(103)$
$\mathcal{F}_{1}^{\frac{5}{2}}$ | $120(135)$ | $2894(1812)$ | $235(199)$

## References

  * Chen  _et al._ (2016) H.-X. Chen, W. Chen, X. Liu, and S.-L. Zhu, Phys. Rept. 639, 1 (2016), arXiv:1601.02092 [hep-ph] .
  * Chen  _et al._ (2017) H.-X. Chen, W. Chen, X. Liu, Y.-R. Liu, and S.-L. Zhu, Rept. Prog. Phys. 80, 076201 (2017), arXiv:1609.08928 [hep-ph] .
  * Esposito  _et al._ (2017) A. Esposito, A. Pilloni, and A. Polosa, Phys. Rept. 668, 1 (2017), arXiv:1611.07920 [hep-ph] .
  * Hosaka  _et al._ (2016) A. Hosaka, T. Iijima, K. Miyabayashi, Y. Sakai, and S. Yasui, PTEP 2016, 062C01 (2016), arXiv:1603.09229 [hep-ph] .
  * Dong  _et al._ (2017) Y. Dong, A. Faessler, and V. E. Lyubovitskij, Prog. Part. Nucl. Phys. 94, 282 (2017).
  * Olsen  _et al._ (2018) S. L. Olsen, T. Skwarnicki, and D. Zieminska, Rev. Mod. Phys. 90, 015003 (2018), arXiv:1708.04012 [hep-ph] .
  * Guo  _et al._ (2018) F.-K. Guo, C. Hanhart, U.-G. Meißner, Q. Wang, Q. Zhao, and B.-S. Zou, Rev. Mod. Phys. 90, 015004 (2018), arXiv:1705.00141 [hep-ph] .
  * Cerri  _et al._ (2019) A. Cerri _et al._ , CERN Yellow Rep. Monogr. 7, 867 (2019), arXiv:1812.07638 [hep-ph] .
  * Altmannshofer  _et al._ (2019) W. Altmannshofer _et al._ (Belle-II), PTEP 2019, 123C01 (2019), [Erratum: PTEP 2020, 029201 (2020)], arXiv:1808.10567 [hep-ex] .
  * Brambilla  _et al._ (2020) N. Brambilla, S. Eidelman, C. Hanhart, A. Nefediev, C.-P. Shen, C. E. Thomas, A. Vairo, and C.-Z. Yuan, Phys. Rept. 873, 1 (2020), arXiv:1907.07583 [hep-ex] .
  * Guo  _et al._ (2020) F.-K. Guo, X.-H. Liu, and S. Sakai, Prog. Part. Nucl. Phys. 112, 103757 (2020), arXiv:1912.07030 [hep-ph] .
  * Liu  _et al._ (2019a) Y.-R. Liu, H.-X. Chen, W. Chen, X. Liu, and S.-L. Zhu, Prog. Part. Nucl. Phys. 107, 237 (2019a), arXiv:1903.11976 [hep-ph] .
  * Yuan and Olsen (2019) C.-Z. Yuan and S. L. Olsen, Nature Rev. Phys. 1, 480 (2019), arXiv:2001.01164 [hep-ex] .
  * Aaij  _et al._ (2015) R. Aaij _et al._ (LHCb), Phys. Rev. Lett. 115, 072001 (2015), arXiv:1507.03414 [hep-ex] .
  * Aaij  _et al._ (2016a) R. Aaij _et al._ (LHCb), Phys. Rev. Lett. 117, 082002 (2016a), arXiv:1604.05708 [hep-ex] .
  * Aaij  _et al._ (2016b) R. Aaij _et al._ (LHCb), Phys. Rev. Lett. 117, 082003 (2016b), [Addendum: Phys.Rev.Lett. 117, 109902 (2016), Addendum: Phys.Rev.Lett. 118, 119901 (2017)], arXiv:1606.06999 [hep-ex] .
  * Aaij  _et al._ (2019) R. Aaij _et al._ (LHCb), Phys. Rev. Lett. 122, 222001 (2019), arXiv:1904.03947 [hep-ex] .
  * Chen  _et al._ (2019a) H.-X. Chen, W. Chen, and S.-L. Zhu, Phys. Rev. D 100, 051501 (2019a), arXiv:1903.11001 [hep-ph] .
  * Chen  _et al._ (2019b) R. Chen, Z.-F. Sun, X. Liu, and S.-L. Zhu, Phys. Rev. D 100, 011502 (2019b), arXiv:1903.11013 [hep-ph] .
  * Guo  _et al._ (2019) F.-K. Guo, H.-J. Jing, U.-G. Meißner, and S. Sakai, Phys. Rev. D 99, 091501 (2019), arXiv:1903.11503 [hep-ph] .
  * Liu  _et al._ (2019b) M.-Z. Liu, Y.-W. Pan, F.-Z. Peng, M. Sánchez Sánchez, L.-S. Geng, A. Hosaka, and M. Pavon Valderrama, Phys. Rev. Lett. 122, 242001 (2019b), arXiv:1903.11560 [hep-ph] .
  * He (2019) J. He, Eur. Phys. J. C 79, 393 (2019), arXiv:1903.11872 [hep-ph] .
  * Guo and Oller (2019) Z.-H. Guo and J. Oller, Phys. Lett. B 793, 144 (2019), arXiv:1904.00851 [hep-ph] .
  * Shimizu  _et al._ (2019) Y. Shimizu, Y. Yamaguchi, and M. Harada, (2019), arXiv:1904.00587 [hep-ph] .
  * Xiao  _et al._ (2019a) C.-J. Xiao, Y. Huang, Y.-B. Dong, L.-S. Geng, and D.-Y. Chen, Phys. Rev. D 100, 014022 (2019a), arXiv:1904.00872 [hep-ph] .
  * Xiao  _et al._ (2019b) C. Xiao, J. Nieves, and E. Oset, Phys. Rev. D 100, 014021 (2019b), arXiv:1904.01296 [hep-ph] .
  * Wang  _et al._ (2020a) F.-L. Wang, R. Chen, Z.-W. Liu, and X. Liu, Phys. Rev. C 101, 025201 (2020a), arXiv:1905.03636 [hep-ph] .
  * Meng  _et al._ (2019) L. Meng, B. Wang, G.-J. Wang, and S.-L. Zhu, Phys. Rev. D 100, 014031 (2019), arXiv:1905.04113 [hep-ph] .
  * Wu  _et al._ (2019) J.-J. Wu, T.-S. Lee, and B.-S. Zou, Phys. Rev. C 100, 035206 (2019), arXiv:1906.05375 [nucl-th] .
  * Xiao  _et al._ (2019c) C. Xiao, J. Nieves, and E. Oset, Phys. Lett. B 799, 135051 (2019c), arXiv:1906.09010 [hep-ph] .
  * Voloshin (2019) M. Voloshin, Phys. Rev. D 100, 034020 (2019), arXiv:1907.01476 [hep-ph] .
  * Sakai  _et al._ (2019) S. Sakai, H.-J. Jing, and F.-K. Guo, Phys. Rev. D 100, 074007 (2019), arXiv:1907.03414 [hep-ph] .
  * Wang and Wang (2020) Z.-G. Wang and X. Wang, Chin. Phys. C 44, 103102 (2020), arXiv:1907.04582 [hep-ph] .
  * Yamaguchi  _et al._ (2020) Y. Yamaguchi, H. García-Tecocoatzi, A. Giachino, A. Hosaka, E. Santopinto, S. Takeuchi, and M. Takizawa, Phys. Rev. D 101, 091502 (2020), arXiv:1907.04684 [hep-ph] .
  * Liu  _et al._ (2019c) M.-Z. Liu, T.-W. Wu, M. Sánchez Sánchez, M. P. Valderrama, L.-S. Geng, and J.-J. Xie, (2019c), arXiv:1907.06093 [hep-ph] .
  * Lin and Zou (2019) Y.-H. Lin and B.-S. Zou, Phys. Rev. D 100, 056005 (2019), arXiv:1908.05309 [hep-ph] .
  * Wang  _et al._ (2019) B. Wang, L. Meng, and S.-L. Zhu, JHEP 11, 108 (2019), arXiv:1909.13054 [hep-ph] .
  * Gutsche and Lyubovitskij (2019) T. Gutsche and V. E. Lyubovitskij, Phys. Rev. D 100, 094031 (2019), arXiv:1910.03984 [hep-ph] .
  * Burns and Swanson (2019) T. Burns and E. Swanson, Phys. Rev. D 100, 114033 (2019), arXiv:1908.03528 [hep-ph] .
  * Du  _et al._ (2020a) M.-L. Du, V. Baru, F.-K. Guo, C. Hanhart, U.-G. Meißner, J. A. Oller, and Q. Wang, Phys. Rev. Lett. 124, 072001 (2020a), arXiv:1910.11846 [hep-ph] .
  * Wang  _et al._ (2020b) G.-J. Wang, L.-Y. Xiao, R. Chen, X.-H. Liu, X. Liu, and S.-L. Zhu, Phys. Rev. D 102, 036012 (2020b), arXiv:1911.09613 [hep-ph] .
  * Xu  _et al._ (2020) H. Xu, Q. Li, C.-H. Chang, and G.-L. Wang, Phys. Rev. D 101, 054037 (2020), arXiv:2001.02980 [hep-ph] .
  * Kuang  _et al._ (2020) S.-Q. Kuang, L.-Y. Dai, X.-W. Kang, and D.-L. Yao, Eur. Phys. J. C 80, 433 (2020), arXiv:2002.11959 [hep-ph] .
  * Peng  _et al._ (2020) F.-Z. Peng, M.-Z. Liu, M. Sánchez Sánchez, and M. Pavon Valderrama, Phys. Rev. D 102, 114020 (2020), arXiv:2004.05658 [hep-ph] .
  * Peng  _et al._ (2021a) F.-Z. Peng, J.-X. Lu, M. Sánchez Sánchez, M.-J. Yan, and M. Pavon Valderrama, Phys. Rev. D 103, 014023 (2021a), arXiv:2007.01198 [hep-ph] .
  * Xiao  _et al._ (2020) C. Xiao, J. Lu, J. Wu, and L. Geng, Phys. Rev. D 102, 056018 (2020), arXiv:2007.12106 [hep-ph] .
  * Dong  _et al._ (2021) X.-K. Dong, F.-K. Guo, and B.-S. Zou, (2021), arXiv:2101.01021 [hep-ph] .
  * Peng  _et al._ (2021b) F.-Z. Peng, M. Sánchez Sánchez, M.-J. Yan, and M. Pavon Valderrama, (2021b), arXiv:2101.07213 [hep-ph] .
  * Ali and Parkhomenko (2019) A. Ali and A. Y. Parkhomenko, Phys. Lett. B 793, 365 (2019), arXiv:1904.00446 [hep-ph] .
  * Zhu  _et al._ (2019) R. Zhu, X. Liu, H. Huang, and C.-F. Qiao, Phys. Lett. B 797, 134869 (2019), arXiv:1904.10285 [hep-ph] .
  * Wang (2020) Z.-G. Wang, Int. J. Mod. Phys. A 35, 2050003 (2020), arXiv:1905.02892 [hep-ph] .
  * Giron  _et al._ (2019) J. F. Giron, R. F. Lebed, and C. T. Peterson, JHEP 05, 061 (2019), arXiv:1903.04551 [hep-ph] .
  * Cheng and Liu (2019) J.-B. Cheng and Y.-R. Liu, Phys. Rev. D 100, 054002 (2019), arXiv:1905.08605 [hep-ph] .
  * Stancu (2019) F. Stancu, Eur. Phys. J. C 79, 957 (2019), arXiv:1902.07101 [hep-ph] .
  * Eides  _et al._ (2020) M. I. Eides, V. Y. Petrov, and M. V. Polyakov, Mod. Phys. Lett. A 35, 2050151 (2020), arXiv:1904.11616 [hep-ph] .
  * Fernández-Ramírez  _et al._ (2019) C. Fernández-Ramírez, A. Pilloni, M. Albaladejo, A. Jackura, V. Mathieu, M. Mikhasenko, J. Silva-Castro, and A. Szczepaniak (JPAC), Phys. Rev. Lett. 123, 092001 (2019), arXiv:1904.10021 [hep-ph] .
  * Guo  _et al._ (2015a) F.-K. Guo, C. Hanhart, Q. Wang, and Q. Zhao, Phys. Rev. D 91, 051504 (2015a), arXiv:1411.5584 [hep-ph] .
  * Dong  _et al._ (2020) X.-K. Dong, F.-K. Guo, and B.-S. Zou, (2020), arXiv:2011.14517 [hep-ph] .
  * Xiao  _et al._ (2013) C. Xiao, J. Nieves, and E. Oset, Phys. Rev. D 88, 056012 (2013), arXiv:1304.5368 [hep-ph] .
  * Liu  _et al._ (2018) M.-Z. Liu, F.-Z. Peng, M. Sánchez Sánchez, and M. P. Valderrama, Phys. Rev. D 98, 114030 (2018), arXiv:1811.03992 [hep-ph] .
  * Pavon Valderrama (2019) M. Pavon Valderrama, Phys. Rev. D 100, 094028 (2019), arXiv:1907.05294 [hep-ph] .
  * Aaron  _et al._ (1968) R. Aaron, R. Amado, and J. Young, Phys. Rev. 174, 2022 (1968).
  * Mai  _et al._ (2017) M. Mai, B. Hu, M. Doring, A. Pilloni, and A. Szczepaniak, Eur. Phys. J. A 53, 177 (2017), arXiv:1706.06118 [nucl-th] .
  * Zyla  _et al._ (2020) P. Zyla _et al._ (Particle Data Group), PTEP 2020, 083C01 (2020).
  * Baru  _et al._ (2017) V. Baru, E. Epelbaum, A. Filin, C. Hanhart, and A. Nefediev, JHEP 06, 158 (2017), arXiv:1704.07332 [hep-ph] .
  * Baru  _et al._ (2016) V. Baru, E. Epelbaum, A. Filin, C. Hanhart, U.-G. Meißner, and A. Nefediev, Phys. Lett. B 763, 20 (2016), arXiv:1605.09649 [hep-ph] .
  * Ericson and Weise (1988) T. E. O. Ericson and W. Weise,  _Pions and Nuclei_ (Clarendon Press, Oxford, UK, 1988).
  * Wang  _et al._ (2018) Q. Wang, V. Baru, A. Filin, C. Hanhart, A. Nefediev, and J.-L. Wynen, Phys. Rev. D 98, 074023 (2018), arXiv:1805.07453 [hep-ph] .
  * Baru  _et al._ (2019) V. Baru, E. Epelbaum, A. Filin, C. Hanhart, A. Nefediev, and Q. Wang, Phys. Rev. D 99, 094013 (2019), arXiv:1901.10319 [hep-ph] .
  * Voloshin (2011) M. Voloshin, Phys. Rev. D 84, 031502 (2011), arXiv:1105.5829 [hep-ph] .
  * Skerbis and Prelovsek (2019) U. Skerbis and S. Prelovsek, Phys. Rev. D 99, 094505 (2019), arXiv:1811.02285 [hep-lat] .
  * Du  _et al._ (2020b) M.-L. Du, V. Baru, F.-K. Guo, C. Hanhart, U.-G. Meißner, A. Nefediev, and I. Strakovsky, Eur. Phys. J. C 80, 1053 (2020b), arXiv:2009.08345 [hep-ph] .
  * Baru  _et al._ (2020) V. Baru, E. Epelbaum, A. A. Filin, C. Hanhart, R. V. Mizuk, A. V. Nefediev, and S. Ropertz, (2020), arXiv:2012.05034 [hep-ph] .
  * Hanhart  _et al._ (2015) C. Hanhart, Y. S. Kalashnikova, P. Matuschek, R. Mizuk, A. Nefediev, and Q. Wang, Phys. Rev. Lett. 115, 202001 (2015), arXiv:1507.00382 [hep-ph] .
  * Albaladejo  _et al._ (2016) M. Albaladejo, F.-K. Guo, C. Hidalgo-Duque, and J. Nieves, Phys. Lett. B 755, 337 (2016), arXiv:1512.03638 [hep-ph] .
  * Guo  _et al._ (2016) F.-K. Guo, C. Hanhart, Y. S. Kalashnikova, P. Matuschek, R. Mizuk, A. Nefediev, Q. Wang, and J. L. Wynen, Phys. Rev. D 93, 074031 (2016), arXiv:1602.00940 [hep-ph] .
  * Wise (1992) M. B. Wise, Phys. Rev. D 45, 2188 (1992).
  * Yan  _et al._ (1992) T.-M. Yan, H.-Y. Cheng, C.-Y. Cheung, G.-L. Lin, Y. Lin, and H.-L. Yu, Phys. Rev. D 46, 1148 (1992), [Erratum: Phys.Rev.D 55, 5851 (1997)].
  * Manohar and Wise (2000) A. V. Manohar and M. B. Wise,  _Heavy quark physics_ , Cambridge Monographs on Particle Physics, Nuclear Physics, and Cosmology (Cambridge University Press, 2000).
  * Detmold  _et al._ (2012) W. Detmold, C. Lin, and S. Meinel, Phys. Rev. D 85, 114508 (2012), arXiv:1203.3378 [hep-lat] .
  * Baru  _et al._ (2015) V. Baru, E. Epelbaum, A. Filin, J. Gegelia, and A. Nefediev, Phys. Rev. D 92, 114016 (2015), arXiv:1509.01789 [hep-ph] .
  * Burns (2015) T. Burns, Eur. Phys. J. A 51, 152 (2015), arXiv:1509.02460 [hep-ph] .
  * Baru  _et al._ (2011) V. Baru, A. Filin, C. Hanhart, Y. Kalashnikova, A. Kudryavtsev, and A. Nefediev, Phys. Rev. D 84, 074029 (2011), arXiv:1108.5644 [hep-ph] .
  * Aaij  _et al._ (2020) R. Aaij _et al._ (LHCb), Phys. Rev. D 102, 112012 (2020), arXiv:2007.11292 [hep-ex] .
  * Lin  _et al._ (2017) Y.-H. Lin, C.-W. Shen, F.-K. Guo, and B.-S. Zou, Phys. Rev. D 95, 114017 (2017), arXiv:1703.01045 [hep-ph] .
  * Piucci (2019) A. Piucci,  _Amplitude analysis of $\Lambda_{b}^{0}\to\Lambda_{c}^{+}\bar{D}^{0}K^{-}$ decays and pentaquark searches in the $\Lambda_{c}^{+}\bar{D}^{0}$ system at the LHCb experiment._, Ph.D. thesis, Heidelberg U. (2019).
  * Guo  _et al._ (2015b) F.-K. Guo, U.-G. Meißner, W. Wang, and Z. Yang, Phys. Rev. D 92, 071502 (2015b), arXiv:1507.04950 [hep-ph] .
  * Liu  _et al._ (2016) X.-H. Liu, Q. Wang, and Q. Zhao, Phys. Lett. B 757, 231 (2016), arXiv:1507.05359 [hep-ph] .
  * Bayar  _et al._ (2016) M. Bayar, F. Aceti, F.-K. Guo, and E. Oset, Phys. Rev. D 94, 074039 (2016), arXiv:1609.04133 [hep-ph] .
  * Oller (2019) J. A. Oller,  _A Brief Introduction to Dispersion Relations_, Springer Briefs in Physics (Springer, 2019).