# The Dual-Edged Sword of Technical Debt: Benefits and Issues Analyzed Through
Developer Discussions

Xiaozhou Li<EMAIL_ADDRESS>Matteo Esposito<EMAIL_ADDRESS>Andrea Janes<EMAIL_ADDRESS>Valentina Lenarduzzi
<EMAIL_ADDRESS>University of Oulu, Finland  Free University of
Bozen-Bolzano, Italy

###### Abstract

Background. Technical debt (TD) has long been one of the key factors
influencing the maintainability of software products. It represents technical
compromises that sacrifice long-term software quality for potential short-term
benefits.
Objective. This work is to collectively investigate the practitioners’
opinions on the various perspectives of TD from a large collection of
articles. We find the topics and latent details of each, where the sentiments
of the detected opinions are also considered.
Method. For such a purpose, we conducted a grey literature review on the
articles systematically collected from three mainstream technology forums.
Furthermore, we adopted natural language processing techniques like topic
modeling and sentiment analysis to achieve a systematic and comprehensive
understanding. However, we adopted ChatGPT to support the topic
interpretation.
Results. In this study, 2,213 forum posts and articles were collected, with
eight main topics and 43 sub-topics identified. For each topic, we obtained
the practitioners’ collective positive and negative opinions.
Conclusion. We identified 8 major topics in TD related to software
development. Identified challenges by practitioners include unclear roles and
a lack of engagement. On the other hand, active management supports
collaboration and mitigates the impact of TD on the source code.

###### keywords:

Technical Debt, Topic Modeling, Opinion Mining, NLP, ChatGPT

††journal: Journal of Systems and Software

##  1 Introduction

Imagine purchasing a high-end product today, enjoying all its benefits
immediately, but deferring the payment to a future date. While convenient in
the short term, this financial strategy often incurs interest and can lead to
a significant burden over time. This common scenario nowadays is called
financial debt [1].

Similarly, in software development, stashing technical debt (TD) allows teams
to deliver features quickly but at the cost of future agility and efficiency
[1, 2]. Just as financial debt can grow and compound, so too can TD,
transforming initial gains into long-term liabilities that can cripple a
project’s progress and sustainability.

Therefore, TD is a commonly adopted metaphor in software engineering that
represents the implied cost of future reworking resulting from the earlier
choices of easy but limited solutions. Since Cunningham first brought up the
concept [3], it has received increasing attention in the last decade.

Though it is often considered an issue to be addressed in software development
and maintenance practice, it is also well-acknowledged that TD can be both
good and bad. One of the main advantages of incurring TD is the reduced
maintenance time and cost in a given short time, which can help critical in-
time deliveries and speed up the initial development [4]. Especially when such
technical debt, different from financial debt, might not require interest to
be paid, which makes it nearly pure gain. Besides such a benefit in
development time saving, TD can also help companies save startup capital and
decrease risks [5]. Furthermore, it is also beneficial that practitioners
introduce TD to focus resources and efforts on achieving time-critical
business goals and enable proper prioritization [6].

On the other hand, the direct and indirect issues caused by the TD are well-
recognized by many studies, e.g., increased maintenance cost [7], reduced code
quality and system reliability [8], slow feature development [9], decreased
team morale [10], and so on.

Nonetheless, past studies focus on the symptom and not on the cause. Our work
departs from the previous empirical studies based on code and metrics and aims
to analyze practitioners’ opinions on technical debt. To the best of our
knowledge, none of the existing literature investigated practitioners’
opinions on TD surveying the grey literature. Our main contributions are as
follows:

  * 1.

large-scale opinion mining on TD by surveying grey literature.

  * 2.

use of large language model (LLM) for data extraction and synthesis.

  * 3.

identification of eight major topics, and 43 sub-topics for future research
and practitioners’ reference.

Future research efforts should focus on the developer’s issues with TD while
consolidating and improving the good practices. Moreover, research efforts
should also focus on AI-driven TD detection and management, as well as an
analysis of the effect of TD on current software development methodologies
such as DevOps.

Paper Structure. Section 2 describes the design study we conducted. Section 3
describes the obtained results, while Section 4 discusses them. Section 5
highlights the threats to validity of this work. Section 6 describes the
related work, while Section 7 draws the conclusion and future works.

##  2 Study Design

This section describes our study’s goals, research questions, and the
methodology for answering them. Generally, to analyze the collective opinions
of software practitioners on TD, we first collect a representative amount of
articles from popular technology forums. With such text data, by adopting
Natural Language Processing (NLP) techniques, e.g., topic modeling and
sentiment analysis, we extract the main topics discussed on TD and the
positive and negative opinions regarding each topic. In particular, we use
ChatGPT to support the effective summarization of each topic based on the
according keywords. Our empirical study follows the guidelines defined by
Wohlin et al. [11].

###  2.1 Goal and Research Questions

The goal of our study is to conduct a large-scale survey of grey literature
with the _purpose_ of identifying practitioners’ most discussed aspects of TD
and their sentiment connected to it. Our perspective is of the practitioners
who face the daunting task of paying or accumulating TD. The context is forum
posts and online articles discussing aspects of TD.

Based on our goal, we defined the following three Research Questions.

RQ1. _What aspects are commonly discussed when practitioners talk about TD?_

Practitioners in the past weren’t always aware of TD concepts and their
impacts on their source code [7]. Nowadays, with the increasing focus on
software quality both from academia and industry [2], practitioners have come
to be knowledgeable about their decisions. Albeit TD sometimes is opaque to
them, the proportion of “conscious” TD, or so-called self-admitted TD, is non-
negligible [12, 13]. Hence, practitioners have opinions on it and, based on
their needs and constraints, they find themselves bound to accumulate TD to
meet expectations and deadlines [14, 15]

Therefore, in RQ1, we aim to find the topics commonly discussed in the
articles and discussion threads posted by practitioners regarding TD to
discover practitioners’ most focused aspects of TD. Identifying only the main
topics does not contribute deeply to the state of the art nor allows for
extracting comprehensive conclusions. Hence, we ask:

RQ2. _What are the issues about TD that the practitioners commonly complain
about?_

Based on the main topics extracted in RQ1, we further identify the sub-topics
for each aspect of TD. In an optimal environment, practitioners have full
knowledge and time to avoid stashing TD [14, 16], but this cannot be further
from reality [16]. Practioners have different sentiments tied to specific TDs
[12] when self-admitting TD. Therefore, as suggested by Fucci et al. [12] use
of sentiment analysis, in RQ2, we analyze the content and the sentiment of the
practitioner’s discussions on TD, particularly focusing on their collectively
negative opinions. Finally, we want to focus on the positive aspects discussed
by the practitioners. Hence we ask:

RQ3. _What benefits can (paying) TD bring?_

Like in finance, paying a debt can have positive impacts [16]. Therefore, in
RQ3, we are interested in investigating the positive opinions regarding TD,
possibly including the benefits of repaying TD and effective strategies to
solve TD issues.

###  2.2 Data and Processing

Here, we introduce how and from where the data is collected and processed for
analysis.

####  2.2.1 Keyword selection

To collect all the relevant texts from the target sources, we use ”TD” as the
search query, which is straightforward but effective. First of all, this study
must include the initial text selection. Therefore, we do not add any other
keywords to limit the results. Meanwhile, we also aim to avoid being overly
inclusive with an unnecessary amount of irrelevant articles. For example, we
did not attach the abbreviation of “TD”, TD, as another keyword using “OR”
because texts on HTML programming can also be included (¡td¿ is the tag
defines a standard data cell in an HTML table). It is nearly universal that
any articles on TD that use the abbreviation mention the full term at least
once. Furthermore, we didn’t consider the query “technical OR debt” for the
same reason.

####  2.2.2 Data Source selection

To obtain the opinions of practitioners, we extract such content of the
discussion and idea-sharing articles from three popular technology forums,
i.e., StackOverflow, Medium and DZone. StackOverflow is the largest forum of
technology-related questions and answers (Q&As) for developers and tech
enthusiasts. Compared to StackOverflow, DZone is also one of the world’s
largest online communities for developers but focuses more on new tech trends,
e.g., DevOps, AI, big data, Microservices, etc. Furthermore, like DZone,
Medium is a well-known technology forum that provides tutorials and review
articles. Comparatively, articles on Medium are more common-reader-friendly
and written in a non-technical style. These three platforms are the largest
tech communities that can be considered to cover a representative school of
opinions described in different styles.

Figure 1: Overview of the followed process (adapted from [17])

####  2.2.3 Data Extraction

Due to the different availability of APIs and different crawling policies of
these three platforms, we applied different data crawling strategies for each
platform accordingly:

  * 1.

StackOverflow. We applied API-based content crawling using the StackExchange
API to retrieve the questions and answers regarding each selected tracing
tool. Specifically, we used the advanced search
API111https://api.stackexchange.com/docs/advanced-search to extract all the
questions that contain the name of the tool in either the title or the body of
the question, together with the according answers. Please note that due to the
API’s daily query limitation, the pagesize parameter was set to the maximum
(i.e., 100 results shown per query) to minimize the crawling time.

  * 2.

Medium: Though Medium’s policy does not allow direct crawling from the forum,
instead, it provides archive for the articles assigned with tags. All the
historical articles with a given tag $T$ can be found at
https://medium.com/tag/T/archive. Therefore, by using tools like
BeautifulSoup222https://www.crummy.com/software/BeautifulSoup/, we could
obtain all the corresponding articles by traversing all the time-stamp URLs.

  * 3.

DZone: We adopted a hybrid crawling approach combining manual search and
BeautifulSoup. Unlike Medium, Dzone allows crawling with BeautifulSoup within
each article but not within the search results list. Hence, we conducted a
hybrid crawling strategy by manually collecting all article URLs for each tool
and then automatically crawling the article content for each URL using
BeautifulSoup.

###  2.3 Data Analysis

By following the steps shown in Figure 2, we can extract the topics commonly
discussed by the practitioners and their collective opinions on each topic.
The process contains five steps, which are described below.

Figure 2: Data Analysis process (adapted from [17])

  * 1.

Step 1: Preprocessing: This step pre-processes the raw text data and prepares
them for further analysis. First, we divide texts from the dataset into
sentence-level instances since each text can contain multiple topics and
various sentiments. Second, we build the bigram and trigram models, which
means we identify the common phrases (e.g., New York instead of new and york).
Subsequently, a series of text processing activities are required for each
sentence, including transforming text into lower cases, removing non-alpha-
numeric symbols, screening stop-words, eliminating extra white spaces, and
lemmatization.

  * 2.

Step 2: Informativeness Filtering: This step filters out the noninformative
sentences with a trained text classifier. By doing so, we shall identify the
sentences that contain useful information and screen out those not relevant.
Aiming to answer the research questions, the informative sentences shall
contain explicit discussion regarding TD. For example, “In the context of
software development, TD is a shortcut taken by the developer to save time and
finish a given project faster than they would otherwise be able to.” is
informative by describing the concept and practice of TD in general; “Do get
in touch with us at<EMAIL_ADDRESS>to inquire about how we can help you build
your product.” is non-informative and should be filtered out.

  * 3.

Step 3: Topic Modeling: Herein, we detect the main topics of the informative
sentences identified from the previous step. We shall determine the main
aspects the articles discuss using topic modeling techniques. Selecting a
topic model that fits the data is recommended to obtain topics that more
accurately reflect the text data. For example, as the texts are tokenized to
sentence level, a short-text topic modeling method, e.g., Gibbs Sampling
Dirichlet Multinomial Mixture (GSDMM), is more effective than traditional
Latent Dirichlet Allocation (LDA). Furthermore, if possible, domain-specific
topic models can also be adopted.

  * 4.

Step 4: Topic Filtering: In this step, by using the first topic model built
with the informative texts in the previous step, we could map each sentence to
the topic it belongs to. Supposedly, when the topics obtained are too general,
we need to investigate further the topics discussed within each of the general
topics, which can be seen as “aspect”. We can adopt the same topic modeling
method as the previous step for this step.

  * 5.

Step 5: Opinion Mining: Finally, using opinion mining techniques on the texts,
we can also know the collective sentiment from the social media concerning
each topic in each aspect. Therefore, the outcome shall provide a detailed
reflection on the percentage of positiveness and negativeness for each topic
regarding TD.

Especially regarding the interpretation of the topics from lists of keywords,
we adapt the ChatGPT-driven techniques used in Janes et al.’s study [17].
Using ChatGPT [18] can support effectively interpreting and summarizing the
topics based on these keywords. ChatGPT is an artificial intelligence chatbot
developed by OpenAI that uses foundational large language models (LLMs) and is
fine-tuned via supervised and reinforcement learning techniques. Though a
newly emerging technique, ChatGPT has quickly gained overwhelmingly worldwide
attention from industry and academia. Specifically, regarding the facilitation
of text summarization, many early-stage studies have investigated the use of
ChatGPT for such tasks [19, 20].

Herein, for this topic extraction task, we adopted the newly released GPT-4
model333https://openai.com/product/gpt-4. Compared to the legacy GPT-3.5
model, the new model has much higher reasoning capacity and conciseness. We
initiate the topic extraction by entering a series of structured requests
formatted as follows.

“Extract a short ¡MAIN TOPIC¿-related topic for each of these lines of
keywords.

¡1st LINE OF KEYWORDS¿

…

¡$n$th LINE OF KEYWORDS¿”

The replies received were also structured to correspond to the following
requests.

“¡1st TOPIC¿: ¡Explanation¿

…

¡$n$th TOPIC¿: ¡Explanation¿”

The first and third authors compare the AI-extracted topics and the original
keyword lists for validation.

###  2.4 Verifiability and Replicability

We have published the complete raw data in the replication package to allow
our study to be replicated444https://zenodo.org/records/13133205.

##  3 Results

By following the steps of the methods provided in Section 2, we present the
results that can answer the proposed research questions.

###  3.1 Step 1: Preprocessing.

We pre-processed the crawled textual data by retaining only the natural
language sentences. Herein, we eliminated unnecessary content, such as the
source code, URLs, publishing date, author info, etc. For Medium articles, we
started eliminating the article heading that includes the publishing date and
author info by splitting the string at the common last character of the part
“min read” and selecting the later part. Subsequently, we used the sentence
tokenizer from the Natural Language Toolkit (NLTK)555http://www.nltk.org/ to
obtain the list of sentences from each article. As the tokenizer does not
identify the source code or URLs, we eliminated them by selecting only the
sentences ending with a period, an exclamation mark, or a question mark.

First, we crawled data from social media, including StackOverflow (319
questions and 510 answers), Medium (1368 articles), and Dzone (121 posts).
Furthermore, we clean the text data by using langdetect Python package
666https://pypi.org/project/langdetect/ to filter the non-English texts,
html2text package 777https://pypi.org/project/html2text/ to filter the source
code and HTML markdowns from each text and customized regular expression based
methods to drop the source code. Thereafter, we have 318 StackOverflow
questions, 509 StackOverflow answers, 1265 Medium articles, and 121 Dzone
posts. Then, we tokenized the texts into sentences using the NLTK sentence
tokenizer and obtained 3044 sentences in StackOverflow questions, 965 in
StackOverflow answers, 49903 in Medium articles, and 6801 in Dzone posts.

###  3.2 Step 2: Informativeness Filtering.

Herein, we identified the informative sentences using a Naïve Bayes (NB)
classifier or the Expectation Maximization for Naïve Bayes (EMNB) classifier
[21]. The study of Janes et al. [17] suggests the two options that provide
very high performance. The selection shall be based on the accuracy comparison
of these two classifiers with the obtained dataset. First, we manually labeled
a sufficient number of training data, including 50% informative sentences and
50% half non-informative ones. The selection criteria for informative
sentences are that the sentence must explicitly discuss TD. With increasing
training and testing data, the two classifiers shall be trained and compared
with the F1-score using a 5-fold cross-validation.

Figure 3: Testing informative text classifier accuracy

In this study, from the 60713 sentences obtained previously, we manually
selected 3 000 training data, including 1500 informative sentences and 1500
non-informative ones. To evaluate the performance of informativeness
filtering, with a series of experiments, we compared the results of the NB
algorithm and the EMNB algorithm with 3000 training data. We inspected the
accuracy comparison of the two classifiers with different amounts of data
starting from 200 to 3000 with an incremental step of 20. The test data ratio
is set as default (0.25). Figure 3 shows that with the given training data, NB
performs better than EMNB, with the accuracy reaching as high as 0.843. The
classifier performs even better than the one used in [17] (0.76). Thus, we
adopted the NB classifier for filtering the informative sentences. Using the
classifier trained by the 3000 training data, we obtained 36523 informative
sentences.

###  3.3 Step 3: Topic Modeling.

To extract the topics from a large volume of textual data, we adopt the
collapsed Gibbs Sampling Dirichlet Multinomial Mixture (GSDMM) [22].
Initially, the textual data points (i.e., the sentences obtained previously)
are randomly assigned to a pre-defined number of clusters with a set of
critical document label information. Then the documents are traversed for
several iterations, in each of which a cluster is re-assigned to each document
according to a conditional distribution where these rules are followed: 1)
choose a cluster with more documents, 2) choose a cluster where the documents
therein have higher similarity. GSDMM has a much better performance on short
texts compared with LDA [23]; therefore, it is the better option for this
study.

Herein, considering the effectiveness of the GSDMM model and the efficiency of
the modeling process, we adopt the common default setting of the model with 30
iterations, $\alpha=0.25$ and $\beta=0.15$. Meanwhile, we set the upper bound
topic number parameter $K=30$. We consider Li et al.’s secondary study [24]
summarized 24 TD-related notions when it would be highly unlikely the total
topic number exceeds such a limit. Furthermore, we set the number of words for
each topic to 30 to ease the topic interpretation.

###  3.4 Step 4: Topic Filtering.

By training with the 36523 informative sentences, we obtained the GSDMM topic
model with 20 topics. Therein, 35521 sentences (97.3%) are associated with
eight topics. Each of the other 12 topics contains no more than 2.3% of the
data; therefore, these topics are omitted. The number of sentences and
extracted topics are reported in Table 1.

Table 1: The Main Topics on TD # |  Interpreted Topic | #Sentences
---|---|---
1 |  The True Cost of TD: Balancing Short-term with Long-term Pain | 2788
2 |  Managing TD: a Team-Centric Approach | 10183
3 |  Paying down TD in Agile Sprints | 1982
4 |  Understanding the Business Impact of TD | 2480
5 |  Measuring TD: Tools and Metrics | 1183
6 |  TD in Product Development: A Necessary Evil? | 5063
7 |  Balancing New Features and TD in Software Development | 7744
8 |  Code Quality and TD: the Role of Testing | 4098

Furthermore, we adopted the same process to train a separate GSDMM topic model
for each subset of sentences associated with each main topic. We used the same
parameters for the main topic model described above for such a process. In
addition, to reduce the bias caused by the limited data volume, we selected
the sub-topics associated with at least 100 sentences. We reached at least 3%

###  3.5 Step 5: Topic Interpretation and Mapping.

Here are the eight popular topics commonly discussed by practitioners
collectively on social media platforms.

[T1] The True Cost of TD: Balancing Short-term Gain with Long-term Pain. One
of the main topics discussed by the practitioners is the nearly unpredictable
cost of TD as well as the trade-off of short-term gain and long-term pain. It
is a common concern that accumulated TD can have insufferable financial
implications in software projects. Although delayed mitigation of such TD can
be costly for maintenance, the in-time quick solution is urgently needed.
Furthermore, the sub-topics in this aspect are as follows.

  * 1.

[T1.1] The Long-Term Consequences of Shortcuts in Software Design: The
potential long-term consequence of TD is commonly the main concern of
practitioners, as the core of TD adoption is to meet the urgent deadline by
taking necessary shortcuts. The unforeseeable consequences can be damaging.
One article mentioned that among the consequences of TD, the loss of clients
is by far the worst. The more direct influence on the near future can also be
with each new feature added, even more efforts are required for a developer to
implement it.

  * 2.

[T1.2] Choosing Quick Solutions and Incurring TD: The other side of the coin
for incurring TD is the benefit of having quick solutions in time. The
apparent benefit of such activity is saving time in the short term when
additional rework and increased complexity can be inevitable to a certain
extent. Acknowledgedly, in this way, issues e.g., developers become less and
less likely to fix systemic issues, instead choosing to implement workarounds,
layers of abstraction, and quick fixes, also occur.

  * 3.

[T1.3] Balancing Speed and Sustainability in TD Payment: The impact of
incurring TD on the sustainability in TD payment is another concern of the
practitioners when the balance between speed and sustainability is needed by
practitioners. For example, one opinion is that shipping imperfect code can be
justified, but business leaders will ultimately shoulder the responsibility
for paying down tech debt over time. And slowly “paying down” tech debt while
investing in writing new code is the common strategy suggested.

  * 4.

[T1.4] Financial Analogies in TD: The financial analogies are also commonly
adopted when the article introduces the concept of TD for a beginner audience.
Meanwhile, much like debt in finance, the burden of TD can be unbearable at a
certain point.

  * 5.

[T1.5] High Cost of TD in Financial Terms: When adopting financial debt as an
analogy or metaphor, the practitioners often express their concern about the
potential high cost in financial terms. Similar to the debt or loan concept in
financial terms, the high interest can cost increasingly more on resources and
development efforts. Such a cost can impact the business through disruption,
revenue, fines, brand equity, among others”.

  * 6.

[T1.6] Short-Term Gains vs. Long-Term Costs in TD: On the other hand, the
short-term benefits are also critical to the TD concept, as it is the main
reason to adopt it. Regarding the motivation behind such tendency, it is
acknowledged by many that this is due to our present bias, the tendency of
humans to prefer payoffs in the short term over those in the long term and
software engineers, in addition to having the same fundamental bias, normally
have a deep desire to see their work shipped.

  * 7.

[T1.7] The Lifecycle of TD in Software Development: The lifecycle of TD in
software development is another concern in terms of the cost and balancing for
TD. Practitioners acknowledge that throughout its lifecycle, a product-
engineering team will need to constantly balance the need to move quickly and
test ideas with building a maintainable and flexible codebase. Meanwhile, it
is also advocated that incorporating regular reduction of technical debt into
the product lifecycle would create a more controlled and viable product that
is more suitable to follow and serve future customer needs.

[T2] Managing TD: A Team-centric Approach. Due to the high cost of accumulated
TD and potential risk in the future, TD management is a critical aspect the
practitioners need to proactively conduct through the project’s lifecycle.
Apparently, managing TD requires team effort where management skills are
required towards the needs of effective communication and problem solving. The
goal is to incorporate the in-time solution for fast delivery while preventing
TD from hindering future development. Furthermore, the sub-topics in this
aspect are as follows.

  * 1.

[T2.1] Solving TD through Team Collaboration: The practitioners indicate team
collaboration is one of the ”best practices” to facilitate the effective
mitigation of TD because finding and fixing the loop is a much harder problem
and would require cross-team effort. Especially, when coming to estimation and
priority, one mentioned the experience of thinking of “priorities” is better,
estimations are still necessary when it comes to team collaborations, which
emphasizes the importance of collaboration towards solving TD.

  * 2.

[T2.2] Tools and Strategies for Reducing TD: In order to properly manage TD,
suitable and effective tools and strategies adoption are important. As
mentioned by practitioners, You need to make tool choices that enable you to
hire properly and allow you to scale appropriately, and give you the
flexibility to do what you need to do as a company. Meanwhile, practitioners
also suggest that isolating messy code is a very effective strategy for
dealing with unavoidable debts. Furthermore, it is also acknowledged that
having a clear understanding of what sort of technical debt you are going to
run up will enable you to have a solid strategy for dealing with it.

  * 3.

[T2.3] Understanding and Managing TD as a Team: In order to solve TD
effectively, understanding TD in the first place is critical, which also
requires team effort. Practitioners mentioned that it is common that most
techies don’t really understand technical debt as a concept. What is worse,
even sometimes the general managers don’t understand why it is important to
invest time on technical debts. Many practitioners emphasize that in order for
the debt to be addressed within the agile process, we need the whole team to
understand the state of the code and how it impacts delivery.

  * 4.

[T2.4] The Role in Team for Perception in TD: Practitioners also need to pay
attention to the different perceptions of TD from the different roles in a
software team. As mentioned by practitioners, the understanding of TD differs
between roles, such as general managers and technical staff. Other roles, such
as architects and company leadership, are also mentioned to have an obligation
to understand TD and its impact.

  * 5.

[T2.5] Team Involvement in Decision-Making on TD: Based on the common and
different understanding of TD across the team, together with the support of
proper tools and strategies, the decision-making activity is also critical in
terms of TD management, which also requires team involvement. Practitioners
mention that communication gaps about the process leading to poor decision
making, … your technical debt load is increasing, you’ll continue to drown and
reduce your team’s velocity. On the other hand, it is also acknowledged that
various decision-making will also lead to TD.

  * 6.

[T2.6] The Role of Engineers in Managing TD: Software engineers (i.e.,
developers) certainly play a crucial role in managing TD within software
development, as they are mostly the direct inducer of TD as mostly, technical
debt occurs because the developers write subpar code. Other roles, like
security engineer and data engineer, can also be influenced by TD management.
For example, technical debt in your data pipeline will impact your
organization in ways that will annoy stakeholders, make the working lives of
analysts tedious, and frustrate data engineers.

  * 7.

[T2.7] Managing TD in Agile Teams: Regarding team collaboration in managing
TD, many practitioners mentioned such practice and its importance in agile
teams, as TD management is a continuous process for agile teams. One of the
advantages is that .. clean code is linked to agile development and continuous
delivery. Meanwhile, adopting proper agile practices could help reduce and
track technical debt.

[T3] Paying Down TD in Agile Sprints. As mentioned by many practitioners
talking about managing TD with team collaboration, they consider ”paying back”
TD with continuous and agile practice is the right way. Therein, agile sprint
planning is important in facilitating the management and reduction of TD by
prioritizing, tracking and addressing them using project backlog. Furthermore,
the sub-topics are as follows.

  * 1.

[T3.1] Allocating Resources for TD in Agile Sprints: In general, many
practitioners have mentioned that the allocation of time and resources of the
project team is important when managing TD in agile sprints. For such a
purpose, sprint planning meeting is an opportunity to decide how to use your
allocated time where the incoming sprint assignment towards solving TD can be
decided collectively. By doing so, a clear allocation of effort for
refactoring, fixing bugs, and improving code quality can be expected with the
progression of well-planned sprints.

  * 2.

[T3.2] Prioritizing TD in Sprint Planning:  When planning the sprints that
integrate TD management activities, it is also important to prioritize the
tasks together with the non-TD-related ones properly. Practitioners value
activities that measure debt continuously in order to control the total cost
of ownership of the application life-cycle and include debt measurement in
project management and prioritization. Meanwhile, one critical prerequisite
towards efficient prioritization is the understanding of TD by stakeholders.

  * 3.

[T3.3] Addressing TD in Sprint Backlogs: When the preparation towards solving
TD is done, including resource allocation and prioritization mentioned above,
TD can be addressed continuously in sprint backlogs, where these tasks should
also be properly tracked and addressed alongside new features and bug fixes.
The team should make sure TD is continuously addressed when it is also
possible to devote an entire sprint to addressing a large amount of technical
debt.

  * 4.

[T3.4] Daily Focus on TD in Agile Teams: From the project mentality
perspective, it is also a good practice to have TD considered in the daily
focus within agile teams. Practitioners mention that the difference between
“eventually repaid” and “sustainable” is the amount of interest you pay on a
daily basis while making changes to the code. And practitioners acknowledge
that it is important that we have to really consider eradicating technical
debt as part of daily work.

  * 5.

[T3.5] Prioritizing TD in Sprint Backlogs: Prioritization of TD-related tasks
in sprint backlog is also mentioned by many. If the debt can be understood by
the owner and prioritized in relation to other system requirements, then it
may reasonably be managed on the product backlog. This means assessing each
item of TD, understanding its impact on the product and the user, and
allocating appropriate resources to address it on time. By consistently
including TD in the sprint backlog with elaborate estimation and
prioritization, teams can ensure that it is paid down regularly.

[T4] Understanding the Business Impact of TD. Another key topic discussed by
the practitioners is the business impact of TD and its understanding. The
unchecked TD, when accumulated, can inevitably impact the development
efficiency, which can eventually influence the software product quality and
customer satisfaction. Furthermore, the sub-topics in this aspect are as
follows.

  * 1.

[T4.1] The Risk Impact of TD for Company and Organization: It is commonly
acknowledged that TD will hurt the business of companies in the long run. With
too much debt, you run a big risk of having to spend months rewriting your
code, which is the worst situation to be in and is commonly the first step to
eventually running out of money and shutting down the company. Furthermore, at
the company level, unchecked TD can result in the company’s suffering costs
from fixing the problem, providing credit monitoring services, reputational
risk, or even facing regulatory action. Individuals are the ones fighting for
months to repair their identity.

  * 2.

[T4.2] The Cost of TD on Business Development: Reducing TD also means limiting
its potential cost on the business level; however, the organization has little
idea of how much technical debt it ’carries’ in its code and paying tech debt
is notoriously difficult to make visible to those setting business level
priorities. Meanwhile, many business managers can only see the value of the
feature and not appreciate (unless you find ways to explain) the technical
debt build-up, which will make everything cost more and take longer. To track
the cost of TD, Stepsize888https://www.stepsize.com/technical-debt is a tool
recommended by the practitioners to help capture and track TD from their
workflow so they can quantify its cost to the business and ultimately
prioritize the most important debt.

  * 3.

[T4.3] TD Impact on Team Morale and Productivity: Regarding the impact of TD,
team morale and productivity is another critical concern of practitioners. It
is commonly acknowledged that leaving technical debt unaddressed can result in
many problems in your organization, such as low team morale and tech debt will
impact every corner of your business, from team morale to customer
satisfaction. Certainly, on the contrary, implementing a process for managing
technical debt will positively influence engineering team morale and customer
satisfaction.

  * 4.

[T4.4] Evaluating the Business Risks of TD: It is critical to assess the risk
of TD on the business level together with the estimation in development
practices, as unchecked TD can pose significant risks to the organization as a
whole. Commonly, as mentioned by the practitioners, the (main) risk (of TD) is
that you will end up with an inferior product that also has more support and
maintenance costs. Some propose a potential good practice is to build a debt
limit, which is essentially a measurement of risk and ability to make timely
“payments.” Meanwhile, another potential risk-reducing practice is quick
feedback means a lot less risk than blocking feature development to slowly
refactor the code in isolation.

[T5] Measuring TD: Tools and Metrics. Tools and metrics that are adapted to
measure TD properly are critical to the management of TD. The tools and
metrics can quantify TD in a codebase, helping teams to track better, manage,
and address code quality issues. Furthermore, the sub-topics in this aspect
are as follows.

  * 1.

[T5.1] Tracking TD with Code Editors and Tools: TD-tracing tools and plugins
within code editors can largely facilitate the practitioners in terms of
efficiency and code quality assurance. Code editors, e.g., VBSCode and
JetBrains, offer extensions that can directly manage and track TD. For
example, Stepsize is one of the tools for such purpose, and it’ll allow you to
quickly report technical issues in your code without leaving your editor (see
the VSCode or JetBrain …). Such tools can provide various useful features
besides TD tracking, e.g., link issues, prioritize workflow, and generate
reports, etc.

  * 2.

[T5.2] Measuring TD with SonarQube: Types of Tools for Addressing Technical
Debt Tools such as Stepsize, Sonarqube, and Klockwork are used to manage
technical debt., as mentioned by practitioners. SonarQube (or SonarCloud) is
one of the most popular tools that practitioners use to measure TD. However,
practitioners also share their experiences (positive or negative) about
SonarQube. For example, In Sonarqube versions prior to 5.5 there was the
possibility to change the way that technical debt is calculated in order to
take into account the complexity, but after 5.5 I can’t see how to change it.

  * 3.

[T5.3] Using Metrics to Measure and Track TD: It is critical for the software
project team to understand the metrics to measure TD in order to manage it. As
mentioned by practitioners, … engineering teams need a clear understanding of
the strategy and metrics for technical debt. The often-mentioned metrics used
for TD tracking and management include the number of bugs, code quality
scores, and impact on the maintenance cost, etc.

  * 4.

[T5.4] Analyzing TD Levels and Violations: When measuring and tracking TD
within the project, it is common to classify TD with priority and severity,
where ”level” is a term adopted correspondingly. Meanwhile, it is also true
when practitioners have concern that it’s hard to calculate the exact level of
Technical debt, you need to know the seriousness of the situation once it gets
unsolvable. On the other hand, the number of violations in terms of standards
or defined rules can be considered useful variables for TD measurement.

  * 5.

[T5.5] Refactoring and Managing TD with SonarQube: SonarQube is a popular tool
adopted commonly by practitioners for measuring and managing TD. SonarQube can
provide features that support efficient refactoring to reduce the identified
TD by suggesting automatic refactoring for code violations.

[T6] TD in Product Development: A Necessary Evil? Debt in a broader sense, …
technical or otherwise, is a necessary dimension of all business and product
development practice. Practitioners also discuss TD-related strategy and
practice on the level of product development, where managing and ”paying down”
TD comes together with maintaining a product focus. Furthermore, the sub-
topics in this aspect are as follows.

  * 1.

[T6.1] Balancing Quality and Speed in Product Development: balancing quality
and development speed is crucial when managing product development, especially
when teams prioritize rapid delivery over quality. Many practitioners discuss
the common phenomenon where at the very beginning of a project, developers
tend to favor speed over quality, and it works… soon it becomes riskier,
harder, and more time-consuming to add features. It is a common claim from the
practitioners that they want to grow their product while maintaining speed and
quality and, therefore, low technical debt. And product managers, engineers,
and product owners must balance this with the need for quality and long-term
value.

  * 2.

[T6.2] The Trade-Offs of TD in Fast-Paced Markets: In fast-paced business
environments, product teams often face the dilemma of delivering new features
quickly to meet market demands or focusing on quality and incurring TD. It is
common when the pressure to speed up the time to market has intensified how
businesses deal with the risk of malfunctions, bugs, and overall defects
throughout the customer experience. Teams need to understand the implications
of these choices and strive to find a balance that maintains quality without
significantly slowing down development.

  * 3.

[T6.3] TD from a Development Perspective: From the development team’s
perspective, the balancing between product quality and fast delivery often
occurs, which likely leads to many shortcuts taken and unattended. Critical
decisions on taking shortcuts or conducting refactoring can

  * 4.

[T6.4] Managing TD in Product Development Teams: In product development, TD is
a concern that requires attention from various stakeholders, e.g., data and
security engineers, managers, and etc. Senior architects and project managers
play a crucial role in understanding the implications of TD and guiding the
team toward sustainable development practices. The balancing mentioned in the
previous sub-topic is the practices executed often based on the collective
expertise.

[T7] Balancing New Features and TD in Software Development. New feature
development is a critical part of software evolution and can provide potential
new attractive designs or functions to the end users, which can maintain and
enhance their satisfaction to the product. Therefore, managing TD requires the
project teams to balance the need for adding new features and that for
addressing TD. Such practice also requires strategic decisions about where to
allocate time and resources. Furthermore, the sub-topics in this aspect are as
follows.

  * 1.

[T7.1] Allocating Resources for New Features vs. TD Reduction: As
practitioners are aware that the problem is that every new feature adds debt
or creates a placeholder for future debt. Especially when the resources are
limited, the team should be careful allocating time and effort to balance the
new feature development and TD reduction. Inevitably, without any means of
keeping on top of that debt, you can be sure that sometime in the future, you
are going to be working on integrating a new feature and be continually
blocked at every turn by brittle/inflexible technical decisions made in the
past that now would require a Herculean effort to fix before even getting to
your feature.

  * 2.

[T7.2] Managing Feature Development and TD in Agile Projects: The challenge of
balancing the addition of new features with the need to reduce the existing TD
and proactively preventing new TD exist commonly in agile projects. It is
common to face situation like …You are supposed to deliver a new feature by
the end of the sprint; … And sometime during the sprint, the development team
realizes that if we want to implement it the correct way - ensuring
engineering best practices, it will overspill and is not feasible to complete
in the earlier planned timelines. Such situation should be properly managed.

  * 3.

[T7.3] Strategic Prioritization of Bug Fixes and Feature Development:
Effective management of TD involves prioritizing maintenance work, e.g., bug
fixes, and code improvements while also working on new features. The project
teams need to assess the long-term impact of TD and allocate sufficient time
to address it alongside ongoing development tasks, where explicit
prioritization is required the most.

  * 4.

[T7.4] Integrating TD Management into Feature Development: It is commonly
acknowledged inevitably every new feature adds debt or creates a placeholder
for future debt, which suggests that it could be a good practice that TD
management proactively integrated when any new feature is development.

  * 5.

[T7.5] Continuous Improvement of Code Quality Amidst Feature Expansion:
Maintaining a balance between feature development and technical debt reduction
is crucial for code quality. It is commonly acknowledged that there is
undoubtedly a trade-off between feature work and the work developers know is
crucial for long-term code quality and stability. Teams must focus on fixing
bugs, conducting thorough testing, and implementing robust development
practices to ensure long-term sustainability and functionality of the
software.

[T8] Code Quality and TD: The Role of Testing. It is also commonly mentioned
by the practitioners that the connection between code quality, TD reduction
and testing in software projects. The potential best practices, automation and
review process to reduce TD are considered important. Furthermore, the sub-
topics in this aspect are as follows.

  * 1.

[T8.1] Data and Testing in Managing TD: Regarding machine learning and data-
driven application, managing TD requires also closer monitoring the data
quality, model performance and so on. Regular testing and improvement of data
pipelines are also crucial to address TD, especially when changes in data
models can significantly impact feature performance and overall cost.
Considering the situation where in order to improve the product, it’d be
useful to consider performance optimization — to decrease response time under
high load and to increase the maximum number of users that could work
simultaneously without bugs, it is also important to consider the management
of TD therein.

  * 2.

[T8.2] Automated Testing in Reducing TD: Automated testing plays a vital role
in managing TD, particularly in development environments with continuous
integration and deployment. The low stakes environment in software is
automated testing before deployment. A unit or integration test is a very
basic yet powerful way for probing and sensing. Good architecture enables such
tests. By automating tests for new code and features, teams can quickly
identify and address bugs, ensuring that the development process does not add
to the TD. This approach is crucial for maintaining product quality and
reducing manual testing efforts.

  * 3.

[T8.3] Testing as a Tool for Quality Assurance in Software Development:
Testing is commonly considered an essential practice for maintaining code
quality when it is also for minimizing TD. Testing can facilitate the practice
of reducing the long-term cost associated with TD. As mentioned by
practitioners, Having .. changes in the code, we should manage the testing
side properly because we have to ensure quality and not cause any kind of
regressions. Furthermore, it is also acknowledged that as technology is
getting more advance, it is now possible to test the quality of code using
automated software tools.

  * 4.

[T8.4] Testing and Refactoring in Microservice Architectures: There are
practitioners discussing the testing and refactoring regarding microservice-
based systems in terms of managing the TD therein. Similarly, in the context
of microservices, testing is also the key to managing TD, particularly when
changes in code and architecture are frequent. After all, if the development
process and integration is properly structured, and the pooled microservices
performs as expected, the overall TD risk of all layers of microservice
structured debt improve. Effective testing practice can also support
refactoring process towards maintaining a clean, efficient and scalable
codebase.

  * 5.

[T8.5] Improving Code Quality through Testing and Review: Towards code quality
enhancement, testing and code review are also critical in terms of TD
identification and management. Both activities can help the development team
to monitor code performance, identify security issues, and improve overall
software quality. Using an automated code review tool can help change the
perception of code quality — especially for beginning developers, who are not
familiar with the concept of TD and necessary actions. Proactively managing TD
and maintain high-quality standards are important and can be achieved through
testing and code reviewing.

  * 6.

[T8.6] The Impact of Testing on TD in Software Architecture: Specially for
software products with complex architecture, lack of testing can lead to an
increase in TD or specifically architecture debt. Good testing practices,
including design and documentation reviews, are essential for identifying
architectural flaws and high-complexity areas in the code. These practices
help in making informed changes and improvements, reducing the long-term TD
associated with poor architectural decisions.

###  3.6 Step 6: Opinion Mining.

Using the VADER method [25], we can assess the sentiment of each informative
sentence and the overall sentiment of each topic. To be noted, herein, we
consider the percentage of positive, neutral, and negative sentences without
considering the according sentiment strength. To determine the benefits and
issues of each main topic, we compared each set of sentiment percentages to
the average sentiment percentage of all sentences.

Figure 4: Topic Sentiment Summary

The proportions of positive, neutral, and negative sentences for the main
topics are shown in Figure 4. We can observe that in topics Managing TD (T2)
and Understanding the Business Impact of TD (T4), the percentage of negative
opinions (60.5% and 50.6%) are overwhelmingly more than the positive.
Meanwhile, topics like The True Cost of TD (T1), Measuring TD (T5), and
Dealing with TD in Agile Development (T7) also have considerably more negative
opinions than positive ones. On the other hand, for the other three topics,
the situation is reversed. Especially for the topic TD in Product Development:
A Necessary Evil? (T6), the positive opinions are dominating.

(a) Sub-Topics Sentiments for Topic 1

(b) Sub-Topics Sentiments for Topic 2

(c) Sub-Topics Sentiments for Topic 3

(d) Sub-Topics Sentiments for Topic 4

(e) Sub-Topics Sentiments for Topic 5

(f) Sub-Topics Sentiments for Topic 6

(g) Sub-Topics Sentiments for Topic 7

(h) Sub-Topics Sentiments for Topic 8

Figure 5: Sentiment Distribution for the Sub-Topics

Furthermore, we also summarize the sentiment of the extracted sub-topics of
each main topic, shown in Figure 5. By comparing the sentiment percentage
between each sub-topic and that of its main topic, we determined each sub-
topic for each main topic to be either a benefit, an issue, or a neutral
opinion according to the following criteria. These criteria were also adopted
by Janes et al. [17].

  * 1.

If the percentage of positive sentences is higher than average and the
percentage of negative ones lower than average, the sub-topic is considered as
a benefit of TD or a positive impact of handling TD.

  * 2.

If the percentage of positive sentences is lower than average and the
percentage of negative ones higher than average, the topic is considered an
issue of TD.

  * 3.

For any other circumstances, the topic is considered disputed.

Table 2: The Positive and Negative Opinions in Sub-topics Topic |  Sub-Topics
---|---
[T1] The True Cost of TD |  [T1.1] The Long-Term Consequences of Shortcuts in Software Design; [T1.2] Choosing Quick Solutions and Incurring TD; [T1.3] Balancing Speed and Sustainability in TD Payment; [T1.4] Financial Analogies in TD; [T1.5] High Cost of TD in Financial Terms; [T1.6] Short-Term Gains vs. Long-Term Costs in TD; [T1.7] The Lifecycle of TD in Software Development;
[T2] Managing TD |  [T2.1] Solving TD through Team Collaboration; [T2.2] Tools and Strategies for Reducing TD; [T2.3] Understanding and Managing TD as a Team; [T2.4] The Role in Team for Perception in TD; [T2.5] Team Involvement in Decision-Making on TD; [T2.6] The Role of Engineers in Managing TD; [T2.7] Managing TD in Agile Teams;
[T3] Paying Down TD in Agile Sprints |  [T3.1] Allocating Resources for TD in Agile Sprints; [T3.2] Prioritizing TD in Sprint Planning; [T3.3] Addressing TD in Sprint Backlogs; [T3.4] Daily Focus on TD in Agile Teams; [T3.5] Prioritizing TD in Sprint Backlogs;
[T4] Understanding the Business Impact of TD |  [T4.1] The Risk Impact of TD for Company and Organization; [T4.2] The Cost of TD on Business Development; [T4.3] TD Impact on Team Morale and Productivity; [T4.4] Evaluating the Business Risks of TD;
[T5] Measuring TD: Tools and Metrics |  [T5.1] Tracking TD with Code Editors and Tools; [T5.2] Measuring TD with SonarQube; [T5.3] Using Metrics to Measure and Track TD; [T5.4] Analyzing TD Levels and Violations; [T5.5] Refactoring and Managing TD with SonarQube;
[T6] TD in Product Development |  [T6.1] Balancing Quality and Speed in Product Development; [T6.2] The Trade-Offs of TD in Fast-Paced Markets; [T6.3] TD from a Development Perspective; [T6.4] Managing TD in Product Development Teams;
[T7] Balancing New Features and TD in Software Development |  [T7.1] Allocating Resources for New Features vs. TD Reduction; [T7.2] Managing Feature Development and TD in Agile Projects:; [T7.3] Strategic Prioritization of Bug Fixes and Feature Development; [T7.4] Integrating TD Management into Feature Development; [T7.5] Continuous Improvement of Code Quality Amidst Feature Expansion;
[T8] Code Quality and TD |  [T8.1] Data Metrics and Testing in Managing TD; [T8.2] Automated Testing in Reducing TD; [T8.3] Testing as a Tool for Quality Assurance in Software Development; [T8.4] Testing and Refactoring in Microservice Architectures; [T8.5] Improving Code Quality through Testing and Review; [T8.6] The Impact of Testing on TD in Software Architecture;

Based on the criteria given above, we can identify the positive and negative
sub-topics within each main topic theme. Shown in Table 5, the positive sub-
topics (i.e., benefits) are marked in green color, while negative ones (i.e.,
issues) are marked red.

Regarding the topic [T1] The true cost of TD: Balancing Short-term Gain with
Long-term Pain, it is inevitable that practitioners have overall negative
opinions about the long-term consequences caused by TD (T1.1) in a landslide.
Meanwhile, the sentiment is overwhelmingly positive when discussing TD as a
quick solution (T1.2). To a certain extent, it approves that the positive
perspective of TD as a quick solution is commonly recognized. Furthermore,
when discussing the balance between the speed for paying back TD and the
sustainability between paying back and quick solutions (T1.3) and using the
financial analogies as the metaphor (T1.4), the collective opinion is
surprisingly negative. On the other hand, surprisingly, practitioners have
positive opinions on the high cost of TD in financial terms (T1.5). Similarly,
the comparison between the short-term gain and long-term cost in TD (T1.6) and
the relevant discussion is also positively received. Though it does not
necessarily indicate that the argument on the short-term benefits has more
ground to stand on than its counterpart, it certainly shows the positive side
of TD as the practitioners collectively value the short-term gains. However,
the practitioners are concerned about the negative effects of TD and its
lifecycle in software development (T1.7).

On the topic [T2] Managing TD: A Team-Centric Approach, practitioners have
overall positive opinions on solving TD through team collaboration (T2.1),
tools and strategies for reducing TD (T2.2) and understanding and managing TD
as a team (T2.3). It is interesting to notice that team collaboration is one
of the positive ways to solve TD and manage its effect. Adopting proper tools
and strategies shall also contribute to the reduction of TD within the TD
management practices. Furthermore, enhancing the collective team’s
understanding of TD is another perspective on which practitioners have
positive opinions. On the other hand, practitioners have negative opinions
regarding the teams’ roles when perceiving TD (T2.4). The unclear definition
of roles related to TD management could cause issues.

[T3] Paying down TD in Agile Sprints is also one of the main topics discussed
heavily, as it is the corresponding metaphor for solving TD. Therein,
allocating resources for TD in agile sprints (T3.1) as an important practice
for paying back TD is positively received. So it is with the adoption of daily
focus on TD in agile teams (T3.4) and TD prioritization in sprint backlog
(T3.5). These ”keys to success” for paying down TD in agile sprints are
commonly recommended by practitioners and likely discussed positively.
However, addressing TD in sprint backlogs (T3.3) is usually discussed as an
issue that most likely results in negative opinions. Regarding prioritizing TD
in sprint planning (T3.2), the opinions are disputed. The practitioners will
likely present the steps and actionable points in this matter more neutrally.

Regarding [T4] Understanding the Business Impact of TD, the two negative
perspectives therein are the risk assessment of TD in terms of business (T4.1)
and the impact of TD on team morale and productivity (T4.3). It is reasonable
to consider that when talking about potential risks caused by TD, the tone of
practitioners is mostly negative. Since TD can negatively influence team
morale and productivity [10], the opinion on this subtopic is also reasonably
negative. Surprisingly, practitioners’ opinions are mostly positive when
talking about the cost of TD on business development (T4.2). The reason is
likely that practitioners collectively mention the cost of TD is a positive
sign in terms of business impact. Meanwhile, evaluating the business risks of
TD (T4.4) is neutral and disputed, as approximately half of the content is on
listing risks and the other half on proposing how to mitigate them.

Regarding [T5] Measuring TD: Tools and Metrics, tracking TD with code editors
and tools (T5.1) and using metrics to measure and track TD (T5.3) are both
positively received by practitioners. These two topics include adopting tools
and metrics to evaluate and measure TD, which can help solve its negative
effects. Therefore, it is reasonable that such topics are positive. On the
other hand, measuring TD with SonarQube (T5.2) is perceived negatively by
practitioners, which reflects that such a function in SonarQube can be
problematic. Furthermore, two other sub-topics on analyzing TD levels and
violations (T5.4) and refactoring and managing TD with SonarQube (T5.5) are
also negative. It is worth noticing that both features of TD measuring and
refactoring with SonarQube are perceived negatively. Meanwhile, it is also
reasonable that the discussion on TD violation has mostly negative tones.

[T6] TD in Product Development: A Necessary Evil? has four different sub-
topics. Therein, practitioners are somewhat pessimistic regarding the balance
between quality and speed in product development (T6.1), as this sub-topic is
perceived negatively. Meanwhile, the general notion of TD from a development
perspective (T6.3) is also negative. However, considering the trade-off of TD
in fast-paced markets (T6.2), practitioners are positive, which shows that the
benefit of introducing TD for the short-term gain targeting the fast-paced
market is well recognized as a good practice. Furthermore, the general notion
of managing TD in product development teams is also positive.

Regarding the topic [T7] Balancing New Features and TD in Software
Development, the only negative sub-topic is regarding handling TD in
continuous software development (T7.1). It reflects that the practitioners
consider such a practice insufficiently considered therein. On the other hand,
the other sub-topics, e.g., balancing perfection and practicality in agile
development (T7.1), managing TD with feature development (T7.2), prioritizing
fixes to manage TD (T7.3), and integrating new features while managing TD
(T7.4) are all received positively by the practitioners, which reflects these
are the practices positively recommended.

Finally, regarding the topic [T8] Code Quality and TD, both data metrics and
testing in managing TD (T8.1) and the impact of testing on TD in software
architecture (T8.6) are received negatively. It reflects that these practices
are poorly conducted in managing TD when code quality can suffer. Meanwhile,
the other sub-topics on testing practices on TD, e.g., testing as a tool for
quality assurance in software development (T8.3), testing and refactoring in
microservice architectures (T8.4), improving code quality through testing and
review (T8.5), are received positively. It reflects that testing, reviewing,
and refactoring are the recommended practices to enhance code quality.
Automated testing in reducing TD (T8.2) received disputed opinions.

##  4 Discussion

Our work adopted an NLP-oriented approach to investigate the practitioners’
collective opinions on TD. To this end, we collected articles and posts from
three majorly popular technical forums and social media platforms (i.e.,
Medium, Stack Overflow, and DZone) for technical discussions. Using GSDMM
topic modeling on properly preprocessed textual data, we extracted eight main
topics about TD that practitioners are discussing.

1\. Common aspects of TD (RQ1). The true cost of TD, team-centric management
approaches, paying down TD in agile sprints, understanding its business
impact, tools and metrics for measuring TD, its role in product development,
dealing with TD in agile development, and the role of testing in code quality.

Moreover, by further modeling the topics (using GSDMM) within the set of
textual data associated with each main topic, we obtained the set of sub-
topics for each, and by using the VADER sentiment analysis method, we can
summarize if any of these sub-topics are positive or negatively perceived by a
collective of practitioners.

Table 2 presents the potential benefits (i.e., the positive sub-topics) in
terms of each main topic and the potential issues (i.e., the negative sub-
topics) for each topic marked respectively in green and red color. Such
results answer respectively RQ2 and RQ3.

In particular, on RQ2 (negative sub-topics) and referring to Table 2, several
practitioners reported feeling frustrated by some recurring issues in managing
TD. Of those, the most important are unclear roles and a lack of involvement
of development teams in TD management. In this respect, when the roles are not
well-defined or, more generally, developers are not actively involved in any
decision related to TD. In most cases, this gives birth to misunderstandings
and ineffective TD management strategies.

Another critical complaint is the overemphasizing of business risk at the
expense of technical. Suppose the management of TD mainly pays attention to
reducing business risks and does not show much interest in technical debts. In
that case, this may demotivate the development teams for the long-term health
of the code.

Furthermore, new features have always been pushed without adequate regard for
the issues about existing TD compounds built up over time. This practice slows
future development cycles and increases maintenance efforts since a code base
will grow in complexity and fragility if more TD is added.

Inadequate tools to track and measure TD further highlight this issue.
According to developers’ opinion, teams can’t keep the code quality and
prioritize the debt’s repayment if they just rely on SonarQube to track TD
metrics. Finally, balancing quality with velocity while paying less attention
to integrating agile practices in general results in too much TD accumulation
and deterioration of code quality.

2\. Common complains on TD (RQ2). Developers complain about recurring issues
with TD management and role definition. Overemphasis on business risk and
unbalance between pushing new features quality and quality. Inadequate tools
and pressure on handling TD under agile development.

Finally, concerning RQ3, the benefits associated with active managing and
paying down TD are more significant for software development teams—quick
delivery and fixes to the code base. Good TD management encourages
collaboration at a high level and common ground among the team members.
Therefore, all stakeholders, such as engineers, managers, and other key roles,
should be included in maintaining sustainable development practices that align
with the goal and quality objectives. Therefore, managing TD in the product
development team, instead of the commercial team or boards, allows for a more
concrete view of the TD status.

Moreover, handling TD regularly in Agile can keep development teams interested
and motivated. No issues are then stashed away, and the development runs at an
even, higher-producing speed with increased satisfaction within teams.

Finally, a more sustainable and productive software development life cycle can
be realized by balancing the need for speed and the ability to keep high code
quality with structured, tool-supported methods that allow fast and effective
code testing, review, and QAS processes.

3\. Benefits of paying TD (RQ3). Developers think that active TD management at
the product development team helps to ground the issues. Furthermore, tools
and comprehensive TD pay-off help automatically balance new features and code
quality while delivering in an agile scenario.

##  5 Threats to Validity

In this section, we discuss these threats and the strategies we adopted to
mitigate them, based on the standard checklist for validity threats proposed
in [26].

Construct validity. Construct validities are concerned with issues that to
what extent the object of study truly represents the theory behind the study
[26]. The RQs and the classification schema adopted might suffer from this
threat. To limit this threat, the authors reviewed independently and then
discussed collaboratively RQs and the related classification schema.

Internal Validity. The source selection approach adopted in this work is
described in Section 2. To enable the replicability of our work, we carefully
identified and reported bibliographic sources adopted to identify the peer-
review literature, search engines, adopted for the gray literature, search
strings as well as inclusion and exclusion criteria.

Possible issues in the selection process are related to the selection of
search terms that could have led to a non-complete set of results. To overcome
to mitigate this risk, we applied a broad search string. This was possible
because of the novelty of the topic. To overcome the limitation of the search
engines, we queried the academic literature from eight bibliographic sources,
while we included the gray literature from Google, Medium Search, Twitter
Search, and Reddit Search. Additionally, we applied a snowballing process to
include all the possible sources.

The application of inclusion and exclusion can be affected by researchers’
opinions and experiences. To mitigate this threat, all the sources were
evaluated by at least two authors independently.

External Validity. External validity is related to the generalizability of the
results of our multivocal literature review. In our study we map the
literature on Open Tracing Tools, considering both the academic and the gray
literature. However, we cannot claim to have screened all the possible
literature, since some documents might have not been properly indexed, or
possibly copyrighted or, even not freely available.

Conclusion validity. Conclusion validity is related to the reliability of the
conclusions drawn from the results [26]. To ensure the reliability of our
treatments, the terminology adopted in the schema has been reviewed by the
authors to avoid ambiguities. All primary sources were reviewed by at least
two authors to mitigate bias in data extraction and each disagreement was
resolved by consensus, involving the third author.

##  6 Related Work

TD is a metaphor that reflects technical compromises that ultimately
negatively affect source code maintenance; various previous studies have
investigated its impact [27, 2] and its evolution and management [24]. For
instance, Li et al. [24] surveys 94 studies to systematically map the debt
type, highlighting ambiguity in the broad definition of TD. Therefore, Ernst
et al. [27] investigated whether the technical debt metaphor goes beyond the
ambiguity found by Li et al. [24]. Earlier studies had dealt mainly with code
metrics and small developer surveys. Building on this, Ernst et al. [27]
surveyed 1,831 participants: software engineers and architects dealing with
long-term and software-intensive projects from three large organizations. They
conducted follow-up interviews with seven software engineers. While the
metaphor is practical for communication, the existing tools cannot handle all
its details. Based on their findings, the authors propose a technical debt
timeline to improve management and tooling strategies.

Similarly, recent studies survey the opinion and technical experience of
developers on the TD. For instance, Rocha et al. [28], present the results of
an online survey with 74 participants from the Brazilian software industry,
aiming to understand why technical debt is introduced, eliminated, and
managed, mainly at the code level. In particular, their survey goes beyond
explicit technical debt management and includes poor code introduced
unintentionally, which is then recognized as debt while the software evolves.
The main reasons for technical debt are overload in the workload, time
pressure, and pressure from management. However, participants also relate
technical debt to inexperience when judging other developers.

Ramač et al. [16] investigated TD by preparing and conducting a family of
surveys. Several questionnaires were launched within the framework of the
InsighTD initiative by a 12-country research team. The questionnaires were
replicated in entire national teams with industry practitioners. The authors
received 653 valid responses from six countries. The results show that 22% of
practitioners have theoretical knowledge about TD, while 47% have practical
experience. More senior practitioners working in organizations or teams with
more people are much more aware of TD management. The most important root
cause of TD is time pressure, whereas among its effects are time delays in
delivery, low maintainability, and rework.

Spínola et al. [14] extracted a list of statements about technical debt from
various online forums, blogs, and literature. Among these, 14 statements were
chosen and assessed through two surveys, to which 37 practitioners answered
the questionnaires. With both consensus and agreement ranking, it became
evident that TD is far from just labeling ”bad code” in software project
management—it is essential.

Our work differentiates from previous related work on surveying developer
opinions [27, 28, 16, 14] for the scope, the methodology, and size of the
surveyed opinions. More specifically, concerning Spínola et al. [14] we
analyzed 2,213 posts and online articles in the grey literature. Moreover, our
study employed two novel methodological choices for the related work: the use
of LLMs for data synthesis and the broader scope. We aimed to investigate,
collect, and classify developers’ opinions into major topics to provide a
comprehensive and concise view of possible new research opportunities and
practitioners’ suggestions for consolidating good practices while averting
non-optimal ones.

##  7 Conclusion

Our study has employed an LLM-focused methodology to investigate the
multivocal topic of TD in the grey literature survey as perceived by
practitioners in software development. We identified eight major topics about
TD from the discussion data, which included its actual cost, approaches to
managing TD, how it is combined with agile practices, its business impacts,
measurement tools, and its place in product development and code quality
matters.

Among the top and continuous TD management challenges that practitioners have
been grumbling about during all these years are unclear team roles and
insufficient developer engagement, thereby affecting proper decision-making
and strategy implementation.

On the other hand, an active management and pay-down of TD is not very cheap.
It facilitates collaboration among its members and major stakeholders toward a
collective approach to sustainable development practices.

Future research efforts should focus on AI-driven TD detection and management,
as well as an analysis of the effect of TD on current software development
methodologies such as DevOps.

## References

  * Allman [2012] E. Allman,  Managing technical debt,  Communications of the ACM 55 (2012) 50–55.
  * Lenarduzzi et al. [2019] V. Lenarduzzi, N. Saarimäki, D. Taibi,  The technical debt dataset,  in: Proceedings of the fifteenth international conference on predictive models and data analytics in software engineering, 2019, pp. 2–11.
  * Cunningham [1992] W. Cunningham,  The wycash portfolio management system,  ACM Sigplan Oops Messenger 4 (1992) 29–30.
  * Guo et al. [2016] Y. Guo, R. O. Spínola, C. Seaman,  Exploring the costs of technical debt management–a case study,  Empirical Software Engineering 21 (2016) 159–182.
  * Besker et al. [2018] T. Besker, A. Martini, R. E. Lokuge, K. Blincoe, J. Bosch,  Embracing technical debt, from a startup company perspective,  in: 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME), IEEE, 2018, pp. 415–425.
  * De Almeida et al. [2021] R. R. De Almeida, R. do Nascimento Ribeiro, C. Treude, U. Kulesza,  Business-driven technical debt prioritization: An industrial case study,  in: 2021 IEEE/ACM International Conference on Technical Debt (TechDebt), IEEE, 2021, pp. 74–83.
  * Seaman and Guo [2011] C. Seaman, Y. Guo,  Measuring and monitoring technical debt,  in: Advances in Computers, volume 82, Elsevier, 2011, pp. 25–46.
  * Bavota and Russo [2016] G. Bavota, B. Russo,  A large-scale empirical study on self-admitted technical debt,  in: Proceedings of the 13th international conference on mining software repositories, 2016, pp. 315–326.
  * Martini et al. [2015] A. Martini, J. Bosch, M. Chaudron,  Investigating architectural technical debt accumulation and refactoring over time: A multiple-case study,  Information and Software Technology 67 (2015) 237–253.
  * Besker et al. [2020] T. Besker, H. Ghanbari, A. Martini, J. Bosch,  The influence of technical debt on software developer morale,  Journal of Systems and Software 167 (2020) 110586.
  * Wohlin et al. [2012] C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, Experimentation in Software Engineering, Springer, 2012.
  * Fucci et al. [2021] G. Fucci, N. Cassee, F. Zampetti, N. Novielli, A. Serebrenik, M. Di Penta,  Waiting around or job half-done? sentiment in self-admitted technical debt,  in: 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR), IEEE, 2021, pp. 403–414.
  * Maldonado and Shihab [2015] E. d. S. Maldonado, E. Shihab,  Detecting and quantifying different types of self-admitted technical debt,  in: 2015 IEEE 7Th international workshop on managing technical debt (MTD), IEEE, 2015, pp. 9–15.
  * Spínola et al. [2013] R. O. Spínola, A. Vetrò, N. Zazworka, C. Seaman, F. Shull,  Investigating technical debt folklore: Shedding some light on technical debt opinion,  in: 2013 4th International Workshop on Managing Technical Debt (MTD), IEEE, 2013, pp. 1–7.
  * Kozanidis et al. [2022] N. Kozanidis, R. Verdecchia, E. Guzmán,  Asking about technical debt: Characteristics and automatic identification of technical debt questions on stack overflow,  in: Proceedings of the 16th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, 2022, pp. 45–56.
  * Ramač et al. [2022] R. Ramač, V. Mandić, N. Taušan, N. Rios, S. Freire, B. Pérez, C. Castellanos, D. Correal, A. Pacheco, G. Lopez, et al.,  Prevalence, common causes and effects of technical debt: Results from a family of surveys with the it industry,  Journal of Systems and Software 184 (2022) 111114.
  * Janes et al. [2023] A. Janes, X. Li, V. Lenarduzzi,  Open tracing tools: Overview and critical comparison,  Journal of Systems and Software (2023) 111793.
  * OpenAI [2023] OpenAI, ChatGPT, https://chat.openai.com/?model=gpt-4, accessed on November 22, 2023.
  * Yang et al. [2023] X. Yang, Y. Li, X. Zhang, H. Chen, W. Cheng,  Exploring the limits of chatgpt for query or aspect-based text summarization,  arXiv preprint arXiv:2302.08081 (2023).
  * Luo et al. [2023] Z. Luo, Q. Xie, S. Ananiadou,  Chatgpt as a factual inconsistency evaluator for abstractive text summarization,  arXiv preprint arXiv:2303.15621 (2023).
  * Nigam et al. [2000] K. Nigam, A. K. McCallum, S. Thrun, T. Mitchell,  Text classification from labeled and unlabeled documents using EM,  Machine learning 39 (2000) 103–134.
  * Yin and Wang [2014] J. Yin, J. Wang,  A dirichlet multinomial mixture model-based approach for short text clustering,  in: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, 2014, pp. 233–242.
  * Weisser et al. [2023] C. Weisser, C. Gerloff, A. Thielmann, A. Python, A. Reuter, T. Kneib, B. Säfken,  Pseudo-document simulation for comparing lda, gsdmm and gpm topic models on short and sparse text using twitter data,  Computational Statistics 38 (2023) 647–674.
  * Li et al. [2015] Z. Li, P. Avgeriou, P. Liang,  A systematic mapping study on technical debt and its management,  Journal of Systems and Software 101 (2015) 193–220.
  * Gilbert and Hutto [2014] C. Gilbert, E. Hutto,  Vader: A parsimonious rule-based model for sentiment analysis of social media text,  in: Eighth International Conference on Weblogs and Social Media (ICWSM-14). Available at (20/04/16) http://comp. social. gatech. edu/papers/icwsm14. vader. hutto. pdf, volume 81, 2014, p. 82.
  * Wohlin et al. [2012] C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, Experimentation in Software Engineering., 2012.
  * Ernst et al. [2015] N. A. Ernst, S. Bellomo, I. Ozkaya, R. L. Nord, I. Gorton,  Measure it? manage it? ignore it? software practitioners and technical debt,  in: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, 2015, pp. 50–60.
  * Rocha et al. [2017] J. C. Rocha, V. Zapalowski, I. Nunes,  Understanding technical debt at the code level from the perspective of software developers,  in: Proceedings of the XXXI Brazilian Symposium on Software Engineering, 2017, pp. 64–73.