11institutetext: Blekinge Institute of Technology, Karlskrona, Sweden
11email: rjb, nal<EMAIL_ADDRESS>

# A Value-driven Approach for Software Process Improvement - A Solution
Proposal

Ramtin Jabbari  Nauman bin Ali  Kai Petersen

###### Abstract

Software process improvement (SPI) is a means to an end, not an end in itself
(e.g., a goal is to achieve shorter time to market and not just compliance to
a process standard). Therefore, SPI initiatives ought to be streamlined to
meet the desired values for an organization.

Through a literature review, seven secondary studies aggregating maturity
models and assessment frameworks were identified. Furthermore, we identified
six proposals for building a new maturity model.

We analyzed the existing maturity models for (a) their purpose, structure,
guidelines, and (b) the degree to which they explicitly consider values and
benefits. Based on this analysis and utilizing the guidelines from the
proposals to build maturity models, we have introduced an approach for
developing a value-driven approach for SPI.

The proposal leveraged the benefits-dependency networks. We argue that our
approach enables the following key benefits: (a) as a value-driven approach,
it streamlines value-delivery and helps to avoid unnecessary process
interventions, (b) as a knowledge-repository, it helps to codify lessons
learned i.e. whether adopted practices lead to value realization, and (c) as
an internal process maturity assessment tool, it tracks the progress of
process realization, which is necessary to monitor progress towards the
intended values.

###### Keywords:

Maturity model, Software development process, Software process assessment and
improvement, Software development practice, Software value.

##  1 Introduction

Increasingly the ability to deliver high-quality software is seen as a source
for achieving and sustaining competitive advantage in the market. The role of
software process to develop high quality software is well recognized [1].
Therefore, several proposals to design, evaluate and improve software
development processes have been presented both from academia and industry.

Several prescriptive and inductive approaches for software process assessment
and improvement have been proposed [2]. The prescriptive approaches like
CMMI111Capability Maturity Model Integration: A framework containing a process
level improvement training and appraisal program that deals with the
integration of maturity models in software development process, initiated by
SEI, Software Engineering Institute [3]. and SPICE222Software Process
Improvement and Capability Determination: A set of standards for software
development process to assess capability level of the processes in accordance
with a reference model [4]. The SPICE standards present a descriptive model,
based on which the process of activities would be assessed against fixed
predefined levels of maturity [5]. tend to recommend a set of best practices
identified from several organizations. The assessment frameworks for these
approaches compare an organization’s compliance with the recommendations in
the standard. On the other hand, the inductive approaches like quality
improvement paradigm (QIP) have a measurement focus. They start with
understanding the current situation and then selecting, applying and
evaluating improvements that are perceived as most useful for the specific
organization.

The prescriptive approaches are criticized for their one-size-fits-all
solution that does not take into account an organization’s unique context. The
inductive approaches by taking the challenges and the goals of an organization
into account address this criticism. However, a problem in the inductive
approaches is the lack of support they provide to practitioners in assessing
their current process against the targeted values and in choosing which
improvement actions to implement and in what order.

The main aim of this paper is to propose a value-driven approach to SPI that
helps to overcome the limitations of inductive and deductive approaches. To
achieve the aim, we built on existing work for developing maturity models with
the following objectives:

  1. 1.

Specifying the generic structure of maturity models presented in the
literature in terms of their type, characteristics and specifications. From
this we gain insights of how to structure a value-driven approach by
understanding the limitations and strengths of the existing structures of
maturity models.

  2. 2.

Identifying the existing approaches, guidelines, and methods for designing a
new maturity model proposed in the literature. Knowing the existing design
approaches gives methodological guidance of how develop a new approach (in our
case a value-driven approach to SPI).

  3. 3.

Evaluating the existing approaches for achieving process maturity and the
process assessment from the perspective of value and benefits. Understanding
the evidence with respect to achieving value and benefits aids in identifying
gaps as well as best practices that can be transferred to our solution.

From a methodological point of view, we conducted a tertiary literature review
to construct our solution. As a proof-of-concept we presented a benefits-
dependency networks (BDN) for DevOps.

The remainder of the paper is structured as follows: Section 2 describes the
background of the study. Thereafter, we present the research questions and
method in Section 3. The results are presented in Section 4. Section 5
presents the results and our solution proposal (VASPI). Section 6 concludes
the paper and presents the directions for future research.

##  2 Background and related work

###  2.1 Maturity Models and SPI

In software engineering, maturity has been interpreted as the capability of
implementing, developing, measuring, and improving software processes [6, 7].
Inspired by Crosby’s quality management maturity grid [8] several maturity
models have been developed for organizations developing software [9, 10].

Broadly, there are two different types of frameworks for software process
assessment and improvement: inductive (descriptive) or prescriptive
(comparative) [6, 2, 11]. The first type, like quality improvement paradigm
(QIP), focuses on a specific organization through realizing the current status
and improving the identified challenges based on desired objectives. The
second type (like CMMI) provide a set of predefined best practices based on
successful experiences in other companies through making comparisons.

CMMI, a typical example of prescriptive frameworks, has been widely used to
assess the maturity and capability level of processes and organizations. It
has two main model representations [12, 13]: (1) a staged representation that
uses levels to assess the overall maturity of an organization, and (2) a
continuous representation that assesses the capability levels of an
organization in particular process areas.

The staged model, sets the requirements for implementing a number of practices
regardless of the nature and context of an organization. To address the
criticism that not all practices are relevant for all companies, CMMI also
suggests individual maturity levels for specific key process areas (KPAs) [9].
Therefore, using a continuous representation, a company can prioritize the
improvement related to a particular area alone. This provides companies some
flexibility compared to the staged model of organizational maturity.

Pikkarainen and Mantyniemi [14] have briefly explained how generic and
specific goals and practices are positioned to measure processes: “Generic
goals and practices apply to multiple process areas, whereas specific goals
and practices apply to individual process areas. The specific goals and
practices of process areas describe what kind of activities need to be carried
out. Generic goals and practices are aimed at finding out how well the
activities are performed”.

The critical decision for a software organization is to select the best
sequence of the ‘process areas’ (risk management, causal analysis, and
resolution) to fulfill the organizational needs and concerns. In addition to
extra workloads, activities, documentation, and changes, an unsuitable set of
the process areas can lead to negative results [15].

####  2.1.1 Assessment Frameworks for Agile Software Development

Lots of practices have been introduced and used as part of agile software
development methods, but the level of applicability of the practices and their
desired benefits has not been properly systematized. Lack of standards or
maturity models that specifically fit the agile practices could be a reason
[16].

To evaluate the applicability of agile practices, few methods and guidelines
have been introduced from different perspectives. McCaffery et al. [17]
assesses the agility of development practices, particularly for small/ medium
sized suppliers, by a combination of the flexibility of agile and traditional
maturity models like CMMI and SPICE.

Further, Hossain et al. [18] have proposed a conceptual framework of
strategies to select agile methods in the context of global software
development (GSD). The framework was built based on challenges reported in the
research literature on the use of agile methods in GSD.

Also, Soundararajan and Arthur [19] have presented a framework to assess the
adequacy, capability, and effectiveness of applying an agile method. The
framework works through “the identification of the agile objectives,
principles that support the achievement of those objectives, and practices
that reflect the spirit of those principles” [19]. To this end, the relations
between objectives and principles, and between principles and practices have
been defined.

The Objectives, Principles and Practices (OPP) framework [19] attempts to
address the gap of other existing ‘agile assessment methods’. OPP framework
assesses software and process artifacts by considering adequacy, capability,
and effectiveness of an agile method through specifying the links between
objective(s), principle(s), and corresponding practice(s).

###  2.2 Value-Based Software Engineering

Traditionally, software development has been done in a value-neutral manner
[20]. In reality, the decisions regarding software development require a
trade-off between competing value attributes [21]. The usefulness of pursuing
a value-driven approach has been recognized and it has lead to several
advancements in key knowledge areas of software engineering such as value-
based requirements engineering and value-based product management [22, 20].
However, to the best of our knowledge no study has investigate the use of a
value-based approach for SPI.

Khurum et al. [22] have presented an extensive catalogue of values for
software development (referred to as software value map (SVM)). They have
categorization the identified value aspects, sub-aspects, and value
components. The catalogue provides a common understanding of values and
presents the interrelationships between value perceptions from multiple
perspectives [22].

In this study, we suggest the use of SVM [22] as a means to identify values
that are important to consider from an SPI perspective. These values will
provide input to structure the knowledge related to process improvement using
a Benefits Dependency Network (BDN) [23].

While the inductive approaches for SPI support the assessment and realization
of the current state, but they do not help identifying improvement actions and
their order. On the other hand, prescriptive approaches recommend one path to
improve to pursue for all organizations [6, 11]. The proposed approach in this
paper has the potential to address these weaknesses. By structuring the
possible paths to the desired values using BDNs, any organization engaging in
SPI can choose, track and pursue their own path for improvement actions.

###  2.3 Benefits Dependency Network

Benefit dependency network (BDN) have been used to manage digital
transformation in organizations. It has been used to identify, manage and
deliver benefits from information technology investments [23]. Figure 1
presents a template of a BDN.

Figure 1: A template of a BDN from Peppard and Ward [23]

We draw inspiration from BDNs to structure process related knowledge and
streamline SPI initiatives. The visual map this approach provides gives the
ability to develop a business case for SPI initiatives, and justifying what
changes and improvements are required. The transparency will also help to
understand, track and manage the actual progress implementing the
improvements.

##  3 Research Method

In this section, we formulate the research questions and describe the method
to address the research questions by the use of a literature review.

###  3.1 Research Questions

Four research questions (RQs) have been formulated for this study, as follows:

RQ1. What is the generic structure of maturity models presented in the
literature?

RQ1 aims to identify the generic structure of maturity models, existing in the
literature, to better understand the content and their relations to
value/benefit-driven approaches. To address the RQ1, we have looked at
existing studies, which explicitly discuss the topic of interest.

RQ2. What are the approaches/guidelines/methods to design a new maturity model
introduced in the literature?

RQ2 aims to identify development processes presented in the literature to
create a new maturity model. To address the RQ2, we looked at studies, which
explicitly present a new or an enhancement of maturity model.

RQ3. To what degree do existing maturity models take value/ benefit into
account for assessment?

The purpose of RQ3 is to investigate those maturity models, in which relevant
values and benefits have been explicitly taken into account.

RQ4. How could a new maturity model be proposed for a value-driven SPI
approach?

RQ4 aims to propose a new approach for creating a model to assess the maturity
on the basis of values and benefits associated with practices in a specific
context.

###  3.2 Research Method

To identify existing guidelines and procedures to develop a maturity model, we
searched if secondary studies on the topic have already been conducted [24].
We searched in Scopus and Google-Scholar using the following general search
string:

(“capability model” OR“maturity model” AND (“systematic review” OR “research
review” OR “research synthesis” OR “research integration” OR “systematic
overview” OR “systematic research synthesis” OR “integrative research review”
OR “integrative review” OR “systematic literature review” OR “systematic
mapping” OR “systematic map”)

From this search, we identified seven secondary studies [6, 25, 26, 27, 28,
29, 30] that are discussed in Section 4.1.

We re-analysed the relevant literature identified in these secondary studies
to answer the research questions RQ1 – RQ4.

##  4 Results

###  4.1 Related Literature Review Studies

Table LABEL:tab:relatedwork presents the identified secondary studies we have
used for extracting relevant information.

Table 1: Related Systematic Studies Focus/Goal |  Type of Study |  #Papers |  Key Conclusion |  Ref.
---|---|---|---|---
Identifying the state of the art of maturity models (The maturity of maturity models). |  Systematic mapping study [24, 31] |  237 |  $\bullet$ F1. Maturity models mostly used in software development and engineering. $\bullet$ F2. The most studies are about the development of maturity models proposing a new or an enhancement of existing ones. $\bullet$ F3. Out of 128 studies related to the development of maturity models, 56 are conceptual and 42 are design-oriented. $\bullet$ F4. The number of empirical or validated studies is scarce. $\bullet$ F5. Those studies proposed new maturity models suffer from not reviewing existing models, not considering other available solutions, and not carefully examining their applicability and suitability in practice that negatively affect the quality and relevance of the new models. |  [25]
Classifying the existing maturity models. |  Systematic literature review [24] |  61 |  $\bullet$ F6. 52 software process capability/maturity models are identified. $\bullet$ F7. The identified models are classified in 29 specified domains like automotive systems, business process, etc. $\bullet$ F8. 96% of them (50 models) are developed based on CMMI and/or SPICE. |  [28]
Identifying the methods, practices, and activities used to propose or build maturity models. |  Systematic literature review |  7 |  $\bullet$ F9. Most of the new maturity models are based on CMMI, and SPICE. $\bullet$ F10. The majority studies are from the system information domain. $\bullet$ F11. Only one paper is about the development of maturity models in software engineering. $\bullet$ F12. The majority of identified models are based on practical experiences, success factors, and practices in industry or company. $\bullet$ F13. Lack of theoretical basis and distinct methodology. $\bullet$ F14. Few proposed models are evaluated to assess the validity, reliability, and generalizability. $\bullet$ F15. The general activities to build new maturity models are specified. |  [6]
Identifying the existing approaches used to develop maturity models and corresponding challenges during the development. |  Systematic literature review |  34 |  $\bullet$ F16. Five generic components identified to describe the structure of maturity models. $\bullet$ F17. Recent studies spent significant effort to standardize the development of maturity models through prescriptive guidelines, standardized vocabulary, and validated procedure. $\bullet$ F18. Three main approaches identified as the development processes of maturity model (Meta models). $\bullet$ F19. New maturity models suffer from lack of empirical validation, operationalising maturity measurement, and theoretical foundations. $\bullet$ F20. The challenges highlight potential gaps for further empirical research and validation on the topic. |  [27]
Investigating the current software process improvement (SPI) approaches using in small and medium firms (SMEs) |  Systematic literature review |  11 |  $\bullet$ F21. Most SMEs suffer from a lack of knowledge and resources to adopt proper SPIs. $\bullet$ F22. Although the majority of SPIs for SMEs are on the basis of CMMI, these standards are developed primarily for larger firms that do not fit the SMEs [29, 32]. $\bullet$ F23. Adopting the mentioned standards in SMEs leads to costly time and resource consumptions. $\bullet$ F24. Lack of generalizability for those SPIs proposed for a specific SME. $\bullet$ F25. There is a vital need for the development of SPIs in particular for SMEs. |  [29]
Studying the strengths and limitations of existing maturity models |  Literature review |  not specified |  $\bullet$ F26. There is no consistent definition and consensual perception of maturity models in the literature. $\bullet$ F27. Crosby’s Quality Management Maturity Grid is one of the first staged maturity models for quality assessment that could recognize the importance of human factors like leadership and collaboration. $\bullet$ F28. Bessant’s Continuous Improvement Capability model, as another staged model, is able to assess the general level of maturity and indicate how to conduct continuous improvement for enhancing the performance by providing a roadmap. $\bullet$ F29. CMMI is the most used staged model, share some similarities in terms of structure and content. $\bullet$ F30. Maturity models which are “step-by-step recipes” suffer from lack of theoretical foundation, empirical basis and reality, validity, and generalization such as Crosby’s Grid, Bessant, and CMMI. $\bullet$ F31. There is a large number of similar models based on CMMI without satisfactory documentation, thoughtful adoption, and economic basis. $\bullet$ F32. A shortage of guidelines for specific steps and activities towards the improvement of maturity levels (Bessant’s Model and CMMI). $\bullet$ F33. The majority of the models are costly and complicated to be applied and they are more suitable for large and bureaucratic organization (CMMI). $\bullet$ F34. CMMI mostly focuses on process and disregards the importance of people, culture, and organization. |  [30]
Understanding the status of Business Process Maturity Models (BPMM) |  Systematic literature review |  61 |  $\bullet$ F35. 20 BPMM identified, out of which 9 models are considered as the leading ones in terms of the rate of attention in the literature. $\bullet$ F36. Only 2 out of 9 models have been empirically studied and validated. $\bullet$ F37. Only 7 studies out of 61 reported the validation. $\bullet$ F38. A lack of validation negatively influences the acceptance level of the models by practitioners and potential audience. $\bullet$ F39. Only two leading models distinguish between maturity model (representing a path for improvement) and assessment model (specifying the current situation) that indicates a lack of consistency in terms of design, scope, and terminology. $\bullet$ F40. A lack of theoretical and empirical studies on existing maturity models. |  [26]

###  4.2 Values/Benefits versus Maturity

####  4.2.1 The Characteristics of Quality for Software Engineering Processes:

Kroeger et al. [33] conducted a study by interviewing 17 experienced software
engineers in addition to reviewing 30 peer-reviewed studies from SPI case
studies. They aimed to identify and characterize the quality criteria of
software engineering processes.

To this end, they have specified three main quality evaluation methods: (1)
Product and process measurement, (2) Software process assessment, and (3)
Process modeling and analysis.

The first category contains methods which indirectly assess the quality of
processes by measuring the developed systems and their outputs. The second
category is an evidence-based approach that compares the processes with a set
of standard best practices (like CMMI). The third determines the quality of
processes by the use of modeling and analysis such as petri-net based
formalism, visual process language, and unified modeling language that deals
with the processes in terms of a formal language. Then, through analysis
techniques, inconsistencies, ambiguities, and potential issues would be
identified before the process is deployed.

In fact, different approaches focus on different aspects of specific quality
attributes. For example, the categories (1) and (3) mostly focus on efficiency
and effectiveness, while the category (2) takes suitability and manageability
into consideration.

To characterize the quality of software processes, Kroeger et al. [33] have
listed the most frequently discussed quality attributes. In total, 15
attributes extracted from both interviews and literature: Effectiveness 88%
111The percentages are only related to the interviews. Though, effectiveness
was also the most frequently discussed one in the literature [33].,
Adaptability 76%, Compatibility 59%, Applicability 53%, Deployability 47%,
Efficiency 47%, Monitorability 41%, Understandability 41%, Controllability
35%, Learnability 29%, Predictability 29%, Analyzability 29%, Accessibility
24%, Modifiability 24%, and Supportability 24%.

###  4.3 Methods/Guidelines to Design Maturity Models

In this section, we attempt to identify studies that propose new maturity
models, present the steps taken to design them, and discuss the motivations or
problems to be addressed. Table 2 shows the identified literature used for
this study.

Table 2: The Studies Proposing Methods/Guidelines to Design Maturity Models

Goal | Steps | Motivation | Ref.
---|---|---|---
Developing a domain-specific maturity model | ✓ | ✓ | [34]
Building SPI in very small enterprises | ✓ | ✓ | [32]
Measuring the maturity of requirements engineering process | ✓ | ✓ | [35]
Developing a maturity model for SPI implementation through adapting CMMI and Beecham’s model.111A requirements process improvement model presented by Beecham et al. [39]. | ✓ | ✓ | [36]
Proposing a maturity model for the assessment and improvement of software maintenance activities. | ✓ | ✓ | [37]
Developing a roadmap for CollabMM222Collaboration Maturity Model (CollabMM) [38]. evolution through the lesson learned of model applications and literature review. | ✓ | ✓ | [38]

##  5 Discussions and Construction of VASPI

This section discusses the research questions based on the results. We also
present our solution proposal for a value-driven approach for software process
improvement (VASPI).

###  5.1 RQ1. What is the generic structure of maturity models presented in
the literature?

First, we reviewed the structure of maturity models presented in the
literature through identifying type, characteristics, and specification
(levels of maturity, descriptors, dimensions, and activities). The majority of
discussions on maturity models have been concentrated on the type staged
models like Crosby, Bessant, and CMMI [40, 41, 9, 10, 27, 30].

In fact, as a key conclusion of the reviewed papers (F8, F9, F22, and F31 in
Table LABEL:tab:relatedwork), the majority of identified models are based on
CMMI, and SPICE. The common characteristics identified are as follows:

  * •

Number of levels. (For example, CMMI has five levels of maturity.)

  * •

Descriptor for each level. (CMMI has five different descriptors: Initial,
Managed, Defined, Quantitatively managed, and Optimizing.)

  * •

Description of each descriptor.

  * •

Number of dimensions (For example, the process areas in CMMI).

  * •

Number of elements or activities for each dimension.

  * •

Description of element or activity performed at each maturity level.

Then, we looked at the structure of CMMI, as the most used staged maturity
model (See section 4.2.1). The existence of boundaries between CMMI process
areas makes it hard to fit, e.g., agile practices in which it is not possible
to separate the areas explicitly. The process areas also could not support
overlaps between activities and dependencies among the areas.

In fact, the staged models with fixed predefined levels of maturity suffer
from a lack of ability for expressing dependencies among processes within a
capability level, while flexible models could provide, e.g., more than five
maturity levels. Hence, flexible models allow the definition of intermediate
status that indicates further details of goals, practices, potential
improvements, and maturity [6, 34, 32]. Although, it brings a new challenge
for organizations to use CMMI (as an indicator for software development
process maturity) and agile practices to take benefits at the same time [17].

As a consequence, a tendency exists towards customizing maturity models or
proposing new models in accordance with specific domains, which means there is
no one model that fits all [28].

###  5.2 RQ2. What are the approaches/guidelines/methods to design a new
maturity model introduced in the literature?

From the literature, we found studies which propose new maturity models most
often suffer from avoiding existing models and solutions, a lack of
theoretical foundation, empirical evaluation, and validity (F5, F13, F14, F19,
F20, F36, F37, F38 and F40 in Table LABEL:tab:relatedwork). Therefore, before
proposing our solution, we attempted to identify the methods presented for
building a new maturity model by the literature. Although the identified
studies do not present detailed steps of the development, or they belong to
the specific context that is not possible to be generalized, we could specify
key steps of the development procedure. We aggregated generic activities
performed to create a new (or an enhancement of) maturity model. To this end,
we identified six studies compared and shown in Table 3. For the comparison,
six characteristics have been defined as follows:

  * •

Context. A context for which a model has been specifically proposed.

  * •

Origination. The basis on which a model has been developed. We have identified
four originations used in the literature to build maturity models: (1)
Practical experience (the most available models are developed based on
practical experience [30]). (2) Standards like CMMI and SPICE. (3) Knowledge
engineering. (4) Design science.

  * •

Source. It specifies an existing model, framework, or structure based on which
a model has been proposed.

  * •

Type. Two main types of maturity model distinguished in the literature: (1)
Staged (fixed models) which have predefined and distinct levels of maturity
(2) Flexible models which can recognize interdependencies among different
levels of maturity.

  * •

Meta-model [27]. Three main approaches identified as procedures for maturity
model development: a 6-phase, 8-step, and 5-step development process based on
the concept of maturity models [27].

  * •

Evaluation. We considered whether or not the proposed model has been evaluated
and validated.

Table 3: Comparison between studies of proposing a maturity model

# | Study |  Context |  Origination[6] |  Source | Type | Meta-model | Evaluation
---|---|---|---|---|---|---|---
1 | [34] |  Focus Area |  Practical experience & Design science |  not reported | Flexible | I111Meta-Model I [27] | Yes
2 | [32] |  Very small enterprises |  Standards |  CMMI, SPICE | Staged | $\bullet$ | Yes
3 | [35] |  Requirements Engineering |  Practical experience |  RMM222Requirements Maturity Model presented by Sommerville et al. [42] | not reported | $\bullet$ | Yes
4 | [36] |  SPI Implementation |  Practical experience & Standards |  CMMI, RPI333Requirements Process Improvement model presented by Beecham et al. [39] | Staged | $\bullet$ | Future planned
5 | [37] |  Maintenance |  Practical experience & Standards |  CMMI | Staged | $\bullet$ | Yes
6 | [38] |  Collaboration |  Practical experience & Standards |  CMMI, BPMM444Business Process Maturity Model, KMMM555Knowledge Management Maturity Model | Staged | I | Yes

To specify a generic procedure for developing maturity models, we have
aggregated the findings from the literature [32, 34, 35, 36, 37, 38]. We
considered the most common activities and corresponding methods to build
maturity models. Consequently, four main phases have been defined:

  * (1)

Motivation: Through evaluating specific contexts and/or existing models, the
motivation has been gained to build a new model.

    * –

Evaluate the context, for which a model would be specifically proposed through
different evaluation methods like survey, literature review, etc.

    * –

Evaluate existing model(s).

  * (2)

Definition:

    * –

Identify gaps.

    * –

Address the gaps.

  * (3)

Construction:

    * –

Propose new model.

    * –

Present an enhancement of existing model(s).

  * (4)

Evaluation:

    * –

Evaluation/Validation.

Figure 2 indicates the generic development procedure for maturity models
identified from the studies. Numbers of the activities show the references
presented in Table 3.

Figure 2: Common procedure of creating maturity models

###  5.3 RQ3. To what degree do existing maturity models take value/ benefit
into account for assessment?

To our knowledge, the number of studies about maturity and actual benefits is
scarce. Most existing discussions interpret the topic as general quality
aspects and attributes of the software. Kroeger et al. [33] stated that the
majority of existing methods used to assess quality attributes of software
process development have been limited to indirect measurement and evidence-
based models (CMMI and formal models). These methods do not consider actual
relations between practices and corresponding values, benefits, and quality
attributes.

Moreover, such approaches mostly focus on general software quality attributes
like efficiency, effectiveness, suitability, and manageability, which are
related to the whole software development process and not associated to every
single practice within the process.

Also, some maturity models like CollabMM proposed by Magdaleno et al. [38]
consider quality attributes and benefits as advantages of the model usage. For
example, it has been claimed that by applying CollabMM, time reduction, less
complexity, learning, satisfaction, and innovation would be achieved. But,
CollabMM does not specify the relations between practices and their benefits.

Some challenges also realized in relation to agile approach which are: A lack
of measures or assessment tools that practically fit agile practices [16] and
a lack of maturity models that consider the relations (dependencies) among
agile practices (for example, iteration review requires iterative development
to be in place [43]) and between the practices and values/benefits.

As mentioned in section 2.1.1, the OPP framework (Objectives, Principles, and
Practices) considers the relation between objectives, principles, and
practices. However, it does not take the relations/dependencies among
practices into account [19].

Figure 3 shows an example of OPP framework that indicates accommodate change
(as an agile principle) is linked to a number of practices. (Incremental
development, Frequent releases, Feature-driven, and Face-to-face
communication). As Figure 3 shows, relations among a group of practices are
missing (i.e., practices dependencies or a sequential order).

Moreover, the level of objectives and principles are too high that does not
consider actual values/benefits which are explicitly coming from a specific
practice. Hence, making decisions regarding applying particular practices
becomes hard and negatively affects the feasibility of the framework.

Figure 3: An example of OPP framework

Therefore, there is a lack of model/framework to assess the whole development
process containing a set of practices following the desired benefits. Also,
the majority of existing models are fixed model type and have rigid levels of
maturity. Consequently, they are ineffective to improve processes practically,
as they cannot express interdependencies among the processes for improving a
maturity level [44, 34]. On top of these challenges, these models are too
expensive and complicated to be implemented [32, 29].

To address the identified gaps of existing maturity models, we realized the
need for a new concept of maturity that meets the lack of actual relations
between values/benefits and practices and has a consolidated view over the
whole process of software development (See section 5.4.2).

###  5.4 RQ4. How could a new maturity model be proposed for a value-based
assessment approach?

This section presents our proposed model/framework as a guideline to design a
new maturity model for software engineering development.

####  5.4.1 Purpose:

We aimed to propose a set of guidelines to design a model/framework for
maturity assessment of applied software development practices based on our
literature review. This guideline considers the relations among practices
(i.e., dependencies among them) and corresponding goals/values/benefits.

####  5.4.2 Need of a new approach for maturity model:

To address the identified gaps come from existing maturity models, the
proposed guideline contributes through creating a value-driven model, as
follows:

  * •

To meet a lack of actual relations between goals/values/benefits and
practices.

Instead of general appraisal characteristics, it considers the relations
between specific practices and desired benefits.

  * •

To provide a holistic perspective of practices that includes
relations/dependencies among the practices and the link to the desired
goals/values/benefits.

  * •

To provide a maturity model/framework to do only what is needed to be done to
achieve the desired goal(s)/value(s)/benefit(s).

Instead of applying general processes of assessment, the proposed guideline in
this study aims to design a new maturity model/framework based on the actual
practices that need to be conducted in accordance with the desired benefits.

####  5.4.3 Development:

Two main approaches are presented by our guideline: Literature approach (L)
and in-Practice approach (P).

Figure 4 shows the guideline proposed in this study to design a new maturity
model. We describe the process of developing the guideline, as follows:

Figure 4: a Value-driven Approach to design a new maturity model, VASPI

####  5.4.4 Literature approach (L)

First, we follow the L approach in order to design a maturity model (L5) based
on findings would be extracted from literature (peer-reviewed literature, gray
literature, or technical reports). To create L5, the following steps have been
taken:

  * •

L1. Identify Value(s)/Benefit(s): According to the context in which a new
model would be designed, the desired values and benefits have to be identified
from relevant literature.

For example, to create a maturity model for a specific agile context e.g.,
software deployment, the desired values and benefits of deployment context
have to be identified.

Several methods exist to conduct a literature study in accordance with needs
and concerns. For example, in software engineering, the most well-known
methods for performing literature reviews are Systematic Literature Reviews
(that could be conducted based on the guidelines proposed by Kitchenham et al.
[24]) and Systematic Mapping Study (two samples of the guideline are [24] by
Kitchenham et al. and [31] by Petersen et al.). Garousi et al. [45] also
present a method, Multivocal Literature Review, that includes gray literature
as well as peer-reviewed literature.

Primary studies (original research), secondary (systematic reviews) and
tertiary literature (handbooks) [24], peer-reviewed literature
(scientific/academic peer-reviewed journals, and peer-reviewed conferences
articles), and nonpeer-reviewed ones like technical reports, and blogs could
be used as sources.

As an example, by the use of systematic literature review [24] and conducting
a study on ‘deployment’, the following benefits related to the deployment
might be identified: Cost saving, Fast and frequent release, Quick responses,
Increase productivity, Predictability, and Repeatability [46, 47, 48, 49]

Then, the identified benefits can be linked to the corresponding software
values to provide a common understanding of different value perspectives
(e.g., customer, business). To this end, Software Value Map (SVM) [22]
presented by Khurum et al. can be used. SVM provides an exhaustive view on
value perspectives (Financial, Customer, Internal Business Process, Innovation
and Learning) as a unified collection that indicates a common understanding to
ensure not missing any perspective [22]. Table 4 shows an example of SVM and a
value perspective and its value aspects, sub aspects and components. Also,
Table 5 shows an example of the identified benefits categorized by the use of
SVM.

Table 4: An example of SVM [22]

Value Perspective |  Value Aspect |  Sub-value Aspect |  Value Components
---|---|---|---
Customer |  Perceived value |  Intrinsic value |  functionality
reliability
usability
Delivery process value |  process w.r.t. time
Customer lifetime value |  Revenue |  upselling revenue

Table 5: An example of benefits categorized by SVM

Benefits (Bn) | Value perspective | Value component
---|---|---
B1. Fast & frequent releases B2. Cost saving B3. Quick responses |  Customer/ perceived value |  $\bullet$ functionality $\bullet$ reliability $\bullet$ usability
B4. Increase productivity |  Internal Business Process/ production value |  $\bullet$ market requirement value $\bullet$ physical value wrt. time
B5. Repeatability B6. Predictiability |  Innovation and learning/ value of technology |  $\bullet$ human capital value $\bullet$ customer capital value $\bullet$ market value size

  * •

L2. Identify Principles/Practices: This step attempts to identify principles
which exist to support the values of the context specified in L1. Also,
corresponding practices that promise the specified benefits would be
identified in L2.

The importance of identifying principles is to characterize the context and
knowing the fundamentals. Also, it assists to better understand values and
recognize practices, in which the principles have been applied [50].
Identifying the principles helps us to find relevant practices and realize
alternative practice sets.

  * •

L3. Identify Dependencies between Practices: A process of software development
contains a number of applied practices. Consequently, to identify the actual
practices that need to be in place to achieve the desired benefits, the
relations among practices and the alternatives should be identified.

For example, a number of practices can be identified such as continuous
deployment [47], automated deployment, and continuous integration [51] for
deployment. But, how these practices should be in place is missing. Figure 5
shows one alternative formation for the practices according to the identified
relations from the literature (R1 [47] and R2 [51]).

Figure 5: An example of identifying the relations between specified practices

  * •

L4. Specify Relations between Practices and Value(s)/Benefit(s): Before step
L4, values/benefits (L1), practices (L2), and relations among practices (L3)
are specified. Step L4 aims to realize the link of practices to the desired
values/benefits through the same procedure performed in L3 (extracting
relevant data from the literature).

By specifying the desired benefit(s) of a practice or a set of practices, it
is possible to trace the link from practice(s) to benefit(s) and vice versa to
pursue the desired goals and values.

One initial strategy to specify relations between practices and
values/benefits can be (1) identifying the expected benefit(s) of each
practice and then, (2) aggregating the identified benefits as an outcome from
a set of practices. Figure 6 shows an example of relations between practices
and benefits.

Figure 6: An example of relations between practices and benefits

For example, Fast and frequent releases (B1), Quick responses (B3), Increase
productivity (B4), Predictability (B6), and Communication/Collaboration have
been identified as the desired benefits of the practice Continuous integration
[47, 48, 49].

To provide a consolidated view of the whole approach and in order to create an
instance of Maturity Model derived from literature (L5), the complete findings
including values/benefits, practices and dependencies, and relations between
practices and benefits have to be formed together. Figure 7 shows a sample of
L5 created by the guideline of Literature approach in this study. It should be
mentioned that this is not an order in which the practices are executed but
rather the dependencies. A dependency indicates how practices have to be in
place to achieve the desired benefits/values.

Figure 7: An example of a maturity model derived from research literature (L5)

####  5.4.5 in-Practice approach (P)

We follow the in-Practice approach to design a maturity model (P5) based on
findings would be from empirical studies and methods such as workshops,
focused groups, and case studies. To create P5 the following steps have to be
taken:

  * •

P1. Set desired Value(s)/Benefit(s): According to the context in which a new
model would be designed, the desired values and benefits have to be set. The
desired values and benefits can come from business plans, organizational
goals, running workshops with stakeholders, etc.

  * •

P2. Identify Principles/Practices: This step attempts to identify principles
which exist to support the values of the context specified in P1. Also,
corresponding practices that promise the specified benefits would be
identified in P2.

P2 supports what has been identified in L2 and indicate what is missing. Gap
analysis, interviews, observations, workshops can be used as a means for data
collection.

  * •

P3. Identify actual Dependencies between Practices: After specifying the
practices, it is turn to position them based on their relations or
dependencies. It helps an organization, project, or product team to
concentrate on real concerns and actual activities according to the desired
goals without any disadvantage of traditional maturity models such as
overheads of cost, time, effort, additional documentations, and resources
[52].

  * •

P4. Specify actual Relations between Practices and Value(s)/Benefit(s): Before
P4, values/benefits (P1), practices (P2), and relations among practices (P3)
have been specified. Step P4 aims to realize the link of practices to the
desired values/benefits to create a practical maturity model (P5) in
accordance with actual case(s) (c1, c2, and c3 in Figure 4).

By specifying the desired benefit(s) of a specific practice or a set of
practices, it would be possible to trace the link from practice(s) to
benefit(s) and vice versa to pursue the desired goals and values.

####  5.4.6 Joint Maturity Model

By mapping relevant instances of L5 and P5 and making comparison between
similarities and differences (LP1) of the context, concerns, values,
practices, and relations, it is possible to create a joint sample of maturity
model.

LP2 aims to bring the state of the art and practice together to provide a
holistic view on development processes. Then, the joint model can be used in
process assessment activities in accordance with real concerns (LP3). Based on
the application of a joint model in a specific context, a corresponding joint
maturity model would be incrementally evolved through learning and feedback
(LP4).

##  6 Conclusion and Future Research

In this paper, we presented VASPI as a value-driven approach for software
process improvement. Through a literature review, we identified existing
approaches for maturity and process assessments. We have analyzed these
existing models, their purpose, structure, and guidelines. Moreover, we
considered the relations and dependencies between practices and their links to
the desired benefits. To this end, four research questions have been answered.

RQ1. What is the generic structure of maturity models presented in the
literature?

To address the RQ1, we reviewed the structure of maturity models presented in
the literature. We identified their type and specification such as levels of
maturity, descriptors, dimensions, and activities. However, the majority of
discussions have been about staged models like Crosby, CMM, and CMMI [40, 41,
9, 10, 27, 30]. The common characteristics (Number of levels, Descriptor of
levels, Descriptions, Dimensions, and Activities) of the maturity models have
also been identified. In addition, we looked at the structure of CMMI as the
most used staged maturity model. The staged models most often suffer from a
lack of ability for expressing dependencies among processes within a
capability level, while flexible models allow the definition of intermediate
status that indicates further details of maturity [6, 32, 34].

RQ2. What are the approaches/guidelines/methods to design a new maturity model
introduced in the literature?

To address the RQ2, we found studies which propose new maturity models.
Through our investigation, we realized that new models most often suffer from
avoiding existing solutions, a lack of theoretical basis, empirical studies,
and validation [6, 25, 27, 30]. We also identified the methods presented in
the literature to design a new maturity model. Although the identified methods
have not been presented in detail or they belong to a specific context, we
have specified the main aspects of the procedure for designing new models. The
generic activities performed to create a new maturity model have been
aggregated through considering six central characteristics (Context,
Origination, Source, Type, Meta-model, and Evaluation).

RQ3. To what degree do existing maturity models consider value/ benefit into
account for assessment?

The majority of existing models have been limited to indirect measurement and
evidence-based models like CMMI and formal models. They do not consider the
relations between practices and desired benefits and quality attributes [33].
These models focus on software quality attributes in general (like efficiency,
effectiveness, and suitability) which are not linked to every single practice
within the software process development. Some maturity models like CollabMM
consider quality attributes and benefits as advantages of using the model but
do not specify the relations between practices and benefits. [38]. We also
identified the OPP framework (Objectives, Principles, and Practices) that
considers the relation between principles and practices but it does not define
the relations among practices [19].

We conclude that there is a lack of models to assess a set of practices by
their desired benefits. The majority of existing models such as CMMI, SPICE,
RPI, BPMM, and KMMM (see Table 3) are fixed model type with rigid levels of
maturity that they cannot define interdependencies among different types of
practices. In addition, they are costly to implement [44, 29, 32, 34].

RQ4. How could a new maturity model be proposed for a value-based assessment
approach?

The concept of maturity models has been mostly concluded in traditional
approaches like CMMI that assess maturity based on predefined standards and
frameworks. However, the movement towards being agile points to a need for
maturity models that fit agile practices. Although a number of models have
been presented in this context, they suffer from a lack of holistic view of
the whole process. Furthermore, these models do not explicitly consider
values, benefits, practices, and relations among them.

To address the gaps identified in this study, we realized the need for a new
approach for the maturity that enables us to meet a lack of actual relations
between practices and desired values/benefits with a consolidated view on the
whole software development process (See section 5.4.2). We have proposed a
value-driven approach for software process Improvement (VASPI) through
conducting literature reviews to design a new maturity model in accordance
with real concerns. By the use of VASPI, it would be possible to look at
practices, desired benefits, and corresponding values at the same time as it
attempts to identify relations among them in particular context.

In the future, we will apply ‘Literature approach’ of VASPI for DevOps by
including gray and peer-reviewed literature (DevOps-VASPI). We also intend to
apply the ‘In-practice approach’ in cooperation with a software company. We
will use empirical research methods to evaluate and validate VASPI.

## Acknowledgment

This work has been supported by ELLIIT, a Strategic Area within IT and Mobile
Communications, funded by the Swedish Government. The work has also been
supported by research grant for the ViTS project (reference number 20180127)
from the Knowledge Foundation in Sweden.

## References

  * [1] Fuggetta, A., Di Nitto, E.:  Software process.  In: Proceedings of the Future of Software Engineering. FOSE 2014, New York, NY, USA, ACM (2014)
  * [2] Pettersson, F., Ivarsson, M., Gorschek, T., Öhman, P.:  A practitioner’s guide to light weight software process assessment and improvement planning.  Journal of Systems and Software 81(6) (2008) 972–995
  * [3] Silva, F.S., Soares, F.S.F., Peres, A.L., de Azevedo, I.M., Vasconcelos, A.P.L.F., Kamei, F.K., de Lemos Meira, S.R.:  Using CMMI together with agile software development: A systematic review.  Information & Software Technology 58 (2015) 20–43
  * [4] Emam, K.E., Melo, W., Drouin, J.N.:  SPICE: The theory and practice of software process improvement and capability determination.  IEEE Computer Society Press (1997)
  * [5] Dorling, A.:  SPICE: Software process improvement and capability determination.  Software Quality Journal 2(4) (1993) 209–224
  * [6] García-Mireles, G.A., Moraga, M.A., García, F.:  Development of maturity models: a systematic literature review.  (2012)
  * [7] Paulk, M.C., Weber, C.V., Curtis, B., Chrissis, M.B., et al.:  The capability maturity model: Guidelines for improving the software process. Volume 441.  Addison-wesley Reading, Massachusetts: (1994)
  * [8] Crosby, P.B.:  Quality is free: The art of making quality certain.  Signet (1980)
  * [9] Fraser, P., Moultrie, J., Gregory, M.:  The use of maturity models/grids as a tool in assessing product development capability.  In: Proceedings of the IEEE International Engineering Management Conference, IEMC’02. Volume 1., IEEE (2002) 244–249
  * [10] Paulk, M., Curtis, B., Chrissis, M., Weber, C.:  Capability maturity model sm for software, version 1.1. relatório técnico cmu.  Technical report, SEI-93-TR-024 ESC-TR-93-177, SEI Joint Program Office Hanscom AFB, MA 01731-2116 (1993)
  * [11] De Bruin, T., Freeze, R., Kaulkarni, U., Rosemann, M.:  Understanding the main phases of developing a maturity assessment model.  (2005)
  * [12] Team, C.P.:  CMMI for development, version 1.2.  (2006)
  * [13] Becker, J., Knackstedt, R., Pöppelbuß, J.:  Developing maturity models for it management.  Business & Information Systems Engineering 1(3) (2009) 213–222
  * [14] Pikkarainen, M., Mantyniemi, A.:  An approach for using CMMI in agile software development assessments: experiences from three case studies.  (2006)
  * [15] Huang, S.J., Han, W.M.:  Selection priority of process areas based on CMMI continuous representation.  Information & Management 43(3) (2006) 297–307
  * [16] Jalali, S., Wohlin, C., Angelis, L.:  Investigating the applicability of agility assessment surveys: A case study.  Journal of Systems and Software 98 (2014) 172–190
  * [17] McCaffery, F., Pikkarainen, M., Richardson, I.:  Ahaa–agile, hybrid assessment method for automotive, safety critical smes.  In: Proceedings of the 30th international conference on Software engineering, ACM (2008) 551–560
  * [18] Hossain, E., Babar, M.A., Verner, J.:  Towards a framework for using agile approaches in global software development.  In: Proceedings of the International Conference on Product-Focused Software Process Improvement, Springer (2009) 126–140
  * [19] Soundararajan, S., Arthur, J.D.:  A structured framework for assessing the” goodness” of agile methods.  In: Proceedings of the 18th IEEE International Conference and Workshops on Engineering of Computer Based Systems (ECBS), IEEE (2011) 14–23
  * [20] Biffl, S., Aurum, A., Boehm, B., Erdogmus, H., Grünbacher, P.:  Value-based software engineering.  Springer Science & Business Media (2006)
  * [21] Barney, S., Petersen, K., Svahnberg, M., Aurum, A., Barney, H.T.:  Software quality trade-offs: A systematic map.  Information & Software Technology 54(7) (2012) 651–662
  * [22] Khurum, M., Gorschek, T., Wilson, M.:  The software value map—an exhaustive collection of value aspects for the development of software intensive products.  Journal of Software: Evolution and Process 25(7) (2013) 711–741
  * [23] Peppard, J., Ward, J., Daniel, E.:  Managing the realization of business benefits from it investments.  MIS Quarterly Executive 6(1) (2007)
  * [24] Kitchenham, B., Charters, S.:  Guidelines for performing systematic literature reviews in software engineering.  Technical Report EBSE 2007-001, Keele University and Durham University Joint Report (2007)
  * [25] Wendler, R.:  The maturity of maturity model research: A systematic mapping study.  Information & Software Technology 54(12) (2012) 1317–1339
  * [26] Tarhan, A., Turetken, O., Reijers, H.A.:  Business process maturity models: a systematic literature review.  Information and Software Technology 75 (2016) 122–134
  * [27] Lasrado, L.A., Vatrapu, R., Andersen, K.N.:  Maturity models development in is research, Information Systems Research Seminar in Scandinavia (2015)
  * [28] von Wangenheim, C.G., Hauck, J.C.R., Salviano, C.F., von Wangenheim, A.:  Systematic literature review of software process capability/maturity models.  In: Proceedings of the International Conference on Software Process Improvement and Capability Determination (SPICE). (2010)
  * [29] Almomani, M.A.T., Basri, S., Mahamad, S., Bajeh, A.O.:  Software process improvement initiatives in small and medium firms: A systematic review.  In: Proceedings of the 3rd International Conference on Advanced Computer Science Applications and Technologies (ACSAT), IEEE (2014) 162–167
  * [30] Albliwi, S.A., Antony, J., Arshed, N.:  Critical literature review on maturity models for business process excellence.  In: Proceedings of the IEEE International Conference on Industrial Engineering and Engineering Management (IEEM), IEEE (2014) 79–83
  * [31] Petersen, K., Vakkalanka, S., Kuzniarz, L.:  Guidelines for conducting systematic mapping studies in software engineering: An update.  Information and Software Technology 64 (2015) 1–18
  * [32] Habra, N., Alexandre, S., Desharnais, J.M., Laporte, C.Y., Renault, A.:  Initiating software process improvement in very small enterprises: Experience with a light assessment tool.  Information and software technology 50(7) (2008) 763–771
  * [33] Kroeger, T.A., Davidson, N.J., Cook, S.C.:  Understanding the characteristics of quality for software engineering processes: A grounded theory investigation.  Information and Software Technology 56(2) (2014) 252–271
  * [34] Van Steenbergen, M., Bos, R., Brinkkemper, S., Van De Weerd, I., Bekkers, W.:  The design of focus area maturity models.  In: Proceedings of the International Conference on Design Science Research in Information Systems, Springer (2010) 317–332
  * [35] Niazi, M.:  An instrument for measuring the maturity of requirements engineering process.  In: Proceedings of the International conference on product focused software process improvement, Springer (2005) 574–585
  * [36] Niazi, M., Wilson, D., Zowghi, D.:  A maturity model for the implementation of software process improvement: an empirical study.  Journal of Systems and Software 74(2) (2005) 155–172
  * [37] April, A., Huffman Hayes, J., Abran, A., Dumke, R.:  Software maintenance maturity model (smmm): the software maintenance process model.  Journal of Software: Evolution and Process 17(3) (2005) 197–223
  * [38] Magdaleno, A.M., de Araujo, R.M., Werner, C.M.L.:  A roadmap to the collaboration maturity model (collabmm) evolution.  In: Proceedings of the 15th International Conference on Computer Supported Cooperative Work in Design (CSCWD), IEEE (2011) 105–112
  * [39] Beecham, S., Hall, T., Rainer, A.:  Building a requirements process improvement model.  Technical report, University of Hertfordshire (2003)
  * [40] Mettler, T., Rohner, P., Winter, R.:  Towards a classification of maturity models in information systems.  In: Management of the interconnected world.  Springer (2010) 333–340
  * [41] Crosby, P.:  B., Quality is Free.  New York, New York, McGraw-Hill, Inc (1979)
  * [42] Sawyer, P., Sommerville, I., Viller, S.:  Requirements process improvement through the phased introduction of good practice.  Software Process: Improvement and Practice 3(1) (1997) 19–34
  * [43] Opperthauser, D.:  Defect management in an agile development environment.  CrossTalk: The Journal of Defense Software Engineering (2003)
  * [44] Baars, T., Mijnhardt, F., Vlaanderen, K., Spruit, M.:  An analytics approach to adaptive maturity models using organizational characteristics.  Decision Analytics (2016)
  * [45] Garousi, V., Felderer, M., Mäntylä, M.V.:  The need for multivocal literature reviews in software engineering: complementing systematic literature reviews with grey literature.  In: Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering, ACM (2016) 26
  * [46] Park, S., Cha, B., Kim, J.:  Preparing and inter-connecting hyper-converged smartx boxes for iot-cloud testbed.  In: Proceedings of the 29th IEEE International Conference on Advanced Information Networking and Applications, AINA. (2015) 695–697
  * [47] Virmani, M.:  Understanding devops bridging the gap from continuous integration to continuous delivery.  In: Innovative Computing Technology (INTECH), 2015 Fifth International Conference on. (2015)
  * [48] Fitzgerald, B., Stol, K.:  Continuous software engineering and beyond: trends and challenges.  In: Proceedings of the 1st International Workshop on Rapid Continuous Software Engineering, RCoSE 2014. (2014) 1–9
  * [49] Farroha, B., Farroha, D.:  A framework for managing mission needs, compliance, and trust in the devops environment.  In: Proceedings of Military Communications Conference (MILCOM), 2014 IEEE. (2014)
  * [50] Boehm, B.W.:  Seven basic principles of software engineering.  Journal of Systems and Software 3(1) (1983) 3–24
  * [51] Wettinger, J., Breitenbücher, U., Leymann, F.:  DevOpSlang - bridging the gap between development and operations.  In: Proceedings of the Third European Conference on Service-Oriented and Cloud Computing, ESOCC 2014. (2014) 108–122
  * [52] Viju, G., Elsalam, M.M.A., Ibrahim, K.A., Jassim, M., Jassim, M.:  The impact of software process improvements in small and medium scale enterprises.  International Journal of Soft Computing and Engineering (USCE) 3(4) (2013)