# On the Usage of Psychophysiological Data in Software Engineering:
An Extended Systematic Mapping Study

Roger Vieira<EMAIL_ADDRESS>Kleinner Farias
<EMAIL_ADDRESS>Graduate Program in Applied Computing (PPGCA),
University of Vale do Rio dos Sinos (Unisinos),
São Leopoldo, Rio Grande do Sul, Brazil

###### Abstract

In recent years, many studies have applied wearable devices to capture
psychophysiological data from software developers. However, the current
literature lacks investigations that classify the studies and point out gaps
to be explored. This article, therefore, seeks to present a comprehensive
overview of the literature by classifying and creating a systematic map of the
works. Besides, it seeks to pinpoint research gaps, challenges, and trends.
Based on widely known guidelines, a systematic mapping of the literature was
designed and run to answer eight research questions. After applying a careful
filtering process, we selected 27 representative studies from a sample of
2,084 potentially relevant works retrieved from seven digital libraries. The
main results are: a classification scheme of the published studies was
produced; there is no predominance of the devices used to capture
psychophysiological data; over 50% of the studies have explored indicators
related to mental states and neural activity; and 80% have analyzed composite
data to understand the cognitive load and in the context of understanding
debugging programs and strategies. Our findings can benefit researchers and
students by creating a systematic map of the literature, being a starting
point for future research.

###### keywords:

Psychophysiological Data , Neuroscience , Software Development

††journal:

##  1 Introduction

Software development is an intellectual process, where every developer applies
knowledge grounded in a set of logical premises to produce solutions designed
to real-world problems. The development needs professionals to be focused and
deeply engaged in their work, paying attention to every detail which offers a
threat to impact negatively the produced solution [1]. Moreover, the
development of real-world applications has happened in increasingly unstable
business environments [55], requiring collaborative work [49, 12, 53, 51],
development effort from different developers’perspectives, including the
effort to integrate software artifacts [17, 22, 23, 20, 19], maintain source
code [14], and evolve software architecture [21].

Nowadays, studies aim to understand which cognitive processes are related to
software development tasks, using technologies as Electroencephalography (EEG)
and Functional Magnetic Resonance Imaging (fMRI) [63, 1, 26, 50, 73]. In these
studies, the authors advocate that it is possible to find a relation between
defect production in software artifacts and the developer’s cognitive
processes presented when source code was written, aiming the development of
mechanisms capable of priory low confidence code refactoring and even
prediction of defective software artifacts. Manica et al. [50] investigate how
software developer’s effort and affective states can be affected by model
composition techniques. Vieira and Farias [73] introduce a psychophysiological
data integrator approach for for the Microsoft Visual Studio code.

The advent of microelectronics brought smaller, faster, and low energy profile
processors, enabling the usage of mobile and accessible devices to collect
biological signals, including in several academic experiments. This is the
case of Mindwave Mobile 2 [61] and Emotiv EPOC and EPOC+ [59], devices that
are used to increase comprehension upon cognitive processes involved in the
coding activities. Allied to the extensibility of current Integrated
Development Environments, the usage of these devices might enable the
development of new functionalities enabling the identification of developer’s
behavior patterns about their productivity and produced software artifacts
quality [28, 59, 78].

In the last decade, psychophysiological data has been widely used as an
improvement resource in the software engineering context. However, the current
literature lacks a formal classification and a systematic mapping of articles
already made and published approaching this topic [26, 31]. Consequently, a
detailed understanding of the usage of that data remains limited. Grounded on
such an argument, the present study aims to answer the following research
question: “How are the psychophysiological data are being used as improvement
resources in the software engineering context?”

Therefore, this work seeks to classify and provides a thematic analysis of
studies that have already been published upon the usage of psychophysiological
data as an improvement resource in the software engineering context. To
fulfill that, a systematic mapping was designed and performed, following a set
of well-established assumptions aiming to answer eight research questions. An
initial number of 2,084 studies were collected within 7 digital databases, of
which 27 were selected to answer the previously formulated research questions.
As result, it notices that most studies are outlined as a controlled
experiment, characterized by the usage of devices to collect
psychophysiological data and integrated development environments (IDEs),
intended to analyze developer’s psychophysiological attributes, mainly when it
was applied in the comprehension of source code and debugging strategies.
Besides, it was not identified mechanisms capable of enabling support for data
integration and analysis natively within IDEs. Lastly, this study has
identified some challenges, which would be explored by the scientific
community for the next years.

This work is an extension of an initial literature review [74]. This extended
article differs by (1) exploring eight research questions, as opposed to 6
questions, (2) presenting a description of essential concepts related to
psychophysiological data and integrated development environments, (3)
detailing the methods used for the extraction of categories and their correct
application for the classification of the answers obtained during the
filtering stage, and (4) introducing the results in more detail. The
presentation of the theoretical framework used, coupled with the addition of
research questions and a more in-depth analysis of the results, therefore
culminated in a better presentation of the identified findings and challenges,
also enabling the making of a heat map capable of illustrating possible gaps
to be treated as opportunities in the development of future studies.

Initially, this study presents a literature revision upon theoretical
background adopted in its development (Section 2), also enumerates and
synthesizes the related works identified about the topic (Section 3). Next,
the adopted steps as the methodology for systematic mapping development are
detailed (Section 4). After that, the performing of selection, filtering and
selected studies data extraction are showing up (Section 6), like the found
results (Section 7) and the identified challenges are discussed grounded by
the current literature (Section 8). Finally, conclusions, final thoughts, and
future works are conferred in Section 9.

##  2 Background

This section presents the main concepts needed to understand the literature
mapping study. The concepts are derived from articles already published
(discussed in Section 3) on the topic of literature review considering the use
of psychophysiological data.

###  2.1 Psychophysiological Metrics and Data

The psychophysiological data111In academic literature, we often find the term
biometric data, however, in this article, the authors have adopted the term
psychophysiological data to avoid ambiguity across the process to assign an
identity to a subject based on the biological data (aka.: biometry) are
measurements produced from metrics concerned with quantifying
psychophysiological aspects of the human being. These measurements seek to
quantify human responses to external stimuli in specific situations, for
example, software developer’s cognitive load produced when reading the source
code in a situation of pressure to deliver a software product to the market.
Some studies [46, 3] have explored the variation of measurements to better
understand the mental states generated when human beings are exposed to
situations produced through controlled experiments. The literature has
explored the use of various metrics in different contexts of software
development [27, 28, 56, 18, 13, 40].

####  2.1.1 Eye Metrics

The eye metrics are often obtained using devices known as eye-trackers. These
devices use laser beams or infrared light to measure a set of attributes
related to the subject’s eyeball, such as eye movement trajectory, pupil
dilation, focus, eye gaze, and eye blinking frequency.

When applied in specific contexts, the collected data can be used to indicate
psychophysiological processes, such as cognitive load, excitation, valency,
attention, anxiety, and stress. The literature points to the viability of the
experimental use of these devices in various contexts, such as monitoring the
execution of software development tasks [27], performing software debugging
strategy [47], and the difficulty of learning a new language [42].

####  2.1.2 Brain Metrics

The brains’ neural activity while executing several tasks chains a series of
fluctuations in potential difference (or voltage), all long over the cranial
surface, which can be measured using a technique called electroencephalography
(EEG) [28, 3].

Researchers have identified the possibility to translate the neural activity
and its voltage fluctuations in a wave spectrum grouped by different frequency
bands, being able to be related to a respective set of activities (Table 1).
Alpha wave ($\alpha$), for instance, often can be observed in subjects who are
in states of meditation or relaxation, being attenuating as physical or mental
activity intensity increases [3, 28]. Electrodermal activity, as well as
skin’s surface temperature measurement, has already been correlated upon
psychological processes, like excitation and some specific emotions [28].

Table 1: Brainwaves spectrum and its frequently related activities. Wave |  Symbol |  Band |  Relationship
---|---|---|---
Alpha |  $\alpha$ |  8 – 12Hz |  relaxation, meditation, reflection
Beta |  $\beta$ |  12 – 30Hz |  alert state, focus
Gamma |  $\gamma$ |  31 – 120Hz |  high brain activity, learning
Delta |  $\delta$ |  0,4 – 4Hz |  deep sleep
Theta |  $\theta$ |  4 – 7Hz |  drowsiness, meditation

As well as voltage fluctuations on the cranial surface, neural activities
trigger biological processes according to the execution of both physical and
mental activities, such processes as the brain blood and oxygen supply, can be
measured employing techniques like functional Near-Infrared Spectroscopy
(fNIRS) and functional Magnetic Resonance Imaging (fMRI). Instead of EEG,
fNIRS and FMRI are capable of generating clear images of brain activity in
well-defined regions, enabling its relation to cognitive and biological
processes [25, 45, 66].

####  2.1.3 Skin Metrics

The set of metrics related to a variation on skin’s surface properties, such
as temperature and Electrodermal Activity (EDA) is known by Skin metrics, the
last one is often found in the literature under the term Galvanic Skin
Response (GSR). The EDA sensor devices usually collect data from the skin’s
electric resistance as a response to some external stimulus, for instance,
when a subject is in a state of excitation, sweat glands rise its activity,
triggering an increase of sweat production, consequently, reducing the
electrical resistance of the skin, enabling a better electrical current
conductibility across the skin [28, 41].

####  2.1.4 Cardio-respiratory Metrics

In the revision process of the studies used as theoretical grounding, there
were identified four main metrics associated to cardiorespiratory aspects
presented by the subjects analyzed in the experiments: Blood Volume Pulse
(BVP), Heart Rate (HR), Heart Rate Variability (HRV) and the Respiratory Rate
(RR). The heart rate refers to the number of cardiac cycles (systolic and
diastolic movements) performed by cardiac muscle on the period of one minute,
on the other hand, the heart rate variability points to time variation between
two consecutive cardiac cycles. As just the HR, the respiratory rate stands
for the cycles performed by the lungs to provide oxygen to the body. The blood
volume pulse, also found in literature as just “pulse”, measures the blood
flow in human body specific areas according to changes drove by Sympathetic
Nervous System (SNS), especially when a subject is exposed to stressful
situations [3, 28]. The presented metrics are strongly linked to cognitive
load and several emotions [28].

###  2.2 Cognitive Load Theory

The human body, besides constantly provides physical resources, does also with
the called mental resources – as memory, attention, and reasoning – aiming the
maintenance of the subject’s vital functions. When necessary, such resources
can be also allocated in specific ways, to enable the execution of harder
tasks started by the subject, for instance, the resolution of logical problems
faced by him in the software development process.

To the number of mental resources, especially the working memory, which
appears while a mental effort is necessary to perform a set of cognitive
activities - -both simple and complex ones – it is attributed the term
Cognitive Load [69]. Due to its kind, the Cognitive Load can still be split
into three specialized categories:

  1. 1.

Intrinsic Cognitive Load: it is related to the level of difficulty associated
with a simple instructional topic. All instructions have an inherent
difficulty level associated with it, which cannot be modified by the
instructional agent. However, the mental schema used to represent a task might
be fractioned in smaller pieces, enabling the possibility to be handled singly
and posteriorly regrouped without compromising semantically the original
schema.

  2. 2.

Extraneous Cognitive Load: such kind of cognitive load can be generated
according to the way an informational content is presented by an agent to one
who will perform an activity. Different approaches to present information can
generate distinct levels of extraneous cognitive load.

  3. 3.

Germane Cognitive Load: different from such other cognitive load categories,
the germane cognitive load can be considered a type of artificial cognitive
load, once it results from the process of mental schema creation used to
represent a collection of instructions, such as automation, analogies or event
deconstruction and construction of concepts, intending to provide easiness in
the comprehension and learning regarding a specific topic.

In short, the developer’s cognitive load, for instance, can be affected by
many factors, such as the source code presentation within integrated
development environments, his expertise regarding a specific programming
language, his knowledge about-faced errors, and his solution comprehension as
a whole [28].

###  2.3 Integrated Development Environment

The Integrated Development Environments (IDEs) can be considered as the main
set of tools used by developers, for both creation tasks and source code
maintenance. This set of tools holds source code editors, file navigation,
compilers, and debuggers, artifacts refactoring assistants, and tools for
syntax highlighting, such as other functionalities [65]. From a developer
perspective, besides the expertise across the programming language which are
being used and the full knowledge regarding logical aspects involved in the
problem, it is essential his domain over the tools used in the development
process [4].

In the last years, some authors have conducted researches about the usage of
IDEs by developers, both in the software development process [4, 78], and its
efficacy as a debugging tool [78]. Moreover, domain-specific modeling
environments have been proposed to allow developers to work with semantically
richer IDEs Farias et al. [24], Gonçalves et al. [36, 37, 35]. Usually
employing eye-trackers to collect quantitative data, the authors evaluate
aspects like usage easiness, user graphical interface organization and
disposition, the set of available tools, such other resources according to the
solution provider. Looking for the evolution of such tools and to facilitate
the development of related studies, some author has been produced articles
aiming to perform a better way to integrate biometric data — such as these
collected by eye-trackers – within integrated development environments, in a
way that data can be analyzed and related to the generated source code, or
even, to the developer’s interaction while executing his tasks, with no need
of external tools [38].

###  2.4 Psychophysiological Data in Software Engineering

Psychophysiological data has great applicability in understanding what
experiences developers are subjected to while performing their tasks, during
an intense, fully development-focused work cycle, as well as when facing some
sort of difficulty understanding a piece of code or the process. solution
design logic [28].

In the context of Software Engineering, the use of psychophysiological data
has been emerging as it refers to the understanding of cognitive processes
involved in software development and their correlates [28]. Studies in this
area have explored from devising new low-cost sensor devices that are capable
of reliably measuring psychophysiological signals to examining how novice and
experienced developers understand source code writing [44, 70] and how this
can help improve software artifact production [59].

Despite the increase in the production of studies in this area, some authors
advocate the low consistency and lack of consensus presented in them, thus
seeking to conduct surveys that bring a current and consolidated view around
the use of psychophysiological data in the context of Software Engineering
[31, 54]. Through the development of mapping and systematic reviews, the
authors aim to identify gaps in the theme that may serve as input for the
preparation of new studies able to address the problems presented.

##  3 Related Work

This section presents a comparative analysis of this article with the current
literature. To this end, several articles were identified after applying the
search string “Psychophysiological Data AND Software Engineering ” to Google
Scholar 222https://scholar.google.com/, seven of which were selected for
similarity to the theme explored in this study. Such studies are briefly
analyzed below.

###  3.1 Synthetic Analysis

Intending to provide a wide point of view involving all related works, each of
them was read and a synthetic analysis was performed to present a brief resume
of them. The following is a list of a synthetic analysis of each related work.

  * •

Fritz et al. [27]: it presents a new approach to the classification of
difficulty in understanding source code that was investigated using biometric
data associated with psychophysiological states. Through an experiment
involving 15 developers, it was used Eye-trackers, electrodermal activity
sensors (EDA), and electroencephalograph (EEG) to verify how the collected
data could provide a form of prediction when a developer would consider a task
as being difficult.

  * •

Fritz and Muller [28]: the dissertation was based on a series of controlled
experiments carried out in previous studies, which applied different sensors
for data collection. They also discussed the strategies adopted for data
collection, preparation, transformation, loading, and analysis, as well as to
elucidate the machine learning algorithms applied to extract relevant features
to the objectives of each study.

  * •

Fucci et al. [29]: the authors replicated experiments previously presented in
the literature [26], using low-cost sensor devices. To do so, they performed
measurements using EEG, EDA, and BVP on a sample of 28 Computer Science
students while performing tasks involving the comprehension of texts written
in natural language and source code.

  * •

Goncales et al. [31]: a systematic mapping was developed to identify gaps in
previous works that addressed the study of cognitive load in the context of
Software Engineering. Thirty-three related articles that intersected the
themes of cognitive load, software engineering, and use of sensor devices for
psychophysiological data collection were analyzed.

  * •

Gui et al. [39]: a literary survey was conducted between 2007 and 2017 on the
state of the art about the use of brain biometric data, the types of sensors
used, how these data were collected, their application contexts, and
statistical approaches, and machine learning to extract features around the
use of such data.

  * •

Radevski et al. [59]: a conceptual architecture model was proposed aiming at
real-time data acquisition from low-cost EEGs. A pilot test was also performed
using Emotiv EPOC and EPOC + devices to evaluate features such as usability
and reliability during application in experiments. Moreover, the authors have
addressed the ethical implications involved in collecting real-time
psychophysiological data for developer productivity assessment.

  * •

Rostami et al. [61]: through an experimental study, the cognitive load and
familiarity of a group of Computer Science students were measured regarding
their interaction with Integrated Development Environments, using a low-cost
EEG device to collect data. psychophysiological data. The authors also pointed
out the advantages of EEG over similar techniques and the technical
limitations of low-cost sensor devices.

###  3.2 Comparative Analysis and Opportunities

Five Comparison Criteria (CC) were defined to assist in the process of
identifying similarities and differences between the proposed work and the
selected articles. This comparison is crucial in making the process of
identifying research opportunities using objective rather than subjective
criteria. The criteria are described below:

  * •

Systematic Mapping (CC01): studies that developed a systematic mapping of the
literature aiming at acquiring a vision of the current state of the art.

  * •

Data Integration (CC02): papers that researched how psychophysiological data
are being supported by integrated development environments.

  * •

Psychophysiological Metrics (CC03): these categories are studies that
researched biological metrics that were linked to cognitive processes.

  * •

Applications in Software Engineering (CC04): studies that sought to evaluate
how psychophysiological data are applied in Software Engineering.

  * •

Integrated Development Environments (CC05): researches related to integrated
development environments, both around its use and its improvement.

Table 2 presents the comparison of the selected works, contrasting them with
the proposed work. Some research gaps and opportunities are noted:

  1. 1.

The proposed work was the only one that completely met all the comparison
criteria;

  2. 2.

Psychophysiological metrics and applications in Software Engineering were the
most explored themes, while systematic mapping and integrated development
environments were the least explored;

  3. 3.

No studies have explored in detail the use of psychophysiological data in the
context of integrated development environments.

Table 2: Comparative analysis among related works

Related Work | Comparison Criteria
---|---
CC1 |  CC2 |  CC3 |  CC4 |  CC5
Proposed Work |  $\CIRCLE$ |  $\CIRCLE$ |  $\CIRCLE$ |  $\CIRCLE$ |  $\CIRCLE$
Fritz et al. [27] |  $\Circle$ |  $\Circle$ |  $\CIRCLE$ |  $\LEFTcircle$ |  $\Circle$
Fritz and Muller [28] |  $\Circle$ |  $\Circle$ |  $\CIRCLE$ |  $\CIRCLE$ |  $\Circle$
Fucci et al. [29] |  $\Circle$ |  $\Circle$ |  $\CIRCLE$ |  $\CIRCLE$ |  $\Circle$
Goncales et al. [31] |  $\CIRCLE$ |  $\Circle$ |  $\CIRCLE$ |  $\CIRCLE$ |  $\Circle$
Gui et al. [39] |  $\Circle$ |  $\Circle$ |  $\LEFTcircle$ |  $\Circle$ |  $\Circle$
Radevski et al. [59] |  $\Circle$ |  $\LEFTcircle$ |  $\LEFTcircle$ |  $\CIRCLE$ |  $\Circle$
Rostami et al. [61] |  $\Circle$ |  $\LEFTcircle$ |  $\LEFTcircle$ |  $\CIRCLE$ |  $\CIRCLE$

$\CIRCLE$ Completely Meets |  $\LEFTcircle$ Partially Meets |  $\Circle$ Don’t Meets
---|---|---

Therefore, the following research opportunity was identified: a holistic
literature review that meets the five established comparison criteria. This
opportunity is explored in the next sections.

##  4 Methodology

This section aims to describe our review protocol. This protocol addresses
essential steps defined in well established guidelines [58] to plan and create
a systematic mapping of the literature. In addition, our study protocol was
also defined based on previously validated systematic mapping studies [9, 52,
10, 34, 33, 7, 74, 67]

###  4.1 Research Questions

In addition to the objectives, the Research Questions (or RQs) will guide the
researcher in the development of his systematic mapping, helping to identify
possible studies that may converge with the theme of the study under
development [58]. The Research Questions used in this study are presented in
Table 3, bringing their purposes and the variables associated with them.

Although systematic mapping has a well-established methodology for conducting
it, its topics extend to the needs of the researcher [58]. Thus, the steps
adopted for the development of the present work were structured as elucidated
in subsequent sections.

Table 3: Research Questions, their description and related variables Research Question |  Description |  Variable
---|---|---
RQ01: How would it be taxonomy to classify studies involving IDEs and psychophysiological data? |  Aims to build up a taxonomy capable of classifying studies involving psychophysiological data and IDEs. |  Taxonomy
RQ02: What kind of devices has been used to collect psychophysiological data through studies? |  List devices used in the selected studies to collect data over the experiments |  Devices
RQ03: What kind of attributes are extracted by devices used in the studies? |  Elucidates the main objective of e the data types that have been extracted over the experiments and what it means within the scope of the study |  Attributes
RQ04: How the integration of extracted data are supported by the IDEs? |  Explains how the extracted data can be integrated inside of an IDE to provide extraction of features. |  Integration Support
RQ05: For what purposes is the data being applied?? |  Brings up what extracted data aims to explain in the IDEs’ and software development contexts. |  Purposes
RQ06: What are the main contribution of the reviewed studies? |  Finds out the contributions of each analyzed study in the context of psychophysiological data applied to software development process. |  Contributions
RQ07: What is the nature of the reviewed studies? |  Understand what methodologies researchers are using within scope of applying physiological data and its cognitive processes into IDEs and in the software development process |  Nature
RQ08: Which publication venues have the selected studies been released? |  Identify venues where studies have been published across last ten years. |  Publication Venues

###  4.2 Search Strategy

The second step in the Systematic Mapping methodology is the definition of the
search strategy, which aims to reach a representative sample of studies that
can generically answer the Research Questions previously defined.

The Search Strategy used in this study was conceived in two stages: (1)
selection of digital databases related to the study theme, and (2)
construction of a search string and its variants, respecting the peculiarities
of the search. search engines present in each database.

####  4.2.1 Data Sources

Table 4 presents the electronic databases used. They were chosen because they
are widely used and previous literature mapping studies have shown their
effectiveness [8, 32]. Also, database selection was based on the search engine
coverage, which returned articles from leading Software Engineering journals.

Table 4: Databases and its digital addresses Database |  Address
---|---
ACM Digital Library |  http://portal.acm.org/
DBLP |  https://dblp.uni-trier.de/
IEEE Xplore |  http://ieeexplore.ieee.org
Science Direct |  https://www.sciencedirect.com/
Scopus |  https://www.scopus.com/
Semantic Scholar |  https://www.semanticscholar.org/
Springer Link |  http://www.springerlink.com/

####  4.2.2 Search String

The search string is a construct derived from the combination of major terms
and their synonyms and is used as an input to database search engines.
selected. This input allows search engines to return a collection of
potentially relevant studies contributing to the development of Systematic
Mapping. Table 5 lists the main terms and their synonyms that were used in the
development of this study.

To produce a functional search string, the following steps were taken to
develop it: (1) define the main terms; (2) search for related synonyms or
semantically equivalent words, and (3) match the main terms to their synonyms
using logical operators such as “ AND ” and “ OR ”. Applying the
characteristics mentioned, we arrived at the following search string:

(EEG OR electroencephalography OR neural OR brain OR cognitive OR psychometric
OR bioinformatic) AND (software OR developer OR programmer OR professional OR
“software engineering”) AND (ide OR code OR editor OR “integrated development
environment”)
---

Due to the syntactical peculiarities that come from each search engine
contained in the databases, the initial search string needed to be
individually tailored to provide a better result without losing its semantic
value.

Table 5: Search string major terms and their synonyms Main Term |  Synonym
---|---
Neuroscience |  eeg, fmri, electroencephalography, “functional magnetic resonance imaging”, neural, brain, cognitive, psychometric, bioinformatic
Software Engineering |  software, developer, programmer, professional, software engineering
IDE |  ide, code, editor, integrated development environment
Data Analysis |  “data analysis”, “signal analysis”, analytic, “signal processing”

###  4.3 Inclusion and Exclusion Criteria

Once the databases were selected and the search string was constructed, the
subsequent step in the Systematic Mapping development process was composed of
the definition of the Inclusion Criteria and the Exclusion Criteria (CI and
EC, respectively).

The Inclusion Criteria were applied to include studies in the filtering
process after their initial collection in databases [58]. The Exclusion
Criteria acted as a mechanism for discarding studies that did not represent
answers. for Research Questions [58]. Therefore, the following items were
defined as Inclusion Criteria:

  * •

IC01: Published between the years of 2009 and 2019;

  * •

IC02: Written in English;

  * •

IC03: Available as full papers in digital databases;

  * •

IC04: Related to the search string and research questions;

Furthermore, to filter previously selected studies by the Inclusion Criteria,
the following Exclusion Criteria were applied:

  * •

EC1: Match the keyword in the search string, but the context is different for
research purposes;

  * •

EC2: Papers with no abstract;

  * •

EC3: Is just an abstract or only a summary, conference call;

  * •

EC4: There is no relation with Software Engineering domain;

  * •

EC5: Is a copy/duplicate or an older version of an already selected article;

  * •

EC6: Was not possible to access the full paper;

  * •

EC7: Motivation related;

###  4.4 Data Extraction Strategy

Once the steps for selecting the studies were established, it was necessary to
define two mechanisms, namely: (1) data extraction form and (2) classification
schemes. Such mechanisms are of paramount importance for the development of
works in the Systematic Mapping format, since, in a standardized way, they
will define how the data contained in the selected works will be extracted and
classified, thus enabling reliable identification of possible answers to the
data previously defined in Research Questions [58].

###  4.5 Data Extraction Form

To standardize the process of extracting the data contained in the articles
and standardize the classification of the answers to the Research Questions, a
data extraction form was developed, as can be observed in Figure 1.

Figure 1: A snippet of data extraction form applied in the study development.

####  4.5.1 Classification Schema

To develop an effective mechanism for classifying studies, among the possible
approaches to category extraction used in the classification scheme, the
following were adopted in this study:

  * •

Topic Independent Scheme: Generic categories are used to provide an overview
of the selected studies and their theme. Usually, the authors recommend the
use of categories already adopted in another systematic mapping;

  * •

Topic Dependent Scheme: Specific questions emerge as the studies are filtered,
thus bringing a detailed view of their content, which is a peculiarity present
in each systematic mapping.

Based on the classification schemes presented, the following categories were
extracted as possible answers for each Research Question:

  * •

Taxonomy (RQ01): This category aims to guide the study in the construction of
a taxonomy capable of assisting in the identification of studies that use
psychophysiological data integrated with IDEs within the scope of Software
Engineering. Unlike the other Research Questions, it was constructed as the
answers of the others emerged..

  * •

Devices (RQ02): Represents which types of devices were used in the studies
selected for the collection of psychophysiological data and other biomedical
metrics. The following items represent the extracted categories used for the
classification of studies in this regard:

    * –

Electroencephalogram (EEG): type of sensing used to measure and record brain
electrical activity;

    * –

Eye-tracker: a device used to measure the eye position of the individual as
well as his movements;

    * –

Functional Magnetic Resonance Imaging (fMRI): measures brain activity in
association with changes in blood flow and variations in oxygen saturation;

    * –

Functional Near-Infrared Spectroscopy (fNIRS): like fMRI, fNIRS collects brain
activity measurements through hemodynamic responses;

    * –

Electrodermal Activity (EDA): performs measurement series around galvanic skin
variation;

    * –

Device Combination: applicable when more than one device was used in the
experiment, also applied to complementary sensing devices such as measuring
bracelets and chest straps equipped with cardiac and respiratory monitoring
sensors;

    * –

No Devices: when the nature of the study did not require sensor devices to be
used.

  * •

Attributes (RQ03): refers to context-based characteristics associated with
data collected through sensor devices. The attributes identified in the
selected studies were classified into:

    * –

Fixation/Attention: aims to indicate how long an individual’s visual focus
remains in an Area of Interest, usually associated with attention [6].
However, without complementary Mental State measures, it is not possible to
infer the relationship of this attribute to focused attention.

    * –

Mental States: represents a set of attributes responsible for the relationship
between cognitive processes (attention, learning, meditation, drowsiness,
etc.) and the electrical activity of neurons, usually expressed through brain
waves such as Alpha, Beta, Gamma, Delta and Theta [70].

    * –

Neural Activity: this category covers sets of neural activation patterns and
their associated semantics [60].

    * –

Stress/Cognitive Load: often associated with an individual’s working memory as
he or she performs a task or even a set of tasks that require context
switching [68].

    * –

Mental Effort: tt intends to explain the collection of biological metrics
linked to the contribution of resources provided by the body during the
execution of cognitive tasks [72].

    * –

Combined Attributes: applicable to studies that have performed the analysis,
three or more combined attributes.

    * –

No Attributes: category adopted in studies in which specific attributes around
psychophysiological data were not evaluated.

  * •

Integration Support (RQ04): this category indicates whether the study analyzed
implemented any mechanism capable of integrating the collected data with the
IDEs used.

    * –

Supported: applicable when the study analyzed used a tool capable of
integrating the psychophysiological data collected and the IDE used;

    * –

Not Supported: when a study used psychological data as well as IDEs, but data
analysis was independent of IDE, needing an external tool to perform it;

    * –

Not Applicable: category adopted in analyzed studies that use
psychophysiological data but do not have in their context the use of IDEs;

  * •

Purposes (RQ05): it brings an overview of the main objectives stipulated by
the authors in each study, regarding the use of the collected data and their
related attributes. Thus, the identified categories were enumerated as
follows:

    * –

Program Comprehension and Debugging Strategy: this category of purposes
attempts to quantify the ease of understanding a program from the developer’s
point of view, in addition to analyzing the strategies adopted by it during
the debugging process;

    * –

Brain-Computer Interface: this category applied when the main purpose of the
study is to develop an interface capable of enabling the interaction between
humans and computers through the use of signals from neural activities;

    * –

Cognitive Load: related to studies that conduct measurements linked to a
developer’s mental effort while performing a complex task;

    * –

Improve Productivity: aims to find ways to provide better tools and processes
in the context of Software Engineering and the quality of life of the
developer;

    * –

Out of Context: applicable where the purpose of the study was not directly
linked to the use of psychophysiological data in the context of Software
Engineering.

  * •

Contributions (RQ06): deviating from the others, this category adopts two
approaches: the first one of subjective nature, will present a synthetic text
with the main contributions identified by the authors for the development of
this work. In the same vein, the second approach will seek to generalize the
contributions identified through the following categories, according to the
value returned for the development of the present study:

    * –

Metric: even indirectly, the study presents some metric derived from
psychophysiological data that could be correlated to some of the identified
attributes and purposes presented above;

    * –

Tool: brought some tools for both data collection and processing/analysis;

    * –

Model: when the study contributes to the presentation of a conceptual or
practical model of architecture or experimentation involving the use of
psychophysiological data;

    * –

Method: presents an isolated method that can be used in the aspects of
collecting, processing or analyzing psychophysiological data;

    * –

Process: similar to the item method, however, this category encompasses a
sequence of methods for collecting, processing and or analyzing the collected
data;

    * –

Integration: It enables the identification of some approach of integrating the
psychophysiological data collected with integrated development environments.

  * •

Nature (RQ07): the nature of a study refers to the type of research developed
as to its main purpose and development. The classification style often adopted
in the literature is the [75], which determine the following categories:

    * –

Proposal of Solution: relating to studies proposing new techniques for solving
existing problems or revising techniques already adopted;

    * –

Evaluation Research: this category applies to studies that evaluate techniques
and practices applied in industry to specific problems;

    * –

Validation Research: studies for the evaluation of innovative techniques, but
not yet validated for their application in the industry;

    * –

Philosophical Paper: studies which aim to bring a new perspective regarding
the already consolidated knowledge structures;

    * –

Personal Experience Paper: This type of work discussed with another author
solved a problem by applying a technique in practice, without the need to also
use the technique during the development of the study;

    * –

Opinion Paper: it encompasses all studies that present opinions based on a
particular research topic;

    * –

Controlled Experiment: in this nature of studies, the authors performed
controlled environment experiments following well-established and reproducible
methodologies.

  * •

Publication Venues (RQ08): it identifies in which publications the selected
studies have been published, such as conferences, workshops, scientific
journals, etc., as the objective of inferring the maturity level of the topic
in the scientific community.

##  5 Threats to Validity

Regarding the development of Systematic Mappings, even if they have a well-
defined and reproducible methodology, the threats to the validity of the study
are inherent in the same [58]. In this regard, three types of threats have
been enumerated: construction, conduction and conclusion.

The first nature of threats – construction – refers to the definitions present
in the methodology, especially the search strings adapted for each database,
which may not represent a return of the same value for each database. To
mitigate this threat, individual search strings were built using the
peculiarities of each search engine.

The subsequent nature of threats – conduction – implies conducting the
classification process, which may have adopted a very generic or, on the other
hand, a very specific categorization schema, compromising the formation of
study groups. In this aspect, an iterative approach was sought, that is,
during the classification of articles, new categories were added to the
exclusion of others, requiring new classification cycles of previously
classified studies.

Finally, the conclusion nature brings with it biases from the authors applied
to the results, and may lead to convenient conclusions inclinations.
Therefore, we sought to use the theoretical basis of similar studies to
corroborate the results, thus reinforcing the conclusions obtained.

##  6 Filtering Process Execution

This section aims to describe how the filtering process of potentially
relevant studies obtained from the electronic databases presented in Table 4
of Section 4.2.1 was conducted.

The filtering process used in this work took place in eight distinct steps,
presented through Figure 2. Except for the Initial Research stage, which only
used the Inclusion Criteria in the screening, the others used groupings of the
Exclusion Criteria as guidelines for the removal of the initially selected
studies. The following are the steps taken as well as their driving
guidelines.

Figure 2: Study Filtering

  * •

Initial Search: this step was characterized by the insertion of the search
string (Section 4.2.2) into the chosen databases (Section 4.2.1), combined
with the composite filters parameterized according to the Inclusion (Section
4.3), with this, the search returned an amount of 2084 articles to be
filtered, grouped according to their respective databases, as illustrated in
Figure 2.

  * •

Filter by Title: at this stage of the filtering process, the titles of the
articles resulting from the Initial Search were analyzed. Exclusion Criteria
EC01 and EC02 were used for filtering (see Section 4.3), thus the initial
amount of 2084 articles was reduced to a total of 175, representing a
reduction of 91.6% of the volume of studies initially selected (Figure 2).

  * •

Filter by Abstract: responsible for the accuracy in choosing relevant studies
Systematic Mapping development, this step consisted of reading the abstracts
of all 175 resulting articles. Exclusion criteria EC03 and EC04 (see Section
4.3) were applied as filters, reducing the volume of studies by 74.28%, thus
reaching a total of 45 articles (Figure 2).

  * •

Studies Combination: as illustrated in Figure 2, the previous filtering steps
applied to each of the electronic databases. Labels that linked articles to
their original databases were removed, making them a single literary corpus,
thus allowing the execution of the subsequent steps.

  * •

Duplicate Removal: after article combination, the Exclusion Criteria EC05 was
applied, responsible for removing duplicates of the studies and/or their old
versions. Such filtering resulted in a reduction of 13.39% of the total number
of studies, thus resulting in a literary corpus of 39 articles (Figure 2).

  * •

Addition by Convenience: to increase the volume of studies analyzed, three
articles related to the theme were added for convenience and within the
Inclusion and Exclusion Criteria previously used, increasing by 7.7% of the
total or 42 articles (Figure 2). Such addition was made by the authors’
knowledge and through linked analysis of the references found in the articles
already analyzed (snowballing), thus following the recommendations of [58] for
selection.

  * •

Filter by Full Text: in this step, the articles from the previous stage were
read in full, applying the Exclusion Criteria EC06 and EC07 (Section 4.3)
resulting in the removal of 35.71% of the articles and generating an amount of
27 papers (Figure 2).

  * •

Work Selection: The final phase of the process resulted in the selection of 27
articles, named in this paper as Primary Studies, which are arranged in B.
Besides, through the strategies described in Section 4.4, the data were
extracted to obtain the answers to the Research Questions.

##  7 Results

This section presents the results obtained by executing the filtering process
described in Section 6 according to the outline discussed in Section 4. To
present quantitative data for each Research Question, we used tables
containing the Primary Study distribution, represented by their identifiers
listed in B, according to their respective answer categories.

###  7.1 RQ01: Taxonomy

Figure 3 introduces a classification scheme (or taxonomy) produced after
reading and examining the primary studies. The taxonomy structure has the
variables explored in the formulated research questions, as well as the
possible classifications (or values) for such variables found in the studies.

It is possible to observe that the second level of taxonomy was developed
based on the classification criteria of the studies covered in the development
of this work. The criteria are (1) devices that were used in the studies to
collect psychophysiological data; (2) the context in which the data collected
were used in the scope of the IDEs; (3) the types of data that were collected
by the devices used and (4) what the collected data is capable of indicating.

Figure 3: Taxonomy of psychophysiological data and IDEs usage.

Although taxonomy can bring an overview within the scope of this work and
facilitate the process of classifying and identifying studies, it is worth
noting that such a tool was developed to be applied in this study, therefore,
it is necessary to validate it and/or adaptation when used in the development
of similar works.

###  7.2 RQ02: Devices

Table 6 shows the results for RQ02. The main feature is that, although the
most used device was the EEG (29.63%, 8 of 27), there is no predominance of
the devices used to capture psychophysiological data. Greater adoption of EEG
could be explained by the improvement of data capture and mainly by the ease
of use and cost reduction of such devices, thus helping in the wider use in a
different context of studies [61]. Furthermore, some studies [61, 27, 28, 59]
indicate that electroencephalography, through brain waves from neural
activation, is suitable for identifying neural activities during the
development of different tasks, including those related to software
development, code understanding, and debugging strategies.

Table 6: List of devices used per study (RQ02) Device |  Studies |  Percentage |  Studies
---|---|---|---
N/A |  3 |  11,11% |  S15, S26, S27
Eye-tracker |  5 |  18,52% |  S02, S03, S12, S16, S20
EEG |  8 |  29,63% |  S01, S04, S11, S14, S18, S19, S23, S25
fMRI |  5 |  18,52% |  S05, S08, S17, S21, S22
fNIRS |  2 |  7,41% |  S13, S24
Device Combination |  4 |  14,81% |  S06, S07, S09, S10

Both eye-trackers and fMRI (Functional Magnetic Resonance Imaging) register
18.52% of the results, followed by fNIRS (Functional Near-Infrared
Spectroscopy), which was found in 7.41% of the studies.

The results also point out that some studies have chosen to use more than one
type of device. The combination of devices was identified in 14.81% of our
sample. We observed that often the use of EEG is combined with eye-trackers or
even complementary devices, such as measuring bracelets and chest bands
equipped with cardio-respiratory frequency sensors. This combination seeks to
complement the measurements in scenarios where eye-tracker and EEG cannot
indicate, in isolation, attributes such as stress and focused attention [6].

Given the nature of some of the Primary Studies, as well as the methodology
adopted, the use of devices for data collection was not found. Such a range of
studies represented 11.11% of our sample.

###  7.3 RQ03: Attributes

Regarding the analyzed attributes, Table 7 clarifies their distributions in
Primary Studies. There is a predominance (29.63%) of studies that had the
subject’s neural activity as an object of study. Such works used devices such
as EEGs, fNIRS, and fMRI to evaluate which brain regions were activated during
the execution of their experiments, in addition to relating certain semantics
to the observed brain activities [60].

Table 7: Distribution of attributes analyzed in each selected study (RQ03) Attribute |  Studies |  Percentage |  Studies
---|---|---|---
N/A |  3 |  11,11% |  S15, S26, S27
Fixation/Attention |  5 |  18,52% |  S02, S03, S12, S16, S20
Mental States |  7 |  25,93% |  S01, S04, S14, S18, S19, S23, S25
Neural Activity |  8 |  29,63% |  S05, S08, S11, S13, S17, S21, S22, S24
Attributes Combination |  2 |  7,40% |  S09, S10
Fixation/Attention, Neural Activity |  2 |  7,40% |  S06, S07

Studies that have lectured around _Mental States_ accounted for 25.93% of
Primary Studies. In these studies, we opted primarily for the use of EEGs for
the observation of mental states, given their intimate connection to brain
waves (Alpha, Beta, Gamma, Delta, and Theta), in addition to being a topic
that was very present in the specialized literature [70].

The _Fixation / Attention_ was the attribute studied in 18.52% of the studies
present in the literary corpus adopted in this systematic mapping. Usually,
the fixation was attributed by the studies to the ocular focus at points of
interest, predominantly measured by _Eye-trackers_ , see data presented in
Table 6 however, in cases where attention was the object studies,
complementary sensors were used concurrently.

In 7.40% of the works, a combined analysis of the attributes of _Fixation /
Attention_ and _Neural Activity_ was performed. In these studies, it was
observed that the authors’ focus permeated the understanding of source code,
for that, _Eye-trackers_ were used to collect data related to ocular
positioning and / or focus and fNIRS, to identify neural activation of brain
areas linked to learning and reading [16, 15].

Studies that analyzed more than three attributes in their experiments
correspond to 7.40% of the total. In these works, situations of greater
specificity were used when compared to others, such as the measurement of
_Fixation / Attention_ , _Mental States_ and complementary attributes
(cardiorespiratory frequency) during the execution of daily tasks by a
developer [27], or even the combination of _Mental States_ with attributes
from myoelectric signals (blink of the eye, temporomandibular tension), to use
them to compare the level of understanding of source code between developers
new and experienced [44].

Although 88.88% of the studies applied analyzes of attributes related to the
collected psychophysiological data, in 11.12% of them there was no analysis of
these attributes. This characteristic was due to the nature of these works, in
addition to the methodological process adopted in their development.

###  7.4 RQ04: Integration Support

This Research Question aimed to obtain as an answer what types of support the
IDEs used in the experiments provided for the integration of the collected
psychophysiological data, thus enabling its analysis in the context of
Software Engineering by the researchers. Figure 4 illustrates how the answers
for this Research Question were divided.

Figure 4: Data integration supporting by IDEs across studies (RQ04)

It is observed that, of the total of 27 studies analyzed, 11.1% of these made
a type of use of psychophysiological data, however, without the use of IDEs in
their experiments, as previously described in Section 4.5.1 and illustrated
through Figure 4. On the other hand, about 88.9% of the studies were
classified in the _Unsupported_ category, that is, they carried out
experiments involving IDEs and also the analysis of psychophysiological data.
Still, none of these studies specified mechanisms capable of integrating the
collected data and the Integrated Development Environments (IDEs), therefore,
the analyzes were carried out externally and using specialized tools.

###  7.5 RQ05: Purposes

In this item, the purposes adopted by the authors in the Primary Studies
regarding the use of psychophysiological data in the context of Software
Engineering are set out. Through the data elucidated in Table 8, it was found
that, in the majority (66.70%), the Primary Studies focused on analyzing the
_Program Comprehension and Debugging Strategy_ used by the developers in the
experiments performed.

Table 8: Purpose of data analysis in each study (RQ05) Purpose |  Studies |  Percentage |  Studies
---|---|---|---
Program Comprehension and Debugging Strategy |  18 |  66,70% |  S02, S03, S05, S06, S07, S08, S09, S12, S13, S14, S16, S17, S20, S21, S22, S25, S26, S27
Mental Workload |  4 |  14,81% |  S04, S11, S19, S24
Human-Computer Interaction |  2 |  7,41% |  S01, S23
Improvement in Productivity |  2 |  7,41% |  S10, S18
N/A |  1 |  3,70% |  S15

It was also observed that, in 14.81% of the Primary Studies, the focal purpose
of the work was the evaluation of the developer while he performed a series of
activities related to his context. It is worth noting that, despite the
possibility of evaluating Source Code Understanding and the debugging strategy
individually, the measurements related to these purposes have greater validity
when linked to the _Cognitive Load_ , according to surveys developed by others
authors [31].

Constituting 7.41% of the Primary Studies were those whose main purpose was
the search for the _Improvement in Productivity_ of the developers through the
use of psychophysiological data and IDEs.

Studies involving the development of _Brain Computer Interfaces_ also
corresponded to 7.41% of Primary Studies. This category includes the
construction of a device prototype for the control of home peripherals
utilizing brain waves [70] and the idealization of a portable EEG capable of
communicating with smartphones [2].

###  7.6 RQ06: Contributions

This section aims to present the main contributions identified in each of the
Primary Studies. Such contributions were extracted during the reading of the
full text (Section 6) and aimed to answer this Research Question. The texts
were synthesized and arranged through the following list, to allow a
structured reading, even if the source of the data was qualitative and with
subjective aspects.

  * •

S01 and S18: Enabled the acquisition and processing of neural activity signals
in real-time, in addition to the extraction of features to use such signals as
BCI (Brain-computer Interface).

  * •

S02: It brought a vision of how the display of code information is interpreted
by both novice and experienced programmers.

  * •

S03: The study indicated that, although the fixation time is longer in cases
where the code is presented in black and white, compared to the same section
with syntax highlighting, this difference does not is significant in terms of
understanding the code.

  * •

S04: In this study, it is worth highlighting two main contributions: (1) the
methodology for using EEG in the assessment of cognitive load and (2) the
evidence that more experienced programmers solve tasks more quickly,
generating a lower cognitive load in the same amount of time as beginners.

  * •

S05: As the main contribution of this study, the authors have brought a
mapping of brain activation during the clearance process.

  * •

S06: The authors proposed methodologies to relate the developer’s cognitive
load to the low quality of the source code.

  * •

S07: This study was important for bringing a review of how other authors are
using the Eye-tracker and fNIRS as tools for measuring the cognitive/mental
load of developers in terms of understanding the source code.

  * •

S08: Among the contributions of this study, two are of great value for the
development of this mapping: (1) it clarifies that the programming language
and the natural language are interpreted in different regions of the brain and
(2) with the increase in the programmer’s expertise, the programming language
begins to be interpreted as a natural language by your brain.

  * •

S09: The work brought as a contribution to the possibility of using
neurophysiological signals to assess the difficulty in programming activities,
in addition to bringing the possibility of developing tools to improve the
software development process.

  * •

S10: The authors brought a revision of great value around psychophysiological
data and their sensing devices. They also presented the techniques used to
analyze the data used and the algorithms for predicting similar cases.

  * •

S11: As a result, the authors have shown that with increasing cognitive load,
productivity degrades over time.

  * •

S12: Although an experiment was carried out in the context of the use of
programming languages, the study presented the participants’ eye movements as
a result, without drawing parallels with the literature to find patterns or
explanations for the identified patterns.

  * •

S13 and S14: The results are useful in understanding what happens in the
programmer’s brain when observing certain aspects (logic, code flow) during
the process of understanding programming languages. By comprehension of the
brain’s activation patterns, it is possible to create metrics for assessing
the skills of both novice and experienced programmers. It is still possible to
determine the factors that make teaching programming more effective in terms
of increasing cognitive ability.

  * •

S15: A working context-sensitive IDE has been developed and tested by the
authors.

  * •

S16 and S20: The authors exposed the state-of-the-art around the use of eye-
tracking in Software Engineering, in addition to advances in technology over
the years.

  * •

S17: Developed guidelines for establishing fMRI as a standard tool for
observing the understanding of source code and other related processes.

  * •

S21: As a contribution of the study, the authors showed that there is an
activation pattern associated with working memory and natural language
processing. As a contribution, the work brought a methodology for using fMRI
in the context of understanding programming languages.

  * •

S22: The authors identified that the use of semantic suggestions reduces brain
activation during the process of understanding the source code, in addition to
reproducing the studies of other authors and validating their results.

  * •

S23: The work in question brought the feasibility of developing a domestic
EEG, both for use as monitoring equipment and for BCI, given the possibility
of analyzing data in real-time. It also elucidated the mental states linked to
the brain wave spectrum.

  * •

S24: The present study contributed with the evidence that, during the teaching
process of object-oriented programming, visual IDEs such as BlueJ, reduce the
cognitive load, making it possible to infer a more facilitated learning
process. The authors also encourage studies on the use of IDEs and fNIRS.

  * •

S25: Although the work evaluated the behavior of the Theta and Alpha waves,
the authors did not bring a psychophysiological perspective of the cognitive
processes involved during the activities.

  * •

S26: He proposed a metric to assess cognitive complexity based on a graph of
method calls.

  * •

S27: The authors showed that the debugging methodologies used by developers
vary according to the features of the IDEs, as opposed to traditional forms of
debugging.

In addition to the results presented in the form of synthetic texts, the
metrics were arranged according to the categories to which the respective
contributions were classified, as shown in Table 9.

Table 9: Distribution of studies by type of Contribution (RQ06) Purpose |  Studies |  Percentage |  Studies
---|---|---|---
Metric |  10 |  37,04% |  S03, S04, S05, S10, S12, S13, S14, S21, S22, S26
Tool |  2 |  7,41% |  S09, S15
Model |  6 |  22,22% |  S01, S16, S18, S19, S20, S23
Method |  6 |  22,22% |  S02, S06, S07, S08, S24, S25
Process |  3 |  11,11% |  S11, S17, S27

There is a greater number (37.04%) of studies that contributed significantly
by providing some kind of _Metrics_ about the use of psychophysiological data
applied to the attributes listed in RQ03 and purposes of RQ05. Therefore, it
is observed that _Method_ and _Model_ presented the same frequencies (22.22%
each), bringing both contributions that could be applied to the definition of
an architectural model or experimental method.

The category _Process_ brought contributions (11.11%) to the definition of
processes that could be adopted concerning the collection, processing, and
analysis of psychophysiological data. Finally, only 2 studies (7.41%)
presented any tool that could be applied in the context of this work.

It is worth noting that none of the studies analyzed were classified in the
category Integration since they did not present any mechanisms capable of
supporting the integration of the data collected with the integrated
development environments or other tools used.

###  7.7 RQ07: Nature

In this Research Question, we sought to provide an overview of the
distribution of each selected study, using the criteria defined in Section
4.5.1. The results obtained by the process of conducting systematic mapping
are organized and arranged in Table 10.

Table 10: Distribution of studies by Nature (QP07) Nature |  Studies |  Percentage |  Studies
---|---|---|---
Controlled Experiment |  15 |  55,56% |  S02, S03, S04, S05, S06, S08, S09, S10, S11, S12, S13, S14, S21, S22, S25
Proposal of Solution |  7 |  25,93% |  S07, S15, S18, S23, S24, S26
Evaluation Research |  1 |  3,70% |  S19
Opinion Article |  4 |  14,81% |  S16, S17, S20, S27

Note that predominantly the selected studies (55.56%) are of nature
_Controlled Experiment_ , that is, studies in which the authors perform some
type of formatting experiment to validate an existing technique or confirm a
concept found in the literature. Next are the studies of nature _Proposal of
Solution_ , representing 25.93% of the total value. In this nature of the
studies, are listed those that the main objective was to present a solution to
an existing problem or an improved version of a solution already published,
through a proof of concept. The _Opinion Article_ articles 14.81%, brings
works that express an opinion of the authors on specific subjects, instigating
a discussion and opening an opportunity for new works. Finally, 3.70% of the
works were of a nature _Evaluation Research_ , where you can evaluate a tool,
methodology, or idea.

###  7.8 RQ08: Publication Venues

This Research Question aimed to quantify the venues in which the Primary
Studies were published. The data regarding the responses acquired through the
development of this study are shown in Table 11

Table 11: Distribution of venues where studies were published (RQ08) Venue |  Studies |  Percentage |  Studies
---|---|---|---
Conference |  16 |  59,26% |  S03, S04, S06, S07, S08, S09, S10, S11, S12, S13, S14, S15, S17, S21, S25, S26
Journal |  6 |  22,22% |  S01, S02, S16, S20, S23, S24
Workshop |  2 |  7,41% |  S18, S19
Symposium |  3 |  11,11% |  S05, S22, S27

It appears that in its majority (59.26%), the Primary Studies were published
in _Conferences_ related to the main area, indicating a greater incipience
about the theme of the selected works. Consequently, there are the works that
were published in _Journals_ (22.22%), featuring subjects consolidated in the
scientific community. Finally, works that have been published in smaller
events are presented, such as _Symposium_ (11.11%) and _Workshops_ (7.41%).

####  7.8.1 Publication Frequency

In addition to the results presented around the _Publication Venues_ , a
survey regarding the distribution of the published studies over the years was
also carried out, as quantified through Figure 5.

Figure 5: Publication of selected studies over the years

It is possible to verify that, during the last decade – period evaluated in
the development of this systematic mapping – there was an increase in the
number of publications regarding the use of psychophysiological data applied
in the context of Software Engineering.

##  8 Discussion and Challenges

Figure 6: Density of studies due to their nature, purpose and contributions.

Through Figure 6 a bubble chart is presented where the primary studies are
organized into three facets: contribution (left horizontal axis), purposes
(vertical axis), and nature (right horizontal axis). Such a chart brings a
combined view of the primary studies to identify the relationships between the
contributions of each study, their nature, and the purposes adopted. The heat
map illustrated in Figure 7, on the other hand, elucidates the density of
studies for each pair of answers to the research questions presented in this
work. Therefore, using these resources, it was possible to identify the
aspects discussed in the following paragraphs.

_Lack of Support for Integrating Psychophysiological Data:_ Studies that
carried out experiments using integrated development environments and
psychophysiological data showed that there are no native mechanisms to support
such types of data on the part of IDEs, nor through extensions that add such
functionality. In this respect, data were collected by proprietary tools that
communicated with the sensor devices, to be subsequently pre-processed and
analyzed. Such a lack of support for integration proves to be opportune for
the use of _Big Data_ tools that work with large volumes of data and/or
processing of data flows in real-time. Furthermore, the production of
infrastructure technology for the integration of psychophysiological data into
integrated development environments through extensible functionalities is
shown to be a necessity present in experimental studies applied to Software
Engineering.

_Source Code Understanding Experiments and Debugging Strategies Applied to
Software Engineering:_ Most of the authors of the studies reviewed in this
work adopted the main purpose of using psychophysiological data to assess
source code understanding. by the developers, in addition to their strategies
adopted for the debugging process. This preference is elucidated by the domain
of the studies analyzed, which usually sought applicability in the context of
Software Engineering. Analyzing Figure 7, it becomes evident that this type of
purpose was adopted in studies that used controlled experiments as the main
methodology.

Figure 7: Heatmap showing study density for each pair of responses to the
applied Research Questions.

Legend:
EEG: Electroencephalography – Eye: Eye-tracker – fMRI: functional Magnetic
Resonance Imaging – DevComb: Device Combination – fNIRS: functional Near-
Infrared Spectroscopy – NoDev: No Device – MentSta: Mental States – Fix/Att:
Fixation/Attention – Act: Neural Activity – Fix/Att,Act: Fixation/Attention,
Neural Activity – AttComb: Attributes Combination – NoAtt: No Attributes –
NoSupp: Not Supported – NoApp: Not Applicable – BCI: Brain-computer
Interaction – Compr,Dbg: Program Comprehension and Debugging Strategy –
CogLoad: Cognitive Load – ImprPrd: Productivity Improvement – OutCtx: Out of
Context – SolProp: Proposal of Solution – CtrlExp: Controlled Experiment –
OpiPpr: Opinion Article – EvalRsc: Evaluation Research – Jnl: Journal – Conf:
Conference – Symp: Symposium – Work: Workshop

_Data Analysis Using External Tools:_ Although studies have performed data
analysis to obtain the results and develop their discussions and conclusions,
there was a need for analysis external to the controlled environment of the
experiment, through the use of tools such as MATLAB, or proprietary solutions
developed by the suppliers of the sensor devices used in the experiments. This
characteristic was evident in the experiments in which integrated development
environments were used (see item NoSupp of Figure 7). In these experiments, in
addition to the data collected by the devices, it was demanded that the
metrics obtained in the IDEs were independently correlated to the
psychophysiological data. In the aspect presented, there is an opportunity for
the development of extensions to IDEs that enable the use of data analytics
techniques in a native way, thus allowing the extraction of insights around
psychophysiological data integrated to IDEs.

_Increasing of Interest in Psychophysiological Data Applied to Software
Engineering:_ It is possible to observe a greater presence of studies
published in conferences related to applied computing. This characteristic
suggests the incipience of this research topic, given the interest in the
development of works of this kind and the increase in publications in recent
years (Figure 5 in Section 7). Notwithstanding, the multidisciplinary
integration between Neuroscience and Software Engineering opens scope for the
production of diverse and innovative studies regarding the applicability of
psychophysiology combined with the software development process. This
phenomenon becomes observable through the plurality of contributions (Item
7.6) from the analyzed articles, as well as by the related works used (Item
3), which served as a theoretical basis during the production of this
systematic mapping.

##  9 Conclusions and Future Works

The development of this work provided the authors with a theoretical glimpse
of the current state of the art regarding the use of psychophysiological data
applied to Software Engineering, since its acquisition, processing, and
extraction of insights on the physical and psychological processes involved in
the production of software artifacts by the developer. It was possible to
observe that, in the majority, the authors opted for the use of
electroencephalography (29.63%), eye-tracking (18.52%) and functional magnetic
resonance imaging (18.52% ) in their experiments, this fact is corroborated by
the maturity of the use of equipment present in the literature, as well as the
reduction of costs and ease in the configuration, operation, and extraction of
data from the measurements.

An expressive number of studies were found that analyzed the principles of
cognitive load theory which, as well as the use of electroencephalography and
eye-tracking, demonstrated to be well consolidated in the specialized
literature, reinforcing the theoretical basis of the authors, especially in
experiments that aimed to understand how the developer understands the source
code and what strategies it uses during the debugging process (66.7%), as well
as in those that sought to assess the levels of the cognitive load presented
by developers during the execution of their tasks (14.81%).

A point that could be observed when reading the articles in the filtering and
selection stage, was the lack of consensus of the authors regarding possible
cognitive processes related to the developer’s activities, except for the
cognitive load, other concepts when applied in the context of Software
Engineering, they presented an ambiguous content in relation, as is the
example of neural activity, in which some studies conceptualized it as being
the intensity of electrical and/or hemodynamic activation in the brain, while
other studies presented this concept relating only with punctual areas of the
brain and assigning semantics to it.

In the aspect of data analysis, the authors mainly used external tools for
processing, extracting features, and obtaining insights about the data sets,
especially in studies of an experimental nature in which the authors applied
the manipulation of integrated development environments as part of the data
sources. In these studies, no ways of integrating data from IDEs with analysis
tools were presented, requiring the use of generic third-party software suites
for the manipulation, processing, and analysis, or tools provided by the
developers of the sensor devices used (88.9%). This factor proved to be an
opportunity for the application of multidisciplinary knowledge, such as the
use of Big Data tools to manipulate the volume of data and, real-time
processing or near real-time, extracting insights using machine learning
algorithms, making wearable and Internet of Things devices to expand data
collection outside a specific environment, enabling understand the
psychophysiological characteristics presented by the developer in his real
work environment, among other applications.

Through the opportunities observed during the production of this systematic
mapping, it is intended to develop, as future works, mechanisms that enable
the collection and processing of psychophysiological data from sensor devices,
as well as their integration into integrated development environments, in a
way to enable the generation of metadata that can be related to the events
generated within the IDEs. In possession of such data, it will still be
possible to apply machine learning techniques to obtain patterns that may
suggest potential failures in the production of software according to the
psychophysiological standard of each developer, enabling the creation of
intelligent and adaptable IDEs to the user, in addition to tactical resources
for prioritizing refactoring in software units.

##  10 Acknowledgments

This work was supported by the Coordenação de Aperfeiçoamento de Pessoal de
Nível Superior – Brasil (CAPES) under Grant 001; National Council for
Scientific and Technological Development (CNPq) under Grant 313285/2018-7.

## References

  * Dur [2016] , 2016. WAP: Understanding the Brain at Software Debugging. International Symposium on Software Reliability Engineering, ISSRE, 87–92.
  * Alshbatat et al. [2014] Alshbatat, A. I. N., Vial, P. J., Premaratne, P., Tran, L. C., 2014. EEG-based Brain-computer Interface for Automating Home Appliances. Journal of Computers 9 (9), 2159–2166.
  * Andreassi [2007] Andreassi, J. L., 2007. Psychophysiology: Human behavior and physiological response, 5th ed. Lawrence Erlbaum Associates Publishers, Mahwah, NJ, US.
  * Astromskis et al. [2017] Astromskis, S., Bavota, G., Janes, A., Russo, B., Di Penta, M., 2017. Patterns of developers behaviour: A 1000-hour industrial study. Journal of Systems and Software 132, 85–97.
URL http://dx.doi.org/10.1016/j.jss.2017.06.072

  * Bednarik [2012] Bednarik, R., 2012. Expertise-dependent visual attention strategies develop over time during debugging with multiple code representations. International Journal of Human Computer Studies 70 (2), 143–155.
URL http://dx.doi.org/10.1016/j.ijhcs.2011.09.003

  * Beelders and Du Plessis [2016] Beelders, T., Du Plessis, J. P., oct 2016. The influence of syntax highlighting on scanning and reading behaviour for source code. In: ACM International Conference Proceeding Series. Vol. 26-28-Sept. Association for Computing Machinery (ACM), pp. 1–10.
  * Bischoff et al. [2019a] Bischoff, V., Farias, K., Gonçales, L. J., Barbosa, J. L. V., 2019a. Integration of feature models: A systematic mapping study. Information and Software Technology 105, 209–225.
  * Bischoff et al. [2019b] Bischoff, V., Farias, K., Gonçales, L. J., Victória Barbosa, J. L., 2019b. Integration of feature models: A systematic mapping study. Information and Software Technology 105 (October 2016), 209–225.
  * Bischoff et al. [2021] Bischoff, V., Farias, K., Menzen, J. P., Pessin, G., 2021. Technological support for detection and prediction of plant diseases: A systematic mapping study. Computers and Electronics in Agriculture 181, 105922.
  * Carbonera et al. [2020] Carbonera, C. E., Farias, K., Bischoff, V., 2020. Software development effort estimation: a systematic mapping study. IET Software 14 (4), 328–344.
  * Crk and Kluthe [2014] Crk, I., Kluthe, T., 2014. Toward using alpha and theta brain waves to quantify programmer expertise. 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2014, 5373–5376.
  * de L. Xavier et al. [2019] de L. Xavier, M. S., Farias, K., Barbosa, J., Gonçales, L., Bishoff, V., 2019\. Umlcollab: A hybrid approach for collaborative modeling of uml models. In: Proceedings of the XV Brazilian Symposium on Information Systems. pp. 1–8.
  * de Oliveira [2012] de Oliveira, K. S. F., 2012. Empirical evaluation of effort on composing design models. Ph.D. thesis, Programa de Pós-Graduação em Informática of the Departamento de Informática ….
  * D’Avila et al. [2020] D’Avila, L. F., Farias, K., Barbosa, J. L. V., 2020. Effects of contextual information on maintenance effort: A controlled experiment. Journal of Systems and Software 159, 110443.
  * Fakhoury [2018] Fakhoury, S., 2018. Moving towards objective measures of program comprehension. ESEC/FSE 2018 - Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 936–939.
  * Fakhoury et al. [2018] Fakhoury, S., Ma, Y., Arnaoudova, V., Adesope, O., 2018. The effect of poor source code lexicon and readability on developers’ cognitive load. Proceedings - International Conference on Software Engineering, 286–296.
  * Farias [2010] Farias, K., 2010. Empirical evaluation of effort on composing design models. In: 2010 ACM/IEEE 32nd International Conference on Software Engineering. Vol. 2. IEEE, pp. 405–408.
  * Farias et al. [2011] Farias, K., Garcia, A., Lucena, C., 2011. Evaluating the effects of stability on model composition effort: an exploratory study. In: VIII Experimental Software Engineering Latin American Workshop collocated at XIV Iberoamerican Conference on Software Engineering, Rio de Janeiro. Citeseer.
  * Farias et al. [2012] Farias, K., Garcia, A., Lucena, C., 2012. Evaluating the impact of aspects on inconsistency detection effort: a controlled experiment. In: International Conference on Model Driven Engineering Languages and Systems. Springer, pp. 219–234.
  * Farias et al. [2014] Farias, K., Garcia, A., Lucena, C., 2014. Effects of stability on model composition effort: an exploratory study. Software & Systems Modeling 13 (4), 1473–1494.
  * Farias et al. [2010] Farias, K., Garcia, A., Whittle, J., 2010. Assessing the impact of aspects on model composition effort. In: Proceedings of the 9th International Conference on Aspect-Oriented Software Development. pp. 73–84.
  * Farias et al. [2015] Farias, K., Garcia, A., Whittle, J., Chavez, C. v. F. G., Lucena, C., 2015. Evaluating the effort of composing design models: a controlled experiment. Software & Systems Modeling 14 (4), 1349–1365.
  * Farias et al. [2013] Farias, K., Garcia, A., Whittle, J., Lucena, C., 2013. Analyzing the effort of composing design models of large-scale software in industrial case studies. In: International Conference on Model Driven Engineering Languages and Systems. Springer, pp. 639–655.
  * Farias et al. [2009] Farias, K., Nunes, I., Silva, V., Lucena, C., 2009. Mas-ml tool: um ambiente de modelagem de sistemas multi-agentes. In: V Workshop on Software Engineering for Agent-oriented Systems. pp. 1–12.
  * Fishburn et al. [2014] Fishburn, F. A., Norr, M. E., Medvedev, A. V., Vaidya, C. J., 2014. Sensitivity of fNIRS to cognitive state and load. Frontiers in Human Neuroscience 8 (1 FEB), 1–11.
  * Floyd et al. [2017] Floyd, B., Santander, T., Weimer, W., 2017. Decoding the Representation of Code in the Brain: An fMRI Study of Code Review and Expertise. Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering, ICSE 2017, 175–186.
  * Fritz et al. [2014] Fritz, T., Begel, A., Müller, S. C., Yigit-Elliott, S., Züger, M., 2014\. Using psycho-physiological measures to assess task difficulty in software development. Proceedings - International Conference on Software Engineering (1), 402–413.
  * Fritz and Muller [2016] Fritz, T., Muller, S. C., 2016. Leveraging Biometric Data to Boost Software Developer Productivity, 66–77.
  * Fucci et al. [2019] Fucci, D., Girardi, D., Novielli, N., Quaranta, L., Lanubile, F., 2019. A replication study on code comprehension and expertise using lightweight biometric sensors. IEEE International Conference on Program Comprehension 2019-May, 311–322.
URL http://arxiv.org/abs/1903.03426

  * Funke et al. [2013] Funke, G., Knott, B., Mancuso, V. F., Strang, A., Estepp, J., Menke, L., Brown, R., Dukes, A., Miller, B., 2013. Evaluation of subjective and EEG-based measures of mental workload. Communications in Computer and Information Science 373 (PART I), 412–416.
  * Goncales et al. [2019] Goncales, L., Farias, K., Da Silva, B., Fessler, J., 2019. Measuring the cognitive load of software developers: A systematic mapping study. IEEE International Conference on Program Comprehension 2019-May (March), 42–52.
  * Gonçales et al. [2019a] Gonçales, L. J., Farias, K., De Oliveira, T. C., Scholl, M., 2019a. Comparison of software design models: An extended systematic mapping study. ACM Computing Surveys 52 (3), 1–41.
  * Gonçales et al. [2019b] Gonçales, L. J., Farias, K., Oliveira, T. C. D., Scholl, M., 2019b. Comparison of software design models: an extended systematic mapping study. ACM Computing Surveys (CSUR) 52 (3), 1–41.
  * Gonçales et al. [2015] Gonçales, L. J., Farias, K., Scholl, M., Roberto Veronez, M., de Oliveira, T. C., 2015. Comparison of design models: A systematic mapping study. International Journal of Software Engineering and Knowledge Engineering 25 (09n10), 1765–1769.
  * Gonçalves et al. [2010] Gonçalves, E. J., Farias, K., Cortés, M. I., da Silva, V. T., Feitosa, R. G., 2010. Modelagem de organizações de agentes inteligentes: uma extensão da mas-ml tool. AUTOSOFT 2010, 1.
  * Gonçalves et al. [2015] Gonçalves, E. J. T., Cortés, M. I., Campos, G. A. L., Lopes, Y. S., Freire, E. S., da Silva, V. T., de Oliveira, K. S. F., de Oliveira, M. A., 2015\. Mas-ml 2.0: Supporting the modelling of multi-agent systems with different agent architectures. Journal of Systems and Software 108, 77–109.
  * Gonçalves et al. [2011] Gonçalves, E. J. T., Farias, K., Cortés, M. I., Feijó, A. R., Oliveira, F. R., da Silva, V. T., 2011. Mas-ml tool-a modeling environment for multi-agent systems. In: ICEIS (2). pp. 192–197.
  * Guarnera et al. [2018] Guarnera, D. T., Bryant, C. A., Mishra, A., Maletic, J. I., Sharif, B., 2018. iTrace: Eye tracking infrastructure for development environments. Eye Tracking Research and Applications Symposium (ETRA), 2015–2017.
  * Gui et al. [2019] Gui, Q., Ruiz-Blondet, M. V., Laszlo, S., Jin, Z., 2019. A survey on brain biometrics. ACM Computing Surveys 51 (6), 1–38.
  * Guimarães et al. [2010] Guimarães, E., Garcia, A., Farias, K., 2010. Analyzing the effects of aspect properties on model composition effort: A replicated study. In: Proceedings of the 15th International Workshop on Aspect-Oriented Modeling (October 2010). Citeseer.
  * Haag et al. [2004] Haag, A., Goronzy, S., Schaich, P., Williams, J., 2004. Emotion recognition using bio-sensors: First steps towards an automatic system. In: André, E., Dybkjær, L., Minker, W., Heisterkamp, P. (Eds.), Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science). Vol. 3068. Springer Berlin Heidelberg, Berlin, Heidelberg, pp. 36–48.
  * Hejmady and Narayanan [2012] Hejmady, P., Narayanan, N. H., 2012. Visual attention switching patterns of programmers debugging with an IDE. ETRA ’12 Proceedings of the Symposium on Eye Tracking Research and Applications, 197–200.
  * Ikutani and Uwano [2014] Ikutani, Y., Uwano, H., 2014. Brain activity measurement during program comprehension with NIRS. 2014 IEEE/ACIS 15th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2014 - Proceedings, 1–6.
  * Lee et al. [2016] Lee, S., Matteson, A., Hooshyar, D., Kim, S., Jung, J., Nam, G., Lim, H., 2016. Comparing Programming Language Comprehension between Novice and Expert Programmers Using EEG Analysis. Proceedings - 2016 IEEE 16th International Conference on Bioinformatics and Bioengineering, BIBE 2016, 350–355.
  * Li and Lu [2009] Li, M., Lu, B. L., 2009. Emotion classification based on gamma-band EEG. In: Proceedings of the 31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society: Engineering the Future of Biomedicine, EMBC 2009. pp. 1323–1326.
  * Li and Jain [2015] Li, S. Z., Jain, A. K. (Eds.), 2015. Encyclopedia of Biometrics, Second Edition. Springer US.
URL https://doi.org/10.1007/978-1-4899-7488-4

  * Lin et al. [2016] Lin, Y. T., Wu, C. C., Hou, T. Y., Lin, Y. C., Yang, F. Y., Chang, C. H., 2016. Tracking Students’ Cognitive Processes during Program Debugging-An Eye-Movement Approach. IEEE Transactions on Education 59 (3), 175–186.
  * Logozzo et al. [2012] Logozzo, F., Barnett, M., Fandrich, M., Cousot, P., Cousot, R., 2012. A semantic integrated development environment. SPLASH’12 - Proceedings of the 2012 ACM Conference on Systems, Programming, and Applications: Software for Humanity, 15–16.
  * Lucas et al. [2017] Lucas, E. M., Oliveira, T. C., Farias, K., Alencar, P. S., 2017. Collabrdl: A language to coordinate collaborative reuse. Journal of Systems and Software 131, 505–527.
  * Manica et al. [2018] Manica, M., Farias, K., Gonçales, L., Bischoff, V., da Silva, B. C., Guimarães, E. T., 2018. Effects of model composition techniques on effort and affective states: A controlled experiment (s). In: SEKE. pp. 304–303.
  * McLyndon et al. [2019] McLyndon, S. d. L., Farias, K., Barbosa, J., Gonçales, L., Bishoff, V., 2019\. Umlcollab: Uma abordagem híbrida para modelagem colaborativa de modelos uml. In: Anais do XV Simpósio Brasileiro de Sistemas de Informação. SBC, pp. 231–238.
  * Menzen et al. [2020] Menzen, J. P., Farias, K., Bischoff, V., 2020. Using biometric data in software engineering: a systematic mapping study. Behaviour & Information Technology, 1–23.
  * Neto et al. [2019] Neto, R. A. C., Farias, K., Bischoff, V., Gonçales, L. J., 2019. Supporting collaborative modelling in uml class diagrams. Revista Brasileira de Computação Aplicada 11 (3), 39–46.
  * Obaidellah et al. [2018] Obaidellah, U., Al Haek, M., Cheng, P. C., 2018. A survey on the usage of eye-Tracking in computer programming. ACM Computing Surveys 51 (1), 1–58.
  * Oliveira et al. [2018] Oliveira, A., Bischoff, V., Gonçales, L. J., Farias, K., Segalotto, M., 2018\. Brcode: An interpretive model-driven engineering approach for enterprise applications. Computers in Industry 96, 86–97.
  * Oliveira et al. [2008] Oliveira, K., Garcia, A., Whittle, J., 2008. On the quantitative assessment of class model compositions: An exploratory study. 1th ESMDE at MODELS.
  * Peitek [2018] Peitek, N., 2018. A neuro-cognitive perspective of program comprehension. Proceedings - International Conference on Software Engineering, 496–499.
  * Petersen et al. [2015] Petersen, K., Vakkalanka, S., Kuzniarz, L., 2015. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology 64, 1–18.
URL http://dx.doi.org/10.1016/j.infsof.2015.03.007

  * Radevski et al. [2015] Radevski, S., Hata, H., Matsumoto, K., 2015. Real-time monitoring of neural state in assessing and improving software developers’ productivity. Proceedings - 8th International Workshop on Cooperative and Human Aspects of Software Engineering, CHASE 2015, 93–96.
  * RD et al. [1997] RD, L., GR, F., PM, C., RJ, D., 1997. Neural activation during selective attention to subjective emotional responses. Neuroreport 8 (18), 3969.
  * Rostami et al. [2015] Rostami, S., Shenfield, A., Sigurnjak, S., Fakorede, O., 2015. Evaluation of Mental Workload and Familiarity in Human Computer Interaction with Integrated Development Environments using Single-Channel EEG. In: Psychology of Programming Interest Group 2015 - 26th Annual Workshop, 2015. No. July. pp. 7–22.
  * Sharafi et al. [2015] Sharafi, Z., Soh, Z., Guéhéneuc, Y. G., 2015. A systematic literature review on the usage of eye-tracking in software engineering. Information and Software Technology 67, 79–107.
URL http://dx.doi.org/10.1016/j.infsof.2015.06.008

  * Siegmund et al. [2014] Siegmund, J., Kästner, C., Apel, S., Parnin, C., Bethmann, A., Leich, T., Saake, G., Brechmann, A., 2014. Understanding understanding source code with functional magnetic resonance imaging. Proceedings - International Conference on Software Engineering (1), 378–389.
  * Siegmund et al. [2017] Siegmund, J., Peitek, N., Parnin, C., Apel, S., Hofmeister, J., Kästner, C., Begel, A., Bethmann, A., Brechmann, A., 2017. Measuring neural efficiency of program comprehension. Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, 140–150.
  * Snipes et al. [2015] Snipes, W., Murphy-Hill, E., Fritz, T., Vakilian, M., Damevski, K., Nair, A. R., Shepherd, D., 2015. A Practical Guide to Analyzing IDE Usage Data. The Art and Science of Analyzing Software Data, 85–138.
  * Solovey et al. [2015] Solovey, E. T., Afergan, D., Peck, E. M., Hincks, S. W., Jacob, R. J., 2015. Designing implicit interfaces for physiological computing: Guidelines and lessons learned using fNIRS. ACM Transactions on Computer-Human Interaction 21 (6), 35:1–35:27.
  * Souza et al. [2020] Souza, T. V. d., Farias, K., Bischoff, V., 2020. Big data analytics applied in supply chain management: A systematic mapping study. In: XVI Brazilian Symposium on Information Systems. pp. 1–8.
  * Sweller [1988] Sweller, J., 1988. Cognitive load during problem solving: Effects on learning. Cognitive Science 12 (2), 257–285.
URL http://www.sciencedirect.com/science/article/pii/0364021388900237

  * Sweller et al. [2011] Sweller, J., Ayres, P., Kalyuga, S., 2011. Cognitive Load Theory, 1st Edition. Springer, London.
URL http://www.springer.com/series/8640

  * Szu et al. [2013] Szu, H., Hsu, C., Moon, G., Yamakawa, T., Tran, B. Q., Jung, T. P., Landa, J., 2013\. Smartphone Household Wireless Electroencephalogram Hat. Applied Computational Intelligence and Soft Computing 2013, 1–8.
  * Uysal [2016] Uysal, M. P., 2016. Evaluation of learning environments for object-oriented programming: measuring cognitive load with a novel measurement technique. Interactive Learning Environments 24 (7), 1590–1609.
  * Veltman and Gaillard [1998] Veltman, J. A., Gaillard, A. W., 1998. Physiological workload reactions to increasing levels of task difficulty. Ergonomics 41 (5), 656–669.
  * Vieira and Farias [2020a] Vieira, R. D., Farias, K., 2020a. Cognide: A psychophysiological data integrator approach for visual studio code. In: Proceedings of the 34th Brazilian Symposium on Software Engineering. pp. 393–398.
  * Vieira and Farias [2020b] Vieira, R. D., Farias, K., 2020b. Usage of psychophysiological data as an improvement in the context of software engineering: A systematic mapping study. In: XVI Brazilian Symposium on Information Systems. pp. 1–8.
  * Wieringa et al. [2006] Wieringa, R., Maiden, N., Mead, N., Rolland, C., 2006. Requirements engineering paper classification and evaluation criteria: A proposal and a discussion. Requirements Engineering 11 (1), 102–107.
  * Yeh et al. [2017] Yeh, M. K., Gopstein, D., Yan, Y., Zhuang, Y., 2017. Detecting and comparing brain activity in short program comprehension using EEG. Proceedings - Frontiers in Education Conference, FIE 2017-Octob (1444827), 1–5.
  * Yin et al. [2010] Yin, M., Li, B., Tao, C., 2010. Using cognitive easiness metric for program comprehension. 2nd International Conference on Software Engineering and Data Mining, SEDM 2010, 134–139.
URL
http://www.scopus.com/inward/record.url?eid=2-s2.0-77956547690&partnerID=40&md5=65cd721fb8d7bf99f56d393b4dba2b53

  * Zayour and Hajjdiab [2013] Zayour, I., Hajjdiab, H., 2013. How much integrated development environments (IDEs) improve productivity? Journal of Software 8 (10), 2425–2431.
  * Zayour and Hamdar [2016] Zayour, I., Hamdar, A., 2016. A qualitative study on debugging under an enterprise IDE. Information and Software Technology 70, 130–139.
URL http://dx.doi.org/10.1016/j.infsof.2015.10.010

##  Appendix A Search String per Database

Database | Search String
---|---
ACM Digital Library |  acmdlTitle: ( +(eeg fmri electroencephalography neural brain cognitive psychometric bioinformatic “functional magnetic resonance imaging”) +(software developer programmer professional) +(ide code editor “integrated development environment” “software engineering”)) OR recordAbstract: ( +(eeg fmri electroencephalography neural brain cognitive psychometric bioinformatic “functional magnetic resonance imagin”) +(software developer programmer professional) +(ide code editor “integrated development environment” “software engineering”))
IEEE Xplore |  (sofware OR developer OR programmer OR professional OR “software engineering”) AND (eeg OR fmri OR electroencephalography OR neural OR brain OR cognitive OR psychometric OR bioinformatic OR “functional magnetic resonance imaging”) AND (ide OR code OR editor OR “integrated development environment”)
Science Direct |  ({“software engineering”} OR developer OR programmer OR code) AND (IDE ) AND (eeg OR Fmri OR cognitive OR bci)
Scopus |  ALL ( ( “software engineering” OR developer OR programmer OR code ) AND ( ide OR “integrated development environment” ) AND ( eeg OR fmri OR cognitive OR bci ) ) AND PUBYEAR >2008 AND ( LIMIT-TO ( SUBJAREA , “COMP” ) OR LIMIT-TO ( SUBJAREA , “ENGI” ) ) AND ( LIMIT-TO ( LANGUAGE , “English” ) )
Springer Link |  (“integrated development environment” OR IDE) AND (“software engineering” OR “software development” OR programmer OR developer) AND (bci OR eeg OR fmri OR cognitive)
DBLP |  IDE—“integrated development environment”
Semantic Scholar |  (IDE OR “integrated development environment”)

##  Appendix B Selected Studies

  1. S01

Alshbatat et al. [2]

  2. S02

Bednarik [5]

  3. S03

Beelders and Du Plessis [6]

  4. S04

Crk and Kluthe [11]

  5. S05

Dur [1]

  6. S06

Fakhoury et al. [16]

  7. S07

Fakhoury [15]

  8. S08

Floyd et al. [26]

  9. S09

Fritz et al. [27]

  10. S10

Fritz and Muller [28]

  11. S11

Funke et al. [30]

  12. S12

Hejmady and Narayanan [42]

  13. S13

Ikutani and Uwano [43]

  14. S14

Lee et al. [44]

  15. S15

Logozzo et al. [48]

  16. S16

Obaidellah et al. [54]

  17. S17

Peitek [57]

  18. S18

Radevski et al. [59]

  19. S19

Rostami et al. [61]

  20. S20

Sharafi et al. [62]

  21. S21

Siegmund et al. [63]

  22. S22

Siegmund et al. [64]

  23. S23

Szu et al. [70]

  24. S24

Uysal [71]

  25. S25

Yeh et al. [76]

  26. S26

Yin et al. [77]

  27. S27

Zayour and Hamdar [79]