# A note on Tsuji’s criterion
for numerical triviality

Shigetaka FUKUDA
Faculty of Education, Gifu Shotoku Gakuen University 2020 Mathematics Subject
Classification: 14C20Key words and phrases: numerically trivial

###### Abstract

In this study, we give an alternative and elementary proof to Tsuji’s
criterion for a Cartier divisor to be numerically trivial.

##  1 Introduction

In this article every algebraic variety is proper over the field of complex
numbers $\mathbf{C}$.

In 1970’s Iitaka ([1]) initiated the classification theory of higer
dimensional algebraic varieties by using the pluricanonical systems. In 1980’s
Mori ([2]) deepened the Iitaka theory by cutting off the subvarieties of
elliptic type.

In [3], Tsuji gave an interesting and useful criterion for a Cartier divisor
to be numerically trivial:

######  Theorem 1 (Tsuji [3, Lemma 5.1], cf. Bauer et al. [4, Theorem 2.4]).

Let $f:M\to B$ a surjective morphism between complete varieties. Let $L$ be a
nef Cartier divisor on $M$, $W$ some subvariety of $M$ such that $f(W)=B$ and
$B_{0}$ a subset of $B$ which is a union of countably many proper Zariski-
closed subsets. Assume that:

(1)

for some $b\in B$, $(L,C)=0$ for every curve $C$ on $f^{-1}(b);$

(2)

$(L,C)=0$ for every irreducible curve $C$ on $W$ such that $f(C)\nsubseteq
B_{0}.$

Then $L$ is numerically trivial.

Tsuji’s criterion for numerical triviality is one of the basic tools to
decompose every algebraic variety into the varieties of elliptic type, of
parabolic type and of hyperbolic type by cutting off the varieties of
parabolic type. (See Ambro ([5], [6]) and Fukuda ([7]).)

Tsuji’s proof ([3]) is analytic and the proof ([4]) by Th. Bauer, F. Campana,
Th. Eckl, S. Kebekus, Th. Peternell, S. Rams, T. Szemberg and L. Wotzlaw is
algebraic.

In this research note we give an alternative and elementary proof to Tsuji’s
criterion (Theorem 1). The argument (see Subcase 7), which uses the following
corollary of the Hodge index theorem, due to Bauer et al. is essential:

######  Lemma 1 (cf. [4, Proposition 2.5]).

Let $f:M\to B$ a surjective morphism from a complete surface $M$ to a complete
curve $B$. Let $L$ be a nef Cartier divisor on $M$. Assume that:

(1)

for some $b\in B$, $(L,C)=0$ for every curve $C$ on $f^{-1}(b);$

(2)

$(L,W)=0$ for some irreducible curve $W$ on $M$ such that $f(W)=B$.

Then $L$ is numerically trivial.

######  Remark 1.

In the statement of Lemma 1, the condition (1) immediately implies that $L$ is
numerically trivial on every general fiber of the morphism $f$, by considering
the flattening. By the normalization, the Stein factorization and the
desingularization, the article ([4, Proposition 2.5]), for an algebraically
fibered surface, implies the assertion of Lemma 1.

##  2 Elementary proof of Main Theorem 1

###### Proof.

We prove the assertion by induction on $(\dim M,\dim B)$.

First we take a commutative diagram Figure 1:

$\begin{CD}M^{\prime}@>{\text{$g$}}>{}>B^{\prime}\\\
@V{\text{$\mu$}}V{}V@V{}V{\text{$\nu$}}V\\\ M@>{\text{$f$}}>{}>B\\\ \end{CD}$
Figure 1: The Stein factorization for $f$

with the following properties:

(1)

$M^{\prime}$ and $B^{\prime}$ are nonsingular projective varieties;

(2)

$\mu$ is a birational morphism;

(3)

$\nu$ is a generically finite morphism;

(4)

$g$ is a morphism with only connected fibers.

There exists some irreducible component $W^{\prime}$ of $\mu^{-1}(W)$ such
that $g(W^{\prime})=B^{\prime}$. We set $L^{\prime}:=\mu^{*}L$.

The locus $g(\bigcup\\{C^{\prime}|C^{\prime}$ is an irreducible curve on
$M^{\prime}$, $g(C^{\prime})$ is a point and the intersection number
$(L^{\prime},C^{\prime})>0\\})$ is included in a union of at most countably
many proper Zariski-closed subsets of $B^{\prime}$ (see Proposition 1). Thus
we obtain a union $B_{0}^{\prime}\supseteq\nu^{-1}(B_{0})$ of countably many
proper Zariski-closed subsets of $B^{\prime}$ with the following two
properties:

(1)

$L^{\prime}$ is numerically trivial on every fiber of $g$ over
$B^{\prime}\setminus B_{0}^{\prime}$;

(2)

$(L^{\prime},C^{\prime}_{W})=0$ for every irreducible curve $C^{\prime}_{W}$
on $W^{\prime}$ such that $g(C^{\prime}_{W})\nsubseteq B_{0}^{\prime}$.

It suffices to prove that $(L^{\prime},C^{\prime})=0$ for every irreducible
curve $C^{\prime}$ on $M^{\prime}$. We fix an irreducible curve $C^{\prime}$
on $M^{\prime}$

######  Case 1.

$g(C^{\prime})\nsubseteq B_{0}^{\prime}$. This case divides into Subcases 1
and 2.

######  Subcase 1.

$g(C^{\prime})\nsubseteq B_{0}^{\prime}$ and $g(C^{\prime})$ is a point.

We have $(L^{\prime},C^{\prime})=0$ from (1).

######  Subcase 2.

$g(C^{\prime})\nsubseteq B_{0}^{\prime}$ and $g(C^{\prime})$ is a curve. This
subcase divides into Subcases 3 and 7.

######  Subcase 3.

$g(C^{\prime})\nsubseteq B_{0}^{\prime}$, $g(C^{\prime})$ is a curve and $\dim
B=1$. This subcase divides into Subcases 4, 5 and 6.

We note that $g(W_{1})=g(C^{\prime})=B^{\prime}$ for some irreducible curve
$W_{1}$ on $W^{\prime}$.

######  Subcase 4.

$g(C^{\prime})\nsubseteq B_{0}^{\prime}$, $g(C^{\prime})$ is a curve, $\dim
B=1$ and $\dim M=1$.

$M^{\prime}=W^{\prime}=W_{1}=C^{\prime}$. Thus $(L^{\prime},C^{\prime})=0$.

######  Subcase 5.

$g(C^{\prime})\nsubseteq B_{0}^{\prime}$, $g(C^{\prime})$ is a curve, $\dim
B=1$ and $\dim M=2$.

Because $(L^{\prime},W_{1})=0$ from (2), Lemma 1 implies that $L^{\prime}$ is
numerically trivial and thus $(L^{\prime},C^{\prime})=0$.

######  Subcase 6.

$g(C^{\prime})\nsubseteq B_{0}^{\prime}$, $g(C^{\prime})$ is a curve, $\dim
B=1$ and $\dim M\geqq 3$.

Because the codimension $\mathop{\mathrm{codim}}(W_{1}\cup
C^{\prime},M^{\prime})\geqq 2$, we have an irreducible hyperplane section $H$
of $M^{\prime}$ that includes $W_{1}$ and $C^{\prime}$ (see Proposition 2).
Then $L^{\prime}\mid_{H}$ is numerically trivial from the induction
hypothesis. Consequently $(L^{\prime},C^{\prime})=0$.

######  Subcase 7 (cf. [4, 2.1.2]).

$g(C^{\prime})\nsubseteq B_{0}^{\prime}$, $g(C^{\prime})$ is a curve and $\dim
B\geqq 2$.

Let $S:=\\{S_{i}\\}$ be the set of irreducible components of
$g^{-1}(g(C^{\prime}))$. We note that $g(W_{1})=g(C^{\prime})$ for some
irreducible curve $W_{1}$ on $W^{\prime}$. Thus $L^{\prime}\mid_{S_{1}}$ is
numerically trivial for some $S_{1}\in S$ such that $S_{1}\supseteq W_{1}$
from the property (2) and from the induction hypothesis.

If $g(\cup_{m\neq 1}S_{m})=g(C^{\prime})$, then $g(S_{1}\cap(\cup_{m\neq
1}S_{m}))=g(C^{\prime})$ from the connectedness of fibers of $g$ and therefore
$g(S_{1}\cap S_{2})=g(C^{\prime})$ for some $S_{2}\in S$.

Thus $\dim g(\cup_{m\neq 1}S_{m})\leqq 0$ or $g(S_{1}\cap
S_{2})=g(C^{\prime})$.

If $g(S_{1}\cap S_{2})=g(C^{\prime})$ and $g(\cup_{m\neq
1,2}S_{m})=g(C^{\prime})$, then $g((S_{1}\cup S_{2})\cap(\cup_{m\neq
1,2}S_{m}))=g(C^{\prime})$ from the connectedness of fibers of $g$ and
therefore $g((S_{1}\cup S_{2})\cap S_{3})=g(C^{\prime})$ for some $S_{3}\in
S$. From this argument, we obtain the following properties:

(1)

$g(S_{1})=g(S_{2})=\cdots=g(S_{k})=g(C^{\prime})$;

(2)

$g((S_{1}\cup S_{2}\cup\cdots\cup S_{i-1})\cap S_{i})=g(C^{\prime})$ for all
$i$ with $1\leqq i\leqq k$;

(3)

$\dim g(\cup_{i\neq 1,2,\cdots k}S_{i})\leqq 0.$

The fact that $L^{\prime}\mid_{S_{1}}$ is numerically trivial and that
$g(S_{1}\cap S_{2})=g(C^{\prime})$ implies that $L^{\prime}\mid_{S_{2}}$ is
numerically trivial from the induction hypothesis. The fact that
$L^{\prime}\mid_{S_{1}\cup S_{2}}$ is numerically trivial and that
$g((S_{1}\cup S_{2})\cap S_{3})=g(C^{\prime})$ implies that
$L^{\prime}\mid_{S_{3}}$ is numerically trivial from the induction hypothesis.
This argument implies that $L^{\prime}\mid_{S_{1}\cup S_{2}\cup\cdots\cup
S_{k}}$ is numerically trivial. Because $\dim g(\cup_{i\neq 1,2,\cdots
k}S_{i})\leqq 0$, we have that $C^{\prime}\subseteq S_{1}\cup
S_{2}\cup\cdots\cup S_{k}$. Consequently $(L^{\prime},C^{\prime})=0$.

######  Case 2.

$g(C^{\prime})\subseteq B_{0}^{\prime}$. This case divides into Subcases 8 and
11.

######  Subcase 8.

$g(C^{\prime})\subseteq B_{0}^{\prime}$ and $\dim M=2$. This subcase divides
into Subcases 9 and 10.

######  Subcase 9.

$g(C^{\prime})\subseteq B_{0}^{\prime}$, $\dim M=2$ and $\dim B=1$.

Lemma 1 implies that $L^{\prime}$ is numerically trivial and thus
$(L^{\prime},C^{\prime})=0$.

######  Subcase 10.

$g(C^{\prime})\subseteq B_{0}^{\prime}$, $\dim M=2$ and $\dim B=2$.

Because $W^{\prime}=M^{\prime}$, we have that $(L^{\prime},H)=0$ for an
irreducible hyperplane section $H$ of $M^{\prime}$ from the property (2) of
the divisor $L^{\prime}$. The Hodge index theorem implies that $L^{\prime}$ is
numerically trivial. Thus $(L^{\prime},C^{\prime})=0$.

######  Subcase 11.

$g(C^{\prime})\subseteq B_{0}^{\prime}$ and $\dim M\geqq 3$.

Because the codimension $\mathop{\mathrm{codim}}(C^{\prime},M^{\prime})\geqq
2$, there exists an irreducible hyperplane section $H$ of $M^{\prime}$ that
includes $C^{\prime}$ (see Proposition 2). We may assume that $g(H)\nsubseteq
B_{0}^{\prime}$. Note that, from Case 1, $(L,C^{\prime\prime})=0$ for every
irreducible curve $C^{\prime\prime}$ on $H$ such that
$g(C^{\prime\prime})\nsubseteq B_{0}^{\prime}$. Thus $L^{\prime}\mid_{H}$ is
numerically trivial from the induction hypothesis. Consequently
$(L^{\prime},C^{\prime})=0$.

∎

##  3 Appendix

In this appendix, we state two elementary propositions and their proofs, which
are well known to the experts, for the readers’ convinience.

######  Proposition 1.

Let $f\colon M\to B$ be a surjective morphism between projective varieties and
$L$ a nef Cartier divisor on $M$. We assume that, for some $b\in B$, the
intersection number $(L,C)=0$ for every irreducible curve $C$ on $f^{-1}(b)$.

Then the locus $f(\bigcup\\{C|C$ is an irreducible curve on $M$, $f(C)$ is a
point and the intersection number $(L,C)>0\\})$ is included in a union of at
most countably many proper Zariski-closed subsets of $B$.

###### Proof.

There exists some ample divisor $A$ on $B$. Assume that $C$ is an irreducible
curve on $M$ such that $(f^{*}A,C)=0$ (i.e. $f(C)$ is a point) and that
$(L,C)>0$. There exists some irreducible component $W$ of the universal scheme
for the Hilbert scheme $\mathop{\mathrm{Hilb}}(M)$ of $M$ such that $W$
includes $p_{2}^{-1}([C])$ where $[C]$ is the point
($\in\mathop{\mathrm{Hilb}}(M)$) which represents the subscheme $C$ of $M$ and
such that the diagram Figure 2 holds:

$\begin{CD}W@>{\text{embedding}}>{}>M\times\mathop{\mathrm{Hilb}}(M)@>{\text{$p_{2}$}}>{}>\mathop{\mathrm{Hilb}}(M)\\\
@V{}V{\text{$p_{1}$}}V\\\ M\end{CD}$ Figure 2: The deformation of $C$ in $M$

with the projections $p_{1}$ and $p_{2}$ and with the property that $\dim
W=\dim p_{2}(W)+1$. We set $T:=p_{2}(W)$.

First we consider the normalization $n_{1}\colon W_{n}\to W$, $n_{2}\colon
T_{n}\to T$ and $n_{3}\colon W_{n}\to T_{n}$ of the morphism $p_{2}|_{W}\colon
W\to T$.

Next consider the Stein factorization $W_{n}\stackrel{{\scriptstyle
s_{1}}}{{\to}}T^{\prime}\stackrel{{\scriptstyle s_{2}}}{{\to}}T_{n}$ of the
morphism $n_{3}\colon W_{n}\to T_{n}$.

Lastly consider the flattening $f_{1}\colon W^{\prime\prime}\to W_{n}$,
$f_{2}\colon T^{\prime\prime}\to T^{\prime}$ and $f_{3}\colon
W^{\prime\prime}\to T^{\prime\prime}$ of the morphism $s_{1}\colon W_{n}\to
T^{\prime}$, where the morphism $f_{2}$ is birational and the variety
$T^{\prime\prime}$ is nonsingular. We note that the morphism $f_{3}\colon
W^{\prime\prime}\to T^{\prime\prime}$ is flat and with only connected fibers.

We put $h\colon=n_{1}f_{1}$.

Thus we have the commutative diagram Figure 3:

$\begin{CD}W^{\prime\prime}@>{\text{$f_{3}$}}>{}>T^{\prime\prime}\\\
@V{\text{$f_{1}$}}V{}V@V{}V{\text{$f_{2}$}}V\\\
W_{n}@>{\text{$s_{1}$}}>{}>T^{\prime}\\\ \Big{\|}@V{}V{\text{$s_{2}$}}V\\\
W_{n}@>{\text{$n_{3}$}}>{}>T_{n}\\\
@V{\text{$n_{1}$}}V{}V@V{}V{\text{$n_{2}$}}V\\\
(p_{2}^{-1}([C])\subset)W\qquad\qquad @>{\text{$p_{2}|_{W}$}}>{}>\quad\qquad
T(\owns[C])\\\ @V{\text{embedding}}V{}V@V{}V{\text{embedding}}V\\\
M\times\mathop{\mathrm{Hilb}}(M)@>{\text{$p_{2}$}}>{}>\mathop{\mathrm{Hilb}}(M)\\\
@V{}V{\text{$p_{1}$}}V\\\ M\\\ @V{}V{\text{$f$}}V\\\ B\end{CD}$ Figure 3: The
flattening of the deformation

From the flatness of the morphism $f_{3}\colon W^{\prime\prime}\to
T^{\prime\prime}$, the intersection number
$(h^{*}p_{1}^{*}f^{*}A,F^{\prime\prime})=0$ for every fiber $F^{\prime\prime}$
of the morphism $f_{3}\colon W^{\prime\prime}\to T^{\prime\prime}$, because
$(p_{1}^{*}f^{*}A,p_{2}^{-1}([C]))=0$. Thus, for every fiber
$F^{\prime\prime}$ of the morphism $f_{3}\colon W^{\prime\prime}\to
T^{\prime\prime}$, the morphism $f$ contracts $p_{1}h(F^{\prime\prime})$ to
one point from the connectedness of $F^{\prime\prime}$. In other words,
$p_{1}h(F^{\prime\prime})$ is included in some fiber of $f$.

There exists some ample divisor $A^{\prime}$ on $W$. Of course
$(A^{\prime},F)>0$ for every curve $F$ on $W$. Because the morphism $h$ is
birational, we have that $h(F^{\prime\prime\prime})$ is not a point (i.e.
$(h^{*}A^{\prime},F^{\prime\prime\prime})>0$) for some fiber
$F^{\prime\prime\prime}$ of $f_{3}\colon W^{\prime\prime}\to
T^{\prime\prime}$. From the flatness, $(h^{*}A^{\prime},F^{\prime\prime})>0$
for every fiber $F^{\prime\prime}$ of $f_{3}\colon W^{\prime\prime}\to
T^{\prime\prime}$. Thus every fiber $F^{\prime\prime}$ of $f_{3}\colon
W^{\prime\prime}\to T^{\prime\prime}$ cannot be contracted to a point by the
morphism $h$.

There exists some fiber $F^{\prime\prime}_{1}$ of $f_{3}\colon
W^{\prime\prime}\to T^{\prime\prime}$ such that $h^{-1}(p_{2}^{-1}([C]))\cap
F^{\prime\prime}_{1}\neq\emptyset$. Then $F^{\prime\prime}_{1}\subset
h^{-1}(p_{2}^{-1}([C])$, because the morphism $p_{2}h$ maps
$F^{\prime\prime}_{1}$ to a point $[C]\in T$. Consequently
$h(F^{\prime\prime}_{1})=p_{2}^{-1}([C])$, because $F^{\prime\prime}_{1}$ does
not contract to a point by the morphism $h$. Thus
$(h^{*}p_{1}^{*}L,F^{\prime\prime}_{1})>0$. From the flatness of the morphism
$f_{3}\colon W^{\prime\prime}\to T^{\prime\prime}$, the intersection number
$(h^{*}p_{1}^{*}L,F^{\prime\prime})>0$ for every fiber $F^{\prime\prime}$ of
the morphism $f_{3}\colon W^{\prime\prime}\to T^{\prime\prime}$.

We note that every fiber of $f_{3}\colon W^{\prime\prime}\to T^{\prime\prime}$
is mapped in some fiber of $p_{2}|_{W}\colon W\to T$. In other words, every
fiber of $p_{2}|_{W}$ is swept out by fibers of $f_{3}$.

So, for every fiber $F$ of $p_{2}|_{W}\colon W\to T$, the locus $p_{1}(F)$ is
swept out by connected curves $C^{\prime}$ such that $f(C^{\prime})$ is one
point and that the intersection number $(L,C^{\prime})>0$. (We note that we
consider $p_{1}h(F^{\prime\prime})$ as $C^{\prime}$ and that
$C^{\prime}=p_{1}h(F^{\prime\prime})=\mathop{\mathrm{Supp}}((p_{1}h)_{*}F^{\prime\prime})$
from the connectedness of $F^{\prime\prime}$.) Thus $b\notin f(p_{1}(F))$.
Consequently $b\notin f(p_{1}(W))$. In other words, $p_{1}(W)$ is disjoint
with $f^{-1}(b)$.

The countability of the irreducible components of the Hilbert scheme
$\mathop{\mathrm{Hilb}}(M)$ of $M$ implies the assertion.

∎

######  Proposition 2.

Let $M$ be a nonsingular projective variety and $C$ a Zariski-closed subset
with codimension $\mathop{\mathrm{codim}}(C,M)\geq 2$. Then there exists some
irreducible hyperplane section $H$ such that $H\supset C$.

###### Proof.

We take some ample divisor $A$ on $M$. We have a birational morphism $f\colon
M^{\prime}\to M$ such that $M^{\prime}$ is a nonsingular projective variety,
that $f^{-1}(C)$ is divisorial with only simple normal crossings and that
there exists an effective divisor $C_{0}$ with the property that
$\mathop{\mathrm{Supp}}(C_{0})=f^{-1}(C)$ and $-C_{0}$ is $f$-ample. Then
$mf^{*}A-C_{0}$ is ample for a sufficiently large integer $m$. For a
sufficiently large and divisible integer $l$, the divisor $lmA$ is very ample
and there exists a member $H_{0}\in|l(mf^{*}A-C_{0})|$ which is very ample and
irreducible. We put $H\colon=f_{*}(H_{0}+lC_{0})$. Then $H\in|lmA|$.

The locus $f^{-1}(H)$ coincides with
$\mathop{\mathrm{Supp}}(f^{*}H)=\mathop{\mathrm{Supp}}(H_{0}+lC_{0})$. Thus
$f^{-1}(H)\supset f^{-1}(C)$. ∎

Data Availability

No data were used to support this study.

Conflict of Interests

The author declares that there are no conflicts of interest.

Acknowledgments

In this series of research, the author was supported by the research grant of
Gifu Shotoku Gakuen University in the years 2019 and 2020. The author would
like to thank the referee who carefully read the paper and gave the
suggestions to improve the explanation.

## References

  * [1] S. Iitaka: On D-dimensions of algebraic varieties. J. Math. Soc. Japan 23(1971), 356–373.
  * [2] S. Mori: Threefolds whose canonical bundles are not numerically effective. Ann. Math. (2) 116(1982), 133–176.
  * [3] H. Tsuji: Numerically trivial fibrations. math. AG/0001023, October 2000.
  * [4] Th. Bauer, F. Campana, Th. Eckl, S. Kebekus, Th. Peternell, S. Rams, T. Szemberg and L. Wotzlaw: A reduction map for nef line bundles. in Complex Geometry, Goettingen 2000, pp. 27–36, Springer, Berlin, Germany, 2002.
  * [5] F. Ambro: Nef dimension of minimal models. Math. Ann. 330(2004), 309–322.
  * [6] F. Ambro: The moduli $b$-divisor of an lc-trivial fibration. Compos. Math. 141(2005), 385–403.
  * [7] S. Fukuda: Tsuji’s numerically trivial fibrations and abundance. Far East J. Math. Sci. (FJMS) 5(2002), 247–257.

Faculty of Education, Gifu Shotoku Gakuen University

Yanaizu-cho-Takakuwa-Nishi, Gifu, Gifu Prefecture 501-6194, Japan

<EMAIL_ADDRESS>


# Exploring the Relationship Between Personality Traits and User Feedback

Volodymyr Biryuk and Walid Maalej  Department of Informatics
Universität Hamburg
Hamburg, Germany
<EMAIL_ADDRESS>
###### Abstract

Previous research has studied the impact of developer personality in different
software engineering scenarios, such as team dynamics and programming
education. However, little is known about how user personality affect software
engineering, particularly user-developer collaboration. Along this line, we
present a preliminary study about the effect of personality traits on user
feedback. 56 university students provided feedback on different software
features of an e-learning tool used in the course. They also filled out a
questionnaire for the Five Factor Model (FFM) personality test. We observed
some isolated effects of neuroticism on user feedback: most notably a
significant correlation between neuroticism and feedback elaborateness; and
between neuroticism and the rating of certain features. The results suggest
that sensitivity to frustration and lower stress tolerance may negatively
impact the feedback of users. This and possibly other personality
characteristics should be considered when leveraging feedback analytics for
software requirements engineering.

###### Index Terms:

user feedback, personality types, personality traits, requirements engineering

##  I Introduction

Personality types, which are combinations of personality traits, have been
studied for decades—not only in psychology but also in software engineering.
Usually, a variety of personality tests such as Myers-Briggs Type Indicator
(MBTI) [1], Big Five/Five Factor Model (BF/FFM) [2], Keirsey Temperament
Sorter (KTS) [3], Sixteen Personality Factor Questionnaire (16 PF) [4],
Adjective Check List (ACL) [5], or Eysenck Personality Inventory (EPI) [6] are
used to group participants into different personality types [7].

In software engineering, research has focused so far on studying the
personality types of developers and their impact on development performance,
team dynamics, leadership performance, and task allocation[7]. The evidence
suggests, e.g., that certain personality types prefer certain software
engineering tasks and that the correct distribution of tasks has an impact on
project success [7]. However, the success of software engineering projects is
also largely dependent on the contribution of other stakeholders, most
notability of users during requirements engineering and maintenance
activities. Yet, little is known on whether and how user personality impact
these activities.

In recent years, the systematic collection and analysis of user feedback to
inform requirements engineering and software evolution tasks has become a
common practice [8]. Especially, since the emergence of social media and app
stores, a significant amount of feedback on software can be collected and used
by development teams, who also increasingly engaging in conversations with
users [9]. Feedback can reveal the users’ perspective on the software and its
features that is not obvious to the developers, for example unexpected usage
scenarios, unreported defects, or ideas for new features [10]. This work
explores whether personality types can effect the feedback behavior of
software users. If yes, personality traits should be taken into account to
“calibrate” and interpret feedback collected online, in workshops, and
interview sessions.

Our intuition is that the personality type might have an impact on how
individuals perceive software, and consequentially get engaged, and frame
their feedback. For example, users with a certain, highly developed
personality trait may get frustrated with certain features quicker than others
or focus more on the negative aspects. Other users might feel obliged to
explain their position, or are able to spot aspects of the software that
others do not. This can be reflected in different properties of feedback such
as software rating, wording, scope, mentioned features, or motivation and
expectation. If such relationships exist, they should be taken into
consideration in the collection, prioritization, and handling of feedback.

We report on a preliminary study in the context of an introductory programming
course, where we analyzed user feedback given by university students to the
online teaching platform Moodle together with the personality traits of the
students. Our research questions are as follows:

  * •

RQ1: Do feature ratings correlate with personality trait scores?

  * •

RQ2: Do personality trait characteristics correlate with the elaborateness of
user feedback?

We introduce the background for personality studies and close related work in
Section II. Then, we present the design of our study in Section III, the
results in Section IV, and threats to validity in Section V. Finally, we
briefly discuss the findings in Section VI and conclude the paper in Section
VII.

##  II Background

###  II-A Personality Traits and Types

Personality traits are relatively enduring patterns in behavior, thoughts, and
feelings, that manifest the tendency to respond in certain ways under certain
circumstances [11]. The constellations of those unique personality trait
features are called personality types and are used to define discrete groups
of individuals [12]. In personality psychology, they are often used to better
understand the diversity of responses from people in similar circumstances
[13].

Three personality tests dominate the scientific studies so far: namely Myers-
Briggs Type Indicator (MBTI) [1], Five Factor Model (FFM), and Keirsey
Temperament Sorter (KTS) [7]. The FFM model is the most suitable for our
research goal, as it produces more comprehensive information on all scales
[14, 15]. Furthermore, MBTI lacks the representation of neuroticism [14],
which is an important dimension when considering frustration during software
usage and the subsequent feedback behavior. The FFM includes the following
five personality traits:

  * •

Agreeableness (A) Individuals scoring high on this dimension tend to be
compliant, cooperative, altruistic, and helpful to others [16, 17].
Individuals who score low on this dimension tend to be irritable, skeptical
towards others, competitive, and egocentric [16, 17].

  * •

Agreeableness (A) Individuals scoring high on this dimension tend to be
compliant, cooperative, altruistic, and helpful to others [16, 17].
Individuals who score low on this dimension tend to be irritable, skeptical
towards others, competitive, and egocentric [16, 17].

  * •

Conscientiousness (C) Individuals scoring high on this dimension tend to be
careful, thorough, responsible, organized, and forward planning [16, 17].
Those who score low on this dimension tend to be irresponsible, disorganized,
and unscrupulous [16, 17]. High levels of conscientiousness manifest
themselves in an achievement-oriented and dependable character. Low levels of
conscientiousness, however, do not imply a lack of work ethics, but rather a
lower ability to apply them [17].

  * •

Extraversion (E) Individuals who score high on this dimension tend to be
sociable, talkative, assertive, and sociable [16, 17]. Introverted individuals
(those who score low on this dimension) are often retiring, reserved,
cautious, and independent [16, 17].

  * •

Neuroticism (N) Individuals who score high on Neuroticism have a tendency to
experience sadness, embarrassment, anxiety, depression, and anger. They are
prone to insecurity and irrational ideas, are less able to control impulses,
and cope poorly with stress [16, 17]. Those who score low on Neuroticism tend
to handle stressful situations well and to be calm, poised, and emotionally
stable [16, 17].

  * •

Openness (O) Those who score high on openness tend to be open-minded,
imaginative, intellectually curious, and independent of judgment from others
[16, 17]. Those who score low on openness tend to be down-to-earth, shy away
from novelty, and seek conformity [16, 17].

The traits can be interpreted in isolation or in combination with each other
and aggregated into personality types [18].

###  II-B Personality Studies of Software Developers and Users

Beyond psychology, personality types have been in the focus of software
engineering research to investigate effects on various phenomena such as pair
programming, team effectiveness, individual performance, software task
allocation, behavior preferences, education, and project management
effectiveness [7]. However, only a few studies have investigated the
personality of software users in contrast to software producers (developers,
managers). Stachl et al. [19] correlated personality types with mobile app
usage. Their results show that the personality type can be predicted by
looking at the frequency, duration, timing of app usage, and feature usage
such as average text length in messages. This suggests that the identification
of personality types does not require users to participate in personality
tests. When submitting feedback, a more realistic scenario is to analyze usage
data than asking feedback providers to take a personality test.

Various studies in human-computer interaction research examined the effects of
personality on user interface preferences. Alves et al. used the FFM scale to
show a difference in the preferences of GUI styles between different
personality types [20]. The authors found that the personality traits have an
effect on preferences of font size, information presentation density, and
color themes. A study by Sarsam and Al-Samarrie [21] shows that users are more
satisfied with the usage of a learning app if it is designed in accordance
with the preferences associated with their strong personality traits.

We are unaware of studies that have examined the impact of personality on user
feedback. To the best of our knowledge, the closest work to our is by Tizard
et al. [22], who studied the demographic properties of feedback providers. The
authors surveyed 1040 software users about their feedback habits, software
usage, and demographic information such as gender, age, ethnicity, education,
and employment status. Their results show that usage duration and gender have
a significant impact on the amount and frequency of feedback, as well as the
positivity of feedback. However, they did not investigate the impact of
personality on the provided feedback.

##  III Methodology

###  III-A Study Design

Our study consisted of two parts: one for collecting user feedback and one for
identifying the personality traits of the feedback providers. The
questionnaires for each part were available to students independently
throughout the last two weeks of the semester. Participants were able to
complete either or both of them in any order. Each part lasted for about 15
minutes.

In the first part, participants were asked to review (in text form) and rate
(from 1-dissatisfied to 5-satisfied) several Moodle features. Moodle is an
open-source learning platform designed to cover all aspects of teaching for
educators, administrators, and students. The software is usually self-hosted
by institutions and has been the main teaching platform at our department
since 2019. We collected feedback on five Moodle features that were essential
for our course:

  * •

Material overview is the main page of the course where all the items such as
course material, assignments, and additional information are listed and
grouped by week and topic.

  * •

Forum is a StackOverflow-like, simplified, Q&A thread-feature, that allows
students to ask questions and get answers from teachers or peers, vote on
answers, and mark questions as resolved.

  * •

Notification feature notifies about incoming chat messages, announcements, and
course calendar deadlines: either on the Moodle web interface or via email.

  * •

Code runner is an IDE-like environment that enables teachers to create small
programming assignments and the students to write and check code. The text
editor in the code runner only provides the students with a minimalistic text
field without any of the usual IDE features such as auto compilation, code
completion, or auto formatting. Before submitting the code, students can check
the correctness of their program by executing the reference unit test. The
code runner records each submission attempt so the number, duration and score
is available for evaluation. Each week the course exercise consisted of 5-10
sub-tasks which must be completed within a week of publication and can be re-
submitted indefinitely before the deadline.

  * •

Quiz supports the creation of different types of questions (such as
multiple/single choice, drag and drop, and gap filling) that are solved by
students and submitted in the same manner as the programming tasks. Both code
runner and quiz are used by students for homework assignments that are graded
automatically.

We focused on those features as they were regularly used during the course:
including solving weekly tasks, accessing learning material, and communicating
with peers and teachers.

We instructed the participants to explicitly state if they don’t have any
written feedback on a particular feature to prevent them from just skipping
questions without reading. The written feedback questions included one part
regarding problems with the respective feature and one regarding wishes and
requests.

The second part included the questions on the Big Five personality traits
using the NEO-FFI-30 questionnaire [23]. We chose the 30 question version of
the NEO-FFI to reduce the overall length of the study and increase the
willingness to participate. Both parts included the same demographic questions
at the end. So participants could correct or complement incorrect or
incomplete answers from the other part. All questions except the personality
test were optional to make the study shorter and potentially attract more
participants.

###  III-B Data Collection and Preparation

The study participants were university students from a large introductory
software engineering course at our department. The course is mandatory for
first semester informatics students. It is also attended by students of other
majors. We advertised the study during the main course session as well as in
the main teaching platform Moodle. As incentive, we raffled online shopping
vouchers, which participants could receive by completing both parts of the
study. The participation was completely voluntary and did not have any impact
on the course performance or grading. Before the study, we collected the
agreement of a local ethics committee at the university. Out of 124 total
participants who enrolled in the study, 56 have completed both parts with all
mandatory answers to calculate their personality traits.

Before answering the research questions, we labeled the feedback into
different categories and score the personality tests as described in the
following.

####  III-B1 Feedback Labeling

Before the labeling, the first author read every piece of feedback to better
understand the underlying data. We observed that most comments have multiple
types of information.

To label the information types in the feedback, we used a labeling scheme
similar to Pagano and Maalej [10]. One of the authors manually labeled the
feedback with 10 non-exclusive, basic information types praise, criticism,
shortcoming, improvement_request, bug_report, feature_request,
content_request, other_app, noise, and rationale.

Praise, criticism are generic categories for feedback that is positive or
negative but non-informative to developers. Shortcoming is a description of
something that is wrong with the software, but without stating a what can be
improved. Improvement_request and feature_request are similar to shortcoming,
but with more specific information on what can be improved or added to the
software. Bug_report is the description of a bug or defect in the software
that is obviously not intentional.

We added content_request to differentiate requests that are not concerning the
software features but rather the course content. other_app denotes that
another software was mentioned or compared to Moodle in the feedback (e.g. by
more experienced users). Rationale is a category that denotes the presence of
justification or elaboration of the feedback. For example, if a feedback text
contains a feature_request and additionally the participant describes why this
feature is needed in their context, we added the label.

####  III-B2 Personality Test Scoring

We have scored the personality test according to the NEO-PI-R/NEO-FFI manual
by Ostendorfer and Angleitner [24]. Each participant received a score in the
interval [0, 50] for each personality trait. When taking a closer look at the
answer patterns, we found that 3 participants answered disagree or partially
disagree more frequently than expected.

We also observed that the answers of 10 participants exhibit concerning
pattern. 8 participants have long streaks of answering questions with the same
value, including two participants who also have agreed or disagreed with the
questions more frequently than expected. Additionally, two participants have
disagreed with the questions more frequently than expected. The NEO-PI-R/NEO-
FFI manual advises treating these cases with caution, but has no strong advice
to discard them entirely. We conducted our subsequent correlation analyses
with and without these 10 cases and did not find any notable difference.

##  IV Results

We first analyzed the demographic data of our participants. We collected data
about age, gender, semester, and study program. The average participant in our
sample is 20 years old and in their first study year. However, a large portion
of participants did not specify their age or study duration. 21 participants
are female, 18 participants are male, and 17 did not specify any gender. 43
participants are studying computer science and 13 have other majors.

TABLE I: Statistical parameters of personality traits in our sample compared to benchmark by Körner et al. [25] Traits | A | C | E | N | O
---|---|---|---|---|---
MEAN from [25] | 30.49 | 32.53 | 26.46 | 19.47 | 24.53
MEAN | 34.43 | 35.11 | 25.86 | 22.29 | 31.74
SD from [25] | 5.68 | 6.59 | 5.92 | 7.41 | 5.54
SD | 7.35 | 7.14 | 7.49 | 12.04 | 8.00

To simplify our analysis, we limit ourselves to the evaluation of personality
traits and not personality types. Figure 1 shows that agreeableness,
conscientiousness, and openness are overall higher than extraversion and
neuroticism in our sample. The means and standard deviations of personality
trait scores, shown in Table I, are close to the original publication by
Körner et al. [25]. Therefore, we assume that our sample is close to being
representative. Notably, the mean of openness and standard deviation of
neuroticism have the largest differences to the statistics of Körner et al.
One possible explanation is that people willing to participate in such studies
might generally expose a high openness score.

Figure 1: Distribution of personality scores (N=56).

Figure 2: Distribution of feature ratings (N=56).

Figure 2 shows that, on average, the features were rated high, with a median
of 4 (solid lines) and a means ranging between 3.23 and 4.13 (dotted lines).
This corresponds to overall average rating on app store [10]. The quiz feature
was rated highest with a mean of 3.66, the notification feature was rated
lowest with a mean of 3.23.

Table II shows the results of the correlation analysis conducted to answer
RQ1. We found one significant correlation between a personality trait and a
Moodle feature: namely between neuroticism and material_overview(Table II)
with a p-value of 0.05. This means that participants who are high on
neuroticism (and thus with tendency to experience sadness, embarrassment,
anxiety, depression and anger) slightly dislike the material_overview.

TABLE II: Pearson’s coefficient of correlations between personality trait score and numeric feature ratings (N=56). Bold values mean a significant correlation (p$\leq$0.05) Moodle feature | A | C | E | N | O
---|---|---|---|---|---
Material overview | -0.03 | 0.07 | 0.05 | -0.26 | 0.14
Forum | -0.02 | 0.07 | -0.10 | -0.16 | 0.01
Notifications | -0.08 | 0.15 | 0.09 | -0.15 | 0.15
Code runner | -0.05 | 0.05 | -0.08 | -0.16 | -0.04
Quiz | -0.14 | 0.01 | -0.14 | -0.07 | -0.01

For RQ2, we assessed the elaborateness of written feedback. We particularly
looked at the text length, number of mentioned features (Table III), and
whether some rationale was provided in the feedback (Figure 3). Overall, the
median word length of the feedback is 34 words. The longest written feedback
is 220 words long. We found a positive but not significant correlation between
neuroticism and the length of written feedback. However, neuroticism has a
significant positive moderate correlation with the number of mentioned
features (Table III). This means that in our sample, neurotic individuals tend
to write longer feedback, that mentions more features.

TABLE III: Pearson’s correlation between personality traits, feedback length, and number of mentioned features in the feedback (N=56) | A | C | E | N | O
---|---|---|---|---|---
| Feedback length (words)
correlation | 0.00 | 0.02 | 0.02 | 0.25 | -0.01
p-value | 0.979 | 0.876 | 0.868 | 0.072 | 0.943
| Number of mentioned features
correlation | -0.15 | 0.08 | -0.20 | 0.31 | -0.16
p-value | 0.266 | 0.571 | 0.134 | 0.020 | 0.245

Additionally, we focus on rationale as RQ2 since it often contains important
details for requirements and is an indicator for a deeper involvement by the
feedback provider [26]. 39 participants provided rationale in their feedback.
Those participants scored higher on personality traits agreeableness,
conscientiousness, extraversion, and openness and lower on neuroticism (Figure
3).

Figure 3: Rationale provided in user feedback by personality trait (N=56).

The study data and code is publicly available with an open source
license.11110.5281/zenodo.8173418

##  V Threats to Validity

###  V-A External Validity

The rather small sample size in our study limits the generalizability of our
results and may be the cause of statistical insignificance. We chose 0.05 as
threshold for the $p$-value due to our small sample size, which can impact
negatively the validity of our results.

Our participant sample is limited to university students from a beginners
programming course and does not represent the general population or the
population. Even though our mean and standard deviation of the personality
trait scores is similar to the more representative sample, tested by Körner et
al. [25].

The participants were asked to provide feedback on certain features of the
used software. The questionnaire design nudged the participants to
differentiate between feedback that voices wishes and feedback that voices
problems. Even though feedback is sometimes explicitly collected in
requirements engineering, a big part of it is provided by the users on their
own behalf. Therefore, our results might be limited to a specific type of
feedback communication.

A survey-based personality study is prone to over- or underestimation of
certain variables. The willingness to participate in such studies often
correlates with high scores on agreeableness and conscientiousness [25]. Our
self-selected sample is potentially affected by this.

###  V-B Internal Validity

We do not have a control group to test how the feedback behavior might be
influenced by other factors, such as experience with user feedback or
requirements in general. We tested for the skill level by looking at the
progress of Moodle task completion (duration and number of attempts). We have
no reliable way to detect pauses in the participants’ work and can not
reliably determine the real duration. However, generally, this should not
impact the actual tasks in our study (give feedback and answer personality
test).

###  V-C Construct Validity

We have implemented our personality test questionnaire as instructed by the
publication of Körner et al. [25]. But we did not consult any psychologists
prior to the admission of the test. We performed the scoring and evaluation of
the NEO-FFI-30 questionnaire following the instructions from the original
publication by Ostendorf and Angleitner [24]. However, we had no supervision
by or consultation with psychologists and can not do an in-depth
interpretation of the results. The questionnaire Körner et al. [25] is a
reduced version of the original 240 or 60 questions inventory, and therefore
possibly less reliable than the original one.

##  VI Discussion

###  VI-A RQ1: Do feature ratings correlate with personality trait scores?

Our results show a weak negative correlation between the personality trait
neuroticism and the rating of specific Moodle feature material_overview. This
is in line with the study by Alves et al. who conclude that individuals high
on neuroticism prefer low information density GUIs. The Moodle GUI, however,
is rather overloaded with different widgets, that are not always necessary for
pursuing the course.

We expected to see more correlations between other personality traits as well.
Especially, we expected a high neuroticism score to have a much higher
negative impact on code runner and quiz rating, since they were used in a
rather stressful course environment. On the other hand, we expected forum and
notification to be rated higher by individuals who are high on agreeableness.
Overall, we did not find a strong universal relationship between personality
traits and rating feedback. But the observed trends should be checked and
explored further.

###  VI-B RQ2: Do personality trait characteristics affect elaborate- ness of
user feedback?

The personality trait neuroticism has a non-significant, weak positive
correlation with the written feedback length, indicating that users who score
higher on the neuroticism personality trait tend to write longer feedback. At
the same time, the number of mentioned features in their feedback is higher
too, which indicates a higher elaborateness and overall engagement. However,
these types of users seem to provide rationale less often than individuals who
are high on other FFM scales. The ones who provide rationale in their feedback
tend to have a higher score on the scales agreeableness, conscientiousness and
extraversion and particularly openness. This makes sense, especially for high
agreeableness and conscientiousness scores, since they stand for
cooperativeness and dependability. We find some limited evidence that
neuroticism may have an impact on the elaborateness of feedback.

##  VII Conclusion

Our results show some sporadic effects of personality traits on user feedback.
However, we can not positively answer RQ1 with our result and only have an
inconclusive answer for RQ2. We expected the correlations to be more
pronounced for certain personality trait. We expected participants who score
high on neuroticism to allocate lower score to Moodle features, as they have
used them in a stressful environment. We expected participants who score high
on conscientiousness, agreeableness to provide longer, more elaborate feedback
due to the urge of helping others, inherent to these personality traits.

We assume that the small sample and the specific context of university
students could have an impact on the results and would like to repeat the
study with a larger, more diverse sample and feedback on a wider range of
software. Regarding the questionnaire, we believe that future work should
include mandatory questions about the participants’ demographics and previous
feedback behavior, to control for those factors when evaluating the impact of
personality. Our results suggest digging deeper into the impact of neuroticism
on user feedback, since it is the only personality trait that shows positive
results. Our study focuses on one type of feedback, that is requested from the
users (pull feedback). To get a more complete understanding of the underlying
phenomena, it is necessary to analyze feedback that was provided by users on
their own behalf. We believe that analyzing user feedback from public sources
and conducting the personality test afterward independently could improve the
external validity of the results.

## References

  * [1] K. C. Briggs, _Myers-Briggs Type Indicator_. Consulting Psychologists Press Palo Alto, CA, 1976.
  * [2] P. T. Costa Jr, “Revised NEO personality inventory and NEO five-factor inventory,” _Professional manual_ , 1992.
  * [3] D. Keirsey, _Please Understand Me II: Temperament, Character, Intelligence_. Prometheus Nemesis Book Company, 1998.
  * [4] R. B. Cattell, H. W. Eber, and M. M. Tatsuoka, _Handbook for the Sixteen Personality Factor Questionnaire (16 PF): In Clinical, Educational, Industrial, and Research Psychology, for Use with All Forms of the Test_. Institute for personality and ability testing, 1970.
  * [5] H. G. Gough and A. B. Heilbrun, _The Adjective Check List Manual_. Consulting Psychologists Press, 1983.
  * [6] V. Zeigler-Hill and T. K. Shackelford, Eds., _Encyclopedia of Personality and Individual Differences_. Cham: Springer International Publishing, 2020.
  * [7] S. Cruz, F. Q. Da Silva, and L. F. Capretz, “Forty years of research on personality in software engineering: A mapping study,” _Computers in Human Behavior_ , vol. 46, pp. 94–113, May 2015.
  * [8] C. Wang, M. Daneva, M. Sinderen, and P. Liang, “A systematic mapping study on crowdsourced requirements engineering using user feedback,” _Journal of Software: Evolution and Process_ , vol. 31, no. 10, Oct. 2019.
  * [9] D. Martens and W. Maalej, “Extracting and analyzing context information in user-support conversations on twitter,” in _2019 IEEE 27th International Requirements Engineering Conference (RE)_. IEEE, 2019, pp. 131–141.
  * [10] D. Pagano and W. Maalej, “User feedback in the appstore: An empirical study,” in _2013 21st IEEE International Requirements Engineering Conference (RE)_ , Jul. 2013, pp. 125–134.
  * [11] B. W. Roberts, “Back to the future: Personality and Assessment and personality development,” _Journal of Research in Personality_ , vol. 43, no. 2, pp. 137–145, Apr. 2009.
  * [12] P. T. Costa, J. H. Herbst, R. R. McCrae, J. Samuels, and D. J. Ozer, “The replicability and utility of three personality types,” _European Journal of Personality_ , vol. 16, no. 1_suppl, pp. S73–S87, Mar. 2002.
  * [13] L. Borghans, B. H. Golsteyn, J. Heckman, and J. E. Humphries, “Identification problems in personality psychology,” _Personality and Individual Differences_ , vol. 51, no. 3, pp. 315–320, Aug. 2011.
  * [14] A. Furnham, “The big five versus the big four: The relationship between the Myers-Briggs Type Indicator (MBTI) and NEO-PI five factor model of personality,” _Personality and Individual Differences_ , vol. 21, no. 2, pp. 303–307, Aug. 1996.
  * [15] R. R. McCrae and P. T. Costa, “A contemplated revision of the NEO Five-Factor Inventory,” _Personality and Individual Differences_ , vol. 36, no. 3, pp. 587–596, Feb. 2004.
  * [16] S. Roccas, L. Sagiv, S. H. Schwartz, and A. Knafo, “The Big Five Personality Factors and Personal Values,” _Personality and Social Psychology Bulletin_ , vol. 28, no. 6, pp. 789–801, Jun. 2002.
  * [17] S. Rothmann and E. P. Coetzer, “The big five personality dimensions and job performance,” _SA Journal of Industrial Psychology_ , vol. 29, no. 1, Oct. 2003.
  * [18] F. A. Sava and R. I. Popa, “Personality types based on the Big Five model. A cluster analysis over the Romanian population.” _Cognitie, Creier, Comportament/Cognition, Brain, Behavior_ , vol. 15, no. 3, 2011.
  * [19] C. Stachl, Q. Au, R. Schoedel, S. D. Gosling, G. M. Harari, D. Buschek, S. T. Völkel, T. Schuwerk, M. Oldemeier, T. Ullmann, H. Hussmann, B. Bischl, and M. Bühner, “Predicting personality from patterns of behavior collected with smartphones,” _Proceedings of the National Academy of Sciences of the United States of America_ , vol. 117, no. 30, pp. 17 680–17 687, Jul. 2020.
  * [20] T. Alves, J. Natálio, J. Henriques-Calado, and S. Gama, “Incorporating personality in user interface design: A review,” _Personality and Individual Differences_ , vol. 155, p. 109709, Mar. 2020.
  * [21] S. M. Sarsam and H. Al-Samarraie, “A First Look at the Effectiveness of Personality Dimensions in Promoting Users’ Satisfaction With the System,” _SAGE Open_ , vol. 8, no. 2, p. 215824401876912, Apr. 2018\.
  * [22] J. Tizard, T. Rietz, and K. Blincoe, “Voice of the Users: A Demographic Study of Software Feedback Behaviour,” in _2020 IEEE 28th International Requirements Engineering Conference (RE)_ , Aug. 2020, pp. 55–65.
  * [23] A. Körner, M. Geyer, M. Roth, M. Drapeau, G. Schmutzer, C. Albani, S. Schumann, and E. Brähler, “Persönlichkeitsdiagnostik mit dem NEO-Fünf-Faktoren-Inventar: Die 30-Item-Kurzversion (NEO-FFI-30),” _PPmP - Psychotherapie $\cdot$ Psychosomatik $\cdot$ Medizinische Psychologie_, vol. 58, no. 6, pp. 238–245, Jun. 2008.
  * [24] F. Ostendorf and A. Angleitner, _Neo-Persönlichkeitsinventar Nach Costa Und McCrae: Neo-PI-R_. Hogrefe Göttingen, 2004.
  * [25] A. Körner, M. Drapeau, C. Albani, M. Geyer, G. Schmutzer, and E. Brähler, “Deutsche Normierung des NEO-Fünf-Faktoren-Inventars (NEO-FFI) [German norms for the NEO Five-Factor Inventory],” _Zeitschrift für Medizinische Psychologie_ , vol. 17, pp. 133–144, Mar. 2008.
  * [26] Z. Kurtanović and W. Maalej, “Mining User Rationale from Software Reviews,” in _2017 IEEE 25th International Requirements Engineering Conference (RE)_ , Sep. 2017, pp. 61–70.