# My part is bigger than yours - assessment within a group of peers using the
pairwise comparisons method11footnotemark: 122footnotemark: 2

Konrad Kułakowski<EMAIL_ADDRESS>Jacek Szybowski
<EMAIL_ADDRESS>AGH University of Krakow, WEAIiIB, Applied Computer
Science Department  AGH University, Faculty of Applied Mathematics

###### Abstract

A project (e.g. writing a collaborative research paper) is often a group
effort. At the end, each contributor identifies his or her contribution, often
verbally. The reward, however, is quite often financial in nature. This leads
to the question of what (percentage) share in the creation of the paper is due
to individual authors. Different authors may have various opinions on the
matter, and, even worse, their opinions may have different relevance. In this
paper, we present a simple models that allows aggregation of experts’ opinions
linking the priority of his preference directly to the assessment made by
other experts. In this approach, the greater the contribution of a given
expert, the greater the importance of his opinion. The presented method can be
considered as an attempt to find consensus among a group of peers involved in
the same project. Hence, its applications may go beyond the proposed study
example of writing a scientific paper.

###### keywords:

consensus finding , group decision-making , aggregation of individual rankings
, pairwise comparisons

††journal: arxiv.org

##  1 Introduction

From time to time, we observe behind-the-scenes discussions about which author
in a scientific paper is the most relevant. The answers can vary. First, last,
correspondence, first three authors, it doesn’t matter, etc. At its core, it
is a question about the value of the contribution, the effort, the effort that
an author has made to the paper. Very often, scientific journals try to help
answer this question by making it possible to determine the workload of a
given author by describing descriptively what that author did. Such a
description makes it possible, to some extent, to form an opinion about the
significance of an author’s contribution.
However, there are times when such a description is insufficient and a
precise, percentage determination of "contribution to the work" for each
author separately is necessary. This is the case when a certain amount of
money is awarded for the creation of a work to be distributed among the
authors. In such a situation, quite often the "distribution of percentages" is
done on a dictatorial basis, with the corresponding author playing the role of
dictator. In a situation where the authority of all authors is comparable, or
worse, where the corresponding author is a less recognized researcher than
some other researchers on the author list, misunderstandings and irritations
can easily arise. On the one hand, the correspondent (or lead) author should
have the best knowledge of the other authors’ contributions to the work. On
the other hand, other authors of greater esteem can easily question his or her
decisions.
A way out of this somewhat uncomfortable situation may be to adopt arbitrary
regulations for the distribution of the award. E.g. dividing the prize equally
between all participants, or setting rigid proportions between the prize for
the first and subsequent authors. These types of solutions, unless they arouse
personal animosity between the authors, leave a certain feeling of injustice.
The question therefore arises as to whether a dictatorial approach on the one
hand and the establishment of a rigid framework for the distribution of the
prize on the other can be avoided. It seems to us that with a team of peers
this is possible.

In this paper we propose an approach based on aggregating the opinions of
individual authors/experts using prioritization of opinions. Since team
members who were more involved in the project usually have a better
understanding of the project and the contribution of other team members to the
work done we propose that prioritization should be related to the size of this
contribution. In this way, we avoid a situation where those whose
participation is small or marginal have the same share in the decision as
those whose participation is significant. The latter, by virtue of their
significant and large contribution, are given greater priority in determining
the distribution of the award.
In doing so, we assume that the work took place in a team of peers i.e. that
there is no obvious dictator whose opinion everyone naturally aligns with,
and, that there is a real opportunity for the contribution of each team member
to be assessed by others. The absence of a dictator results in the need for
team decisions on the distribution of the prize. The possibility of mutual
evaluation, on the other hand, enables group decision-making. This is the
case, for example, when a team of researchers of similar experience and
reputation work together on a project having from time to time stand-up
meetings where the progress of the work is discussed.
The problem outlined above does not only apply to the creation of scientific
articles. A similar situation occurs with the distribution of awards for
various other team achievements, such as grants, organizational projects or
software projects in IT companies.

The solution proposed in this paper is based on the aggregation procedure used
in group decision-making (GDM) process as used in the pairwise comparison
method. The basics of classical aggregations method are described by Saaty and
Aczel [1], Saaty [34] and later by Forman and Peniwati [17]. The problem of
aggregating expert opinion for the pairwise comparison method has been studied
many times. Aggregation procedures have been adapted for models using fuzzy
sets [10], hesitant probabilistic fuzzy linguistic set [36] or intervals [39].
Among GDM studies, consensus building methods have a special place. In [29] a
method is proposed that in parallel considering the inconsistency of expert
judgements and acceptable consensus. Gong et al. [19] considers the problem of
consensus in social networks, where an additional element that can help or
hinder consensus building is the actions of the moderator. The proposed
maximum fairness consensus model uses decision-maker fairness assessment
inspired by social comparison and Gini coefficient. Large-scale group
decision-making (LSGDM) is the subject of another paper [27]. Liang et al.
propose the framework for LSGDM including clustering, discovering and managing
manipulative behavior and consensus reaching. Zhao et al. [41] pay attention
to individual participation and satisfaction in the decision-making process.
They propose a new consensus model based on optimization of utility aggregated
using additive Choquet integral. The issue of consensus, often understood as
the minimization of the difference between the preferences of decision-makers
and the collective opinion, is described in more detail in [13].

The pairwise comparison of alternatives approach has also resulted in a number
of studies in the area of group decision-making. Traditionally, there are two
opinion aggregation procedures in the Analytic Hierarchy Process (AHP) [33]
method; aggregation of individual judgments (AIJ) and aggregation of
individual priorities (AIP) [1, 35, 17]. We will present the latter in more
detail in Section 2.2.2. Both procedures are commonly accepted and used for
preference aggregation given in the form of pairwise comparisons (PC) matrices
(see Sec. 2.1), whereby AIJ is recommended when experts have previously agreed
on a common assessment framework [17]. Ramanathan and Ganesh [32] argue in
favor of AIP as it meets the Pareto principle when using eigenvector
prioritization method (EVM) [33], whereas AIJ does not. Both methods are
equivalent when using the geometric mean method (GMM) (or logarithmic least
square method (LLSM)333Both GMM and LLSM are equivalent for complete and
incomplete PC matrices [25].) for the ranking calculation [4]. In addition to
the two mentioned, other methods of aggregating preferences have emerged.
Group euclidean distance (GED) minimization method has been proposed by
Blagojevic et al. [5]. The process of progressively modifying the preference
matrix of individual experts so as to minimize the defined geometric
consensual consistency index (GCCI) lies at the heart of the idea from Dong et
al. [14]. Altuzarra et al. proposed Bayesian prioritization (BPP) allowing the
aggregation procedure to be extended to preferences (utilities, priorities)
between the alternatives that are given as probability distributions [2].
Escobar and Moreno-Jimenez consider [16] a new aggregation method called
Aggregation of Individual Preference Structures (AIPS). The aggregation of
more diverse preferential information allows phenomena such as judgement
uncertainty, problem vision or interdependence to be taken into account. Lin
and Kou [28] focus on the Bayesian revision method allowing for PC matrices
improvement. Lin et al. [29] propose aggregation of the nearest consistent
matrices with the additional adjustment connected with the acceptable
consensus. Kulakowski et al. [26] introduce an aggregation method equipped
with heuristically adjusted priorities preventing manipulative behavior by
some decision-makers.

##  2 Preliminaries

###  2.1 Pairwise comparisons

Many methods including AHP, HRE, MACBETH, BWM, outranking methods like e.g.
PROMETHEE [25, 20] and various hybrid solutions use pairwise comparisons (PC)
as a source of information about decision makers’ preferences. The approach
has its extensions to cover different data representations including intervals
[21], fuzzy numbers [23] or grey numbers [15]. The best-known method based on
pairwise comparisons of alternatives is AHP. It is also the most criticized,
although the criticism is often constructive and leads to many improvements
and extensions of the original proposal. Of the more important areas of
improvement, it is worth mentioning: methods for calculating the vector of
weights [22] including calculating rankings for incomplete matrices [6, 7,
24], methods for measuring inconsistencies [8] and others. An interesting
critical analysis of the AHP method can be found in Munier and Hontoria [30].

In the PC method, experts, also known as decision-makers, compare alternatives
in pairs. Let $A=\\{a_{1},\ldots,a_{n}\\}$ be finite set of alternatives and
$E=\\{e_{1},\ldots,e_{k}\\}$ be a set of experts. Each expert compares the
alternatives with each other to form a set of comparisons represented in the
form of a square matrix $C_{q}=\\{c_{ijq}\in\mathbb{R}_{+}:i,j=1,\ldots,n\\}$
where $1\leq q\leq k$, and $q$ indicates the number of expert. A single
$c_{ijq}$ denotes the relative importance of the i-th alternative compared to
the j-th alternative according to the experts opinion $e_{q}$. For the sake of
legibility of notation, whenever possible we will omit the expert index $q$ by
writing the PC matrix as $C=[c_{ij}]$. PC matrix is used to synthesize a
vector of weights. Let us define the function resulting from this calculation.

######  Definition 1.

Let $A$ be a set of alternatives. The priority function for $A$ is the mapping
$w:A\rightarrow\mathbb{R}_{+}$ assigning a real and positive number to each
alternative.

The priority vector for $A$ resulting from $C_{q}$ takes the form:

$w_{q}=\left[w_{q}(a_{1}),\ldots,w_{q}(a_{n})\right]^{T}.$ (1)

Many methods have been described in the literature to calculate the value of
priority vector $w$. The first and so far quite popular is the eigenvalue
method (EVM). This approach was proposed by Saaty in his seminal paper [33],
and consists in calculating a principal eigenvector, and then normalizing it
so that the sum of entries is one. Another method is based on calculating the
geometric means444Referred to as geometric mean method (GMM). of the matrix
rows and taking the result of the i-th row as the priority of the i-th
alternative [12]. The result, thus, obtained is also subject to normalization
at the end of the calculation. In addition to the two mentioned, there are
quite a few other methods for calculating the priority vector. A comparison of
the performance of the six popular priority deriving methods can be found in
[37]. Another overview of prioritization methods can be found in [9].

###  2.2 Group decision-making

####  2.2.1 Aggregation procedures

When more than one expert participates in the decision-making process, their
opinions need to be aggregated. As with a single PC matrix, the end result
should be a vector of weights assigning real numbers to each alternative.
Aggregation of expert opinions can be done both at the level of individual
comparisons and at the level of individual weight vectors. An example of the
first approach can be the AIJ (aggregating of individual judgments) [17]
procedure, according to which each comparison of the i-th and j-th
alternatives by each of $k$ experts is averaged and then a vector of weights
is calculated for the matrix composed of such averaged values. In addition to
AIJ, there are a number of optimization methods that directly use comparison
results from different experts. The previously mentioned model based on
minimization of group Euclidean distance (GED) values [5] can serve as an
example. A different approach is the AIP procedure according to which priority
vectors for individual experts are calculated first and only then are the
results aggregated. Compared to AIJ, the AIP approach has several significant
advantages. The second does not impose the need for a group of experts to act
as a unit, leaving them more freedom to make individual judgments [17]. It can
be easily implemented in the situation of having incomplete data from
different experts. In such a case, it is enough to use appropriate
prioritization methods for incomplete PC matrices [6, 24] to calculate
individual weight vectors. The weight vectors obtained in this way may then be
subjected to aggregation. In addition, AIP satisfies the Pareto principle with
an arithmetic or geometric mean [17].
Aggregation procedures allow each expert’s opinion to be assigned different
strengths (priorities). Thus, experts who are considered less competent can
have less influence on the final opinion, while experts who are more
recognized can have a greater impact on the final outcome. In the next two
sections, we will look at two variants of the AIP method that allow for
assigning different priorities to individual experts.

####  2.2.2 Multiplicative aggregation of individual priorities

The most popular AIP approach is based on the use of a weighted geometric
mean. Let the vector of weights calculated from the pairwise comparison matrix
$C_{q}$ provided by the q-th expert be denoted as follows:

$w_{q}=\left(\begin{array}[]{c}w_{q}(a_{1})\\\ w_{q}(a_{2})\\\ \vdots\\\
w_{q}(a_{n})\end{array}\right),\,\,\,\text{for}\,\,q=1,\ldots,k.$ (2)

Then, let $W$ be a matrix of weights vectors so that

$W=\left(w_{1},\ldots,w_{k}\right),$

i.e.

$W=\left(\begin{array}[]{ccccc}w_{1}(a_{1})&w_{2}(a_{1})&\cdots&\cdots&w_{k}(a_{1})\\\
w_{1}(a_{2})&w_{2}(a_{2})&\cdots&\cdots&w_{k}(a_{2})\\\
w_{1}(a_{3})&\vdots&\ddots&\vdots&\vdots\\\
\vdots&\vdots&\vdots&\ddots&\vdots\\\
w_{1}(a_{n})&w_{2}(a_{n})&\cdots&\cdots&w_{k}(a_{n})\end{array}\right),$ (3)

and, let $p$ be the vector of priorities for individual experts i.e.

$p=\left(\begin{array}[]{c}p_{1}\\\ p_{2}\\\ \vdots\\\
p_{k}\end{array}\right),$ (4)

so that $\sum_{i=1}^{k}p_{i}=1$. Then

$\text{GAIP}(W,p)=\left(\begin{array}[]{c}\prod_{i=1}^{k}w_{i}^{p_{i}}(a_{1})\\\
\prod_{i=1}^{k}w_{i}^{p_{i}}(a_{2})\\\ \vdots\\\ \vdots\\\
\prod_{i=1}^{k}w_{i}^{p_{i}}(a_{n})\end{array}\right)$ (5)

denotes AIP procedure using a weighted geometric mean, where $W$ is the matrix
of priority vectors derived by individual experts and $p$ is the vector of
priorities of those experts.

####  2.2.3 Additive profile aggregation

An alternative way to aggregate the resulting weight vectors is to use a
weighted arithmetic mean. Let priority vector provided by q-th expert be
$w_{q}$ (2), matrix of priority vectors be given as $W$ (3) and the vector of
expert priorities is $p$ (4) so that so that $\sum_{i=1}^{k}p_{i}=1$. Then

$\text{AAIP}(W,p)=\left(\begin{array}[]{c}\sum_{i=1}^{k}p_{i}w_{i}(a_{1})\\\
\sum_{i=1}^{k}p_{i}w_{i}(a_{2})\\\ \vdots\\\ \vdots\\\
\sum_{i=1}^{k}p_{i}w_{i}(a_{n})\end{array}\right)$ (6)

denotes AIP procedure using a weighted arithmetic mean.

###  2.3 Constrained global optimization

Ranking methods very often use optimization methods to determine the best
vector of weights corresponding to the alternatives under consideration. If
the formulated optimization problem is linear in nature then the Simplex
algorithm can be used and the problem is expressed using linear programming
[38]. The approach has a rich literature and many extensions. Linear
programming underlies many other methods and solutions. An example of a method
that grew out of the idea of linear programming is Data Envelopment Analysis
(DEA) [11]. When the problem is nonlinear then, depending on the properties of
the function under analysis (e.g. convexity), other optimization techniques
can be used. For the purpose of testing the solution proposed in this paper we
propose the three most popular, well established, heuristic constrained global
optimization techniques: Nelder-Mead method, Differential Evolution and
Simulated Annealing. The goal of each of the three algorithms is to find the
minimum of some function $f:\mathbb{R}^{n}\rightarrow\mathbb{R}$. Finding the
minimum is equivalent to proposing the desired vector of weights.

The first of these, the _Nelder-Mead_ method [31], is a direct search method
i.e. in its search it does not take into account information, for example,
about the direction of the decreasing gradient. For function $f$, in every
iteration the algorithm keeps $n+1$ points $x_{1},x_{2},\ldots,x_{n+1}$
forming a polytope. The points are ordered so that $f(x_{1})\leq
f(x_{2})\leq\ldots\leq f(x_{n+1})$. Then, the centroid of the $n$ worst points
is generated, and the next point is generated by reflecting the worst point
through the centroid. Depending on the quality of the generated solution, the
procedure can iteratively continue (other points may be selected, or other
points removed) or end if the obtained solution is good enough.

The second method, _Differential Evolution_ [40, p. 187], minimizes a function
using the genetic programming paradigm. The algorithm works with a population
of $m$ candidate solutions, denoted as
$\\{x_{1},x_{2},\ldots,x_{j},\ldots,x_{m}\\}$, where usually $m$ is much
larger than the number of variables (denoted by $n$). During each iteration, a
new population of $m$ points is generated. For each j-th new point, three
random points, $x_{u},x_{v}$, and $x_{w}$, are chosen from the current
population. The new point $x_{s}$ is calculated following the formula
$x_{s}=x_{w}+s(x_{u}-x_{v})$, with $s$ being the scaling factor. Subsequently,
a new candidate $x_{\textit{new}}$ is formed by selecting the i-th coordinate
from $x_{s}$ with probability $\rho$, or selecting the i-th coordinate from
$x_{j}$. If the objective function value at $x_{\textit{new}}$ is less than
that at $x_{j}$, then $x_{\textit{new}}$ replaces $x_{j}$ in the population.

The third used method, _Simulated Annealing_ [18, p. 5] is a straightforward
stochastic optimization technique inspired by the annealing process in
metallurgy. This process involves heating a metal to a high temperature and
then gradually cooling it, allowing its atomic structure to settle into a
lower energy state, making the metal stronger. In optimization terms,
annealing helps the solution evade local minima and search for a better,
potentially global minimum. During each iteration of the algorithm, a new
point $x_{\textit{new}}$ is generated within the vicinity of the current point
$x$, with the radius of this neighborhood shrinking over time. The algorithm
keeps track of the best point encountered so far, $x_{\text{best}}$.

If the objective function value at $x_{\text{new}}$ is better than at
$x_{\text{best}}$, $x_{\text{new}}$ replaces both $x_{\text{best}}$ and $x$.
If not, $x_{\text{new}}$ can still replace $x$ with a probability
$e^{b(i,\Delta f,f_{0})}$, where $b$ depends on the current iteration number
$i$, the change in the objective function value $\Delta f$, and the objective
function value from the previous iteration $f_{0}$. Similar to the Random
Search method, Simulated Annealing uses multiple starting points to enhance
the search for an optimum solution. For each starting point, the process
continues until either the maximum number of iterations is reached, the
solution converges, or the algorithm remains at the same point for the
specified number of iterations.

The popularity of these three methods translates into their availability in
various packages and computing environments including R, Mathworks Matlab™ or
Wolfram Mathematica™ and others555List of optimization software:
https://en.wikipedia.org/wiki/List_of_optimization_software..

##  3 Peer ranking aggregation

###  3.1 Problem description

The immediate inspiration to take up the topic of aggregation of opinions in a
group of peers was the introduction of changes in the regulations for
rewarding employees of a certain university, according to which the reward
depends on the level of contribution of each author of a paper. Of course,
this level must be numerical (percentage) and allow for an unambiguous
distribution of money between the parties involved. It must also be based on
some compromise because all authors must sign a declaration in which they
agree to the assigned share. In practice, most teams deal with this problem by
selecting one person who, acting as "dictator", proposes the distribution of
"percentages" i.e. funds for the prize. Depending on the custom, team members
may (or may not) negotiate the proposed share. In the case of teams working on
a scientific article, the role of dictator is played by the correspondent
author, first author or simply the head of the research group.

From the point of view of the university’s financial management, the advantage
of the model with "one person in charge" is the relative simplicity of the
solution. The entire burden of responsibility for the proper allocation of
funds is transferred to the person designated for this purpose. The problem,
of course, is most often the lack of a methodology to make an informed
decision on the distribution of the award, which boils down to adopting the
simplest decision-making model, i.e. dictatorship. The obvious disadvantage of
the dictatorial approach is the arbitrariness of the decisions made. The
decision-maker likes some members of the team more and others less, which can
directly translate into the evaluation of their work. He also often
overestimates his own participation in the project. On the other hand, the
decision-maker is usually the person most involved in the project and, what
goes hand in hand, the best informed as to the participation and nature of
others’ contributions.

###  3.2 Model

The above observations led us to propose a peer review model in a group of
peers based on pairwise comparison of alternatives. Group members have a dual
role. They are simultaneously the object of comparison, i.e. the alternatives
$A=\\{a_{1},\ldots,a_{n}\\}$ in the PC method, and the experts providing their
judgments i.e. $E=\\{e_{1},\ldots,e_{n}\\}$. Each of the experts, i.e. team
members, provides comparisons in the form of a PC matrix $C=[c_{ij}]$, where
$c_{ij}\in\mathbb{R}_{+}$ in which he assesses the relative level of
contribution of the other members of the group. Based on the collected ratings
$C_{1}$,…,$C_{n}$, vectors of weights $w_{1},\ldots,w_{n}$ for individual
experts are created. The calculated vectors are then subjected to aggregation,
resulting in the formation of a final priority vector

$w=\left(\begin{array}[]{c}w(a_{1})\\\ \vdots\\\ \vdots\\\
w(a_{n})\end{array}\right)$

reflecting the share of each evaluated person in the award. However, in line
with the observation that those more involved in a project are often more
competent to assess the engagement of others, the priorities given to expert
opinions during aggregation may not be identical. They must reflect this
regularity. Let $p:A\rightarrow\mathbb{R}$ be an expert priority function such
that for two experts $e_{i}$ and $e_{j}$ if their final evaluation share meets
$w(a_{i})\geq w(a_{j})$ then also their priorities during the aggregation
process meet $p(a_{i})\geq p(a_{j})$. This regularity comes out of the
observation that the one who contributed more to the success than the other
has more right than the latter to decide on the distribution of the reward. In
the adopted solution, we adopted the simplest possible function $p$ i.e.
$p(a_{i})=w(a_{i})$. It grants the right to decide to a team member directly
proportional to his level of involvement. In many cases, it also seems to be
the most equitable. Below, following [17], we have proposed two aggregation
models to support the proposed solution: multiplicative and additive.

###  3.3 Multiplicative profile aggregation

When proceeding to aggregate the results, we assume that each team member
$e_{q}\in E$ provided a PC matrix $C_{q}$ based on which the ranking vector
$w_{q}=\left(w_{q}(a_{1}),\ldots,w_{q}(a_{n})\right)^{T}$ was calculated.
These vectors form a matrix $W$ of the form:

$W=\left(\begin{array}[]{ccccc}w_{1}(a_{1})&w_{2}(a_{1})&\cdots&\cdots&w_{n}(a_{1})\\\
w_{1}(a_{2})&w_{2}(a_{2})&\cdots&\cdots&w_{n}(a_{2})\\\
w_{1}(a_{3})&\vdots&\ddots&\vdots&\vdots\\\
\vdots&\vdots&\vdots&\ddots&\vdots\\\
w_{1}(a_{n})&w_{2}(a_{n})&\cdots&\cdots&w_{n}(a_{n})\end{array}\right).$

Denoting vector of experts’ priorities as
$p=\left(p(a_{1}),\ldots,p(a_{n})\right)^{T}$ we get an extra condition that
the final solution must satisfy:

$\text{GAIP}(W,p)=\left(\begin{array}[]{c}\prod_{i=1}^{k}w_{i}^{p(a_{i})}(a_{1})\\\
\prod_{i=1}^{k}w_{i}^{p(a_{i})}(a_{2})\\\ \vdots\\\ \vdots\\\
\prod_{i=1}^{k}w_{i}^{p(a_{i})}(a_{n})\end{array}\right)=\left(\begin{array}[]{c}p(a_{1})\\\
p(a_{2})\\\ \vdots\\\ \vdots\\\ p(a_{n})\end{array}\right).$ (7)

For the purposes of the calculation, we will also want $p(a_{i})>0$ for
$i=1,\ldots,n$ and $\sum_{i=1}^{n}p(a_{i})=1$. The introduction of (7) leads
directly to the formulation of a nonlinear optimization problem in the form:

$\displaystyle\min g(W,p)\,\,\,\,\text{s.t.}$ $\displaystyle
g(W,p)=\sum_{i=1}^{n}\left(\prod_{j=1}^{n}w_{j}^{p(a_{j})}(a_{i})-y_{i}\right)^{2}$
(8) $\displaystyle
p(a_{i})=\frac{y_{i}}{\sum_{j=1}^{n}y_{j}}\,\,\,\text{and}\,\,p(a_{i})>0\,\,\text{for}\,\,i=1,\ldots,n.$

The above (8) can be solved using constrained global optimization methods
(Section 2.3).

One can also attempt to find the final vector of weights using a direct
iterative algorithm (listing 3.3) bearing in mind, however, that it may not
always converge. The iterative algorithm follows a simple scheme. It takes as
input a matrix $W$ and three parameters: the maximum number of iterations
$\gamma$, the maximal acceptable difference of consecutive results $\delta$
and maximal permissible error $\epsilon$. Then, in each subsequent iteration,
a new vector of weights $w^{\prime\prime}$ is calculated using GAIP where the
previously calculated weights vector $w^{\prime}$ is used as the priorities of
the individual experts (line: 6). When using the GAIP procedure for the first
time (line: 2), the experts’ priority values may be equal. The algorithm ends
when the maximum number of iterations allowed is reached (line: 5), or the
difference between successive calculated vectors becomes less than $\delta$
(line: 7). In the first case (line: 14) the algorithm did not find a solution.
In the second case, however, the algorithm selected a vector that can be a
solution to the problem (7). In the second case, although the vector
$w^{\prime\prime}$ is calculated, we need to check whether the calculation is
accurate enough. For this purpose, the value $g(W,w^{\prime\prime})$ of the
criterion function is calculated (8) and compared against the maximum
permissible error $\epsilon$ (line: 8).

1:function DIA($W$, $\gamma$, $\delta$, $\epsilon$)

2: $w^{\prime}\leftarrow\text{GAIP}(W,(1/n,\ldots,1/n)^{T})$

3: $w^{\prime}\leftarrow\text{normalize}(w^{\prime})$

4: $\text{i}\leftarrow 1$

5: for i $\leq$ $\gamma$ do

6: $w^{\prime\prime}\leftarrow GAIP(W,w^{\prime})$

7: if $\left\|w^{\prime\prime}-w^{\prime}\right\|<\delta$ then

8: if $g(W,p)\leq\epsilon$ then

9: return solution not precise enough

10: end if

11: return $w^{\prime\prime}$

12: end if

13: end for

14: return solution not found

15:end function

Direct Iterative Algorithm (DIA)

If the algorithm DIA returns the correct value of the vector of weights, this
result can be taken as the correct solution. If the calculated result is too
inaccurate you can increase the number of iterations or the delta parameter
and try to run the algorithm again. However, if the calculation does not
converge despite increasing the parameters $\gamma$, $\delta$ and $\epsilon$
it is necessary to use more advanced optimization methods (Section: 2.3).

###  3.4 Additive profile aggregation

The aggregation model outlined above (Section 3.3) in which experts’ priority
is linked to their ranking score naturally has an additive version. Thus, the
solution must satisfy an additional condition of the form:

$\text{AAIP}(W,p)=\left(\begin{array}[]{c}\sum_{i=1}^{k}p(a_{i})w_{i}(a_{1})\\\
\sum_{i=1}^{k}p(a_{i})w_{i}(a_{2})\\\ \vdots\\\ \vdots\\\
\sum_{i=1}^{k}p(a_{i})w_{i}(a_{n})\end{array}\right)=\left(\begin{array}[]{c}p(a_{1})\\\
p(a_{2})\\\ \vdots\\\ \vdots\\\ p(a_{n})\end{array}\right).$ (9)

As before, we also assume that $p(a_{i})>0$ for $i=1,\ldots,n$ and
$\sum_{i=1}^{n}p(a_{i})=1$. The optimization problem induced by condition (9)
gets the form:

$\displaystyle\min h(W,p)\,\,\,\,\text{s.t.}$ $\displaystyle
h(W,p)=\sum_{i=1}^{n}\left(\sum_{j=1}^{n}p(a_{j})w_{j}(a_{i})-y_{i}\right)^{2}$
(10) $\displaystyle
p(a_{i})=\frac{y_{i}}{\sum_{j=1}^{n}y_{j}}\,\,\,\text{and}\,\,p(a_{i})>0\,\,\text{for}\,\,i=1,\ldots,n.$

DIA algorithm itself undergoes a cosmetic change of replacing GAIP with AAIP
in the lines 2 and 6.

###  3.5 Pareto principle

######  Theorem 2.

Both modified aggregation methods GAIP and AAIP satisfy the Pareto principle
[3] i.e. if for any two alternatives $i,j$ all experts prefer the i-th
alternative over the j-th alternative i.e. $w_{k}(a_{i})>w_{k}(a_{j})$ for
$k=1,\ldots,n$ then this relationship holds in the aggregated vector of
weights i.e. $w(a_{i})>w(a_{j})$.

###### Proof.

Let GAIP and AAIP be as defined in (5) and (6) correspondingly. Then

$\displaystyle||GAIP(W,p)||=$ $\displaystyle=\sqrt{(w_{1}^{p_{1}}(a_{1})\cdots
w_{n}^{p_{n}}(a_{1}))^{2}+\ldots+(w_{1}^{p_{1}}(a_{n})\cdots
w_{n}^{p_{n}}(a_{n}))^{2}}\leq$
$\displaystyle\leq\sqrt{(p_{1}w_{1}(a_{1})+\ldots+p_{n}w_{n}(a_{1}))^{2}+\ldots+(p_{1}w_{1}(a_{n})+\ldots+p_{n}w_{n}(a_{n}))^{2}}=$
$\displaystyle=||AAIP(W,p)||=$ $\displaystyle=\sqrt{\langle
p,w_{1}\rangle^{2}+\ldots\langle
p,w_{n}\rangle^{2}}\leq\sqrt{||p||^{2}||w_{1}||^{2}+\ldots+||p||^{2}||w_{n}||^{2}}=$
$\displaystyle=||p||\sqrt{||w_{1}||^{2}+\ldots+||w_{n}||^{2}}\leq||p||\sqrt{n}\leq\sqrt{n}.$

Obviously, the mapping $AAIP(W,\cdot)$ is linear, so for each
$p,\hat{p}\in\mathbb{R}^{n}$

$AAIP(W,p)-AAIP(W,\hat{p})=AAIP(W,p-\hat{p})\leq\sqrt{n}||p-\hat{p}||.$

∎

Notice that for $p=(\frac{1}{n},\ldots,\frac{1}{n})^{T}$ the norm

$||p||=\frac{1}{\sqrt{n}},$

so

$||GAIP(W,p)||\leq||AAIP(W,p)||\leq 1.$

On the other hand, if for each $i\in\\{1,\ldots,n\\}$ we have

$w_{1}(a_{i})=w_{2}(a_{i})=\ldots=w_{n}(a_{i}),$

then for each $j\in\\{1,\ldots,n\\}$ and each $p$ it follows that

$GAIP(W,p)=AAIP(W,p)=(w_{j}(a_{1}),\ldots,w_{j}(a_{n}))^{T}.$

Now fix $i,j\in\\{1,\ldots,n\\}$. Both mappings satisfy the Pareto principle,
i.e. for each $p\in\mathbb{R}^{n}$ and for each $k\in\\{1,\ldots,n\\}$

$w_{k}(a_{i})\leq w_{k}(a_{i})\Rightarrow\left(w_{k}^{p_{k}}(a_{i})\leq
w_{k}^{p_{k}}(a_{j})\textnormal{ and }p_{k}w_{k}(a_{i})\leq
p_{k}w_{k}(a_{j})\right),$

which implies that

$w_{1}^{p_{1}}(a_{i})\cdot w_{2}^{p_{2}}(a_{i})\cdots w_{n}^{p_{n}}(a_{i})\leq
w_{1}^{p_{1}}(a_{j})\cdot w_{2}^{p_{2}}(a_{j})\cdots w_{n}^{p_{n}}(a_{j})$

and

$p_{1}w_{1}(a_{i})+p_{2}w_{2}(a_{i})+\cdots+p_{n}w_{n}(a_{i})\leq
p_{1}w_{1}(a_{j})+p_{2}w_{2}(a_{j})+\cdots+p_{n}w_{n}(a_{j}).$

This means that if all the experts scored the $i$-th one equally or lower than
the $j$-th one, then the resulting ranking of the $i$-th expert cannot be
higher than the ranking of the $j$-th one.

##  4 Summary

In the work presented here, we proposed two new priority aggregation models
for group decision-making in the pairwise comparison method, in which experts’
priorities are linked to the aggregation score. Both models satisfy the Pareto
principle, according to which, for two alternatives, $a_{i}$ and $a_{j}$, if
each group member prefers $a_{i}$ to $a_{j}$, then the group must also prefer
$a_{i}$ to $a_{j}$.

These models correspond to the situation in which the experts being evaluated
are the object of evaluation. A study example is determining the distribution
of a team achievement award. Such an achievement could be the preparation and
publication of a scientific article. However, the presented approach can have
many other applications, such as the distribution of bonuses among team
members for achieving milestones in an IT project.

We have also indicated methods for calculating the solution to the problem
described by such models. Very often, such a solution can be calculated using
an iterative process. When this fails, global optimization methods such as the
Nelder-Mead method and others come to the rescue. In future research, we will
focus on the properties of the proposed models. In particular, we intend to
research the safety and robustness of such models against manipulative
behavior.

## Acknowledgment

The research has been supported by the National Science Centre, Poland, as a
part of the project SODA no. 2021/41/B/HS4/03475. Jacek Szybowski has also
been partially supported by the Polish Ministry of Science and Higher
Education within the internal task of AGH University of Krakow no.
11.11.420.004. This research was funded in whole or in part by National
Science Centre, Poland 2021/41/B/HS4/03475. For the purpose of Open Access,
the author has applied a CC-BY public copyright license to any Author Accepted
Manuscript (AAM) version arising from this submission.

## References

  * Aczél and Saaty [1983] Aczél, J., Saaty, T.L., 1983\.  Procedures for synthesizing ratio judgements.  Journal of Mathematical Psychology 27, 93–102.  URL: http://dx.doi.org/10.1016/0022-2496(83)90028-7, doi:10.1016/0022-2496(83)90028-7.
  * Altuzarra et al. [2007] Altuzarra, A., Moreno-Jiménez, J.M., Salvador, M., 2007.  A Bayesian priorization procedure for AHP-group decision making.  European Journal of Operational Research 182, 367–382.  URL: https://linkinghub.elsevier.com/retrieve/pii/S0377221706008034, doi:10.1016/j.ejor.2006.07.025.
  * Arrow [1977] Arrow, K.J., 1977.  Social Choice and Individual Values.  Cowles Foundation Monographs. 2 ed., Yale University Press, New Haven, CT.
  * Barzilai and Golany [1994] Barzilai, J., Golany, B., 1994\.  AHP rank reversal, normalization and aggregation rules.  INFOR - Information Systems and Operational Research 32, 57–64.
  * Blagojevic et al. [2016] Blagojevic, B., Srdjevic, B., Srdjevic, Z., Zoranovic, T., 2016\.  Heuristic aggregation of individual judgments in AHP group decision making using simulated annealing algorithm.  Information Sciences 330, 260–273.
  * Bozóki et al. [2010] Bozóki, S., Fülöp, J., Rónyai, L., 2010.  On optimal completion of incomplete pairwise comparison matrices.  Mathematical and Computer Modelling 52, 318 – 333.  doi:http://dx.doi.org/10.1016/j.mcm.2010.02.047.
  * Bozóki and Tsyganok [2019] Bozóki, S., Tsyganok, V., 2019\.  The (logarithmic) least squares optimality of the arithmetic (geometric) mean of weight vectors calculated from all spanning trees for incomplete additive (multiplicative) pairwise comparison matrices.  International Journal of General Systems 48, 362–381.  doi:10.1080/03081079.2019.1585432.
  * Brunelli [2018] Brunelli, M., 2018.  A survey of inconsistency indices for pairwise comparisons.  International Journal of General Systems 47, 751–771.
  * Choo and Wedley [2004] Choo, E.U., Wedley, W.C., 2004\.  A common framework for deriving preference values from pairwise comparison matrices.  Computers and Operations Research 31, 893 – 908.  URL: http://www.sciencedirect.com/science/article/pii/S030505480300042X, doi:10.1016/S0305-0548(03)00042-X.
  * Coffey and Claudio [2021] Coffey, L., Claudio, D., 2021\.  In defense of group fuzzy AHP: A comparison of group fuzzy AHP and group ahp with confidence intervals.  Expert Systems with Applications 178, 114970.  URL: https://www.sciencedirect.com/science/article/pii/S0957417421004115, doi:https://doi.org/10.1016/j.eswa.2021.114970.
  * Cooper et al. [2011] Cooper, W.W., Seiford, L.M., Zhu, J. (Eds.), 2011.  Handbook on Data Envelopment Analysis.  Springer US.  URL: http://dx.doi.org/10.1007/978-1-4419-6151-8, doi:10.1007/978-1-4419-6151-8.
  * Crawford [1987] Crawford, G.B., 1987.  The geometric mean procedure for estimating the scale of a judgement matrix.  Mathematical Modelling 9, 327 – 334.  URL: http://www.sciencedirect.com/science/article/pii/0270025587904891, doi:http://dx.doi.org/10.1016/0270-0255(87)90489-1.
  * Dong and Xu [2016] Dong, Y., Xu, J., 2016.  Consensus Building in Group Decision Making: Searching the Consensus Path with Minimum Adjustments.  Springer.  doi:10.1007/978-981-287-892-2.
  * Dong et al. [2010] Dong, Y., Zhang, G., Hong, W., Xu, Y., 2010.  Consensus models for AHP group decision making under row geometric mean prioritization method.  Decision Support Systems 49, 281–289.
  * Duleba et al. [2022] Duleba, S., Çelikbilek, Y., Moslem, S., Esztergár-Kiss, D., 2022\.  Application of grey analytic hierarchy process to estimate mode choice alternatives: A case study from Budapest.  Transportation Research Interdisciplinary Perspectives 13, 100560.  URL: https://linkinghub.elsevier.com/retrieve/pii/S2590198222000239, doi:10.1016/j.trip.2022.100560.
  * Escobar and Moreno-Jiménez [2007] Escobar, M.T., Moreno-Jiménez, J.M., 2007\.  Aggregation of Individual Preference Structures in AHP-Group Decision Making.  Group Decision and Negotiation 16, 287–301.
  * Forman and Peniwati [1998] Forman, E., Peniwati, K., 1998\.  Aggregating individual judgments and priorities with the analytic hierarchy process.  European Journal of Operational Research 108, 165–169.
  * Gendreau and Potvin [2019] Gendreau, M., Potvin, J.Y. (Eds.), 2019\.  Handbook of Metaheuristics. volume 272 of International Series in Operations Research & Management Science.  Springer International Publishing, Cham.  URL: http://link.springer.com/10.1007/978-3-319-91086-4, doi:10.1007/978-3-319-91086-4.
  * Gong et al. [2024] Gong, G., Zhou, X., Zha, Q., 2024.  Managing fairness and consensus based on individual consciousness of preventing manipulation.  Information Fusion 102, 102047\.  URL: http://dx.doi.org/10.1016/j.inffus.2023.102047, doi:10.1016/j.inffus.2023.102047.
  * Greco et al. [2016] Greco, S., Ehrgott, M., Figueira, J. (Eds.), 2016.  Multiple Criteria Decision Analysis: State of the Art Surveys.  Springer.
  * Grošelj and Dolinar [2023] Grošelj, P., Dolinar, G., 2023\.  Group AHP framework based on geometric standard deviation and interval group pairwise comparisons.  Information Sciences , S002002552300035XURL: https://linkinghub.elsevier.com/retrieve/pii/S002002552300035X, doi:10.1016/j.ins.2023.01.034.
  * Hefnawy and Mohammed [2014] Hefnawy, E.A., Mohammed, A.S., 2014\.  Review of different methods for deriving weights in The Analytic Hierarchy Process.  International Journal of Analytic Hierarchy Process 6, 92 – 123.
  * Karczmarek et al. [2021] Karczmarek, P., Pedrycz, W., Kiersztyn, A., 2021.  Fuzzy Analytic Hierarchy Process in a Graphical Approach.  Group Decision and Negotiation URL: https://doi.org/10.1007/s10726-020-09719-6, doi:10.1007/s10726-020-09719-6. publisher: Springer Netherlands ISBN: 0123456789.
  * Kułakowski [2020a] Kułakowski, K., 2020a.  On the geometric mean method for incomplete pairwise comparisons.  Mathematics 8, 1–12.
  * Kułakowski [2020b] Kułakowski, K., 2020b.  Understanding the Analytic Hierarchy Process.  Chapman and Hall / CRC Press.  doi:10.1201/b21817.
  * Kułakowski et al. [2024] Kułakowski, K., Szybowski, J., Mazurek, J., Ernst, S., 2024\.  Resilient heuristic aggregation of judgments in the pairwise comparisons method.  Information Sciences 657, 119979\.  URL: https://www.sciencedirect.com/science/article/pii/S0020025523015645, doi:https://doi.org/10.1016/j.ins.2023.119979.
  * Liang et al. [2024] Liang, X., Guo, J., Liu, P., 2024.  A consensus model considers managing manipulative and overconfident behaviours in large-scale group decision-making.  Information Sciences 654, 119848\.  URL: http://dx.doi.org/10.1016/j.ins.2023.119848, doi:10.1016/j.ins.2023.119848.
  * Lin and Kou [2015] Lin, C., Kou, G., 2015.  Bayesian revision of the individual pair-wise comparison matrices under consensus in AHP–GDM.  Applied Soft Computing 35, 802–811.  URL: https://linkinghub.elsevier.com/retrieve/pii/S1568494615001428, doi:10.1016/j.asoc.2015.02.041.
  * Lin et al. [2022] Lin, C., Kou, G., Peng, Y., Alsaadi, F.E., 2022.  Aggregation of the nearest consistency matrices with the acceptable consensus in AHP-GDM.  Annals of Operations Research 316, 179–195.  URL: https://link.springer.com/10.1007/s10479-020-03572-1, doi:10.1007/s10479-020-03572-1.
  * Munier and Hontoria [2021] Munier, N., Hontoria, E., 2021\.  Uses and Limitations of the AHP Method: A Non-Mathematical and Rational Analysis.  Management for Professionals, Springer International Publishing, Cham.  URL: http://link.springer.com/10.1007/978-3-030-60392-2, doi:10.1007/978-3-030-60392-2.
  * Nelder and Mead [1965] Nelder, J.A., Mead, R., 1965\.  A Simplex-Method for Function Minimization.  Computer Journal 7, 308–313.
  * Ramanathan and Ganesh [1994] Ramanathan, R., Ganesh, L., 1994\.  Group preference aggregation methods employed in AHP: An evaluation and an intrinsic process for deriving members’ weightages.  European Journal of Operational Research 79, 249 – 265.  URL: https://doi.org/10.1016/0377-2217(94)90356-5, doi:10.1016/0377-2217(94)90356-5.
  * Saaty [1977] Saaty, T.L., 1977.  A scaling method for priorities in hierarchical structures.  Journal of Mathematical Psychology 15, 234 – 281.  doi:10.1016/0022-2496(77)90033-5.
  * Saaty [1989] Saaty, T.L., 1989.  Group Decision Making and the AHP. Springer Berlin Heidelberg, Berlin, Heidelberg.  pp. 59–67.  URL: https://doi.org/10.1007/978-3-642-50244-64, doi:10.1007/978-3-642-50244-64.
  * Saaty and Hu [1998] Saaty, T.L., Hu, G., 1998.  Ranking by eigenvector versus other methods in the analytic hierarchy process.  Applied Mathematics Letters 11, 121–125.
  * Singh et al. [2020] Singh, A., Beg, I., Kumar, S., 2020.  Analytic Hierarchy Process for Hesitant Probabilistic Fuzzy Linguistic Set with Applications to Multi-criteria Group Decision-Making Method.  International Journal of Fuzzy Systems 22, 1596–1606.  URL: https://link.springer.com/10.1007/s40815-020-00874-1, doi:10.1007/s40815-020-00874-1.
  * Srdjevic and Srdjevic [2023] Srdjevic, B., Srdjevic, Z., 2023\.  Prioritisation in the analytic hierarchy process for real and generated comparison matrices.  Expert Systems with Applications 225, 120015.  URL: https://www.sciencedirect.com/science/article/pii/S0957417423005171, doi:https://doi.org/10.1016/j.eswa.2023.120015.
  * Vanderbei [2008] Vanderbei, R.J., 2008.  Linear programming - Foundations and Extensions.  International series in operations research and management science, Springer, Boston, MA.
  * Zadnik and Groselj [2013] Zadnik, S.L., Groselj, P., 2013\.  Estimating priorities in group AHP using interval comparison matrices.  Multiple Criteria Decision Making , 143–159.
  * Zelinka et al. [2013] Zelinka, I., Snášel, V., Abraham, A. (Eds.), 2013.  Handbook of Optimization: From Classical to Modern Approach. volume 38 of Intelligent Systems Reference Library.  Springer Berlin Heidelberg, Berlin, Heidelberg.  URL: https://link.springer.com/10.1007/978-3-642-30504-7, doi:10.1007/978-3-642-30504-7.
  * Zhao et al. [2023] Zhao, Y., Gong, Z., Wei, G., Słowiński, R., 2023.  Consensus modeling with interactive utility and partial preorder of decision-makers, involving fairness and tolerant behavior.  Information Sciences 638, 118933\.  URL: http://dx.doi.org/10.1016/j.ins.2023.118933, doi:10.1016/j.ins.2023.118933.