# A Strong Sustainability Paradigm Based Analytical Hierarchy Process (SSP-
AHP) Method to Evaluate Sustainable Healthcare Systems

Jarosław Wątróbski  Aleksandra Bączkiewicz  Iga Rudawska  Institute of
Management, University of Szczecin, ul. Cukrowa 8, 71-004 Szczecin, Poland
National Institute of Telecommunications, ul. Szachowa 1, 04-894 Warsaw,
Poland  Doctoral School of University of Szczecin, ul. Mickiewicza 16, 70-383
Szczecin, Poland  Institute of Economics and Finance, University of Szczecin,
ul. Mickiewicza 64, 71-101 Szczecin, Poland

###### Abstract

The recent studies signify the growing concern of researchers towards
monitoring and measuring sustainability performance at various levels and in
many fields, including healthcare. However, there is no agreed approach to
assessing the sustainability of health systems. Moreover, social indicators
are less developed and less succinct. Therefore, with this work, authors seek
to map sustainable reference values in healthcare and propose a conceptual and
structured framework that can guide the measurement of the social
sustainability-oriented health systems. Based on a new multi-criteria method
called Strong Sustainability Paradigm based Analytical Hierarchy Process (SSP-
AHP), the presented approach opens the availability for systems’ comparison
and benchmarking. The Strong Sustainability Paradigm incorporated into the
multi-criteria evaluation method prevents the exchangeability of criteria by
promoting alternatives that achieve good performance values on all criteria,
implying sustainability. The research results offer insights into the core
domains, sub-domains, and indicators supporting a more comprehensive
assessment of the social sustainability of health systems. The framework
constructed in this study consists of five major areas: equity, quality,
responsiveness, financial coverage, and adaptability. The proposed set of
indicators can also serve as a reference instrument, providing transparency
about core aspects of performance to be measured and reported, as well as
supporting policy-makers in decisions regarding sectoral strategies in
healthcare. Moreover, the presented approach can be helpful while performing
cross-national comparisons. Our findings suggest that the most socially
sustainable systems are Nordic countries. They offer a high level of social
and financial protection, achieving very good health outcomes. Furthermore,
their social sustainability is safeguarded by balancing all identified values.
On the other hand, the most unsustainable systems, located in central and
eastern European countries, have a high level of unmet needs, experiencing
shortages in the workforce and underfunding of healthcare. A key challenge for
them is to achieve sustainability by increasing the share of public
expenditure on health and limiting the substantial dependence on out-of-pocket
payments.

###### keywords:

Strong sustainability paradigm , MCDA , SSP-AHP , Multi-criteria healthcare
systems assessment

††journal: Ecological Indicatorse-maile-mailfootnotetext:
<EMAIL_ADDRESS>(A. Bączkiewicz)e-maile-
mailfootnotetext<EMAIL_ADDRESS>(I. Rudawska)

##  1 Introduction

In the new millennium sustainability is top on the agenda of international
bodies and agencies such as United Nations, European Commission and
Organization for Economic Cooperation and Development (OECD). Owning to Agenda
2030 (UN, 2015) and its 17 goals, the whole systems, industries and
organizations have been called to rethink, manage and reduce the impact they
have on economies, people and the planet.

The concept of sustainability has been well established in manufacturing
industries (Ali et al., 2021; Gouda and Saranga, 2020) and is gaining more and
more recognition in service sector (Hussain et al., 2016). Its understanding
is not limited to environmental issues (e.g. waste management and pollution)
and economic issues (e.g. cost-efficiency), but covers a wide spectrum of
problems concerning welfare and wellbeing of various stakeholders.
Nevertheless, relatively extensive concern has been expressed so far towards
economic (Seuring and Müller, 2008; Visser and Sunter, 2002) and environmental
(Saad, 2003; Sarkis et al., 2010)goals of industries’ development. The
importance of social aspect of sustainability increases with the complexity of
systems and the multiplicity of stakeholders who are involved (Ajmal et al.,
2018; Colantonio, 2009). The inclination towards obtaining social
sustainability in the long run is reinforced by political agendas, global
transformations (Murphy, 2012), and humanitarian crisis such as COVID
pandemic. It refers predominantly to healthcare sector, that is responsible
for providing services to promote, restore, and improve the health of
population (Murray and Frenk, 2000). Moreover, apart from health improvement,
actions undertaken within health systems should be responsive and fair in
financial contribution (Popescu et al., 2018). These intrinsic goals
formulated towards health systems seem to correlate with the people-oriented
dimension of social pillar of sustainability. It emphasizes the wellbeing of
individuals and the fair distribution of resources and that is why it seems to
be the most relevant for the environment of healthcare (Maghsoudi et al.,
2020).

To obtain and maintain social sustainability in the healthcare context, a
health system ought to provide sufficient resources (human, financial, and
physical), and processes to meet individual and public health and wellbeing.
In spite of the growing interests expressed by many researchers towards the
concept of social pillar of sustainability (Hajirasouli and Kumarasuriyar,
2016; Huq et al., 2014), also in the context of healthcare (Capolongo et al.,
2016; Macassa and Tomaselli, 2020; Maghsoudi et al., 2020), there is still a
little research on the systemic, macro-level factors contributing to the
development of sustainability-oriented heath systems, highlighting the role of
the social dimension. Moreover, there is no consensus so far how to monitor
and measure the health system performance in terms of the sustainability. Most
of the previous research focus on the organizational factors and explore the
various motivators across healthcare supply chain (Hussain et al., 2019) and
within the individual care facilities (AlJaberi et al., 2020; Cristiano et
al., 2021; Jahani Sayyad Noveiri and Kordrostami, 2021). All of them take a
micro perspective and abstract from the sector-wide level.

The literature review clearly demonstrates that MCDA methods have proven their
usefulness in the area of sustainable healthcare systems assessment (Puška et
al., 2021; Öztürk et al., 2020; Zamora et al., 2021). Furthermore, the
analysis of the works mentioned above shows a great practical potential of
MCDA methods from the so-called American school (Dehe and Bamford, 2015),
including the AHP method (Rahman et al., 2021). The reasons for this are
undoubtedly the simplicity of the algorithms themselves (Aktas et al., 2015)
and easy adaptation of the AHP method to any form of the real evaluation
problem (AlJaberi et al., 2017; Singh et al., 2014). On the other hand, it is
possible to point out a significant disadvantage of the AHP method: the linear
compensation criteria effect. The problem indicated in the practice term means
that this method supports only the weak sustainability paradigm (Ziemba et
al., 2017). It is particularly important in the practical development of
sustainability assessment (Ziemba, 2019) based models, where in practice
(depending on the preference of the decision-maker or the purpose of the
assessment), there is a need to model both strong and weak sustainability
paradigms (Ziemba, 2020).

Therefore, the purpose of this work is to: a) map the sustainable values that
would fit the context of the health system in terms of its social goals, b)
introduce a conceptual and structured framework that can guide the evaluation
and measurement of the social sustainability-oriented health systems, c)
develop of a multi-criteria method for evaluating healthcare systems oriented
towards social sustainability, considering the strong sustainability paradigm.

In the methodological term, an additional challenge undertaken by the authors
is to take into account the dynamic influence of the decision-maker on the
degree of criteria compensation, which in practice makes the method
independent of the given evaluation problem and allows to model the level of
strong sustainability paradigm arbitrarily. In order to prove the reliability
of the proposed SSP-AHP method, its performance for expert weights determined
is compared using the functions of the AHP method and Entropy and CRITIC. The
functions included in the implemented AHP method, such as the eigenvector
method, the normalized column sum method, and the geometric mean method, can
be used both to determine the criteria weights based on decision-makers
subjective indications and for multi-criteria evaluation of the sustainability
of alternatives as well as a modification of the SSP-AHP with the strong
sustainability paradigm (Duleba, 2022).

Rest of the paper is organised as follows. Section 2 provides literature
review. In section 3 the conceptual framework with indicators relevant in
sustainable healthcare is detailed. Section 4 introduces research methodology
and framework. Obtained results are presented in section 5 and discussed in 6.
Finally, conclusions and future work directions are drawn in section 7.

##  2 Literature review

###  2.1 Sustainability and social sustainability in healthcare context

There are many definitions for sustainability in the literature. The most
general one was proposed by World Commission on Environment and Development in
1987 and perceives sustainability as ”meeting the needs of the present without
compromising the ability of future generations to meet their own needs” (WCED,
1987). This interpretation lacks any criteria that could be indicative of the
sustainability phenomenon. Since 1987 many other interpretations have been
developed (Sneddon et al., 2006), that can be found suitable also for
healthcare industry. However, there is currently no consensus in the field on
which definition of sustainability to operationalise (Lennox et al., 2020).
Sustainability can be considered at various levels. In narrow sense, it may
refer to a program, clinical intervention, or strategy that continue to be
delivered after a defined period of time and keep to produce benefits for the
target group (Moore et al., 2017). Many research findings demonstrate the
application of sustainability practices, across a range of healthcare
settings, including primary, secondary, tertiary and community healthcare
(Lennox et al., 2020). In a broader sense, health system sustainability refers
to the maintenance and constant adaptation to a changing environmental, social
and economic context, assuring the effective and responsible use of limited
healthcare resources in order to keep and improve the health and well-being of
the population and of each individual (Maghsoudi et al., 2020). It is in line
with the WHO’s goals for health systems, namely: improving health and health
equity in ways that are responsive, financially fair, and make most efficient
use of available resources (WHO, 2007) (WHO, 2000c, 2010).

Given the above mentioned objectives of health systems social sustainability
seems to be the most paramount in healthcare context. It is traditionally
understood as a human-oriented dimension, that emphasizes the wellbeing of
individuals and the fair distribution of resources (Vuong et al., 2017).
According to literature, this concept encompasses social issues, such as
justice, equity and access to basic needs (Awan et al., 2018). Maghsoudi et
al. indicate that such interpretation is appropriate in healthcare context, as
it embodies both the improvement of the wellbeing of patients and the calls
for justice in the distribution of resources (Maghsoudi et al., 2020).
Additionally, Capolongo et al. (Capolongo et al., 2016) highlight that social
sustainability implies also professionals’ well-being, safety, security and
satisfaction. Other authors (Chiu, 2003) stress the importance of providing
equal opportunities to access health and safety resources and addressing the
expectations of stakeholders, mainly patients while assessing the social
sustainability in healthcare. Nevertheless, the issue what measures to apply
to sector-wide perspective and how to choose the relevant indicators remains
unsolved.

Of course, there are other approaches that incorporate MCDA methods and
analyze the dynamics of variation in outcomes over time (Pianosi et al.,
2016). An example is a work in which we considered data collected for five
years for several major cities in the United States to temporally assess
integrated health status using a composite index based on maximum entropy
networks (MENets) (Servadio and Convertino, 2018). There are also approaches
in multi-criteria sustainability assessment considering several indicators
based on machine learning models (Lawal et al., 2021) and stochastic
optimization algorithms in multi-criteria analytical decision models
(Convertino et al., 2019).

###  2.2 The assessment of sustainability in healthcare

Sustainability assessment seeks to monitor and evaluate the extent to which
various aspects of a given organization, industry or system meets key goals of
sustainable development (Bond and Morrison-Saunders, 2013). In that sense it
is a kind of performance evaluation that is seen as a process of collecting,
computing and communicating quantified constructs for the managerial purposes
(Kollberg et al., 2005; Lizarondo et al., 2014), at various levels: global,
national, sector-wide and company-level. The goal of this process is always
the same - to follow up, monitor and improve the performance of the given
system or organization. Although, there is a growing evidence on how to assess
corporate sustainability (Hallstedt, 2017; Hugé et al., 2013; Moldavska and
Welo, 2016, 2019; Morrison-Saunders et al., 2015; Pope et al., 2015, 2017;
Singh et al., 2014; Waas et al., 2014; Ziout et al., 2013), and sustainability
of public services (Bandeira et al., 2018), there is no agreed approach to
measuring sustainability in healthcare context, especially at the sector-wide
level. Few works in this field have focused on the organizational level and
aimed at developing the assessment criteria that were linked to strategic
planning of the health service organisation (Hussain et al., 2016). AlJaberi
et al. (AlJaberi et al., 2020) introduced an interesting assessment framework,
based on analytical hierarchical process (AHP), proposing factors that cover
core business functions of healthcare organization. The concept is very
interesting, but does not correspond strictly to the sector-wide level. In
turn, Noveiri and Kordrostami (Jahani Sayyad Noveiri and Kordrostami, 2021)
proposed an approach, based on data envelopment analysis (DEA), applied to
measure the sustainability performance of hospitals in the presence of
imprecise measures. Although, the findings disclose informative details about
overall sustainability performance in hospitals, they cannot be directly
applied at the sector-wide level. The similar conclusion can be drawn from the
framework introduced by Sarriot et al. (Sarriot et al., 2004), that places
sustainability at the center of primary health care programming. This
promising methodology concerns the assessment of sustainability in NGO primary
health care and community-based projects.

Although, social sustainability and its measurement are gaining more and more
recognition in the literature dedicated to manufacturing (Rajak et al., 2016)
and services (Kumar and Anbanandam, 2019), and even cities and regions
(Dempsey et al., 2011; Yuan et al., 2020), a few studies have contributed to
delineating the significance of social sustainability in healthcare industry.
For instance, Hussian et al. (Hussain et al., 2019) presented an AHP model to
explore and assess the motivators of social sustainability in healthcare
supply chain context. The work captures various motivating practices such as
media and reputation, excellence and awards, organizational practice,
technology and innovation, and attitudes. On the other hand, core categories
like humanization, comfort and distribution were covered by Capolongo et. al.
(Capolongo et al., 2016) to assess the social sustainability of hospital
facility. In other studies, the social dimension has been designed for
assessing customer, employee, and community well-being impacted mainly by
manufacturing activities and manufactured products (Eslami et al., 2019). The
range and the scope of the assessment of social pillar was wide and diverse,
indicating the categories that reflected the attitude of a company towards its
stakeholders and societal impact of the product (for example customer
satisfaction, occupational health and safety, security and wages of employees,
their satisfaction, education and training, product responsibility (Eastlick
and Haapala, 2012; Huang and Badurdeen, 2017; Lu et al., 2011; Rezvan et al.,
2014)). All works cited demonstrate valued contribution to the understanding
of sustainability at the organizational level, but their findings have poor
adaptability to the context of a whole-sector assessment. Therefore, with this
work authors seek to identify multiple criteria which are broad in scope and
help to obtain comprehensive picture of social sustainability of health
system.

###  2.3 Multi-criteria methods in healthcare domain

As it was presented in the previous section, different quantitative multi-
criteria approaches are applied in healthcare assessment (AlJaberi et al.,
2017; Puška et al., 2021; Torkayesh et al., 2021). In this section, MCDA-based
approaches will be analyzed in detail in the terms of typology of models and
methods used. An overview of MCDA methods used in the healthcare domain is
summarized in Table 1 considering methods from American school, namely AHP
(Németh et al., 2019), MAVT (Angelis and Kanavos, 2017), CRADIS (Puška et al.,
2021), fuzzy TOPSIS and VIKOR (Öztürk et al., 2020), MACBETH (Pereira et al.,
2020), CoCoSo (Torkayesh et al., 2021), DEMATEL (Ortíz et al., 2016), MAUT
(Zamora et al., 2021), and methods representing European school, such as
ELECTRE TRI-NC (Rocha et al., 2021) and PROMETHEE (Makan and Fadili, 2021).
Most of the works cited are based on methods from the American school. Among
them, the AHP method is one of the most popular and widely used methods for
sustainability assessment in the healthcare field, both for the location of
medical sites and health systems assessment, especially when a hierarchical
model with a large number of criteria is to be evaluated (Dell’Ovo et al.,
2018). Weights are most often determined by experts and decision-makers using
methods such as AHP, SWING, BWM (Mühlbacher and Kaczynski, 2016). The
advantage of the AHP method is an easy adaptation to hierarchical structure
often found in models of sustainable healthcare problems easy use even for
many alternatives under evaluation (Dehe and Bamford, 2015). The AHP method is
also used in healthcare fields such as assessing sustainable service quality
(Aktas et al., 2015; Singh and Prasher, 2019), assessing sustainable
electronic services quality (Büyüközkan and Çifçi, 2012), general measurement
of sustainability (AlJaberi et al., 2017) and selecting sustainable healthcare
infrastructure site location (Dehe and Bamford, 2015).

However, there are also papers presenting the application of methods from the
European school. Among them is PROMETHEE, which application for assessment of
the sustainable healthcare waste treatment system is presented in paper (Makan
and Fadili, 2021). PROMETHEE belongs to the outranking methods family, similar
to ELECTRE family methods. Among the examples of applying methods belonging to
the ELECTRE family in healthcare systems assessment, it is worth mentioning
the application of the ELECTRE TRI-NC method for assessing hospital
sustainability and quality (Rocha et al., 2021). A low level of criteria
compensation characterizes both ELECTRE and PROMETHEE families of MCDA
methods, so they are suitable for evaluating multi-criteria problems that
focus on sustainability.

Table 1: MCDA methods applied to sustainability healthcare assessment problem
in reviewed literature.

Authors |  Problem |  Methods applied |  Number of Criteria/Sub-criteria
---|---|---|---
(Dehe and Bamford, 2015) |  Selection of sustainable healthcare infrastructure site location |  ER (Evidential Reasoning) and AHP (Analytical Hierarchy Process) for determining criteria weights |  7/28
(Aktas et al., 2015) |  Assessment of sustainable service quality in healthcare field |  AHP |  7/28
(Singh and Prasher, 2019) |  Assessing the quality of sustainable health services |  Fuzzy AHP |  6/30
(AlJaberi et al., 2017) |  Measurement of sustainability in health systems |  AHP |  7/31
(Büyüközkan and Çifçi, 2012) |  Assessment of sustainable electronic services quality in healthcare industry |  Fuzzy AHP and fuzzy TOPSIS |  6/20
(Angelis and Kanavos, 2017) |  Sustainable health technology assessment including new medicines evaluation |  MAVT (Multi-Attribute Value Theory) |  4/14 (preliminaries based on literature review) and 5/11/28 (advanced practical assessment)
(Puška et al., 2021) |  Evaluation of sustainable healthcare waste |  FUCOM (FUll COnsistency Method) for determining weights and CRADIS (Compromise Ranking of Alternatives from Distance to Ideal Solution) |  4/16
(Öztürk et al., 2020) |  Selecting the most sustainable health technology |  AHP for determining weights, fuzzy TOPSIS (Technique for Order of Preference by Similarity to Ideal Solution), fuzzy VIKOR (VlseKriterijumska Optimizacija I Kompromisno Resenje), goal programming |  9/45/115/113
(Pereira et al., 2020) |  Evaluation sustainability of European health systems |  MACBETH (Measuring Attractiveness by a Categorical Based Evaluation Technique) |  3/6/9
(Németh et al., 2019) |  Sustainable decision making in healthcare |  SWING, SMART (Simple Multiattribute Rating Technique), AHP (Analytical Hierarchy Process), MACBETH, DCE (Discrete Choice Experiments), PAPRIKA (Potentially All Pairwise Rankings of All Possible Alternatives), CA (Conjoint Analysis) for criteria weighting |  Not specified
(Torkayesh et al., 2021) |  Assessment of a sustainable health care sector in selected European countries |  BWM (Best-Worst Method) and LBWA (Level Based Weight Assessment) for criteria weights determination, CoCoSo (Combined Compromise Solution) for evaluation of healthcare performances of several countries |  7
(Ortíz et al., 2016) |  Selecting the most sustainable allied hospital variant for an integrated network of laboratory services |  DEMATEL (Decision Making Trial and Evaluation Laboratory) for evaluation the independence between factors of the same category, AHP for calculating weights of criteria and categories |  4/14
(Rocha et al., 2021) |  Assessing hospital sustainability and quality |  ELECTRE TRI-NC |  5/24
(Zamora et al., 2021) |  Assessing sustainable health profit in the context of financial attributes affecting consumption |  MAUT (Multi-Attribute Theory) |  3/7/2
(Makan and Fadili, 2021) |  Assessment of the sustainability of healthcare waste treatment systems |  surrogate weighting and PROMETHEE (Preference Ranking Organization METHod for Enrichment Evaluations) |  4/16

###  2.4 Strong sustainability paradigm in MCDA methods

An important aspect in a reliable sustainability assessment with MCDA methods
is the proper modeling of criteria compensation (Ziemba et al., 2017). In
practice, there exist two paradigms: strong and weak sustainability (Ziemba et
al., 2017). The weak sustainability paradigm assumes criteria compensation,
implying that individual evaluation criteria are interchangeable and can
replace each other (Norouzi and Fani, 2021). On the other hand, strong
sustainability considers the complementarity of criteria but not
interchangeability (Ziemba et al., 2017). High compensation means that
favorable performance values of several criteria can compensate for the poor
performance of other criteria. Therefore, methods with high criteria
compensation support a weak sustainability paradigm. This problem occurs to a
higher or lower level in existing MCDA methods (Ziemba, 2019). Most MCDA
methods are based on utility (Rahman et al., 2021), or value (Stewart and
Durbach, 2016) theory and these methods are usually based on a single
synthesizing criterion. Among them, the popular AHP (Seyedmohammadi et al.,
2018) method, TOPSIS, VIKOR, DEMATEL, SMARTER (Ziemba, 2020) methods can be
mentioned. However, these methods belonging to the so-called American school
(Wątróbski et al., 2019), assuming full substitution of all criteria in the
model, therefore only fulfill a weak sustainability paradigm, which for
example, in the evaluation of healthcare is undesirable (Oppio et al., 2018).
The use of criteria weights partially reduces the criteria compensation
problem, but this is insufficient to achieve the strong sustainability
paradigm (Ziemba, 2019). Methods in which the compensation of criteria is
limited include methods of the so-called European school (Ezbakhe and Perez-
Foguet, 2018) using the outranking relation, for example, ELECTRE or PROMETHEE
(Makan and Fadili, 2021). However, methods from this group are characterized
by a complex computational procedure (Ziemba, 2019) and often recommend
alternatives ranked in non-quantitative form (Rocha et al., 2021). Moreover,
in the considered problem (healthcare sustainability assessment), often a
large number of criteria is taken into account, forming a complex hierarchical
model (Dehe and Bamford, 2015), which also significantly limits the
possibility of model structuring (Ortíz et al., 2016) in the case of methods
from the European school.

It is also worth noting that, from the point of view of the strong/weak
sustainability paradigm, in methods from the European school, the degree of
criteria compensation (although it is less than in the case of the AHP method)
is still significant (Ziemba et al., 2017). This fact encourages modifying the
algorithms of the methods to reduce the degree of criteria compensation and
thus introduce the paradigm of stronger sustainability to the known and widely
used methods. For example, such an attempt was made in the case of the
PROMETHEE II method, based on which the authors proposed the PROSA method
(Ziemba et al., 2017) and its variations, namely generalized PROSA-G (Ziemba,
2019), PROSA-C (Ziemba, 2020) to obtain a more sustainable evaluation of the
variants (Chmielarz and Zborowski, 2022). Successfully results of this attempt
encourage to take analogous efforts, especially concerning methods with
significant compensation of criteria, such as the widely used AHP method.
Furthermore, the wide popularity of the AHP method in sustainable healthcare
systems assessment motivated the authors to develop an extension of the AHP
method supporting a strong sustainability paradigm called SSP-AHP, which is
proposed in this paper.

##  3 The conceptual framework - factors affecting social sustainability in
health systems

There are many criteria that could be used to assess the health system
sustainability performance. Although, the recent studies signify the growing
concern of researchers towards this issue (Lennox et al., 2018, 2020), there
is no standard framework that would provide the universal set of criteria and
indicators. The proposed cluster of criteria (see Table 2) corresponds with
the overall goals of health system defined by WHO and social sustainability
objectives.

Table 2: The summary of main criteria and sub-criteria in health system’s
social-sustainability assessment.

Main criteria | Sub-criteria | Relevant studies | Criterion |  Name |  Unit | Goal
---|---|---|---|---|---|---
$G_{1}$ \- Equity |  Service access |  (Awan et al., 2018; Capolongo et al., 2013, 2016; Hurst and Jee-Hughes, 2001; Levesque and Sutherland, 2020; Murray and Frenk, 2000) | $C_{1}$ |  Cataract surgery performed in hospitals |  [Number per 100 000 inhabitants] | $\uparrow$
$C_{2}$ |  Hip replacement performed in hospitals |  [Number per 100 000 inhabitants] | $\uparrow$
Availability of human resorces | $C_{3}$ |  Practising physicians |  [Density per 1 000 population] | $\uparrow$
$C_{4}$ |  Practising nurses |  [Density per 1 000 population] | $\uparrow$
Availability of health infrastructure | $C_{5}$ |  Acute care beds |  [Per 1 000 population] | $\uparrow$
$C_{6}$ |  Computed Tomography scanners |  [Per 1 million population] | $\uparrow$
$C_{7}$ |  Magnetic Resonance Imaging units |  [Per 1 million population] | $\uparrow$
$G_{2}$ \- Quality of care |  Effectiveness of treatment |  (Bankauskaite and Dargent, 2007; Caunic et al., 2019; de Bienassis et al., 2020; Giovanelli et al., 2015; Hurst and Jee-Hughes, 2001; Kim and Jeon, 2020; Sarriot et al., 2004; Vuong et al., 2017) | $C_{8}$ |  AMI 30 days standardised mortality |  [Age-sex standardised rate per 100 patients] | $\downarrow$
$C_{9}$ |  Breast cancer five year net survival |  [%] | $\uparrow$
$C_{10}$ |  Lung cancer five year net survival |  [%] | $\uparrow$
$C_{11}$ |  Infant mortality |  [Deaths per 1 000 live births] | $\downarrow$
Patient safety | $C_{12}$ |  Observed percentage of hospitalized patients with at least one healthcare-associated infections |  [%] | $\downarrow$
Health outcomes | $C_{13}$ |  Life expectancy |  [Years at birth] | $\uparrow$
$C_{14}$ |  Avoidable mortality |  [Deaths per 100 000 population] | $\downarrow$
$G_{3}$ \- Responsiveness |  Patients’ experience with economic burden |  (ARĀJA and PRIEDE, 2018; Busse, 2013; Fujisawa and Klazinga, 2017; Levesque and Sutherland, 2020) | $C_{15}$ |  Self-reported unmet needs for health care due to financial reasons |  [%] | $\downarrow$
Patients’ experience with non-economic burden | $C_{16}$ |  Self-reported unmet needs for health care due to distance or transportation |  [%] | $\downarrow$
$C_{17}$ |  Self-reported unmet needs for health care due to waiting list |  [%] | $\downarrow$
$G_{4}$ \- Financial coverage |  Risk protection |  (Hurst and Jee-Hughes, 2001; Murray and Frenk, 2000; Popescu et al., 2018) | $C_{18}$ |  Social health insurance (total healthcare) |  [%] of total population | $\uparrow$
Financial contribution | $C_{19}$ |  Public expenditures on health - share of current expenditure on health |  [%] | $\uparrow$
$C_{20}$ |  Public expenditure on health |  [US Dollar per capita] | $\uparrow$
$G_{5}$ \- Adaptability |  Investments in public health |  (Demartini and Trucco, 2017; Lennox et al., 2018; Levesque and Sutherland, 2020; Sarriot et al., 2004; Urquhart et al., 2020) | $C_{21}$ |  Public expenditure on health - share of GDP |  [%] | $\uparrow$
$C_{22}$ |  Capital expenditure on health as a share of GDP |  [%] | $\uparrow$
Investments in human resources | $C_{23}$ |  Medical graduates |  [Per 100 000 population] | $\uparrow$
$C_{24}$ |  Nursing graduates |  [Per 100 000 population] | $\uparrow$
New technologies uptake | $C_{25}$ |  Proportion of primary care physician offices using electronic medical records |  [%] | $\uparrow$

The last ones are high on the political agenda of many international
initiatives and health policies, focusing on strengthening health system
accountability. For instance, Tallinn Charter signed by 53 states, promotes
the Health 2020 policy framework to ensure sustainability of the system and
the efficiency of healthcare expenditure (Tello and Baez-Camargo, 2015). The
proposed framework is grounded in the literature on management by objectives
(Drucker, 2012) and goal setting (Locke and Latham, 2013) and perceives the
definite goals as a core to assess performance. Table 2 presents the
hierarchical structure of indicators of sustainable development of healthcare
system. The model builds on the following dimensions which correspond to the
social sustainability targets: 1) equity), 2) quality, 3) responsiveness, 4)
financial coverage, and 5) adaptability. Moreover, we assume that the criteria
included in the framework have to meet the WHO standards (WHO, 2018a), namely:
a) are prominent in monitoring of major international declarations (here :
Agenda 2030, SG 3), b) are meeting the SMART formula c) there is a strong
track record of extensive measurement experience, preferably supported by an
international database (here: OECD Health Statistics and Eurostat), d) they
are used by countries in monitoring of national plans.

Universal access to health services has been formally accepted as the main
goal of health systems (WHO, 2000a), encompassing both distribution of
payments for health services and distribution of access to healthcare across
population. Thus, equity can be considered as a cross-cutting dimension
(Barsanti and Nuti, 2014) (Levesque and Sutherland, 2020). One of the most
common metric that is widely accepted in WHO (Murray and Frenk, 2000) and OECD
(Hurst and Jee-Hughes, 2001) frameworks to evaluate equity is access to
healthcare.In our study we distinguish service access and availability of
healthcare resources (both human and physical) as sub-criteria, which is in
line with the findings by Capolongo et al. (Capolongo et al., 2013, 2016) and
Awan et al. (Awan et al., 2018) It is here understood as assuring distribution
of health resources and providing equal opportunities to access these
resources.

Quality is another most used domain of health systems’ performance assessment,
widely recognized in the literature (Bankauskaite and Dargent, 2007;
Giovanelli et al., 2015; Kim and Jeon, 2020) (Hurst and Jee-Hughes, 2001)
(WHO, 2018b). In the revised OECD framework, released in 2006, quality of care
is the core dimension and incorporates effectiveness, safety and
responsiveness (Caunic et al., 2019). Quality indicators are often discussed
in relation to health improvements, that is why we distinguished health
outcomes as a sub-criterion in our model. Patient safety remains an important
indicator of quality of care, that is high on health policy agenda. Risk
mitigation, learning-based health systems, and healthcare environment design
taking human factors into account remain in the main focus (Fujisawa and
Klazinga, 2017). Thus, enhancement of patient safety can be seem an essential
component of social sustainability development in healthcare.

The growing concern towards patient’s centeredness, expressed by international
bodies (Fujisawa and Klazinga, 2017), allow us to distinguish system’s
responsiveness as a separate criterion. It is here understood as ”the ability
of the health system to meet the population’s legitimate expectations
regarding their interaction with the health system, apart from expectations
for improvements in health or wealth” (WHO, 2000b). The concept of
responsiveness was introduced by WHO in 2000 in order to strengthen the social
aspect of the health system and to underline the importance of human
relationships within it (ARĀJA and PRIEDE, 2018). Responsiveness refers to
ethical dimension of health system and covers the non-medical expectations of
patients and their experience with the system (Busse, 2013; Papanicolas et
al., 2008). Such features as respect, patient satisfaction, patient
engagement, person centered care, quality of communication as well as
patients’ insights into unmet needs are taken into consideration (Levesque and
Sutherland, 2020). Assessment of responsiveness is possible through patients’
experience surveys, presenting their insights (Matters, 2019). Therefore, we
proposed to measure responsiveness relying on two indicators derived from
public opinion surveys across countries.

There is a general consensus, that healthcare should be provided without
contributing to impacting financial strain and poverty for the individuals and
population (Popescu et al., 2018). Unfortunately, as reported by WHO (WHO,
2022) ”currently, at least half of the people in the world do not receive the
health services they need. About 100 million people are pushed into extreme
poverty each year because of out-of-pocket spending on health”. Thus,
protecting people from the financial burden in scope of healthcare should be
an important goal of social-sustainable health systems. Fairness in financial
contribution plays a central role in WHO and OECD health system performance
framework (Murray and Frenk, 2000) (Hurst and Jee-Hughes, 2001), highlighting
the importance of financial aspects. Our model encompasses both elements,
defined as sub-criteria: risk protection and financial contribution.

Last but not least, adaptability of a health system appears to be a crucial
factor affecting its social sustainability. It is also an important element in
performance assessment (Demartini and Trucco, 2017). Adaptability is here
understood as the system’s ability to flexible adjustment to the changing
trends in healthcare market, across diverse contexts of delivery. As the
health needs change, so the technologies available to meet them, systems need
to be able to adapt (Levesque and Sutherland, 2020). Thus, pace of increase in
public expenditure, investment in R&D programs, and uptake of effective new
technologies should be considered. Especially, the last issue seems to be the
key to sustainability (Lennox et al., 2018; Urquhart et al., 2020). This also
agrees with Sarriot et al. (Sarriot et al., 2004) who see management
strategies that promote continual adaptability as a feature that guides system
towards sustainable outcomes.

##  4 Research Methodology

This section presents the new multi-criteria method called Strong
Sustainability Paradigm based Analytical Hierarchy Process (SSP-AHP) as well
as the flow of the complete research framework with its particular steps in
detail. In addition, data sources and methods for modeling criteria weights
are also defined here.

###  4.1 The SSP-AHP Method

The proposed SSP-AHP method is based on the well-established and widely used
AHP method, an aggregation method developed in 1980 by Saaty (Dehe and
Bamford, 2015). Its advantages are its ease of use and scalability, which
allows it to be easily applied even for complex hierarchical decision models.
Moreover, the AHP method enables the determination of criteria weights besides
evaluating alternatives. The usefulness of AHP arises from its usefulness in
multi-criteria evaluation for both criteria weighting and evaluation of
alternatives. For this reason, several online tools and software have been
implemented and made open access, for example, AHP-OS (Goepel, 2018; Chan and
Lee, 2019) and pyrepo-mcda (Wątróbski et al., 2022a, b). The authors of this
article have developed and made available a repository with the AHP method
extended to model the compensation reduction of the SSP-AHP criteria for
sustainability assessment (energyinpython, 2022). The mentioned tools made
available to a broad scientific audience significantly enhance analytical
capabilities using the AHP method and contribute to its expansion with
additional functionalities. On the other hand, the disadvantages of AHP
include the phenomenon of criteria compensation presented in the section 2.4
and the possibility of reversal of ranking when an alternative is added or
removed from the evaluated set. However, in the case of the sustainable
healthcare evaluation problem, the set of alternatives, i.e., countries, is
stable, so the rank reversal phenomenon need not be considered here. On the
other hand, solving the problem of criteria compensation in AHP is the attempt
to involve the SSP-AHP method development undertaken by the authors in this
paper.

Because the proposed SSP-AHP method is the main research method for
investigation presented in this paper, the formal assumptions and the
algorithm of the proposed method are presented. The SSP-AHP method (the Strong
Sustainability Paradigm based Analytical Hierarchy Process method) is an
extension of the classical AHP method with the possibility of modeling the
degree of compensation of individual criteria indicated by decision-maker,
which is in accordance with the paradigm of strong sustainability. The main
advantage of this method is recommending more sustainable decision variants
and enabling investigating the sustainability of alternatives considering all
criteria or their groups. This method complements the classical AHP with the
measure of the mean absolute deviation $MAD$ of alternatives’ performances
extended by the value of the sustainability coefficient $s$. Coefficient $s$
represents the degree of reduction in criteria compensation. The subsequent
steps of the SSP-AHP method are as follows.

Step 1. Normalization of the decision matrix. The SSP-AHP method, similarly to
each MCDA method, requires a decision matrix demonstrated by Equation (1)

$X=[x_{ij}]_{m\times
n}=\left[\begin{array}[]{cccc}x_{11}&x_{12}&\cdots&x_{1n}\\\
x_{21}&x_{22}&\cdots&x_{2n}\\\ \vdots&\vdots&\vdots&\vdots\\\
x_{m1}&x_{m2}&\cdots&x_{mn}\\\ \end{array}\right]$ (1)

where $m$ denotes the number of alternatives, and $n$ represents the number of
criteria. The SSP-AHP method proposed by the authors takes as input a matrix
containing the quantitative performance values of the criteria, as the
research addresses a problem in which such values are made available. The
normalization procedure can be performed by applying any normalization method.
This research employs the Minimum-Maximum normalization method, which is
conducted using the Equation (2) for profit criteria, while for cost criteria
Equation (3) is applied.

$r_{ij}=\frac{x_{ij}-min_{j}(x_{ij})}{max_{j}(x_{ij})-min_{j}(x_{ij})}$ (2)

$r_{ij}=\frac{max_{j}(x_{ij})-x_{ij}}{max_{j}(x_{ij})-min_{j}(x_{ij})}$ (3)

Step 2. The next step involves calculating $MAD_{ij}$ of each value in the
matrix $R$, according to Equation (4)

$MAD_{ij}=|\overline{r_{j}}-r_{ij}|s_{j}$ (4)

where $R=[r_{ij}]_{m\times n}$ denotes normalized decision matrix, $m$
represents number of alternatives ($i=1,2,\ldots m$) and $n$ denotes number of
criteria ($j=1,2,\ldots n$) and $s$ denotes sustainability coefficient. This
coefficient can take values from 0% implying no criteria compensation
reduction at all, which is equivalent to classical AHP, to 100%, denoting full
criteria compensation reduction. A high $s$ coefficient indicates a high
degree of criteria compensation reduction.

Step 3. Then from the value of the normalized decision matrix, $MAD_{ij}$ is
subtracted, like Equation (5) demonstrates.

$b_{ij}=r_{ij}-MAD_{ij}$ (5)

Step 4. This step involves an aggregation procedure involving calculating the
utility function values for each alternative performing an aggregation using a
weighted sum of $b_{ij}$ values, as Equation 6 shows.

$U_{i}=\sum_{j=1}^{n}b_{ij}w_{j}$ (6)

where $w$ denotes weight value for each $j$th criterion ($j=1,2,\ldots,n$).
Weights are determined using methods provided in A.1.

Step 5. The last step of the method is to order the alternatives in descending
order according to the utility function values of the $i$th alternative
obtained in the previous step. The alternative with the highest value of
$U_{i}$ is the best option.

###  4.2 The Research Framework

This subsection demonstrates the research framework and results provided by
the SSP-AHP method. To evaluate the European countries listed in Table 3, the
authors used a hierarchical set of criteria.

Table 3: Countries selected for social sustainability-oriented health systems
evaluation.

$A_{i}$ | Country | $A_{i}$ | Country | $A_{i}$ | Country | $A_{i}$ | Country
---|---|---|---|---|---|---|---
$A_{1}$ | Belgium | $A_{5}$ | Germany | $A_{9}$ | Luxembourg | $A_{13}$ | Slovak Republic
$A_{2}$ | Czech Republic | $A_{6}$ | Hungary | $A_{10}$ | Netherlands | $A_{14}$ | Slovenia
$A_{3}$ | Finland | $A_{7}$ | Iceland | $A_{11}$ | Norway | $A_{15}$ | Sweden
$A_{4}$ | France | $A_{8}$ | Latvia | $A_{12}$ | Poland | $A_{16}$ | United Kingdom

Based on the framework and assumptions presented in section 3 regarding the
completeness of public data made available by websites of OECD Health
Statistics, Eurostat, and World Health Organization, accessed on 21 January
2022, after collecting the most recent data available and merging them, as a
results authors received an evaluation decision matrix containing the
performance values of 25 selected criteria for the sixteen alternatives
evaluated in this study. This decision matrix is available to a broader
audience in a repository available on GitHub (energyinpython, 2022). In
addition, the full Python code containing the framework of the performed
research along with Step-by-step instructions with examples based on the
problem presented in this article, complete input data and supporting methods,
and the SSP-AHP method in a separate class is made available on GitHub, which
thus can be easily used also for other decision problems than the problem
presented in this paper. Table 2 provided in section 3 presents the brief
summary of all considered criteria structured in the hierarchical model
containing main criteria $G_{1}$–$G_{5}$ and sub-criteria in the health
system’s social-sustainability assessment with names for main dimensions,
subdimensions, detailed lowest level criteria, and objective of criteria,
which is represented by 1 for profit criteria to be maximized and on the other
hand -1 denotes objective to minimize for cost criteria. In our framework, we
propose three sub-criteria to assess the health system’s adaptability:
investments in public health, investments in human resources, and new
technologies uptake.

Criteria weights were determined by experts using the AHP-based relative
weighting method, which is described in detail in A.1. In addition, a
comparative analysis was also performed using objective weighting methods for
benchmarking purposes, namely Entropy (Lotfi and Fallahnejad, 2010) and the
Criteria Importance Through Inter criteria Correlation (CRITIC) (Tuş and Aytaç
Adalı, 2019) weighting methods. It should be noted primarily that these are
methods for determining objective weights, which in our case results in the
ability to determine the objective value of the indicator without the
involvement of experts, and are often used by researchers in MCDA where there
is a need to determine an objective MCDA ranking without the use of expert
knowledge (Abdel-Basset and Mohamed, 2020; Du et al., 2020; Li et al., 2020).
These techniques were chosen as reference methods, a justification of which
can be found in (Zhao et al., 2020; Rostamzadeh et al., 2018). The CRITIC
method has the advantage of considering two dimensions of information
contained in the data when determining the weights. The first dimension is the
contrast intensity between the criteria, measured using the standard
deviation. The second dimension is the conflict between criteria measured
using the linear correlation coefficient. Considering the two dimensions
mentioned above improves the accuracy of multi-criteria decision analysis
methods (Li et al., 2022a). In the case of the Entropy weighting method, a
significant difference between criterion values represents a high amount of
information in the data. The lower the entropy value, the higher the
criterion’s weight. This method is based on Shannon entropy, which measures
the uncertainty or variability contained in the data. The lower the entropy
value, the lower the degree of chaos (Li et al., 2022b; Sidhu et al., 2022).
Of course, expert values can also be used; hence our comparison with expert
weights was determined using an AHP function called the eigenvector method
(Duleba, 2022). These objective weighting methods are detailed in A.1. In the
first part of this research, a comparative analysis of the results provided by
the classical AHP method with six benchmarking MCDA methods was conducted to
confirm the reliable results of the AHP approach. Then, a comparative analysis
with objective weighting methods was conducted to confirm the reliability of
weights determined subjectively in the classical AHP approach. In the
following research stage, an investigation using SSP-AHP increasing the
compensation reduction under each main criteria was performed. The detailed
research framework is presented in Figure 1, and the individual steps of the
framework are described in detail below.

Figure 1: Flowchart presenting application of the SSP-AHP method in
sustainable healthcare system assessment.

Step 1. Selection of alternatives and criteria.

The selection of the countries for social sustainability-oriented health
systems analysis listed in Table 3 has been based on the healthcare financing
model observed in Europa (Beverage model, Bismarck model, and hybrid one - the
national health insurance model).

Step 2. Determination of criteria weights.

In this research, to provide reliable results, the authors used weights
determined subjectively with expert involvement and weights determined from
performance values contained in the decision matrix using objective Entropy
and CRITIC weighting methods in separate framework flows. For both approaches
of determining weights, the rest of the framework steps are identical.

Step 2.1 Determination of criteria weights with the subjective AHP-based
relative weighting method involving experts.

In the first part of the research, the weights for each criteria dimension,
namely $G_{1}$-$G_{5}$, were determined based on the opinion of the domain
expert using the scale proposed by Saaty. According to AHP methodology (Rahman
et al., 2021) small sample size is acceptable, as AHP involves expert opinion
rather than surveys. In our case, the research sample consisted of sixteen
experts, which is considered to be satisfactory and comparable to other
studies in the field (AlJaberi et al., 2017). Experts in senior managerial
positions at various healthcare entities and organizations (hospitals,
insurance schemes, national health funds, ministry of health) from the
surveyed countries have been invited to participate in an online
questionnaire. The selected respondents had at least ten years of expert
experience in the health sector. In addition, they all have been engaged in
decision-making processes while setting and prosecuting health policies and
strategies. Details of the experts and the methods used to recruit them for
the study are provided in B.

In the form of pairwise competitions, experts’ judgments served as input
information for AHP analysis. The structured AHP model expresses the social
sustainability priority in respect of a set of relative importance weights for
the five major categories of social-sustainable health systems. In line with
AHP methodology (Dehe and Bamford, 2015), the geometric mean approach was
preferred over the arithmetic mean to combine the individual pairwise
comparison judgments of the 16 healthcare experts into the consensus pairwise
comparison matrix. The criteria pairwise comparison matrix provided by an
expert is displayed in Table 4.

Table 4: Pairwise comparison matrix of criteria significance.

| Equity | Quality | Responsiveness | Financial coverage | Adaptability
---|---|---|---|---|---
Equity | 1 | 1 | 5 | 3 | 9
Quality | 1 | 1 | 3 | 5 | 7
Responsiveness | 1/5 | 1/3 | 1 | 1 | 9
Financial coverage | 1/3 | 1/5 | 1 | 1 | 7
Adaptability | 1/9 | 1/7 | 1/9 | 1/7 | 1

The next stage was to check the consistency of the pairwise comparison matrix
of criteria significance, which determines whether the pairwise relationship
is consistent, using the $CR$ presented in Equation (9) coefficient. For this
matrix, the $CR$ value is equal to 0.08. A value less than 0.1 means that the
pairwise comparison result is acceptable, so this matrix containing pairwise
comparison criteria is consistent. Then the vector of weights included in
Table 5 for criteria $G_{1}$-$G_{5}$ was determined using the eigenvector
method proposed by Saaty.

Table 5: Main criteria weights determined with the AHP-based relative
weighting method.

Main criterion | $G_{1}$ | $G_{2}$ | $G_{3}$ | $G_{4}$ | $G_{5}$
---|---|---|---|---|---
Weights | 0.3689 | 0.3546 | 0.1292 | 0.1191 | 0.0282

It can be observed that in the opinion of experts, the groups of criteria
$G_{1}$ (Equity) and $G_{2}$ (Quality of care) turned out to be the most
important, as they received the highest values of weights. The weights
determined for the main criteria were then distributed equally to the sub-
dimensions and particular criteria, as shown in Table 6.

Table 6: Weights determined for dimensions, sub-dimensions, and particular
criteria by distributing equally main criteria weights to sub-criteria.

Criterion | Dimension weights | Subdimension weights | Criterion weights
---|---|---|---
$C_{1}$ | 0.3689 | 0.1230 | 0.0615
$C_{2}$ | 0.0615
$C_{3}$ | 0.1230 | 0.0615
$C_{4}$ | 0.0615
$C_{5}$ | 0.1230 | 0.0410
$C_{6}$ | 0.0410
$C_{7}$ | 0.0410
$C_{8}$ | 0.3546 | 0.1182 | 0.0296
$C_{9}$ | 0.0296
$C_{10}$ | 0.0296
$C_{11}$ | 0.0296
$C_{12}$ | 0.1182 | 0.1182
$C_{13}$ | 0.1182 | 0.0591
$C_{14}$ | 0.0591
$C_{15}$ | 0.1292 | 0.0646 | 0.0646
$C_{16}$ | 0.0646 | 0.0323
$C_{17}$ | 0.0323
$C_{18}$ | 0.1191 | 0.0595 | 0.0595
$C_{19}$ | 0.0595 | 0.0298
$C_{20}$ | 0.0298
$C_{21}$ | 0.0282 | 0.0094 | 0.0047
$C_{22}$ | 0.0047
$C_{23}$ | 0.0094 | 0.0047
$C_{24}$ | 0.0047
$C_{25}$ | 0.0094 | 0.0094

Step 2.2 Determination of criteria weights using two objective weighting
methods: Entropy and CRITIC weighting methods.

To confirm the reliability of results obtained applying the relative weighting
method with expert participation, weights were also determined with two
objective weighting methods for comparative analysis, namely Entropy and
CRITIC weighting. These objective weighting methods are based on performance
values in the dataset, have proven their usefulness in many studies involving
MCDA assessment, and are well-established alternatives to expert weights.

The obtained values of the weights by both objective weighting techniques show
that the CRITIC method more closely reflects the values of the weights
determined subjectively with the experts since it also identified criteria
$G_{1}$ and $G_{2}$ as the most relevant. Thus, it can be assumed that the
rankings provided by the SSP-AHP method using expert weights will be more
consistent with the results obtained using the weights determined by the
CRITIC method than Entropy. Criteria weights determined by all the weighting
methods used in this paper are included in Table 16 in C.

Step 3. MCDA evaluation of alternatives using the SSP-AHP method without
criteria compensation reduction.

In the following research stage, the ranking of the SSP-AHP method is
generated for the subjective weights of the criteria without reduction of
criteria compensation. The SSP-AHP method applied without criteria
compensation reduction is equivalent to classical AHP. To confirm results’
reliability, obtained ranking is compared with rankings provided by
benchmarking methods using correlation coefficients. The set of benchmarking
methods includes well-established and widely used MCDA methods, namely TOPSIS
(Chmielarz and Zborowski, 2022), MABAC (Pamučar and Ćirović, 2015), CODAS
(Badi et al., 2018), SPOTIS (Dezert et al., 2020), and PROMETHEE II (Sałabun
et al., 2020; Sotiropoulou and Vavatsikos, 2021). The basics, algorithmic
assumptions of these methods, and details of their steps can be found in the
given references. The motivation for selecting the indicated set of reference
methods was their several advantages. The PROMETHEE II method belongs to the
European methods and is an outranking method with low criteria compensation
(Sotiropoulou and Vavatsikos, 2021). The other methods represent the American
school. The MABAC method is characterized by high stability for changes in the
model, for example, within weights (Pamučar and Ćirović, 2015). The TOPSIS
method is a popular and established method that measures the Euclidean
distance of alternatives from reference points, which are the ideal and anti-
ideal solutions, to determine the best alternative (Chmielarz and Zborowski,
2022). The CODAS method, like TOPSIS, evaluates alternatives based on the
distance from the reference points, but it uses the Euclidean and taxicab
distances from the anti-ideal solution (Badi et al., 2018). The SPOTIS method
also uses the concept of reference objects. However, instead of reference
objects, it uses performance value boundaries as the ideal and anti-ideal
solution, which prevents ranking reversal in the case of adding or removing an
alternative from the evaluated set (Dezert et al., 2020).

Then, also to confirm the reliability of results, the convergence of the SSP-
AHP rankings obtained for subjective weights of criteria is compared with SSP-
AHP rankings obtained for objective weights of criteria. The correlation of
SSP-AHP ranking with the rankings of reference methods for objective weights
is also examined.

Step 4. MCDA evaluation of alternatives using the SSP-AHP method with criteria
compensation reduction.

The following research step is to apply the SSP-AHP method with a stepwise
increasing value of sustainability coefficient, gradually reducing the
criteria compensation. The procedure is performed for the expert criteria
weights and to confirm the reliability of results for the objective weights.
The convergence of the results is measured using two ranking correlation
coefficients: Weighted Spearman’s Rank Correlation Coefficient ($r_{w}$)
(Sałabun et al., 2020), and Pearson coefficient (Deng et al., 2021) detailed
in D.

Step 5. Comparative analysis of results with a set of benchmark MCDA and
weighting methods.

For this purpose, the rankings were compared using two measures of
correlation, namely the $r_{w}$ and Pearson coefficient.

Step 6.  Formulating managerial implications.

This step includes determination based on the received rankings, which
countries are the most sustainable and take the leading positions in terms of
the sustainable healthcare system. Finally, implications for the
sustainability of the healthcare system of analyzed countries are generated
with a particular focus on the effect of reducing the degree of criteria
compensation on the final rankings.

##  5 Results

This section presents and discusses the results of multi-criteria evaluating
the alternatives in terms of a sustainable healthcare system. For this
purpose, with the help of SSP-AHP without criteria compensation reduction
using the criteria weights determined by the AHP-based relative weighting
method, a multi-criteria evaluation of alternatives was carried out, resulting
in utility function values. For the SSP-AHP method, the highest utility
function value score indicates the best-scored alternative. Rankings were
generated concerning the sustainable healthcare system by sorting the utility
function values in descending order. Then the received rankings to confirm
their reliability were compared with the rankings obtained using a set of five
benchmarking MCDA methods. The SSP-AHP results were also compared with the
results gained for two different objective weighting methods: Entropy and
CRITIC weighting methods, to confirm the reliability of weights determined
with the AHP-based relative weighting method involving experts. The next stage
of the research involved a stepwise reduction of criteria compensation in the
SSP-AHP method by gradually increasing the value of $s$ sustainability
coefficient. This stage aimed to identify the most sustainable countries in
terms of the healthcare system, which remain stable despite reducing criteria
compensation. This procedure was performed for weights determined with AHP-
based relative weighting method involving experts, as well as for weights
calculated with objective weighting methods to confirm the reliability of AHP-
based relative weights.

###  5.1 Results of the SSP-AHP evaluation with using weights determined with
AHP-based relative weighting method

The essence of the SSP-AHP method, as with most MCDA methods, is to generate a
quantitative ranking of the evaluated alternatives. First, a ranking was
generated for the SSP-AHP method without criterion compensation reduction
using criterion weights determined by the AHP-based relative weighting method
involving experts, which was compared with rankings obtained by applying five
benchmark MCDA methods to confirm the results reliability and relevance of the
AHP method to the problem under investigation. The results of the SSP-AHP
method without reduction of criteria compensation and five other multi-
criteria methods selected as reference methods, namely TOPSIS MABAC, CODAS,
SPOTIS, and PROMETHEE II, are presented in Table 7. The provided table
contains the utility function values and rankings for each alternative.
Received rankings were also visualized in the column chart shown in Figure 2.

Table 7: Utility function values and rankings provided by the SSP-AHP method
without criteria compensation reduction and reference methods using criteria
weights determined with AHP-based relative weighting method.

$A_{i}$ | Country | SSP-AHP | TOPSIS | MABAC | CODAS | SPOTIS | PROMETHEE II | SSP-AHP | TOPSIS | MABAC | CODAS | SPOTIS | PROMETHEE II
---|---|---|---|---|---|---|---|---|---|---|---|---|---
$A_{1}$ | Belgium | 0.6243 | 0.5530 | 0.1199 | 1.2297 | 0.3757 | 0.1164 | 6 | 6 | 6 | 6 | 6 | 6
$A_{2}$ | Czech Republic | 0.4920 | 0.4852 | -0.0124 | -0.9183 | 0.5080 | -0.1216 | 11 | 9 | 11 | 10 | 11 | 10
$A_{3}$ | Finland | 0.5072 | 0.4579 | 0.0028 | -0.5450 | 0.4928 | -0.0645 | 9 | 12 | 9 | 9 | 9 | 9
$A_{4}$ | France | 0.5776 | 0.5528 | 0.0732 | 0.4678 | 0.4224 | 0.0962 | 7 | 7 | 7 | 7 | 7 | 7
$A_{5}$ | Germany | 0.6775 | 0.6385 | 0.1731 | 2.7486 | 0.3225 | 0.3337 | 3 | 3 | 3 | 3 | 3 | 3
$A_{6}$ | Hungary | 0.3272 | 0.4003 | -0.1772 | -3.5673 | 0.6728 | -0.3599 | 15 | 15 | 15 | 15 | 15 | 15
$A_{7}$ | Iceland | 0.6338 | 0.5671 | 0.1294 | 1.7713 | 0.3662 | 0.1934 | 5 | 5 | 5 | 5 | 5 | 5
$A_{8}$ | Latvia | 0.3589 | 0.4444 | -0.1455 | -1.6453 | 0.6411 | -0.2231 | 14 | 14 | 14 | 13 | 14 | 12
$A_{9}$ | Luxembourg | 0.5360 | 0.5403 | 0.0316 | 0.1882 | 0.4640 | 0.0332 | 8 | 8 | 8 | 8 | 8 | 8
$A_{10}$ | Netherlands | 0.6391 | 0.6257 | 0.1347 | 2.0197 | 0.3609 | 0.2486 | 4 | 4 | 4 | 4 | 4 | 4
$A_{11}$ | Norway | 0.7311 | 0.6408 | 0.2267 | 3.6190 | 0.2689 | 0.4893 | 1 | 2 | 1 | 1 | 1 | 1
$A_{12}$ | Poland | 0.3132 | 0.3454 | -0.1912 | -4.3183 | 0.6868 | -0.4865 | 16 | 16 | 16 | 16 | 16 | 16
$A_{13}$ | Slovak Republic | 0.4001 | 0.4456 | -0.1043 | -2.1426 | 0.5999 | -0.2830 | 13 | 13 | 13 | 14 | 13 | 14
$A_{14}$ | Slovenia | 0.5020 | 0.4779 | -0.0024 | -1.1802 | 0.4980 | -0.1422 | 10 | 11 | 10 | 12 | 10 | 11
$A_{15}$ | Sweden | 0.6863 | 0.6625 | 0.1819 | 3.2536 | 0.3137 | 0.4183 | 2 | 1 | 2 | 2 | 2 | 2
$A_{16}$ | United Kingdom | 0.4813 | 0.4806 | -0.0231 | -0.9808 | 0.5187 | -0.2483 | 12 | 10 | 12 | 11 | 12 | 13

In order to confirm the reliability of the SSP-AHP result, the authors checked
the convergence of the SSP-AHP ranking with the other rankings provided by the
reference methods. Two coefficients, namely Pearson’s correlation coefficient
and Weighted Spearman’s Rank Correlation Coefficient $r_{w}$, were used to
measure rankings’ convergence. The values of the calculated correlations
between the rankings are visualized in Figure 3. The obtained results show
that the rankings identical to those provided by the SSP-AHP method gave the
MABAC and SPOTIS methods. Besides, strong correlation values were observed
when comparing the rankings provided by the SSP-AHP method with those of
PROMETHEE II and CODAS. In the case of TOPSIS, the correlation values were the
lowest of all obtained reference rankings. However, their values are still
high. The ranking leader of the SSP-AHP and the other reference methods except
TOPSIS is Norway ($A_{11}$). For the TOPSIS method ranking, the first place
went to Sweden. For the second place TOPSIS ranking, it is exactly in the
opposite order, i.e., for the AHP and the other benchmark methods except for
TOPSIS, Sweden ($A_{15}$) is second, and for the TOPSIS ranking, Norway is
second. Then from the third place until eighth place, the rankings of SSP-AHP
and all benchmark methods are identical. It is easy to notice that the
countries that received the best scores are Norway, Sweden, Germany ($A_{5}$),
the Netherlands ($A_{10}$), and Iceland ($A_{7}$).

Figure 2: Comparison of rankings provided by the SSP-AHP method without
reduction of criteria compensation and reference MCDA methods for criteria
weights determined with AHP-based relative weighting method.

Figure 3: Correlations of rankings provided by the SSP-AHP method without
criteria compensation reduction and other MCDA methods for criteria weights
determined with AHP-based relative weighting method.

On the other hand, the worst-ranked countries in all the rankings are Hungary
in the second-to-last place and Poland ($A_{12}$) in the last place. In the
case of the TOPSIS method, the number of different places and the range of
differences are higher than in the case of SSP-AHP comparison with other
reference methods, which is reflected in the form of lower values of
correlation coefficients. In conclusion, the determined correlations are
sufficiently high to confirm the high convergence of all rankings, which
proves the reliability of SSP-AHP in the investigated problem. In the next
part of the research to model the strength of the criteria compensation
reduction with the SSP-AHP method, an analysis of increasing the value of
sustainability coefficient $s$ in increments of 5% starting from a value of 0%
up to 100% was performed for criteria belonging to all possible combinations
of the main dimensions, thus reducing the criteria compensation progressively.
An increase of sustainability factor is equal to the stepwise reduction of
criteria compensation starting from full criteria compensation ($s$ equal to
0%) and ending with full criteria compensation reduction ($s$ equal to 100%).
Table 8 and Figure 4 demonstrates the changes in rankings while increasing the
value of $s$ for all criteria dimensions.

Table 8: SSP-AHP rankings for criteria compensation reduction with $s$
coefficient for all combinations of criteria dimensions $G_{1}$–$G_{5}$.

G | $A_{1}$ | $A_{2}$ | $A_{3}$ | $A_{4}$ | $A_{5}$ | $A_{6}$ | $A_{7}$ | $A_{8}$ | $A_{9}$ | $A_{10}$ | $A_{11}$ | $A_{12}$ | $A_{13}$ | $A_{14}$ | $A_{15}$ | $A_{16}$
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
| 6 | 11 | 9 | 7 | 3 | 15 | 5 | 14 | 8 | 4 | 1 | 16 | 13 | 10 | 2 | 12
$G_{5}$ | 6 | 11 | 9 | 7 | 3 | 15 | 5 | 14 | 8 | 4 | 1 | 16 | 13 | 10 | 2 | 12
$G_{4}$ | 5 | 11 | 9 | 7 | 6 | 15 | 3 | 14 | 8 | 4 | 1 | 16 | 13 | 10 | 2 | 12
$G_{4}$ $G_{5}$ | 5 | 11 | 9 | 7 | 6 | 15 | 3 | 14 | 8 | 4 | 1 | 16 | 13 | 10 | 2 | 12
$G_{3}$ | 7 | 11 | 10 | 6 | 2 | 14 | 5 | 15 | 8 | 4 | 1 | 16 | 13 | 9 | 3 | 12
$G_{3}$ $G_{5}$ | 7 | 11 | 10 | 6 | 2 | 14 | 5 | 15 | 8 | 4 | 1 | 16 | 13 | 9 | 3 | 12
$G_{3}$ $G_{4}$ | 6 | 11 | 10 | 7 | 3 | 14 | 4 | 15 | 8 | 5 | 1 | 16 | 13 | 9 | 2 | 12
$G_{3}$ $G_{4}$ $G_{5}$ | 6 | 11 | 10 | 7 | 3 | 14 | 4 | 15 | 8 | 5 | 1 | 16 | 13 | 9 | 2 | 12
$G_{2}$ | 5 | 11 | 12 | 7 | 2 | 16 | 6 | 15 | 8 | 3 | 1 | 14 | 13 | 9 | 4 | 10
$G_{2}$ $G_{5}$ | 5 | 11 | 12 | 7 | 2 | 16 | 6 | 15 | 8 | 3 | 1 | 14 | 13 | 9 | 4 | 10
$G_{2}$ $G_{4}$ | 4 | 11 | 12 | 7 | 2 | 16 | 6 | 15 | 8 | 3 | 1 | 14 | 13 | 10 | 5 | 9
$G_{2}$ $G_{4}$ $G_{5}$ | 4 | 11 | 12 | 7 | 2 | 16 | 6 | 15 | 8 | 3 | 1 | 14 | 13 | 10 | 5 | 9
$G_{2}$ $G_{3}$ | 7 | 11 | 12 | 5 | 2 | 15 | 6 | 16 | 9 | 3 | 1 | 14 | 13 | 8 | 4 | 10
$G_{2}$ $G_{3}$ $G_{5}$ | 7 | 11 | 12 | 5 | 2 | 15 | 6 | 16 | 9 | 3 | 1 | 14 | 13 | 8 | 4 | 10
$G_{2}$ $G_{3}$ $G_{4}$ | 7 | 11 | 12 | 5 | 2 | 15 | 6 | 16 | 9 | 3 | 1 | 14 | 13 | 8 | 4 | 10
$G_{2}$ $G_{3}$ $G_{4}$ $G_{5}$ | 7 | 11 | 12 | 5 | 2 | 15 | 6 | 16 | 9 | 3 | 1 | 14 | 13 | 8 | 4 | 10
$G_{1}$ | 4 | 10 | 11 | 7 | 6 | 15 | 5 | 14 | 8 | 2 | 3 | 16 | 13 | 9 | 1 | 12
$G_{1}$ $G_{5}$ | 4 | 10 | 11 | 7 | 6 | 15 | 5 | 14 | 8 | 2 | 3 | 16 | 13 | 9 | 1 | 12
$G_{1}$ $G_{4}$ | 4 | 10 | 11 | 6 | 7 | 15 | 5 | 14 | 8 | 2 | 3 | 16 | 13 | 9 | 1 | 12
$G_{1}$ $G_{4}$ $G_{5}$ | 4 | 10 | 11 | 6 | 7 | 15 | 5 | 14 | 8 | 2 | 3 | 16 | 13 | 9 | 1 | 12
$G_{1}$ $G_{3}$ | 7 | 10 | 11 | 6 | 4 | 14 | 5 | 15 | 8 | 2 | 3 | 16 | 13 | 9 | 1 | 12
$G_{1}$ $G_{3}$ $G_{5}$ | 7 | 10 | 11 | 6 | 4 | 14 | 5 | 15 | 8 | 2 | 3 | 16 | 13 | 9 | 1 | 12
$G_{1}$ $G_{3}$ $G_{4}$ | 6 | 10 | 11 | 5 | 7 | 14 | 4 | 15 | 8 | 2 | 3 | 16 | 13 | 9 | 1 | 12
$G_{1}$ $G_{3}$ $G_{4}$ $G_{5}$ | 6 | 10 | 11 | 5 | 7 | 14 | 4 | 15 | 8 | 2 | 3 | 16 | 13 | 9 | 1 | 12
$G_{1}$ $G_{2}$ | 4 | 10 | 12 | 6 | 5 | 15 | 7 | 16 | 8 | 1 | 2 | 14 | 13 | 9 | 3 | 11
$G_{1}$ $G_{2}$ $G_{5}$ | 4 | 10 | 12 | 6 | 5 | 15 | 7 | 16 | 8 | 1 | 2 | 14 | 13 | 9 | 3 | 11
$G_{1}$ $G_{2}$ $G_{4}$ | 3 | 11 | 12 | 5 | 7 | 16 | 6 | 15 | 8 | 1 | 2 | 14 | 13 | 9 | 4 | 10
$G_{1}$ $G_{2}$ $G_{4}$ $G_{5}$ | 3 | 11 | 12 | 5 | 7 | 15 | 6 | 16 | 8 | 1 | 2 | 14 | 13 | 9 | 4 | 10
$G_{1}$ $G_{2}$ $G_{3}$ | 6 | 10 | 12 | 5 | 4 | 15 | 7 | 16 | 9 | 1 | 2 | 14 | 13 | 8 | 3 | 11
$G_{1}$ $G_{2}$ $G_{3}$ $G_{5}$ | 6 | 10 | 12 | 5 | 4 | 15 | 7 | 16 | 9 | 1 | 2 | 14 | 13 | 8 | 3 | 11
$G_{1}$ $G_{2}$ $G_{3}$ $G_{4}$ | 5 | 11 | 12 | 4 | 6 | 15 | 7 | 16 | 9 | 1 | 2 | 14 | 13 | 8 | 3 | 10
$G_{1}$ $G_{2}$ $G_{3}$ $G_{4}$ $G_{5}$ | 5 | 11 | 12 | 4 | 7 | 15 | 6 | 16 | 9 | 1 | 2 | 14 | 13 | 8 | 3 | 10

The most significant decrease with increasing $s$ coefficient was observed for
Germany ($A_{5}$). Germany ranked third for $s$ between 0 and 30%. Then, when
$s$ values were 35-70%, it took fourth place, for $s$ in the 75-95% range it
dropped to sixth place, and for the full compensation reduction implied by $s$
equal 100%, this country decreased to seventh place. A fall of 3 positions
from 9th to 12th place with increasing $s$ coefficient was noticed for Finland
($A_{3}$). When considering the decreases, it is worth noting Iceland
($A_{7}$) (down two positions) from the 5th to seventh place, but with the
last step of increasing $s$ from 95% to 100%, it finally moved up to the sixth
place. Latvia ($A_{8}$) also dropped two places from 14th to the last 16th.
Besides drops, promotions towards better places in the ranking were also
observed as the value of $s$ was increased. For example, France ($A_{4}$)
moved from 7th to fourth place. Netherlands ($A_{10}$) moved up significantly
from fourth to first place. Poland ($A_{12}$) moved up from 16th to 14th,
Slovenia ($A_{14}$) from 10th to 8th, and the United Kingdom ($A_{16}$) from
12th to 10th.

Figure 4: Changes in rankings with modification of $s$ coefficient in all
criteria for criteria weights determined with the SSP-AHP method.

An analogous analysis was then conducted by increasing the sustainability
factor among all 32 possible combinations of 5 main dimensions (groups) of
criteria. The simulations were performed to identify countries that
demonstrate the greatest variability, that is, reducing compensation causes
them to move up or down, and countries with high stability, including leaders
and countries in the last positions of the rankings. Germany has shown the
most significant susceptibility to a gradual reduction in criteria
compensation. In the case of an unchanged $s$ coefficient, this country ranks
third. However, during the sustainability analysis, the reduction in the
compensation of criteria belonging to dimensions $G_{1}$ (Equity) in
conjunction with $G_{4}$ (Financial coverage) was enough to register a drop of
5 positions for this country to seventh place. It is denoted in the graph
displayed in Figure 5.

Figure 5: Demonstration of the decrease in Germany ($A_{5}$) in the SSP-AHP
ranking as the $s$ coefficient increases for the criteria belonging to the
main criteria $G_{1}$ and $G_{4}$.

However, when reducing the compensation of criteria belonging to dimensions
$G_{2}$ or $G_{3}$, Germany moved up from third to second place, as shown in
the graph in Figure 6.

Figure 6: Demonstration of Germany’s ($A_{5}$) advancement in the SSP-AHP
ranking as the $s$ coefficient increases for the criteria belonging to the
main criterion $G_{2}$.

In the case of criterion dimension $G_{1}$ for Germany, criteria $C_{2}$ (Hip
replacement), $C_{5}$ (acute care beds), $C_{7}$ (number of MRI machines) have
the best, outstanding values of all analyzed countries, so it implies that
they can compensate for lower values of other criteria. Thus, their ranking
decreases when this compensation is reduced by increasing the $s$ coefficient.
On the other hand, promotion is often observed when the $s$ is increased in
the $G_{2}$ (Quality of care) and $G_{3}$ (Responsiveness) criteria groups.

Sweden ($A_{15}$) is another country for which variability in ranking position
was noted during the reduction in criteria compensation. Sweden dropped from
second to fifth place after increasing the coefficient $s$ for dimensions
$G_{2}$ in conjunction with $G_{4}$. For $G_{2}$, Sweden has the best value
for $C_{12}$ (% of hospitalized patients with healthcare-associated infection)
and $C_{13}$ (life expectancy). For $G_{4}$, the compensatory criterion for
Sweden is $C_{18}$ (social health insurance in % of the total population). In
contrast, Sweden is promoted to the leader position as the coefficient $s$
increases in, for example, $G_{1}$ with no reduction in compensation in
$G_{2}$. The variability in ranking due to decreasing compensation for
criteria was also observed for Belgium ($A_{1}$), which rises from sixth to
third place after decreasing compensation for dimensions $G_{1}$ in
conjunction with $G_{2}$ and $G_{4}$. However, Belgium drops to seventh place
when criteria compensation in dimension $G_{3}$ is reduced. Nevertheless,
during the analysis, promotions were observed more often than decreases and
were more significant, allowing Belgium to be considered a sustainable
country.

Another country that rises or drops in the ranking depending on which criteria
compensation is reduced for is Iceland ($A_{7}$). Iceland rises from fifth to
third place when compensation is reduced for criteria in $G_{4}$ but decreases
to seventh place when compensation is reduced for dimensions $G_{1}$ in
conjunction with $G_{2}$. Netherlands ($A_{10}$) jumps from fourth to first
place when compensation for criteria groups $G_{1}$ in conjunction with
$G_{2}$ is reduced and falls to fifth place when compensation in the criteria
group $G_{3}$ in conjunction with $G_{4}$ is reduced. Finland drops from 9th
to 12th place when criteria compensation is reduced, especially $G_{2}$. For
Finland, no promotions were observed with reduced criteria compensation during
the analysis.

In contrast to Finland, France ($A_{4}$) is the country for which only
promotions were observed when analyzing with increasing reductions in criteria
compensation. The most significant advancement was observed for France when
decreasing compensation for criteria belonging to conjoined dimensions
$G_{1}$, $G_{2}$, $G_{3}$, and $G_{4}$ and for all dimensions. France then
moved up from seventh to fourth place. Norway ($A_{11}$) takes the leading
position in the ranking of the SSP-AHP method without criteria compensation
reduction and the rankings of the four reference methods. When reducing
compensation, decreases to the second position were observed when reducing
compensation in criteria groups $G_{1}$ in conjunction with $G_{2}$ and to the
third place when reducing compensation in criteria group $G_{1}$. It is worth
noting that Norway remains in the first place when we do not limit the
compensation in $G_{1}$.

Norway has the best value for $C_{3}$ (practicing physicians) and $C_{4}$
(practicing nurses), and these criteria compensate other criteria that do not
have as good values as the other countries. The leading position is maintained
when there are no changes within the group of $G_{1}$ criteria. Besides,
Norway has the highest compensating values for $C_{16}$, $C_{17}$ belonging to
$G_{3}$, and $C_{18}$, $C_{19}$, $C_{20}$ involved in $G_{4}$. The country
that has kept its position stable during the criteria compensation reduction
analysis is the Slovak Republic ($A_{13}$). As can be observed, this country
occupies a far thirteenth place in the ranking. The stability of this country
in the ranking is because, for any criterion, it does not achieve an
outstanding performance value, which could compensate for the worse values
achieved for other criteria. Thus, the sustainability of this country finally
results in stability and robustness of its rank evident during the analysis.

The worst-ranked countries are Hungary ($A_{6}$), Latvia ($A_{8}$), and Poland
($A_{12}$). Hungary drops from 15th to 16th place when reducing compensation
for $G_{2}$ and $G_{2}$ in conjunction with $G_{4}$ dimensions but jumps to
14th place when reducing compensation for $G_{3}$ criteria. Hungary has the
lowest number of $C_{6}$ CTs and $C_{7}$ MRIs of all countries analyzed.
Latvia ($A_{8}$) decreases from 14th to 16th place when reducing the
compensation of criteria belonging to dimensions $G_{2}$ in conjunction with
$G_{3}$ or $G_{1}$ in conjunction with $G_{2}$. Poland ($A_{12}$) is last in
the ranking but advances to 14th place if compensation for $G_{2}$ criteria is
reduced. Poland has the worst performance value for $C_{3}$ (practicing
physicians) and $C_{25}$ (new technologies).

###  5.2 Comparative analysis of obtained results with using objective
weighting methods

In the following part of the research, a comparative analysis of the results
of the SSP-AHP without criteria compensation reduction for the subjective
weights determined by the AHP-based relative weighting method with two other
objective criteria weighting methods, that is, Entropy weighting and CRITIC
weighting method was performed. This investigation was conducted to the
convergency of the results provided by the compared approaches and whether the
use of weights determined subjectively based on expert knowledge yields
similar results as the use of weights calculated using objective methods.

For this purpose, rankings were additionally generated using the SSP-AHP
without criteria compensation reduction and benchmark methods for the criteria
weights determined by objective methods. Figure 7 displays the correlations
determined for the rankings obtained with the SSP-AHP method without criteria
compensation reduction for the criteria weights determined by the compared
methods, while Table 9 contains the utility function values and rankings
obtained by the SSP-AHP method without criteria compensation reduction for the
compared criteria weighting techniques.

Figure 7: Comparison of SSP-AHP rankings without criteria compensation
reduction constructed with weights determined with different weighting
methods.

From results displayed in Figure 7 and contained in Table 9, it can be
concluded that the criteria weights determined by the objective CRITIC method
reflect the subjective preferences of the expert more accurately than the
weights determined by the objective Entropy method.

Table 10 shows the utility function values and rankings obtained using the
SSP-AHP method without criteria compensation reduction for the criteria
weights determined by the Entropy method. The rankings are further visualized
in Figure 8. For the criterion weights determined by the Entropy method, the
correlation values of the AHP ranking with the compared rankings of the
reference methods are high, as was the case with the expert weights. Similar
to the use of expert criteria weights, the ranking leader is Norway
($A_{11}$). However, Sweden ($A_{15}$), in the case of Entropy weighting
criteria, did not get the second-ranking position like it was noticed for the
use of expert weights, but in most of the rankings, it ranked only sixth.
Germany ($A_{5}$), Belgium ($A_{1}$), Netherlands ($A_{10}$), and Iceland
($A_{7}$) were ranked high next. Thus, it can be observed that in the case of
involving Entropy weighting criteria top of the six countries, the ranking
includes the same alternatives, namely Norway, Sweden, Germany, Netherlands,
Iceland, and Belgium.

Table 9: The SSP-AHP results without criteria compensation reduction for
criteria weights determined with different weighting methods.

|  | Utility function values | Rankings
---|---|---|---
| Country | SSP-AHP + RELATIVE weights | SSP-AHP + ENTROPY weights | SSP-AHP + CRITIC weights | SSP-AHP + RELATIVE weights | SSP-AHP + ENTROPY weights | SSP-AHP + CRITIC weights
$A_{1}$ | Belgium | 0.6243 | 0.6514 | 0.6645 | 6 | 3 | 5
$A_{2}$ | Czech Republic | 0.4920 | 0.4214 | 0.4789 | 11 | 12 | 12
$A_{3}$ | Finland | 0.5072 | 0.4820 | 0.5654 | 9 | 9 | 8
$A_{4}$ | France | 0.5776 | 0.4936 | 0.5861 | 7 | 8 | 7
$A_{5}$ | Germany | 0.6775 | 0.6823 | 0.6891 | 3 | 2 | 2
$A_{6}$ | Hungary | 0.3272 | 0.3178 | 0.3251 | 15 | 15 | 14
$A_{7}$ | Iceland | 0.6338 | 0.5503 | 0.6553 | 5 | 5 | 6
$A_{8}$ | Latvia | 0.3589 | 0.2867 | 0.3228 | 14 | 16 | 15
$A_{9}$ | Luxembourg | 0.5360 | 0.3652 | 0.4847 | 8 | 13 | 11
$A_{10}$ | Netherlands | 0.6391 | 0.6333 | 0.6670 | 4 | 4 | 4
$A_{11}$ | Norway | 0.7311 | 0.7095 | 0.7679 | 1 | 1 | 1
$A_{12}$ | Poland | 0.3132 | 0.3349 | 0.3129 | 16 | 14 | 16
$A_{13}$ | Slovak Republic | 0.4001 | 0.4306 | 0.4014 | 13 | 11 | 13
$A_{14}$ | Slovenia | 0.5020 | 0.4978 | 0.5553 | 10 | 7 | 9
$A_{15}$ | Sweden | 0.6863 | 0.5423 | 0.6751 | 2 | 6 | 3
$A_{16}$ | United Kingdom | 0.4813 | 0.4449 | 0.5100 | 12 | 10 | 10

Table 10: Utility function values and rankings provided by the SSP-AHP method
without criteria compensation reduction and reference methods using weights
determined with an objective Entropy weighting method.

$A_{i}$ | Country | SSP-AHP | TOPSIS | MABAC | CODAS | SPOTIS | PROMETHEE II | SSP-AHP | TOPSIS | MABAC | CODAS | SPOTIS | PROMETHEE II
---|---|---|---|---|---|---|---|---|---|---|---|---|---
$A_{1}$ | Belgium | 0.6514 | 0.6415 | 0.1870 | 3.3776 | 0.3486 | 0.3521 | 3 | 2 | 3 | 3 | 3 | 2
$A_{2}$ | Czech Republic | 0.4214 | 0.4416 | -0.0429 | -1.0090 | 0.5786 | -0.1943 | 12 | 12 | 12 | 12 | 12 | 12
$A_{3}$ | Finland | 0.4820 | 0.4607 | 0.0177 | -0.1666 | 0.5180 | -0.0431 | 9 | 11 | 9 | 8 | 9 | 9
$A_{4}$ | France | 0.4936 | 0.4862 | 0.0292 | -0.3927 | 0.5064 | 0.0660 | 8 | 8 | 8 | 9 | 8 | 7
$A_{5}$ | Germany | 0.6823 | 0.6322 | 0.2179 | 3.6040 | 0.3177 | 0.3406 | 2 | 4 | 2 | 2 | 2 | 4
$A_{6}$ | Hungary | 0.3178 | 0.3561 | -0.1466 | -3.2870 | 0.6822 | -0.3389 | 15 | 15 | 15 | 15 | 15 | 14
$A_{7}$ | Iceland | 0.5503 | 0.5143 | 0.0859 | 0.8097 | 0.4497 | 0.1037 | 5 | 5 | 5 | 5 | 5 | 6
$A_{8}$ | Latvia | 0.2867 | 0.3183 | -0.1776 | -3.7885 | 0.7133 | -0.3680 | 16 | 16 | 16 | 16 | 16 | 15
$A_{9}$ | Luxembourg | 0.3652 | 0.3898 | -0.0992 | -2.4052 | 0.6348 | -0.2516 | 13 | 13 | 13 | 13 | 13 | 13
$A_{10}$ | Netherlands | 0.6333 | 0.6344 | 0.1689 | 2.9944 | 0.3667 | 0.3475 | 4 | 3 | 4 | 4 | 4 | 3
$A_{11}$ | Norway | 0.7095 | 0.6572 | 0.2451 | 4.1099 | 0.2905 | 0.5086 | 1 | 1 | 1 | 1 | 1 | 1
$A_{12}$ | Poland | 0.3349 | 0.3821 | -0.1294 | -3.0782 | 0.6651 | -0.3695 | 14 | 14 | 14 | 14 | 14 | 16
$A_{13}$ | Slovak Republic | 0.4306 | 0.4714 | -0.0338 | -0.5317 | 0.5694 | -0.1292 | 11 | 9 | 11 | 11 | 11 | 10
$A_{14}$ | Slovenia | 0.4978 | 0.4922 | 0.0334 | -0.0458 | 0.5022 | -0.0301 | 7 | 7 | 7 | 7 | 7 | 8
$A_{15}$ | Sweden | 0.5423 | 0.4922 | 0.0779 | 0.3248 | 0.4577 | 0.2000 | 6 | 6 | 6 | 6 | 6 | 5
$A_{16}$ | United Kingdom | 0.4449 | 0.4679 | -0.0195 | -0.5155 | 0.5551 | -0.1936 | 10 | 10 | 10 | 10 | 10 | 11

Figure 8: Comparison of rankings provided by the SSP-AHP method without
criteria compensation reduction and reference MCDA methods for criteria
weights determined with Entropy method.

The described similarity of results for the subjective and objective criteria
weights is confirmed by the high correlation coefficients values of the
rankings displayed in Figure 9.

Figure 9: Correlations of rankings provided by the SSP-AHP method without
criteria compensation reduction for criteria weights determined with an
Entropy weighting method.

Table 11 contains the utility function values and rankings obtained applying
the SSP-AHP method without criteria compensation reduction and the reference
methods using objective weights calculated by the CRITIC method.

Table 11: Utility function values and rankings provided by the SSP-AHP method
without criteria compensation reduction and reference methods using weights
determined with an objective CRITIC weighting method.

$A_{i}$ | Country | SSP-AHP | TOPSIS | MABAC | CODAS | SPOTIS | PROMETHEE II | SSP-AHP | TOPSIS | MABAC | CODAS | SPOTIS | PROMETHEE II
---|---|---|---|---|---|---|---|---|---|---|---|---|---
$A_{1}$ | Belgium | 0.6645 | 0.6427 | 0.1506 | 2.0106 | 0.3355 | 0.1992 | 5 | 2 | 5 | 6 | 5 | 6
$A_{2}$ | Czech Republic | 0.4789 | 0.4898 | -0.0350 | -1.1498 | 0.5211 | -0.1698 | 12 | 12 | 12 | 12 | 12 | 11
$A_{3}$ | Finland | 0.5654 | 0.5510 | 0.0515 | 0.7856 | 0.4346 | 0.0219 | 8 | 9 | 8 | 7 | 8 | 8
$A_{4}$ | France | 0.5861 | 0.5744 | 0.0722 | 0.7154 | 0.4139 | 0.0719 | 7 | 7 | 7 | 8 | 7 | 7
$A_{5}$ | Germany | 0.6891 | 0.6307 | 0.1752 | 2.4737 | 0.3109 | 0.2962 | 2 | 5 | 2 | 3 | 2 | 3
$A_{6}$ | Hungary | 0.3251 | 0.3669 | -0.1888 | -4.0349 | 0.6749 | -0.3616 | 14 | 15 | 14 | 15 | 14 | 15
$A_{7}$ | Iceland | 0.6553 | 0.6143 | 0.1414 | 2.3248 | 0.3447 | 0.2238 | 6 | 6 | 6 | 4 | 6 | 5
$A_{8}$ | Latvia | 0.3228 | 0.3752 | -0.1911 | -3.7687 | 0.6772 | -0.3467 | 15 | 14 | 15 | 14 | 15 | 14
$A_{9}$ | Luxembourg | 0.4847 | 0.4952 | -0.0292 | -0.6698 | 0.5153 | -0.1110 | 11 | 11 | 11 | 11 | 11 | 10
$A_{10}$ | Netherlands | 0.6670 | 0.6417 | 0.1531 | 2.0438 | 0.3330 | 0.2676 | 4 | 3 | 4 | 5 | 4 | 4
$A_{11}$ | Norway | 0.7679 | 0.7074 | 0.2540 | 3.9955 | 0.2321 | 0.5310 | 1 | 1 | 1 | 1 | 1 | 1
$A_{12}$ | Poland | 0.3129 | 0.3535 | -0.2010 | -4.3977 | 0.6871 | -0.4843 | 16 | 16 | 16 | 16 | 16 | 16
$A_{13}$ | Slovak Republic | 0.4014 | 0.4323 | -0.1125 | -2.5637 | 0.5986 | -0.2906 | 13 | 13 | 13 | 13 | 13 | 13
$A_{14}$ | Slovenia | 0.5553 | 0.5623 | 0.0414 | 0.3020 | 0.4447 | -0.0166 | 9 | 8 | 9 | 9 | 9 | 9
$A_{15}$ | Sweden | 0.6751 | 0.6320 | 0.1612 | 2.4964 | 0.3249 | 0.3774 | 3 | 4 | 3 | 2 | 3 | 2
$A_{16}$ | United Kingdom | 0.5100 | 0.5199 | -0.0039 | -0.5631 | 0.4900 | -0.2086 | 10 | 10 | 10 | 10 | 10 | 12

In addition, the obtained rankings are visualized in the column graph shown in
Figure 10.

Figure 10: Comparison of rankings provided by the SSP-AHP method without
criteria compensation reduction and reference MCDA methods for criteria
weights determined with CRITIC method.

When using the CRITIC method weights, the multi-criteria evaluation with the
classic AHP and reference methods identified Norway as the leader in the
sustainable healthcare system, with Germany, Sweden, Netherlands, Belgium, and
Iceland among the best-scored countries. However, a swap of one position was
observed for Sweden and Germany and Iceland and Belgium compared with ranking
provided with subjective weights.

The values of the correlation coefficients calculated for the rankings
provided by the SSP-AHP method without criteria compensation reduction and
reference methods for the weights determined by the CRITIC method, visualized
in Figure 11, are high, as in the case of the AHP-based relative and Entropy
weighting methods.

Figure 11: Correlations of rankings provided by the SSP-AHP method without
criteria compensation reduction and reference methods for criteria weights
determined with CRITIC weighting method.

The next phase of the research aimed to compare rankings obtained by the SSP-
AHP method, using criteria weights determined subjectively by the expert with
the AHP method based on the Saaty scale with SSP-AHP rankings obtained for
weights determined using objective Entropy weighting and CRITIC weighting
methods. The changes in rankings as a result of reducing the compensation of
all criteria by gradually increasing the coefficient $s$ are visualized in
Figure 12.

It is worth noting that Germany ($A_{5}$) drops from second to fifth place
when the compensation of the main conjoined criteria $G_{1}$, $G_{4}$, and
$G_{5}$ is reduced but moves up to a first position when the compensation of
criteria belonging to $G_{3}$ is reduced. Variability with a reduction in
criteria compensation is also shown by Sweden ($A_{15}$) advancing from sixth
place to fourth place when compensation of conjoined criteria $G_{1}$,
$G_{4}$, and $G_{4}$ is reduced and falling to eighth place when compensation
of conjoined criteria $G_{2}$, $G_{3}$, and $G_{4}$ is reduced. In contrast,
France ($A_{4}$) tends to climb from eighth place to fifth place. The best-
ranked and most stable countries are the Netherlands ($A_{10}$), which jumps
from fourth to second place when criteria compensation is reduced, and Norway
($A_{11}$), the stable leader of the ranking. On the other hand, the lowest-
ranked countries for which no significant change was observed in the reduction
of criteria compensation in the SSP-AHP method are Latvia ($A_{8}$), which is
stable in last place, Hungary ($A_{6}$), Poland ($A_{12}$), Slovak Republic
($A_{13}$).

Figure 12: Changes in SSP-AHP ranking with modification of $s$ coefficient in
all criteria for criteria weights determined with Entropy method.

For the analogous investigation using CRITIC-determined weights, the highest
variability when reducing criterion compensation with increasing $s$ is also
observed for Germany ($A_{5}$). Germany drops from second to eighth place when
reducing the compensation of conjoined criteria $G_{1}$, $G_{3}$, $G_{4}$,
$G_{5}$. Netherlands ($A_{10}$) and France ($A_{4}$) tend to move up when
reducing criteria compensation.

Norway ($A_{11}$) is a stable leader, falling only in the case of maximum
compensation reduction for all criteria to second place. Sweden ($A_{15}$)
drops from third to fifth place when reducing the compensation of criteria
belonging to $G_{2}$ but moves up to second place when reducing the
compensation of conjoined criteria $G_{5}$, $G_{4}$ and $G_{5}$, $G_{3}$ in
conjunction with $G_{5}$, $G_{3}$ in conjunction with $G_{4}$. In the last,
stable places are Latvia ($A_{8}$), Poland ($A_{12}$), and Slovak Republic
($A_{13}$). The changes in rankings as a result of reducing the compensation
of all criteria by gradually increasing the coefficient $s$ are visualized in
Figure 13.

Figure 13: Changes in SSP-AHP ranking with modification of $s$ coefficient in
all criteria for criteria weights determined with CRITIC method. Table 12:
Correlation coefficient values of SSP-AHP rankings obtained for subjective
weights applying criteria compensation with SSP-AHP rankings obtained using
weights determined by Entropy and CRITIC objective methods.

$G$ |  $r_{w}$ for SSP-AHP + CRITIC weights | Pearson for SSP-AHP + CRITIC weights |  $r_{w}$ for SSP-AHP + Entropy weights | Pearson for SSP-AHP + Entropy weights
---|---|---|---|---
| 0.9706 | 0.9676 | 0.8877 | 0.8853
$G_{5}$ | 0.9685 | 0.9618 | 0.9100 | 0.9029
$G_{4}$ | 0.9403 | 0.9441 | 0.8593 | 0.8706
$G_{4}$ $G_{5}$ | 0.9685 | 0.9618 | 0.8760 | 0.8853
$G_{3}$ | 0.9696 | 0.9618 | 0.9035 | 0.8882
$G_{3}$ $G_{5}$ | 0.9645 | 0.9588 | 0.9031 | 0.8765
$G_{3}$ $G_{4}$ | 0.9524 | 0.9500 | 0.8936 | 0.9000
$G_{3}$ $G_{4}$ $G_{5}$ | 0.9569 | 0.9529 | 0.8818 | 0.8647
$G_{2}$ | 0.9550 | 0.9500 | 0.9455 | 0.9353
$G_{2}$ $G_{5}$ | 0.9550 | 0.9500 | 0.9320 | 0.9147
$G_{2}$ $G_{4}$ | 0.9438 | 0.9412 | 0.9450 | 0.9324
$G_{2}$ $G_{4}$ $G_{5}$ | 0.9388 | 0.9353 | 0.9391 | 0.9176
$G_{2}$ $G_{3}$ | 0.9559 | 0.9559 | 0.9144 | 0.9118
$G_{2}$ $G_{3}$ $G_{5}$ | 0.9683 | 0.9647 | 0.9292 | 0.9235
$G_{2}$ $G_{3}$ $G_{4}$ | 0.9521 | 0.9529 | 0.9140 | 0.9118
$G_{2}$ $G_{3}$ $G_{4}$ $G_{5}$ | 0.9471 | 0.9500 | 0.9360 | 0.9294
$G_{1}$ | 0.9574 | 0.9618 | 0.8689 | 0.8794
$G_{1}$ $G_{5}$ | 0.9526 | 0.9500 | 0.8740 | 0.8794
$G_{1}$ $G_{4}$ | 0.9375 | 0.9441 | 0.8638 | 0.8735
$G_{1}$ $G_{4}$ $G_{5}$ | 0.9526 | 0.9500 | 0.8964 | 0.8971
$G_{1}$ $G_{3}$ | 0.9410 | 0.9441 | 0.8609 | 0.8735
$G_{1}$ $G_{3}$ $G_{5}$ | 0.9450 | 0.9441 | 0.8817 | 0.8706
$G_{1}$ $G_{3}$ $G_{4}$ | 0.9353 | 0.9382 | 0.8348 | 0.8529
$G_{1}$ $G_{3}$ $G_{4}$ $G_{5}$ | 0.9426 | 0.9412 | 0.8296 | 0.8500
$G_{1}$ $G_{2}$ | 0.9415 | 0.9412 | 0.9170 | 0.9176
$G_{1}$ $G_{2}$ $G_{5}$ | 0.9503 | 0.9471 | 0.9170 | 0.9176
$G_{1}$ $G_{2}$ $G_{4}$ | 0.9415 | 0.9412 | 0.9045 | 0.9059
$G_{1}$ $G_{2}$ $G_{4}$ $G_{5}$ | 0.9637 | 0.9588 | 0.9102 | 0.9118
$G_{1}$ $G_{2}$ $G_{3}$ | 0.9472 | 0.9500 | 0.8938 | 0.9118
$G_{1}$ $G_{2}$ $G_{3}$ $G_{5}$ | 0.9649 | 0.9618 | 0.9260 | 0.9206
$G_{1}$ $G_{2}$ $G_{3}$ $G_{4}$ | 0.9495 | 0.9529 | 0.8926 | 0.9088
$G_{1}$ $G_{2}$ $G_{3}$ $G_{4}$ $G_{5}$ | 0.9559 | 0.9559 | 0.8984 | 0.9029

The results obtained using the two objective criteria weighting methods are
not identical to the results for the subjectively determined weights, but they
reflect the locations and trends obtained for the subjective weights. The
final step was to objectively determine the convergence of the rankings
provided by the SSP-AHP method using a subjective criterion weighting method
for reducing the compensation of individual combinations of the main
$G_{1}$–$G_{5}$ criterion groups compared to the corresponding results
obtained using weights calculated with the Entropy and CRITIC methods. In
order to ensure the reliability of results, correlations were determined using
two coefficients: $r_{w}$ and Pearson correlation coefficient. SSP-AHP using
AHP weighting method correlation values compared to employing Entropy and
CRITIC weighting are included in Table 12. Similar to the SSP-AHP method
without criteria compensation reduction, for the SSP-AHP method with reduction
of criteria compensation, a higher correlation of rankings obtained using
subjective weights was observed for comparisons with rankings obtained for the
CRITIC weighting method. Besides, the high values of all obtained measurements
confirm the reliability of the expert weights used in the research and the
SSP-AHP method.

##  6 Discussion

Our understanding of social sustainability shares the perspective proposed by
Rachelle et al. (Hollander et al., 2016), highlighting the quality of society
that encourages durable environment for human well-being. The present paper is
among the first studies to map social sustainability values in health systems
from the national level. The proposed framework offers a comprehensive
assessment of social sustainable approach of health systems and serves as an
analytical instrument for cross-national comparisons. It has been based on
five major domains: equity, quality, responsiveness, financial coverage and
adaptability. The first four of these are in line with the broadly recognized
health system performance frameworks, introduced by WHO and OECD (Murray and
Frenk, 2000). Although the proposed framework covers the most crucial, well-
known dimensions, reflecting the principal aims of health system, it also
captures up-to-date dimension i.e. adaptability. It seems extremely important
to boost resilience and flexibility of health systems, especially in times of
crisis, such as COVID pandemic. This opinion is shared among others by Macassa
and Tomaselli (Macassa and Tomaselli, 2020).

The proposed framework has some potential for scientific contribution,
embedding novel approach towards sustainability assessment methodology. It
also enables cross-national comparisons and thus can be applied as an
effective instrument measuring the level of social sustainability of health
systems. Thus, it has practical implications for health policy leaders and
policy makers. The innovative value of the study is also based on dynamic
approach towards assessing social sustainability phenomenon, which remains in
contract to static frameworks introduced by WHO and OECD. The dynamism of our
approach involves the flexibility and substitutability of core five domains.
The previous studies concentrated on the organizational perspective, focusing
on themes related to corporate management (AlJaberi et al., 2020; Cristiano et
al., 2021; Jahani Sayyad Noveiri and Kordrostami, 2021) and supply chain
management (Hussain et al., 2019) and its impact on social sustainability.
Another work by Ajmal et al. (Ajmal et al., 2018) adopted both external,
societal prism and internal, companies’ perspective, introducing six major
social sustainability areas. The areas related to external perspective, namely
social development, social growth and social justice can be compared to our
core domains: financial coverage, quality (in terms of health outcomes), and
equity (in terms of accessibility).

The findings reveal that the most sustainable health systems, no matter which
MCDM method has been applied, are those representing so called the Nordic
model. It contains consistent features, such as tax-based funding, universal
access on the basis of residency and comprehensive coverage of health services
(Magnussen et al., 2009). The Nordic countries uphold the principles of equity
and universalism, which are deeply rooted in the welfare tradition. From the
axiological point of view, social sustainability is thus underpinned in
fundamental social values of freedom and equality (Garcés et al., 2003). Our
findings support the thesis that in the Nordic countries universal access to
high-quality healthcare is a priority goal of the health system, shared
commonly across the political parties (Magnussen et al., 2009). Health
policies in Nordic countries are largely guided by egalitarian rule, which
proves to be effective in attaining improvements in health outcomes. The
responsibility for providing healthcare to the population is decentralized and
lies at the regional level. The stable position of Nordic health systems in
terms of social sustainability provides clear evidence that decentralized
distribution of resources, their needs-based allocation and regional propriety
setting result in effective strategies, reducing inequalities in health
(Arnesen et al., 2002; Lahelma et al., 2002). Decentralization of resource
allocation is also a mean to improve responsiveness through the “short
distance” between patients and local authorities, responsible for providing
care. All Nordic countries have introduced measures to strengthen the role of
patients (Andersson et al., 2007) and to involve them in decision making
regarding treatment.

Our empirical evidence is in line with the findings of other authors (Carey et
al., 2015; Greve, 2016), supporting the thesis that the Nordic priority
setting models safeguard equity, remaining effective in offering high-quality
and responsive medical services. On the other hand, the knowledge-based
solutions underpinning the measurement of outcomes of different medical
interventions support systems’ accountability. The Nordic countries are
actively involved in technology assessment movements and quality programs that
give grater scope for results. Life expectancy in Norway, the leader of the
social sustainable health system ranking, is the highest in Europe, at 83.3
years in 2020 and still increasing (by 0.3 years in 2020), despite the
COVID-19 pandemic (OECD, 2021d). The level of system’s adaptability in Nordic
countries is also high, resulting in the strong resilience. The perfect
illustration is COVID crisis, and the relative success of handling it,
attributed to factors such as rapidly implemented containment measurements,
high level of trust in government and broad social welfare arrangements (OECD,
2021d). This matches the research results obtained by applying novel SSP-AHP
method.

The study findings suggest that there is a group of health systems, that
manifests the most significant susceptibility to a gradual reduction in
criteria of social sustainability. Among these countries are Germany, France,
Netherlands, Belgium and Slovenia. However, this group is not homogeneous. The
greatest shift in the ranking has been observed in case of France, and the
biggest drop in case of Germany. The variations in above mentioned
susceptibility can be explained on the basis of differences in the functioning
of these systems, although there are certain common aspects, such as social
insurance based on the Bismarck’s model. French and Slovenian health systems
have a more centralized character with the more powerful position for the
state than German and Dutch statutory health insurance (SHI). Another crucial
difference is that the French SHI has not had the management responsibilities
assigned to SHI funds and regimes, such as those observed in Germany (Chevreul
et al., 2015). In France financial and operational management is located
mainly at the state level. French system receives one of the highest public
financing of healthcare expenditure in Europe (11,1% of GDP in 2019, above the
EU average of 9,9%), while out-of-pocket spending is among the lowest (9,3%
compared to 15,4% EU average) (OECD, 2021a). Our investigation suggests that
French system is relatively stable in all dimensions and the most significant
advancement has been noted while decreasing compensation for all criteria
except adaptability. This is in line with the previous findings, supporting
the discussion on the significant role of voluntary health insurance (VHI) in
ensuring equity in access and financial risk protection. French VHI provides
complementary insurance, for instance co-payments and better coverage for
medical goods and services that are poorly covered by SHI. It finances 13.8%
of total health expenditure and covers more than 90% of the population
(Chevreul et al., 2015). Relative sound responsiveness of the system results
in very low level of unmet needs for medical care, reported by only 1,2% of
the population in 2019 (OECD, 2021a). These numbers support the empirical
evidence based on SSP-AHP.

Furthermore, according to our findings German heath system demonstrates the
most significant susceptibility to a gradual reduction in criteria
compensation such as equity in access and financial coverage. This result can
be attributed to the broad benefits package within universal SHI, offered by
German health system. Works by other authors support these findings,
highlighting that Germany scores very well in accessibility to medical
services (Osborn et al., 2016). Public surveys reveal at the same time, that
unmet needs for healthcare are driven less by financial reasons than by other
factors such as waiting times. The share of patients reporting cost burden for
a medical examination was only 0.1% in 2019 (Blümel et al., 2021).

Finally we have observed that Poland, Latvia and Hungary compose the group of
countries that score very low in all dimensions. Their health systems lack
social sustainability. Despite some differences in financing and provision of
care (single-payer health insurance system in Poland and in Hungary; general
taxation in Latvia), they all face common, ongoing challenges, affecting
social sustainability. Although statutory health cover is, by definition,
universal there, inequalities in care use, long waiting times for certain
services and a high proportion of population with unmet medical needs let us
to conclude that the “universality” formula is not assured. Similar remarks
have been presented by other researchers (Behmane et al., 2019; Gaál et al.,
2011; Sowada et al., 2019). Current health expenditures (as shares of GDP,
2019) are much below EU average (6,4% in Hungary, 6,5% in Poland, 6,6% in
Latvia, compared to 9,9% across the UE (OECD, 2021e)) and out-of-pocket
spending (excluding informal payments) still accounts for over a third of
total. Moreover, the high amenable mortality rate and the low cancer survival
rates are further indicators of deficiencies in the quality of curative care
(OECD, 2021b, c, f). All discussed countries experience persistent workforce
shortages that lead to low systems’ responsiveness. It can be concluded that a
key challenge in ensuring sustainable health systems in central-eastern Europe
is to increase the share of public expenditure on health and reduce the
substantial dependence on out-of-pocket payments.

##  7 Conclusions

Health system performance assessment is a fundamental instrument for evidence-
based governance in health sectors (Blümel et al., 2020). A lot of effort has
been put so far into the development of frameworks measuring the
accountability of health systems. The publication of "Our common future”
(WCED, 1987) in 1987 triggered a rethinking on environment, development, and
governance, also in the health sector. In this perspective, the design of a
sustainable performance evaluation framework seems to be fundamental to
aligning health systems’ operation with fulfilling the sustainability goals
(SG) of Agenda 2030, especially SG 3. With this work, authors seek to map
social sustainability values in health systems from the national level. The
multi-stage model links the classical AHP method with the new approach called
SSP-AHP into a unique methodological framework that allows prioritization of
the identified values in social-sustainable health systems. The proposed
conceptual and structured framework has been built upon five pillars (equity,
quality, responsiveness, financial coverage, and adaptability) and 25
indicators that can guide the measurement of the social sustainability-
oriented health systems. Thus, a proposed architecture can be treated as a
decision support tool that directs health policy- and decision-makers toward
more sustainable health systems. It can inform health policy makers and
support them with formulating sustainable strategies at a national level. The
agreed set of criteria can help to answer the question how well a health
system is progressing towards social sustainability, and map its strengths and
weaknesses in order to improve the performance.

The obvious popularity and simplicity of the AHP method in several decision
problems and evaluation of various sustainability issues (Dehe and Bamford,
2015) motivated the authors to extend the AHP method towards supporting the
strong sustainability paradigm, namely Strong Sustainability Paradigm-AHP
(SSP-AHP). According to the authors’ best knowledge, this is the first attempt
to limit the linear compensation of criteria in this very popular method. The
SSP-AHP method proposed in the paper additionally provides a tool for modeling
the strength of sustainability itself. As shown in the paper, SSP-AHP allows
adjusting the influence of the decision-maker on the degree of reduction in
criteria compensation depending on the problem that may require consideration
of a more or less strong sustainability paradigm. The proposed method has
great potential for use in any sustainability assessment problems in this
context. Using the sustainability coefficient to model the strength of the
reduction in the degree of criteria compensation is a useful tool that
provides decision-makers with new analytical capabilities.

The authors also proved the usefulness of the SSP-AHP method in healthcare in
practical terms. Based on the SSP-AHP method, a dedicated framework was
developed to map sustainable value in the healthcare system. Its usefulness
was demonstrated by evaluating selected European countries. Additionally,
practical capabilities of the proposed/developed tool were presented,
including modeling the strength of sustainability paradigm support. Besides
structuring the model for Europe, the author’s measure also includes the
framework, where a set of criteria and weights is made available for a wider
audience, which other researchers and decision-makers can use.

The direction for further work is to implement the strong sustainability
paradigm in the remaining MCDA methods. Additionally, the adaptation of fuzzy
theory-based extensions of MCDA methods towards supporting a strong
sustainability paradigm seems to be a promising research direction. From the
methodical and practical point of view, the adaptation of the DEMATEL method
to the proposed model and data structure and possible analysis of inter-
criteria relationships can be a particularly interesting direction of further
work. Adaptation of the hybrid method based on AHP and selected method from
European school seems to be also interesting future work avenue.

##  Appendix A The Weighting Methods

###  A.1 The AHP-based relative weighting method

The AHP-based relative weighting method was detailed based on (Ben Moallem et
al., 2018). In the classical AHP method, the domain expert or decision-maker
determines the criteria weights with a pairwise comparison of particular
criteria regarding their respective significance.

Step 1.  Decision-maker using the scale proposed by Saaty provided in Table 13
expresses the degree of comparability between the two criteria.

Table 13: The fundamental Saaty scale.

Intensity of importance | Meaning
---|---
1 | Equal importance
2 | Weak
3 | Moderate importance
4 | Moderate plus
5 | Strong importance
6 | Strong plus
7 | Very strong or demonstrated importance
8 | Very, very strong
9 | Extreme importance

The results of the comparisons are collected in a pairwise comparison matrix
$X=[x_{ij}]_{n\times n}$ presented in the Equation (7)

$X_{n\times
n}=\left[\begin{array}[]{cccccc}1&x_{12}&\cdots&\cdots&\cdots&x_{1n}\\\
\frac{1}{x_{12}}&1&\cdots&\cdots&\cdots&\cdots\\\
\cdots&\cdots&1&x_{ij}&\cdots&\cdots\\\
\cdots&\cdots&\frac{1}{x_{ij}}&1&\cdots&\cdots\\\
\cdots&\cdots&\cdots&\cdots&1&\cdots\\\
\frac{1}{x_{1n}}&\cdots&\cdots&\cdots&\cdots&1\\\ \end{array}\right]$ (7)

where $x_{ij}$ denotes result of comparison between criterion $i$ and $j$ and
obviously $x_{ij}=\frac{1}{x_{ji}}$ and $x_{ji}=\frac{1}{x_{ij}}$. Of course
$x_{ii}=1$ because each criterion is equally important to itself.

Step 2. Checking the consistency of the comparison matrix created in the
previous step with index proposed by Saaty given in Equation (8)

$CI(X)=\frac{\lambda_{max}-n}{n-1}$ (8)

where $\lambda_{max}$ denotes the maximum eigenvalue of the matrix presented
in Equation (7) and $n$ represents criteria number.

Then, the consistency ratio $CR$ is determined using Equation (9)

$CR(X)=\frac{CI(X)}{RI_{n}}$ (9)

and $RI_{n}$ means the random indices calculated and recommended by Saaty.
Values of $RI_{n}$ are provided in Table 14

Table 14: Random indices $RI$ recommended by Saaty.

n | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10
---|---|---|---|---|---|---|---|---|---|---
RI | 0 | 0 | 0.58 | 0.90 | 1.12 | 1.24 | 1.32 | 1.41 | 1.45 | 1.49

Following Saaty, matrices with $CR\leq 0.1$ are consistent and acceptable. On
the other hand, $CR>0.1$ indicates inconsistency of the considered matrix.

Step 3.  Calculation of the priority vector of criteria. In this research, a
vector was determined using the eigenvector method proposed by Saaty. A
priority vector is the principal eigenvector of $X$ in this technique. The
matrix $X$ contains ratios between weights and has to be multiplied by $w$, as
Equation 10 presents.

$Xw=\left[\begin{array}[]{cccc}\frac{w_{1}}{w_{1}}&\frac{w_{1}}{w_{2}}&\cdots&\frac{w_{1}}{w_{n}}\\\
\frac{w_{2}}{w_{1}}&\frac{w_{2}}{w_{2}}&\cdots&\frac{w_{2}}{w_{n}}\\\
\vdots&\cdots&\cdots&\vdots\\\
\frac{w_{n}}{w_{1}}&\frac{w_{n}}{w_{2}}&\cdots&\frac{w_{n}}{w_{n}}\\\
\end{array}\right]\left[\begin{array}[]{c}w_{1}\\\ w_{2}\\\ \vdots\\\ w_{n}\\\
\end{array}\right]=\left[\begin{array}[]{c}nw_{1}\\\ nw_{2}\\\ \vdots\\\
nw_{n}\\\ \end{array}\right]=nw$ (10)

Equation $Xw=nw$ implies that $n$ and $w$ denote an eigenvalue and an
eigenvector of $X$ and $n$ is the largest eigenvalue of $X$. Vector $w$ can be
received from a pairwise comparison matrix $X$ by solving the below equation
system, as Equation 11 demonstrates.

$\left\\{\begin{array}[]{c}Xw=\lambda_{max}\\\
w^{T}=[1,1,\ldots,1]^{T}=1\end{array}\right.$ (11)

The principal eigenvector of $X$ can be calculated simply using a Python
software package provided in codes available in supplementary materials at
GitHub (energyinpython, 2022).

###  A.2 The Entropy weighting method

The subsequent steps of Entropy weighting method are provided below, based on
(Lotfi and Fallahnejad, 2010).

Step 1.  Normalization of the decision matrix $X=[x_{ij}]_{m\times n}$

$p_{ij}=\frac{x_{ij}}{\sum_{i=1}^{m}x_{ij}}$ (12)

Step 2.  Calculation of the entropy $h_{j}$

$h_{j}=-(ln\;m)^{-1}\sum_{i=1}^{m}p_{ij}\cdot ln\;p_{ij}$ (13)

where $p_{ij}\cdot ln\;p_{ij}$ is established as 0 if $p_{ij}=0$

Step 3.  Calculation of weights values

$w_{j}=\frac{1-h_{j}}{\sum_{j=1}^{n}(1-h_{j})}$ (14)

###  A.3 The CRITIC weighting method

The subsequent stages of Criteria Importance Through Inter criteria
Correlation weighting method (CRITIC) are in following steps based on (Tuş and
Aytaç Adalı, 2019). Decision matrix $X={[x_{ij}]}_{m\times n}$ contains $m$
alternatives and $n$ criteria, where $x_{ij}$ represents the performance
measure of $i^{th}$ alternative with regard to $j^{th}$ criterion. To
determine the weight of the $j^{th}$ criterion $w_{j}$ using CRITIC the
calculations provided below are conducted

Step 1. Decision matrix is normalized by Equation (15) as for benefit criteria

$r_{ij}=\frac{x_{ij}-min_{j}(x_{ij})}{max_{j}(x_{ij})-min_{j}(x_{ij})}$ (15)

Step 2. Calculation of the Pearson correlation coefficient among pairs of
criteria as Equation (16) demonstrates.

$\rho_{jk}=\frac{\sum_{i=1}^{m}(r_{ij}-\overline{r}_{j})(r_{ik}-\overline{r}_{k})}{\sqrt{\sum_{i=1}^{m}(r_{ij}-\overline{r}_{j})^{2}\sum_{i=1}^{m}(r_{ik}-\overline{r}_{k})^{2}}}.$
(16)

Step 3. Calculation of criteria weights using Equation (17) and (18),

$c_{j}=\sigma_{j}\sum_{k=1}^{n}(1-\rho_{jk});$ (17)

$w_{j}=\frac{c_{j}}{\sum_{k=1}^{n}c_{k}},$ (18)

where $i=1,2,\ldots,m;\;j,k=1,2,\ldots,n.$ In the formulas given above $c_{j}$
represents the quantity of information contained in $j^{th}$ criterion,
$\sigma_{j}$ express the standard deviation of the $j^{th}$ criterion and
$\rho_{jk}$ is the correlation coefficient between the $j^{th}$ and $k^{th}$
criteria. A high standard deviation and low correlation of given criterion
with the others determine a high criterion weight. Thus, a high value of
$C_{j}$ provides more information from the considered criterion

##  Appendix B Characteristics of the healthcare experts involved in the
research

A total of 30 health care experts were invited to the study. The recruitment
of experts was preceded by the creation of address database, which included
health sector professionals with experience in assessing health systems.
Social media (like Linkedin) have been used to contact potential respondents.
Finally 16 experts agreed to take part in on-line survey in October 2021. The
most represented professional group was the management of medical institutions
(5 people) and public administration staff (4 people). The vast majority of
the invited health sector professionals held doctoral degrees (8 people). They
were experts with extensive professional experience in the health sector, most
of them more than 20 years. Among the participants of the study, men and those
working in the public sector predominated. The sample structure is presented
in Table 15 provided below.

Table 15: Demographic characteristics of the sample group (N)

Gender | Male | 9
---|---|---
Female | 7
Age | <40 | 2
40-50 | 6
>50 | 8
Dominant background | Healthcare entities - managerial position | 5
Healthcare sector- administrative position | 4
Healthcare unit- medical staff | 2
Insurance scheme | 1
Government – department of healthcare; ministry | 3
Third party payer | 2
Education | MBA | 5
Ph.D. | 8
Professor | 1
M.Sc. | 2
Workplace | Public sector employee | 13
Private sector employee | 2
Entrepreneur, self-employed | 1
Years of experience | 10-15 | 2
16-20 | 5
> 20 | 9
Country of origin | Poland | 2
Hungary | 2
Latvia | 2
France | 1
Belgium | 1
Finland | 1
Sweden | 1
Norway | 1
Slovenia | 2
United Kingdom | 2
Germany | 1

##  Appendix C Criteria weights

Table 16: Criteria weights determined using different weighting methods.

$C_{i}$ | AHP-based relative | CRITIC | Entropy
---|---|---|---
$C_{1}$ | 0.0615 | 0.0330 | 0.0253
$C_{2}$ | 0.0615 | 0.0317 | 0.0170
$C_{3}$ | 0.0615 | 0.0328 | 0.0100
$C_{4}$ | 0.0615 | 0.0330 | 0.0408
$C_{5}$ | 0.0410 | 0.0476 | 0.0300
$C_{6}$ | 0.0410 | 0.0355 | 0.0596
$C_{7}$ | 0.0410 | 0.0289 | 0.0671
$C_{8}$ | 0.0296 | 0.0482 | 0.0762
$C_{9}$ | 0.0296 | 0.0407 | 0.0010
$C_{10}$ | 0.0296 | 0.0410 | 0.0124
$C_{11}$ | 0.0296 | 0.0500 | 0.0356
$C_{12}$ | 0.1182 | 0.0381 | 0.0347
$C_{13}$ | 0.0591 | 0.0409 | 0.0003
$C_{14}$ | 0.0591 | 0.0567 | 0.0508
$C_{15}$ | 0.0646 | 0.0401 | 0.1064
$C_{16}$ | 0.0323 | 0.0464 | 0.0496
$C_{17}$ | 0.0323 | 0.0415 | 0.0694
$C_{18}$ | 0.0595 | 0.0471 | 0.0003
$C_{19}$ | 0.0298 | 0.0377 | 0.0024
$C_{20}$ | 0.0298 | 0.0380 | 0.0550
$C_{21}$ | 0.0047 | 0.0411 | 0.0237
$C_{22}$ | 0.0047 | 0.0368 | 0.0869
$C_{23}$ | 0.0047 | 0.0357 | 0.0570
$C_{24}$ | 0.0047 | 0.0420 | 0.0728
$C_{25}$ | 0.0094 | 0.0353 | 0.0156

##  Appendix D The Correlation Coefficients

###  D.1 Weighted Spearman’s Rank Correlation Coefficient

The calculation of the weighted Spearman correlation coefficient is shown by
the Equation 19, where $N$ is a sample size, $x_{i}$, and $y_{i}$ are the
values in the compared rankings (Sałabun et al., 2020).

$r_{w}=1-\frac{6\sum_{i=1}^{N}(x_{i}-y_{i})^{2}((N-x_{i}+1)+(N-y_{i}+1))}{N^{4}+N^{3}-N^{2}-N}$
(19)

###  D.2 The Pearson Correlation Coefficient

The Pearson correlation coefficient is represented by the Equation 20

$r_{xy}=\frac{N\sum x_{i}y_{i}-\sum x_{i}\sum y_{i}}{\sqrt{N\sum
x_{i}^{2}-(\sum x_{i})^{2}}\sqrt{N\sum y_{i}^{2}-(\sum y_{i})^{2}}}$ (20)

where $N$ is a sample size, $x_{i}$ and $y_{i}$ are the individual sample
elements indexed with $i$ (Deng et al., 2021).

## Acknowledgments

This research was funded in part by National Science Centre, Poland
2022/45/B/HS4/02960. For the purpose of Open Access, the author has applied a
CC-BY public copyright licence to any Author Accepted Manuscript (AAM) version
arising from this submission

## References

  * Abdel-Basset and Mohamed (2020) Abdel-Basset, M., Mohamed, R., 2020\.  A novel plithogenic TOPSIS-CRITIC model for sustainable supply chain risk management.  Journal of Cleaner Production 247, 119586.  doi:https://doi.org/10.1016/j.jclepro.2019.119586.
  * Ajmal et al. (2018) Ajmal, M.M., Khan, M., Hussain, M., Helo, P., 2018\.  Conceptualizing and incorporating social sustainability in the business world.  International Journal of Sustainable Development & World Ecology 25, 327–339.  doi:https://doi.org/10.1080/13504509.2017.1408714.
  * Aktas et al. (2015) Aktas, A., Cebi, S., Temiz, I., 2015.  A new evaluation model for service quality of health care systems based on AHP and information axiom.  Journal of Intelligent & Fuzzy Systems 28, 1009–1021.  doi:10.3233/IFS-141385.
  * Ali et al. (2021) Ali, H., Chen, T., Hao, Y., 2021.  Sustainable Manufacturing Practices, Competitive Capabilities, and Sustainable Performance: Moderating Role of Environmental Regulations.  Sustainability 13, 10051\.  doi:https://doi.org/10.3390/su131810051.
  * AlJaberi et al. (2017) AlJaberi, O.A., Hussain, M., Drake, P.R., 2017.  A framework for measuring sustainability in healthcare systems.  International Journal of Healthcare Management doi:https://doi.org/10.1080/20479700.2017.1404710.
  * AlJaberi et al. (2020) AlJaberi, O.A., Hussain, M., Drake, P.R., 2020.  A framework for measuring sustainability in healthcare systems.  International Journal of Healthcare Management 13, 276–285.  doi:https://doi.org/10.1080/20479700.2017.1404710.
  * Andersson et al. (2007) Andersson, E., Creasy, S., Tritter, J., 2007.  Does patient and public involvement matter?  Health Democracy: The Future of Involvement in Health and Social Care. London: NHS National Centre for Involvement .
  * Angelis and Kanavos (2017) Angelis, A., Kanavos, P., 2017\.  Multiple criteria decision analysis (MCDA) for evaluating new medicines in health technology assessment and beyond: the advance value framework.  Social Science & Medicine 188, 137–156.  doi:https://doi.org/10.1016/j.socscimed.2017.06.024.
  * ARĀJA and PRIEDE (2018) ARĀJA, D., PRIEDE, J., 2018\.  OPPORTUNITIES FOR ASSESSMENT OF HEALTH CARE SYSTEM’S EFFICIENCY: THREE-LEVELS EVALUATION MODEL.  Journal of Economics and Management Research 7, 26–48.  doi:http://doi.org/10.22364/jemr.7.02.
  * Arnesen et al. (2002) Arnesen, K.E., Erikssen, J., Stavem, K., 2002.  Gender and socioeconomic status as determinants of waiting time for inpatient surgery in a system with implicit queue management.  Health policy 62, 329–341.  doi:https://doi.org/10.1016/S0168-8510(02)00052-0.
  * Awan et al. (2018) Awan, U., Kraslawski, A., Huiskonen, J., 2018.  Governing interfirm relationships for social sustainability: the relationship between governance mechanisms, sustainable collaboration, and cultural intelligence.  Sustainability 10, 4473\.  doi:https://doi.org/10.3390/su10124473.
  * Badi et al. (2018) Badi, I., Ballem, M., Shetwan, A., 2018.  Site selection of desalination plant in Libya by using Combinative Distance-based Assessment (CODAS) Method.  International Journal for Quality Research 12, 609.  doi:10.18421/IJQR12.03-04.
  * Bandeira et al. (2018) Bandeira, R.A., D’Agosto, M.A., Ribeiro, S.K., Bandeira, A.P., Goes, G.V., 2018.  A fuzzy multi-criteria model for evaluating sustainable urban freight transportation operations.  Journal of cleaner production 184, 727–739.  doi:https://doi.org/10.1016/j.jclepro.2018.02.234.
  * Bankauskaite and Dargent (2007) Bankauskaite, V., Dargent, G., 2007\.  Health systems performance indicators: methodological issues.  Presupuesto y gasto publico 49, 125–137.
  * Barsanti and Nuti (2014) Barsanti, S., Nuti, S., 2014\.  The equity lens in the health care performance evaluation system.  The International journal of health planning and management 29, e233–e246.  doi:https://doi.org/10.1002/hpm.2195.
  * Behmane et al. (2019) Behmane, D., Dudele, A., Villerusa, A., Misins, J., Klavina, K., Mozgis, D., Scarpetti, G., Organization, W.H., et al., 2019.  Latvia: Health system review.  European Observatory on Health Systems and Policies URL: https://apps.who.int/iris/handle/10665/331419.
  * Ben Moallem et al. (2018) Ben Moallem, M., Derbel, C., Ammar, M.H., Béji, N., Dhouib, D., 2018.  Risk Prioritization Using the Analytic Hierarchy Process (AHP) in a Tunisian Healthcare Department: A Real-World Case Study.  Multiple Criteria Decision Making 13, 9–28.  doi:10.22367/mcdm.2018.13.01.
  * de Bienassis et al. (2020) de Bienassis, K., Kristensen, S., Burtscher, M., Brownwood, I., Klazinga, N., 2020.  OECD Health Working Papers.
  * Blümel et al. (2020) Blümel, M., Röttger, J., Köppen, J., Achstetter, K., Busse, R., 2020.  Integrating the population perspective into health system performance assessment (IPHA): study protocol for a cross-sectional study in Germany linking survey and claims data of statutorily and privately insured.  International Journal of Health Policy and Management 9, 370–379.  doi:10.15171/ijhpm.2019.141.
  * Blümel et al. (2021) Blümel, M., Spranger, A., Achstetter, K., Maresso, A., Busse, R., 2021.  Germany: Health system review.  Health systems in transition 22, 1–272.
  * Bond and Morrison-Saunders (2013) Bond, A., Morrison-Saunders, A., 2013\.  Challenges in determining the effectiveness of sustainability assessment.  Routledge, Taylor & Francis Group.
  * Busse (2013) Busse, R., 2013.  Understanding satisfaction, responsiveness and experience with the health system.  Health system performance comparison: an agenda for policy, information and research , 255–280.
  * Büyüközkan and Çifçi (2012) Büyüközkan, G., Çifçi, G., 2012.  A combined fuzzy AHP and fuzzy TOPSIS based strategic analysis of electronic service quality in healthcare industry.  Expert systems with applications 39, 2341–2354.  doi:https://doi.org/10.1016/j.eswa.2011.08.061.
  * Capolongo et al. (2013) Capolongo, S., Buffoli, M., Oppio, A., Rizzitiello, S., et al., 2013\.  Measuring hygiene and health performance of buildings: a multidimensional approach.  Ann Ig 25, 15–17.
  * Capolongo et al. (2016) Capolongo, S., Gola, M., Di Noia, M., Nickolova, M., Nachiero, D., Rebecchi, A., Settimo, G., Vittori, G., Buffoli, M., 2016.  Social sustainability in healthcare facilities: a rating tool for analysing and improving social aspects in environments of care.  Annali dell’Istituto Superiore di Sanità 52, 15–23.  doi:10.4415/ANN_16_01_06.
  * Carey et al. (2015) Carey, G., Crammond, B., De Leeuw, E., 2015.  Towards health equity: a framework for the application of proportionate universalism.  International journal for equity in health 14, 1–8.  doi:10.1186/s12939-015-0207-6.
  * Caunic et al. (2019) Caunic, R.E., et al., 2019.  Frameworks and measures for health systems performance assessment.  SEA–Practical Application of Science 7, 205–212.
  * Chan and Lee (2019) Chan, P., Lee, M.H., 2019.  Prioritizing sustainable city indicators for Cambodia.  Urban Science 3, 104\.  doi:https://doi.org/10.3390/urbansci3040104.
  * Chevreul et al. (2015) Chevreul, K., Brigham, B., Durand-Zaleski, I., Hernández-Quevedo, C., 2015.  France: health system review.  Health systems in transition 17, 1–218.
  * Chiu (2003) Chiu, R., 2003.  Social Sustainability, Sustainable Development and Housing Development: The Experience of Hong Kong, in: Housing and Social Change: East-West Perspectives. Routledge New York, NY, USA. volume 221. chapter 12, pp. 233–251.
  * Chmielarz and Zborowski (2022) Chmielarz, W., Zborowski, M., 2022\.  On the Assessment of e-Banking Websites Supporting Sustainable Development Goals.  Energies 15, 378\.  doi:https://doi.org/10.3390/en15010378.
  * Colantonio (2009) Colantonio, A., 2009.  Social sustainability: a review and critique of traditional versus emerging themes and assessment methods, in: The Second International Conference on Whole Life Urban Sustainability and Its Assessment: Conference Proceedings, Loughborough University: Loughborough, UK, 2009. pp. 865–885.
  * Convertino et al. (2019) Convertino, M., Annis, A., Nardi, F., 2019.  Information-theoretic portfolio decision model for optimal flood management.  Environmental Modelling & Software 119, 258–274.  doi:https://doi.org/10.1016/j.envsoft.2019.06.013.
  * Cristiano et al. (2021) Cristiano, S., Ulgiati, S., Gonella, F., 2021.  Systemic sustainability and resilience assessment of health systems, addressing global societal priorities: Learnings from a top nonprofit hospital in a bioclimatic building in Africa.  Renewable and Sustainable Energy Reviews 141, 110765.  doi:https://doi.org/10.1016/j.rser.2021.110765.