# Ontology in Hybrid Intelligence: a concise literature review

Salvatore Flavio Pileggi
<EMAIL_ADDRESS>University of Technology Sydney
15 Broadway, Ultimo NSW 2007, Australia

(…)

###### Abstract

In a context of constant evolution and proliferation of AI technology, Hybrid
Intelligence is gaining popularity to refer a balanced coexistence between
human and artificial intelligence. On the other side, the concept has been
extensively used in the past two decades to define models of intelligence
involving more than one technology. This paper aims to provide (i) a concise
and focused overview of the adoption of Ontology in the broad context of
Hybrid Intelligence regardless of its definition and (ii) a critical
discussion on the possible role of Ontology to reduce the gap between human
and artificial intelligence within hybrid intelligent systems. Beside the
typical benefits provided by an effective use of ontologies, at a conceptual
level, the analysis conducted has pointed out a significant contribution to
quality and accuracy, as well as a more specific role to enable extended
interoperability, system engineering and explainable/transparent systems. On
the other side, an application-oriented analysis has shown a significant role
in present systems (70+% of the cases) and, potentially, in future systems.
However, a proper holistic discussion on the establishment of the next
generation of hybrid-intelligent environments with a balanced co-existence of
human and artificial intelligence is fundamentally missed in literature. Last
but not the least, there is currently a relatively low explicit focus on
automatic reasoning and inference.

Keywords: Artificial Intelligence, Hybrid Intelligence, Ontology, Semantic
Web, Interoperability, Explainability, Knowledge Representation.

##  1 Introduction

Artificial Intelligence is constantly evolving [60] towards a more and more
sophisticated technology [52] that is generating a disruptive innovation in
most application domains such as, among others, medicine [29] / healthcare
[87], education [13], finance [2], transportation [1] and manufacturing [46].
Indeed, the last generation of AI-powered systems is having a clear impact to
solve real-world problems, including also tangible contributions in situations
of major crisis [81], and, in general terms, is generating probably
unprecedented expectations.

The current socio-technological climate is characterised by a imperishable mix
of enthusiasm and concern. A concrete example is generative AI [84], such as
ChatGTP, an AI program that has become a cultural sensation in less than 2
months [78]. By automatically creating text based on written prompts, ChatGTP
has revealed the potentiality of AI also to non-expert users, while has
further increased concerns on possible and probable mis-use, as well as on
collateral effect for society. The classic debate involving experts from
academia and industry around ethics and possible regulations, is now somehow
extended to involve a much broader audience in a kind of public debate.

The rising of AI has also led to an increasing popularity of hybrid solutions,
normally referred to as Hybrid Intelligence [17]. That is an evidently less
radical concept that assumes a model of intelligence based on a balanced co-
existence of humans and machines. On the other side, the concept has been
extensively used in the past two decades to define models of intelligence
involving more than one technology.

Born in philosophy, Ontology is a relatively consolidated concept also in
Computer Science [26], where it is extensively adopted to formalize a given
domain within computer systems [11]. In practice, Ontologies are rich data
models that allow the specification of machine-processable ”semantics”. While
their primary purpose is machine-to-machine interaction, ontological modelling
enables extended capabilities also in terms of human-computer interaction by
fostering the same representation of knowledge for humans and machines. While
ontology actively contributes in general in the context of Intelligent Systems
at a different level, its specific potential within Hybrid Intelligence is
probably not fully exploited and could play a significant role within the next
generation systems.

In a context of constant evolution and proliferation of AI technology, this
paper aims to provide (i) a concise and focused overview of the adoption of
Ontology in the broad context of Hybrid Intelligence regardless of its
definition and (ii) a critical discussion on the possible role of Ontology to
reduce the gap between human and artificial intelligence.

The relevance of this study resides mostly in a need for a consolidated
approach for Hybrid Intelligence that may assure an evolution according to the
current society needs.

##### Structure of the paper.

The introductory part follows with a brief presentation of the background
concepts (Section 2); methodological aspects are discussed in Section 3, while
Section 4 provides an overview of the contributions on the topic in
literature; such a body of knowledge is discussed in quantitative and
qualitative terms in Section 5; finally, the paper ends with conclusions
section.

##  2 Background concepts

The root of this review is intrinsically related to the very generic concepts
of intelligence and semantics, which have an extremely broad scope in the
context of modern systems. The focus is explicitly on more narrowed concepts,
more concretely on the intersection between Hybrid Intelligence and Ontology.
Such an approach allows the identification of a body of knowledge. However,
hybrid intelligent systems may be defined in a different ways and quite often
present blurred boundaries; furthermore, ontologies may refer to the more
generic and conceptual knowledge representation as well as to the more
technology-specific Semantic Web.

While methodological aspects are proposed later on (Section 3), this section
aims to overview the central concepts in context. An in-depth exhaustive
discussion is out of the paper scope.

###  2.1 Defining ”intelligence”

The definition of ”intelligence” has been historically object of discussion
and controversy. While it is not probably possible to provide universally
accepted formal definitions, it is interesting to look at the understanding of
intelligence in the context of a more and more technology-intensive society.

####  2.1.1 Human and Artificial Intelligence

Human (or natural) intelligence is normally associated with the ability to
generate an abstracted mental model of a given reality and perform mental
simulations accordingly. It underlays the common notions of thinking and
reasoning [53] enabling human skills in terms of problem analysis/solving and
decision making [39].

Although such considerations may be considered a common assumption, it is also
well-known that human intelligence goes beyond its ”analytical” component to
embrace also ”emotional” intelligence [56], which is based on emotions (e.g.
empathy) to enable a unique and strictly human way to behave, reason and
decide resulting from a mix of rational elements and ”gut feelings” [38]. This
more comprehensive understanding of intelligence plays a key role given the
increasing dehumanization within society, which is pushed also by technology
[63].

There are at least two critical complementing considerations: on one side, all
human beings have their own consciousness and unique ways of thinking [53]; on
the other side, individuals belong to a social context, which affects the way
reality is perceived and consequently the way to think. It naturally leads to
the concept of ”collective” intelligence [44], which puts emphasis on the
added, if not determinant, value of the collectivity to establish a level of
intelligence higher than the correspondent to any individual. Collective
intelligence becomes even more critical in the context of an always connected
world with unprecedented interaction possibilities enabled by the evolution of
the Web [64] and social Web sites operating at a global scale [36].

Computers have intrinsically generated the idea of machines able to ”reason”
like humans. Indeed, Alan Turing, often considered the ”father of computer
science” formulated the question ”can machines think?” that led to the famous
”Turing Test”, where a human tries to distinguish between a computer and human
response in a conversation [54]. Since then, the intuitive concept of
Artificial Intelligence has evolved in the scientific context and very many
definitions have been provided. For instance, ”AI is the science and
engineering of making intelligent machines, especially intelligent computer
programs” [57].

Given the huge availability of data and computational resources, the most
recent advances in the field are producing a actual new generation of
solutions, which are often converted to services or tools potentially
available to the most, such as the already cited ChatGPT. Main stakeholders
are actively involved in a constant debate, with a mix of excitement and
concern. It is triggering different feeling and perceptions in the broader
social context.

####  2.1.2 Hybrid Intelligence

As part of the ongoing debate, more and more questions are rising up. Among
them, and because of a number of evident concerns, how human and artificial
intelligence are going to co-exist?; how such a ”relationship” is going to
evolve in the future?

A an interesting (and also very intuitive) concept in this broader context is
probably Hybrid Intelligence, that focuses on expanding the human intellect
with AI, instead of replacing it [32]. Authors in [17] characterise Hybrid
Intelligence through a comparative analysis.

This more recent understanding and definition of Hybrid Intelligence as a
combination of human and machine intelligence also reflects an evolution of
the concept. Indeed, that same term has been extensively adopted more or less
informally in the last two decades to indicate the engineering of systems
whose ”intelligence” results from the combined application of more than one
technology.

###  2.2 Semantics, Ontology and Semantic Web

Born as the branch of metaphysics dealing with ”the nature of being”, ontology
is a philosophical concept often associated with a formal conceptualization of
a given domain. According to that same holistic approach, ontology became part
of the computer world [26] to provide machine-processable conceptualizations.

An ontology-based approach to Knowledge Representation allows to specify
”semantics” within computer systems and, indeed, ontology is a key asset
within the Semantic Web [5]. Semantic Web adopts the Web infrastructure and
standard languages to formalise a common understanding of data and to achieve
”intelligence” via interoperability and inference. Ontologies are extensively
adopted in wide range of domains and applications, as well as they play a role
in establishing advanced data ecosystems, such as Open Data [61] and Linked
Data [6].

The primary application context for ontology is machine-to-machine
interaction. However, through a proper knowledge engineering process, it is
possible to assume a ”shared” vision of a given knowledge for humans and
machines. It can be further fostered by different kind of abstractions and
visualizations [70].

##  3 Methodology and approach

This paper aims to provide (i) a concise and focused overview of the adoption
of Ontology in the broad context of Hybrid Intelligence and (ii) a critical
discussion on the possible role of Ontology to reduce the gap between human
and artificial intelligence within hybrid intelligent systems. A simplified
conceptualization of main objectives and scope by adopting an enriched
conceptual map [9] is proposed in Figure 1. The underlying idea assumes
intelligence enabled in a heterogeneous context, either involving multiple
technologies or a coexistence of human and artificial intelligence.

Figure 1: Conceptualization of main objectives and scope.

Because of the potentially broad scope, looking at that specific context, the
literature review has been conducted assuming the following selection
criteria:

SC.1

Selected papers have an explicit focus on Hybrid Intelligence, regardless of
its contextual definition

SC.2

Selected papers explicitly address the role of Ontology

SC.3

The role of ontology is relevant in the context of the contribution and its
value can be identified

While searching criteria have been designed accordingly to systematically
retrieve papers which simultaneous contain the keywords ”Hybrid Intelligence”
and ”Ontology”, acceptance is based on a qualitative assessment by the author
of SC.1, SC.2 and SC.3. However, as ontology and hybrid intelligence can be
indirectly referred in different ways within the current technological
context, the query has been extended to include possible common
equivalent/strictly related concepts as follows:

    ("Hybrid Intelligence" OR "Hybrid Intelligent") AND ("Ontology" OR "Semantic Web")

On the other side, very generic keywords, such as ”Knowledge Representation”
have not been used for retrieval.

In order to further enhance a critical discussion in context, selected papers,
wherever possible, are associated with a generic domain and even with a more
specific application. Additionally, as part of the adopted framework, the
different contributions are classified depending on their focus
(conceptual/theoretical or application-focused) and on the provided value in a
given system. This light classification model contributes to generate a
structured overview and a conceptualised analysis. On the other side,
classifications are arbitrary and may present some overlapping.

The combined application of the mentioned selection criteria results in a
relatively restrictive retrieval strategy that limits significantly the number
of papers to include in the literature review. In this specific case, this
approach assures a focused discussion within a broad topic, mostly
characterised by the blurred boundaries of Hybrid Intelligence.

##  4 Ontology in Hybrid Intelligence

An overview of the major contributions retrieved from the most common
scientific databases is proposed in Table 1 and 2, which report conceptual and
application-focused papers respectively. In order to provide a grounded
discussion, they are addressed separately in the next two sub-sections.

###  4.1 A conceptual perspective

Table 1 reports the contributions in literature that present a more
theoretical/conceptual approach. Although they have not a specific
application-focus and should allow a certain generalization of the insight,
where specified, the original domain/application is still reported.

Contr. | Domain | Application | Focus |  Value (main)
---|---|---|---|---
[18] | Business | Decision Support System | Conceptual | Interoperability
[42] | N\A | N\A | Conceptual | Interoperability
[41] | N\A | N\A | Conceptual | Interoperability
[75] | Healthcare | Explainable models | Conceptual | Explainability
[32] | N\A | Explainable models | Conceptual | Explainability
[8] | Education | System Thinking | Conceptual | System Engineering
[22] | Smart Systems | Ambient Assisting Living | Conceptual | System Engineering
[72] | N\A | N\A | Conceptual | Explainability
[67] | N\A | Collective Intelligence | Conceptual | Quality and Accuracy
[79] | N\A | Knowledge Graph | Conceptual | Explainability
[59] | N\A | Collective Intelligence | Conceptual | Quality and Accuracy
[89] | N\A | N\A | Conceptual | Quality and Accuracy
[82] | N\A | N\A | Conceptual | System Engineering
[76] | N\A | N\A | Conceptual | System Engineering
Table 1: Overview of the selected contributions with a conceptual focus.

In [18], authors propose a design principle: ”Provide the Hybrid Intelligence
DSS with an ontology-based representation to transfer an entrepreneur’s
assumptions and create a shared understanding among the mentors, the machine,
and the entrepreneur”. Despite the domain/application specific focus, the use
of ontologies to create a shared understanding among all stakeholders
(including also machines) can be applied in general terms.

Another principle assumes interoperability as one of the key factors behind
the development of hybrid intelligence [42]. An effective machine-to-machine
model to assure interoperability among different systems leads to a central
role of ontologies to enable sophisticated approaches (Semantic
Interoperability [62]). Additionally, also for interoperability, a multi-
stakeholder vision fundamentally applies to real-world scenarios that
typically involve machine both with designers, subject matter experts and
final users [41].

Looking at the current technological climate, an aspect of increasing interest
is the potential role of ontology to reduce fear of AI by pushing more
intelligible explainable models [75][72]. It would enable more transparent and
trusting environments for an effective co-existence of human and artificial
intelligence, especially if considered both with Knowledge Graphs [79].
Moreover, a wider perspective to possible evolutions of AI that implies an
increasing convergence of data, such as AI 2.0 [67], is worthy of mention.

A relatively more classic contribution is in the engineering of systems
adopting hybrid intelligence, where ontology can contribute to formalize the
specification of requirements [22]. This is in line with typical benefits of
Ontology in the generic System [86] / Software [69] Engineering, as well as in
the more specific field of Requirement Engineering [19]. Given the recent
advancement in AI and the peculiarities of Hybrid Intelligence, the role of
Ontology throughout the design process could be remarkable (e.g. contributing
to formalization of problems [76]) and can eventually provide also linking to
the different principles, such as ethics. In [8], authors further extend the
intent and the extent within system engineering by dealing with requirements
and challenges. It addresses the need for automatic reason.

Last but not least, ontologies may contribute to enable and sustain the
quality of knowledge-based systems. For instance, argumentation can be adopted
to address inconsistencies in knowledge bases [32].

[59] deals with the interesting concept of Hybrid Collective Intelligence and
the consequent need for knowledge convergence, as well as [89] discusses the
augmentation of Hybrid Intelligence by collaboration and cognition with an
extensive use of semantic artefacts. In general terms, ontology-based models
are extensively used in hybrid learning [82].

###  4.2 An application perspective

An overview of applications is summarised in Table 2. Such an application
perspective shows a clear generic focus on Knowledge Representation.

Contr. | Domain | Application | Focus |  Value (main)
---|---|---|---|---
[21] | N\A | Blockchain | Application | Automatic Reasoning
[85] | Software | Programming | Application | Knowledge Representation
[66] | Agriculture | Recommender Systems | Application | Semantic Similarity
[35] | Security | Intrusion Detection/ Prevention | Application | Modelling
[31] | Security | Critical Infrastructure Protection | Application | Information Standardization
[34] | Smart Systems | Assistive Technology | Application | Knowledge Representation
[43] | Healthcare | Drug Discovery | Application | Analysis
[58] | Management | Decision Support Systems | Application | Knowledge Representation
[71] | Business | Automated Enterprise Modeling | Application | Automated Reasoning
[45] | Healthcare | Decision Support Systems | Application | Knowledge Representation
[65] | Science | N\A | Application | Knowledge Representation
[68] | N\A | Data Integration | Application | Knowledge Representation
[25] | Business | Assistive Technology | Application | Semantic Similarity
[77] | Law | Text mining | Application | Knowledge Representation
[15] | N\A | Information Systems | Application | Knowledge Representation
[14] | N\A | Internet of Things | Application | Knowledge Representation
[51] | N\A | Multi-agent Systems | Application | Semantic Similarity
[74] | N\A | Data Analysis | Application | Knowledge Representation
[3] | Healthcare | Automated Learning | Application | Knowledge Representation
[37] | N\A | Multi-agent Systems | Application | Automated Reasoning
[28] | Smart Systems | Control Systems | Application | Knowledge Representation
[48] | N\A | Multi-agent Systems | Application | Semantic Similarity
[49] | N\A | Multi-agent Systems | Application | Knowledge Representation
[88] | Business | Multi-agent Systems | Application | Knowledge Representation
[50] | N\A | Multi-agent Systems | Application | Knowledge Representation
[7] | N\A | Multi-agent Systems | Application | Knowledge Representation
[73] | N\A | Multi-agent Systems | Application | Knowledge Representation
[27] | N\A | Multi-agent Systems | Application | Automated Reasoning
[23] | N\A | Decision Support Systems | Application | Knowledge re-use and Sharing
[12] | N\A | Data Mining | Application | Automated Reasoning
[80] | Healthcare | Recommender Systems | Application | Modelling
[4] | N\A | Information Systems | Application | Knowledge Representation
[30] | Agriculture | Control Systems | Application | Knowledge Representation
[83] | N\A | Collaborative Systems | Application | Knowledge Representation
[47] | Business | Recommender Systems | Application | Knowledge Representation
[20] | Software | Software Engineering | Application | Automated Reasoning
[90] | N\A | Data Mining | Application | Knowledge Representation
[55] | Manufacturing | Industry 4.0 | Application | Knowledge Representation
[16] | Smart Systems | Energy Systems | Application | Knowledge Representation
[24] | N\A | Information Systems | Application | Knowledge Representation
[10] | Management | Decision Support Systems | Application | Knowledge Representation
[40] | Business | Business Intelligence | Application | Automated Reasoning
Table 2: Overview of the selected contributions with an application focus.

Indeed, Knowledge Representation is seen as an essential asset to trace the
provenance of data and knowledge and to link the accountability of humans and
machine-based agents [21]. Such kind of integrated approach is a major
prerequisite to establish trust in hybrid systems [21]. These generic
considerations apply in a different way within concrete application domains
(e.g. Science [65]). For instance, in certain programming environments based
on semantically enriched compound objects, ”the confidence of users in making
decisions is based on access to previous decisions of others, on quick
perception and understanding these decisions and on their transformations
leading to new models and algorithms” [85]. Additionally, the adoption of
shared ontologies for different purposes - such as upper, domain,
communication and state annotation - is a relatively established pattern [34].

Within the broad Knowledge representation context, it is possible to define a
set of more fine-grained capabilities. For instance, [66] proposes a model
based on Semantic Similarity to generate a knowledge base in the field of
Agriculture from multiple sources. Solutions for agriculture are also proposed
in [30]. Semantic Similarity [51][48] and negotiation techniques [37][7] are
considered in multi-agent systems. More in general, multi-agent systems
extensively adopt ontology-based representations [49][88][50][73].

[35] adopts an ontological model for intrusion detection. The work proposed in
[31] addresses the significant issues related to information standardization
and ontology-based representation in the security domain, while in [43] the
emphasis is on analysis.

In general terms, automation plays a key role in modern systems. For instance,
[71] proposes an approach for automated Enterprise Modeling, [27] presents a
technique for component integration, [68] approaches problems related to
metadata management in data exchange and integration, while [45] deals with
automated annotation and ontological mapping of clinical texts. Additionally,
the system proposed in [28] has an explicit focus on automated control, as
well as [12] and [90] addresses hybrid-intelligent data mining.

In line with the more theoretical analysis previously proposed, the work
presented in [58] focuses on the explanation of hybrid intelligence systems
with an extensive use of Semantic Web technology, as well as the value of
semantics in human-machine collaboration is highlighted in [25]. Semantic Web
technology is adopted to support hybrid systems in the Law domain [77], within
more generic Information Systems [15][4][24] and Internet of Things [14].

The system proposed in [74] adopts ontology to generate and classify vehicle
driver profiles. Ontology matching is a central concept in the automated
medical learning approach proposed in [3].

[23] implicitly refers to Hybrid Intelligence and addresses the relevance of
knowledge re-use and sharing.

Fuzzy-logic and ontological modelling are combined in [80] to implement
sophisticated recommendation in healthacare. Solutions for Recommendation
Systems are also object of other works (e.g. [47]).

Ontology-based meta-modelling is adopted in [83] to support machine-to-machine
as well as human-machine interaction. Remarkable also applications in the area
of Software Engineering [20] and advanced manufacturing (e.g. Industry 4.0
[55]), Energy Systems [16], crisis management/prevention [10] and Business
Intelligence [40].

##  5 Gap Identification and Challenges

This section aims to quantitatively and qualitatively analyse the identified
body of knowledge and to critically discuss major gaps, with an outlook on
challenges.

###  5.1 A quantitative analysis

The proposed analysis has been conducted on 56 different papers on the topic.
As expected, most contributions (75%) present an application focus. However,
the relatively consistent amount of more conceptual papers has provided a
significant insight (Section 5.2).

The classification by macro domain (Figure 2) is relatively distributed and
does not allow the identification of major patterns or trends. Almost an half
of the papers cannot be clearly associated with a macro-domain. There is a
prevalence of contributions in business (11%), healthcare (9%) and smart
systems (7%).

Similar considerations apply also to a quantitative analysis by application
(Figure 3), which presents a low percentage (13%) of unclassified papers and
shows a major number of contributions in Multi-agent Systems (16%) and
Decision Support Systems (9%).

Figure 2: Quantitative analysis by macro-domain.

Figure 3: Quantitative analysis by application.

An analysis by ”provided value” is reported in Figure 4. In quantitative
terms, the papers object of analysis deal with the four different identified
categories uniformly. Such contributions are discussed in qualitative terms
later on in the paper. An application-oriented perspective points out a high
number of works on generic knowledge representation, while there is much less
explicit focus on other aspects, including also automatic reasoning.

Figure 4: Quantitative analysis of the provided value from a conceptual and an
application perspective.

.

###  5.2 A qualitative analysis

From a conceptual point of view, four key aspects have been identified as
follows:

CP.1

Interoperability as a key factor. It includes interoperability among systems
as well as a shared understanding between humans and machines.
Interoperability is a concept classically associated with Ontology. However,
in the specific field of Hybrid Intelligence, there is a clear emphasis on
human-machine. It may have implications on Knowledge Engineering, as well as
on the evolution of ontology-related technology.

CP.2

Explainable and transparent models. The use of ontologies to support some kind
of intelligible synthesis of a given analysis or process is not an absolute
novelty. For instance, Knowledge Graphs [33] are extensively adopted in
several disciplines and applications and may be underpinned by formal
ontologies to deal with the underlying complexity [70]. In most cases, such a
presentation level is ad-hoc designed to optimally match the requirements
within a system. Explainable and transparent models in AI (and Hybrid AI)
require probably a more systematic approach.

CP.3

System Engineering. Also the contribution of Ontology within System/Software
Engineering is well known and extensively documented in literature. This
review, conducted in the specific context of Hybrid Intelligence, has pointed
out a potential extended and enhanced scope to include linking to different
aspects, such as design and ethic principles, and challenges.

CP.4

Quality and Accuracy (including evolution). This last aspect has a less
specific scope than the previous ones as it addresses the contributions of
Ontology to do ”better”. In practice, it may refer to support advanced
functionalities and features or to drive the evolution of a system. A typical
practical example emerged from the conducted review is the collaborative
approach which, in general terms, requires extended and more sophisticated
semantics, as well as a certain level of interoperability. That is, normally,
a value provided by ontologies.

A more extended qualitative analysis of applications aims to estimate the
relevance of Ontology in a given context. A summary is reported in Table 3.
The relevance of knowledge representation, automated reasoning or other
features has been informally evaluated without specific criteria independently
for each pertinent system to support a more holistic assessment. At least in
the 70+% of cases, the role of ontology may be considered relevant or
critical.

This evaluation is merely indicative and reflects a fundamental lack of detail
about the semantic infrastructure and their actual role in many proposed
systems.

Ontology Relevance
---
Title | KR | AR | Other | Overall
Supporting Trust in Hybrid Intelligence Systems Using Blockchains [21] | $\bullet\bullet\bullet$ | $\bullet\bullet\bullet$ | - | $\bullet\bullet\bullet$
Hybrid intelligence aspects of programming in* AIDA algorithmic pictures [85] | $\bullet\bullet$ | - | - | $\bullet\bullet$
HIAS: Hybrid Intelligence Approach for Soil Classification and Recommendation of Crops [66] | $\bullet\bullet\bullet$ | - | $\bullet\bullet\bullet$ | $\bullet\bullet$
Towards ontology-based intelligent model for intrusion detection and prevention [35] | $\bullet\bullet\bullet$ | $\bullet\bullet\bullet$ | - | $\bullet\bullet\bullet$
Hybint: a hybrid intelligence system for critical infrastructures protection [31] | - | - | $\bullet$ | $\bullet$
Hybrid Intelligence for Driver Assistance [34] | $\bullet$ | - | - | $\bullet$
Stargazer: A hybrid intelligence platform for drug target prioritization and digital drug repositioning using streamlit [43] | - | - | $\bullet$ | $\bullet$
Explaining Scientific and Technical Emergence Forecasting [58] | $\bullet$ | - | - | $\bullet$
Automatic generation of conceptual enterprise models [71] | $\bullet$ | - | - | $\bullet$
Assessing the impact of automated suggestions on decision making: Domain experts mediate model errors but take less initiative [45] | $\bullet\bullet$ | - | - | $\bullet\bullet$
The noosphere paradigm of the development of science and artificial intelligence [65] | $\bullet\bullet\bullet$ | - | - | $\bullet\bullet\bullet$
Combining OWL ontology and schema annotations in metadata management [68] | $\bullet\bullet\bullet$ | - | - | $\bullet\bullet\bullet$
Human-machine collaboration in online customer service–a long-term feedback-based approach [25] | $\bullet$ | - | $\bullet\bullet\bullet$ | $\bullet\bullet\bullet$
The text fragment extraction module of the hybrid intelligent information system for analysis of judicial practice of arbitration courts [77] | $\bullet\bullet\bullet$ | - | - | $\bullet\bullet\bullet$
The hybrid intelligent information system approach as the basis for cognitive architecture [15] | $\bullet\bullet\bullet$ | - | - | $\bullet\bullet\bullet$
The Social Web of Things (SWoT)-Structuring an Integrated Social Network for Human, Things and Services [14] | $\bullet\bullet\bullet$ | - | - | $\bullet\bullet\bullet$
Similarity measure of agents’ ontologies in cohesive hybrid intelligent multi-agent system [51] | $\bullet\bullet$ | - | $\bullet\bullet\bullet$ | $\bullet\bullet\bullet$
Context aware ontology-based hybrid intelligent framework for vehicle driver categorization [74] | $\bullet\bullet\bullet$ | - | - | $\bullet\bullet\bullet$
Hybrid intelligent framework for automated medical learning [3] | $\bullet\bullet\bullet$ | - | - | $\bullet\bullet\bullet$
Agents’ ontologies negotiation in cohesive hybrid intelligent multi-agent systems [37] | $\bullet\bullet$ | $\bullet\bullet$ | - | $\bullet\bullet\bullet$
HVAC control via hybrid intelligent systems [28] | $\bullet\bullet\bullet$ | - | - | $\bullet\bullet\bullet$
Estimating of the similarity of agents’ goals in cohesive hybrid intelligent multi-agent system [48] | $\bullet\bullet$ | - | $\bullet\bullet\bullet$ | $\bullet\bullet\bullet$
Cohesive hybrid intelligent multi-agent system architecture [49] | $\bullet$ | - | - | $\bullet$
An agent-based hybrid intelligent system for financial investment planning [88] | $\bullet$ | - | - | $\bullet$
Modeling team cohesion using hybrid intelligent multi-agent systems [50] | $\bullet$ | - | - | $\bullet$
Ontology support for communicating agents in negotiation processes [7] | $\bullet\bullet\bullet$ | - | - | $\bullet\bullet\bullet$
Visualization of team cohesion in hybrid intelligent multi-agent systems [73] | $\bullet$ | - | - | $\bullet$
Integration of Knowledge Components in Hybrid Intelligent Control Systems [27] | $\bullet\bullet\bullet$ | $\bullet\bullet\bullet$ | - | $\bullet\bullet\bullet$
Using a CBR approach based on ontologies for recommendation and reuse of knowledge sharing in decision making [23] | $\bullet\bullet\bullet$ | - | $\bullet\bullet\bullet$ | $\bullet\bullet\bullet$
Ontology-guided intelligent data mining assistance: Combining declarative and procedural knowledge [12] | $\bullet$ | $\bullet\bullet$ | $\bullet\bullet\bullet$ | $\bullet\bullet$
A hybrid fuzzy-ontology based intelligent system to determine level of severity and treatment recommendation for Benign Prostatic Hyperplasia [80] | $\bullet\bullet\bullet$ | $\bullet$ | $\bullet\bullet\bullet$ | $\bullet\bullet\bullet$
Using hybrid intelligent information system approach for text question generation [4] | $\bullet$ | - | - | $\bullet$
A Hybrid Intelligent Multiagent System for the Remote Control of Solar Farms [30] | $\bullet\bullet\bullet$ | $\bullet\bullet$ | - | $\bullet\bullet\bullet$
Ontology-based meta-model for hybrid collaborative scheduling [83] | $\bullet\bullet\bullet$ | - | - | $\bullet\bullet\bullet$
An Agent-Based Hybrid Intelligent System for Financial Investment Planning [47] | $\bullet$ | - | - | $\bullet$
A generic architecture for hybrid intelligent test systems [20] | $\bullet$ | $\bullet\bullet\bullet$ | - | $\bullet\bullet\bullet$
Constructing hybrid intelligent systems for data mining from agent perspectives [90] | $\bullet$ | - | - | $\bullet$
Efficient services in the industry 4.0 and intelligent management network [55] | $\bullet\bullet\bullet$ | $\bullet\bullet$ | - | $\bullet\bullet\bullet$
A fog-based hybrid intelligent system for energy saving in smart buildings [16] | $\bullet$ | - | - | $\bullet$
DSS-based ontology alignment in solid reference system configuration [24] | $\bullet\bullet\bullet$ | - | - | $\bullet\bullet\bullet$
Design and Conceptual Development of a Novel Hybrid Intelligent Decision Support System Applied towards the Prevention and Early Detection of Forest Fires [10] | $\bullet\bullet$ | - | - | $\bullet\bullet$
A hybrid reasoning architecture for business intelligence applications [40] | $\bullet\bullet\bullet$ | $\bullet\bullet$ | - | $\bullet\bullet\bullet$
KR = Knowledge Representation, AR = Automatic Reasoning
($\bullet\bullet\bullet$) = critical, ($\bullet\bullet$) = relevant,
($\bullet$) = supporting
Table 3: Assessment of the relevance of Ontology within the considered system.

###  5.3 Major Research Gaps

The literature review conducted has also allowed the identification and the
consequent formulation of a number of research gaps as follows:

G.1

The understanding of Hybrid Intelligence in a specific context or system is
not always explicitly defined but rather intuitive. This makes hard to
understand the effective relevance and role of ontologies within the broader
system to solve a given problem or address a given challenge.

G.2

Many contributions address and empathise the added value provided by
ontologies. However, there is often a fundamental lack of detail to support
the corresponding claims.

G.3

The contribution of ontologies to achieve hybrid environments where human and
artificial intelligence co-exist and co-operate is not fully addressed,
although the review has pointed out a relevant insight at a conceptual level.

G.4

An implicit role of ontologies as an interface between humans and machines in
hybrid systems emerged from the conducted literature review. However, looking
at existing solutions, the actual potential seems largely unexplored,
especially at an application level.

G.5

Lack of focus on automatic reasoning and inference.

G.6

Lack of focus on ontological modelling. It reflect a fundamental lack of big
picture, meaning that within certain systems ontologies are seen within
specific components rather than at a more holistic level.

##  6 Conclusions

In a context of constant evolution and proliferation of AI technology, Hybrid
Intelligence is gaining popularity to refer a balanced coexistence between
human and artificial intelligence. That same concept has been often used in
the recent past to define a model of intelligence resulting from multiple
technologies.

By adopting a soft methodology, this paper proposes a literature review that
aims to provide (i) a concise and focused overview of the adoption of Ontology
in the broad context of Hybrid Intelligence regardless of its definition and
(ii) a critical discussion on the possible role of Ontology to reduce the gap
between human and artificial intelligence within hybrid intelligent systems.

Beside the typical benefits provided by an effective use of ontologies, at a
conceptual level, the analysis conducted has pointed out a significant
contribution to quality and accuracy, as well as a more specific role to
enable extended interoperability, system engineering and
explainable/transparent systems. On the other side, an application-oriented
analysis has shown a significant role in present systems (70+% of the cases)
and, potentially, in future systems.

As extensively discussed in the paper, the concept of Hybrid Intelligence is
evolving to meet the requirements of a new generation of systems that may
present blurred boundaries. A proper holistic discussion on the establishment
of the next generation of hybrid-intelligent environments with a balanced co-
existence of human and artificial intelligence is fundamentally missed in
literature. So is the consequent analysis on the role of ontology.

## References

  * [1] Rusul Abduljabbar, Hussein Dia, Sohani Liyanage, and Saeed Asadi Bagloee.  Applications of artificial intelligence in transport: An overview.  Sustainability, 11(1):189, 2019.
  * [2] Arash Bahrammirzaee.  A comparative survey of artificial intelligence applications in finance: artificial neural networks, expert system and hybrid intelligent systems.  Neural Computing and Applications, 19(8):1165–1195, 2010.
  * [3] Asma Belhadi, Youcef Djenouri, Vicente Garcia Diaz, Essam H Houssein, and Jerry Chun-Wei Lin.  Hybrid intelligent framework for automated medical learning.  Expert Systems, 39(6):e12737, 2022.
  * [4] M Belyanova, V Chernenkiy, Y Kaganov, and Y Gapanyuk.  Using hybrid intelligent information system approach for text question generation.  In CEUR Workshop Proceedings, volume 2782, pages 194–201, 2020\.
  * [5] Tim Berners-Lee, James Hendler, and Ora Lassila.  The semantic web.  Scientific american, 284(5):34–43, 2001.
  * [6] Christian Bizer, Tom Heath, and Tim Berners-Lee.  Linked data: The story so far.  In Semantic services, interoperability and web applications: emerging concepts, pages 205–227. IGI global, 2011.
  * [7] Maricela C Bravo, Joaquin Pérez, Victor J Sosa, Azucena Montes, and Gerardo Reyes.  Ontology support for communicating agents in negotiation processes.  In Fifth International Conference on Hybrid Intelligent Systems (HIS’05), pages 6–pp. IEEE, 2005.
  * [8] Bert Bredeweg and Marco Kragten.  Requirements and challenges for hybrid intelligence: A case-study in education.  Frontiers in Artificial Intelligence, 5, 2022.
  * [9] Alberto J Cañas, Roger Carff, Greg Hill, Marco Carvalho, Marco Arguedas, Thomas C Eskridge, James Lott, and Rodrigo Carvajal.  Concept maps: Integrating knowledge and information visualization.  Knowledge and information visualization: Searching for synergies, pages 205–219, 2005.
  * [10] Manuel Casal-Guisande, José-Benito Bouza-Rodríguez, Jorge Cerqueiro-Pequeño, and Alberto Comesaña-Campos.  Design and conceptual development of a novel hybrid intelligent decision support system applied towards the prevention and early detection of forest fires.  Forests, 14(2):172, 2023.
  * [11] Balakrishnan Chandrasekaran, John R Josephson, and V Richard Benjamins.  What are ontologies, and why do we need them?  IEEE Intelligent Systems and their applications, 14(1):20–26, 1999\.
  * [12] Michel Charest and Sylvain Delisle.  Ontology-guided intelligent data mining assistance: Combining declarative and procedural knowledge.  In Artificial Intelligence and Soft Computing, volume 2006, pages 9–14, 2006.
  * [13] Lijia Chen, Pingping Chen, and Zhijian Lin.  Artificial intelligence in education: A review.  Ieee Access, 8:75264–75278, 2020.
  * [14] Cheng Cheng, Chunhong Zhang, Xiaofeng Qiu, and Yang Ji.  The social web of things (swot)-structuring an integrated social network for human, things and services.  J. Comput., 9(2):345–352, 2014.
  * [15] Valeriy Chernenkiy, Yuriy Gapanyuk, Valeriy Terekhov, Georgiy Revunkov, and Yuriy Kaganov.  The hybrid intelligent information system approach as the basis for cognitive architecture.  Procedia computer science, 145:143–152, 2018.
  * [16] Alessandra De Paola, Pierluca Ferraro, Giuseppe Lo Re, Marco Morana, and Marco Ortolani.  A fog-based hybrid intelligent system for energy saving in smart buildings.  Journal of Ambient Intelligence and Humanized Computing, 11:2793–2807, 2020.
  * [17] Dominik Dellermann, Philipp Ebel, Matthias Söllner, and Jan Marco Leimeister.  Hybrid intelligence.  Business & Information Systems Engineering, 61(5):637–643, 2019\.
  * [18] Dominik Dellermann, Nikolaus Lipusch, Philipp Ebel, and Jan Marco Leimeister.  Design principles for a hybrid intelligence decision support system for business model validation.  Electronic markets, 29:423–441, 2019.
  * [19] Diego Dermeval, Jéssyka Vilela, Ig Ibert Bittencourt, Jaelson Castro, Seiji Isotani, Patrick Brito, and Alan Silva.  Applications of ontologies in requirements engineering: a systematic review of the literature.  Requirements engineering, 21:405–437, 2016.
  * [20] Asem Eltaher, Thomas Form, Mohamed Ayeb, and Markus Maurer.  A generic architecture for hybrid intelligent test systems.  In 2008 7th IEEE International Conference on Cybernetic Intelligent Systems, pages 1–8. IEEE, 2008.
  * [21] Hans-Georg Fill and Felix Härer.  Supporting trust in hybrid intelligence systems using blockchains.  In AAAI Spring Symposium: Combining Machine Learning with Knowledge Engineering (1), 2020.
  * [22] Iván García-Magariño and Jorge J Gómez-Sanz.  An ontological and agent-oriented modeling approach for the specification of intelligent ambient assisted living systems for parkinson patients.  In Hybrid Artificial Intelligent Systems: 8th International Conference, HAIS 2013, Salamanca, Spain, September 11-13, 2013. Proceedings 8, pages 11–20. Springer, 2013.
  * [23] José Luis Garrido, María Visitación Hurtado, Manuel Noguera, and José Manuel Zurita.  Using a cbr approach based on ontologies for recommendation and reuse of knowledge sharing in decision making.  In 2008 Eighth International Conference on Hybrid Intelligent Systems, pages 837–842. IEEE, 2008.
  * [24] Alexandre Gouveia, Paulo Maio, Nuno Silva, and Rui Lopes.  Dss-based ontology alignment in solid reference system configuration.  In Hybrid Intelligent Systems: 18th International Conference on Hybrid Intelligent Systems (HIS 2018) Held in Porto, Portugal, December 13-15, 2018 18, pages 530–539. Springer, 2020.
  * [25] Roland Graef, Mathias Klier, Kilian Kluge, and Jan Felix Zolitschka.  Human-machine collaboration in online customer service–a long-term feedback-based approach.  Electronic Markets, 31:319–341, 2021.
  * [26] Nicola Guarino.  Formal ontology, conceptual analysis and knowledge representation.  International journal of human-computer studies, 43(5-6):625–640, 1995.
  * [27] MB Hadjiski and VG Boishina.  Integration of knowledge components in hybrid intelligent control systems.  Innovative Issues in Intelligent Systems, pages 57–109, 2016.
  * [28] Mincho Hadjiski, Vassil Sgurev, and Venelina Boishina.  Hvac control via hybrid intelligent systems.  Cybernetics and Information Technologies, 7(1):71–94, 2007.
  * [29] Pavel Hamet and Johanne Tremblay.  Artificial intelligence in medicine.  Metabolism, 69:S36–S40, 2017.
  * [30] María Dolores Hernández, MC Romero-Ternero, Francisco Sivianes, Alejandro Carrasco, and Jorge Ropero.  A hybrid intelligent multiagent system for the remote control of solar farms.  Applied Artificial Intelligence, 33(2):124–136, 2019.
  * [31] Javier Hingant, Marcelo Zambrano, Francisco J Pérez, Israel Pérez, and Manuel Esteve.  Hybint: a hybrid intelligence system for critical infrastructures protection.  Security and Communication Networks, 2018, 2018.
  * [32] Loan Ho, Victor de Boer, M Birna van Riemsdijk, Stefan Schlobach, and Myrthe Tielman.  Argumentation for knowledge base inconsistencies in hybrid intelligence scenarios.  In KR4HI First International Workshop on Knowledge Representation for Hybrid intelligence, 2022.
  * [33] Aidan Hogan, Eva Blomqvist, Michael Cochez, Claudia d’Amato, Gerard de Melo, Claudio Gutierrez, Sabrina Kirrane, José Emilio Labra Gayo, Roberto Navigli, Sebastian Neumaier, et al.  Knowledge graphs.  ACM Computing Surveys (CSUR), 54(4):1–37, 2021.
  * [34] Chung Hee Hwang, Noel Massey, Bradford W Miller, and Kari Torkkola.  Hybrid intelligence for driver assistance.  In FLAIRS Conference, pages 281–285, 2003.
  * [35] Gustavo Isaza, Andrés Castillo, Manuel López, and Luis Castillo.  Towards ontology-based intelligent model for intrusion detection and prevention.  In Computational Intelligence in Security for Information Systems: CISIS’09, 2nd International Workshop Burgos, Spain, September 2009 Proceedings, pages 109–116. Springer, 2009.
  * [36] Won Kim, Ok-Ran Jeong, and Sang-Won Lee.  On social web sites.  Information systems, 35(2):215–236, 2010.
  * [37] IA Kirikov, SV Listopad, and AS Luchko.  Agents’ ontologies negotiation in cohesive hybrid intelligent multi-agent systems.  In Journal of Physics: Conference Series, volume 2094, page 032033\. IOP Publishing, 2021.
  * [38] Gary Klein.  The power of intuition: How to use your gut feelings to make better decisions at work.  Currency, 2004.
  * [39] Gary A Klein.  Sources of power: How people make decisions.  MIT press, 2017.
  * [40] Hans-Ulrich Krieger, Bernd Kiefer, and Thierry Declerck.  A hybrid reasoning architecture for business intelligence applications.  In 2008 Eighth International Conference on Hybrid Intelligent Systems, pages 843–848. IEEE, 2008.
  * [41] Kirill Krinkin and Yulia Shichkina.  Cognitive architecture for co-evolutionary hybrid intelligence.  In Artificial General Intelligence: 15th International Conference, AGI 2022, Seattle, WA, USA, August 19–22, 2022, Proceedings, pages 293–303. Springer, 2023.
  * [42] Kirill Krinkin, Yulia Shichkina, and Andrey Ignatyev.  Co-evolutionary hybrid intelligence is a key concept for the world intellectualization.  Kybernetes, (ahead-of-print), 2022.
  * [43] Chiyun Lee, Junxia Lin, Andrzej Prokop, Vancheswaran Gopalakrishnan, Richard N Hanna, Eliseo Papa, Adrian Freeman, Saleha Patel, Wen Yu, Monika Huhn, et al.  Stargazer: A hybrid intelligence platform for drug target prioritization and digital drug repositioning using streamlit.  Frontiers in Genetics, 13, 2022.
  * [44] Jan Marco Leimeister.  Collective intelligence.  Business & Information Systems Engineering, 2:245–248, 2010.
  * [45] Ariel Levy, Monica Agrawal, Arvind Satyanarayan, and David Sontag.  Assessing the impact of automated suggestions on decision making: Domain experts mediate model errors but take less initiative.  In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–13, 2021.
  * [46] Bo-hu Li, Bao-cun Hou, Wen-tao Yu, Xiao-bing Lu, and Chun-wei Yang.  Applications of artificial intelligence in intelligent manufacturing: a review.  Frontiers of Information Technology & Electronic Engineering, 18:86–96, 2017.
  * [47] Chunsheng Li, Yatian Gao, and Kan Li.  An agent-based hybrid intelligent system for financial investment planning.  In 2010 International Conference on E-Product E-Service and E-Entertainment, pages 1–4. IEEE, 2010.
  * [48] S Listopad.  Estimating of the similarity of agents’ goals in cohesive hybrid intelligent multi-agent system.  In Proceedings of the 8th International conference Fuzzy Systems, Soft Computing and Intelligent Technologies (FSSCIT-2020), pages 180–185, 2020.
  * [49] Sergey Listopad.  Cohesive hybrid intelligent multi-agent system architecture.  In 2020 26th Conference of Open Innovations Association (FRUCT), pages 262–269. IEEE, 2020.
  * [50] Sergey Listopad.  Modeling team cohesion using hybrid intelligent multi-agent systems.  In 2020 2nd International Conference on Control Systems, Mathematical Modeling, Automation and Energy Efficiency (SUMMA), pages 416–421. IEEE, 2020.
  * [51] SV Listopad and IA Kirikov.  Similarity measure of agents’ ontologies in cohesive hybrid intelligent multi-agent system.  In Journal of Physics: Conference Series, volume 1679, page 032061\. IOP Publishing, 2020.
  * [52] Yang Lu.  Artificial intelligence: a survey on evolution, models, applications and future trends.  Journal of Management Analytics, 6(1):1–29, 2019.
  * [53] Jianhua Ma, Jie Wen, Runhe Huang, and Benxiong Huang.  Cyber-individual meets brain informatics.  IEEE Intelligent Systems, 26(5):30–37, 2011.
  * [54] Computing Machinery.  Computing machinery and intelligence-am turing.  Mind, 59(236):433, 1950.
  * [55] Antonio Martín-Montes, Mauricio Burbano, and Carlos León.  Efficient services in the industry 4.0 and intelligent management network.  In 2017 IEEE 26th International Symposium on Industrial Electronics (ISIE), pages 1495–1500. IEEE, 2017.
  * [56] John D Mayer, Richard D Roberts, and Sigal G Barsade.  Human abilities: Emotional intelligence.  Annu. Rev. Psychol., 59:507–536, 2008.
  * [57] John McCarthy.  What is artificial intelligence.  2007\.
  * [58] James R Michaelis, Deborah L McGuinness, Cynthia Chang, John Erickson, Daniel Hunter, and Olga Babko-Malaya.  Explaining scientific and technical emergence forecasting.  Applications of Social Media and Social Network Analysis, pages 177–192, 2015.
  * [59] Mohammad Moradi, Morteza Moradi, and Farhad Bayat.  In the search of web of intelligence.  In 2019 5th International Conference on Web Research (ICWR), pages 215–220. IEEE, 2019.
  * [60] Vincent C Müller and Nick Bostrom.  Future progress in artificial intelligence: A survey of expert opinion.  Fundamental issues of artificial intelligence, pages 555–572, 2016\.
  * [61] Peter Murray-Rust.  Open data in science.  Nature Precedings, pages 1–1, 2008.
  * [62] Leo Obrst, Werner Ceusters, Inderjeet Mani, Steve Ray, and Barry Smith.  The evaluation of ontologies: Toward improved semantic interoperability.  Semantic web: Revolutionizing knowledge discovery in the life sciences, pages 139–158, 2007.
  * [63] Sharon Oviatt.  Technology as infrastructure for dehumanization: three hundred million people with the same face.  In Proceedings of the 2021 International Conference on Multimodal Interaction, pages 278–287, 2021.
  * [64] Tim O’reilly.  What is web 2.0, 2005.
  * [65] AV Palagin, AF Kurgaev, and AI Shevchenko.  The noosphere paradigm of the development of science and artificial intelligence.  Cybernetics and Systems Analysis, 53:503–511, 2017.
  * [66] S Palvannan and Gerard Deepak.  Hias: Hybrid intelligence approach for soil classification and recommendation of crops.  In Electronic Governance with Emerging Technologies: First International Conference, EGETC 2022, Tampico, Mexico, September 12–14, 2022, Revised Selected Papers, pages 81–94. Springer, 2023.
  * [67] Yunhe Pan.  Heading toward artificial intelligence 2.0.  Engineering, 2(4):409–413, 2016.
  * [68] Tadeusz Pankowski.  Combining owl ontology and schema annotations in metadata management.  In Hybrid Artificial Intelligent Systems: 6th International Conference, HAIS 2011, Wroclaw, Poland, May 23-25, 2011, Proceedings, Part I 6, pages 255–262. Springer, 2011.
  * [69] Salvatore F Pileggi, Antonio A Lopez-Lorca, and Ghassan Beydoun.  Ontology in software engineering.  In ACIS 2018-29th Australasian Conference on Information Systems, 2018.
  * [70] Salvatore Flavio Pileggi.  Getting formal ontologies closer to final users through knowledge graph visualization: Interpretation and misinterpretation.  In Computational Science–ICCS 2022: 22nd International Conference, London, UK, June 21–23, 2022, Proceedings, Part IV, pages 611–622. Springer, 2022.
  * [71] Benedikt Reitemeyer.  Automatic generation of conceptual enterprise models.  In 2020 IEEE 24th International Enterprise Distributed Object Computing Workshop (EDOCW), pages 74–79. IEEE, 2020.
  * [72] Víctor Rodríguez-Doncel, Monica Palmirani, Michał Araszkiewicz, Pompeu Casanovas, Ugo Pagallo, and Giovanni Sartor.  Introduction: a hybrid regulatory framework and technical architecture for a human-centered and explainable ai.  In AI Approaches to the Complexity of Legal Systems XI-XII: AICOL International Workshops 2018 and 2020: AICOL-XI@ JURIX 2018, AICOL-XII@ JURIX 2020, XAILA@ JURIX 2020, Revised Selected Papers XII, pages 1–11. Springer, 2021.
  * [73] Sophiya Rumovskaya.  Visualization of team cohesion in hybrid intelligent multi-agent systems.  In 2020 2nd International Conference on Control Systems, Mathematical Modeling, Automation and Energy Efficiency (SUMMA), pages 620–623. IEEE, 2020.
  * [74] Sohail Sarwar, Saad Zia, Zia ul Qayyum, Muddesar Iqbal, Muhammad Safyan, Shahid Mumtaz, Raúl García-Castro, and Konstantin Kostromitin.  Context aware ontology-based hybrid intelligent framework for vehicle driver categorization.  Transactions on Emerging Telecommunications Technologies, 33(8):e3729, 2022.
  * [75] Yulia Shichkina and Kirill Krinkin.  Principles of building personalized intelligent human assistants.  In 2022 Fourth International Conference Neurotechnologies and Neurointerfaces (CNN), pages 148–151. IEEE, 2022.
  * [76] Daniil Shunkevich.  Ontology-based design of hybrid problem solvers.  In Open Semantic Technologies for Intelligent Systems: 11th International Conference, OSTIS 2021, Minsk, Belarus, September 16–18, 2021, Revised Selected Papers, pages 101–131. Springer, 2022.
  * [77] Maria O Taran, Georgiy I Revunkov, and Yuriy E Gapanyuk.  The text fragment extraction module of the hybrid intelligent information system for analysis of judicial practice of arbitration courts.  In Advances in Neural Computation, Machine Learning, and Cognitive Research IV: Selected Papers from the XXII International Conference on Neuroinformatics, October 12-16, 2020, Moscow, Russia, pages 242–248. Springer, 2021.
  * [78] H Holden Thorp.  Chatgpt is fun, but not an author, 2023.
  * [79] Ilaria Tiddi and Stefan Schlobach.  Knowledge graphs as tools for explainable machine learning: A survey.  Artificial Intelligence, 302:103627, 2022.
  * [80] Abolfazl Doostparast Torshizi, Mohammad Hossein Fazel Zarandi, Ghazaleh Doostparast Torshizi, and Kamyar Eghbali.  A hybrid fuzzy-ontology based intelligent system to determine level of severity and treatment recommendation for benign prostatic hyperplasia.  Computer methods and programs in biomedicine, 113(1):301–313, 2014\.
  * [81] Raju Vaishya, Mohd Javaid, Ibrahim Haleem Khan, and Abid Haleem.  Artificial intelligence (ai) applications for covid-19 pandemic.  Diabetes & Metabolic Syndrome: Clinical Research & Reviews, 14(4):337–339, 2020.
  * [82] Michael van Bekkum, Maaike de Boer, Frank van Harmelen, André Meyer-Vitali, and Annette ten Teije.  Modular design patterns for hybrid learning and reasoning systems: a taxonomy, patterns and use cases.  Applied Intelligence, 51(9):6528–6546, 2021.
  * [83] Leonilde Varela, Goran Putnik, Vijaya Manupti, Ana Madureira, André Santos, Gabriela Amaral, and Luís Ferreirinha.  Ontology-based meta-model for hybrid collaborative scheduling.  In Hybrid Intelligent Systems: 18th International Conference on Hybrid Intelligent Systems (HIS 2018) Held in Porto, Portugal, December 13-15, 2018 18, pages 408–417. Springer, 2020.
  * [84] W Patrick Walters and Mark Murcko.  Assessing the impact of generative ai on medicinal chemistry.  Nature biotechnology, 38(2):143–145, 2020.
  * [85] Yutaka Watanobe and Nikolay Mirenkov.  Hybrid intelligence aspects of programming in* aida algorithmic pictures.  Future Generation Computer Systems, 37:417–428, 2014.
  * [86] Lan Yang, Kathryn Cormican, and Ming Yu.  Ontology-based systems engineering: A state-of-the-art review.  Computers in Industry, 111:148–171, 2019.
  * [87] Kun-Hsing Yu, Andrew L Beam, and Isaac S Kohane.  Artificial intelligence in healthcare.  Nature biomedical engineering, 2(10):719–731, 2018.
  * [88] Zili Zhang and Chengqi Zhang.  An agent-based hybrid intelligent system for financial investment planning.  In PRICAI 2002: Trends in Artificial Intelligence: 7th Pacific Rim International Conference on Artificial Intelligence Tokyo, Japan, August 18–22, 2002 Proceedings 7, pages 355–364. Springer, 2002.
  * [89] Nan-ning Zheng, Zi-yi Liu, Peng-ju Ren, Yong-qiang Ma, Shi-tao Chen, Si-yu Yu, Jian-ru Xue, Ba-dong Chen, and Fei-yue Wang.  Hybrid-augmented intelligence: collaboration and cognition.  Frontiers of Information Technology & Electronic Engineering, 18(2):153–179, 2017.
  * [90] Ning Zhong, Jiming Liu, Zili Zhang, and Chengqi Zhang.  Constructing hybrid intelligent systems for data mining from agent perspectives.  Intelligent Technologies for Information Analysis, pages 333–359, 2004.