2022

This version of the article has been accepted for publication, after peer
review but is not the Version of Record and does not reflect post-acceptance
improvements, or any corrections. The Version of Record is available online
at: https://doi.org/10.1007/s12652-022-03849-2.

These authors contributed equally to this work.

[1]Silvia García-Méndez These authors contributed equally to this work.

These authors contributed equally to this work.

These authors contributed equally to this work.

1]Information Technologies Group, atlanTTic, School of Telecommunications
Engineering, University of Vigo, Campus Lagoas-Marcosende, Vigo, 36310, Spain

# Automatic detection of cognitive impairment in elderly people using an
entertainment chatbot with Natural Language Processing capabilities

Francisco de Arriba-Pérez<EMAIL_ADDRESS><EMAIL_ADDRESS>Francisco
J. González-Castaño<EMAIL_ADDRESS>Enrique Costa-Montenegro
<EMAIL_ADDRESS>[

###### Abstract

Previous researchers have proposed intelligent systems for therapeutic
monitoring of cognitive impairments. However, most existing practical
approaches for this purpose are based on manual tests. This raises issues such
as excessive caretaking effort and the white-coat effect. To avoid these
issues, we present an intelligent conversational system for entertaining
elderly people with news of their interest that monitors cognitive impairment
transparently. Automatic chatbot dialogue stages allow assessing content
description skills and detecting cognitive impairment with Machine Learning
algorithms. We create these dialogue flows automatically from updated news
items using Natural Language Generation techniques. The system also infers the
gold standard of the answers to the questions, so it can assess cognitive
capabilities automatically by comparing these answers with the user responses.
It employs a similarity metric with values in [0, 1], in increasing level of
similarity. To evaluate the performance and usability of our approach, we have
conducted field tests with a test group of 30 elderly people in the earliest
stages of dementia, under the supervision of gerontologists. In the
experiments, we have analysed the effect of stress and concentration in these
users. Those without cognitive impairment performed up to five times better.
In particular, the similarity metric varied between $0.03$, for stressed and
unfocused participants, and $0.36$, for relaxed and focused users. Finally, we
developed a Machine Learning algorithm based on textual analysis features for
automatic cognitive impairment detection, which attained accuracy, F-measure
and recall levels above 80%. We have thus validated the automatic approach to
detect cognitive impairment in elderly people based on entertainment content.
The results suggest that the solution has strong potential for long-term user-
friendly therapeutic monitoring of elderly people.

###### keywords:

Intelligent systems, Natural language processing, Monitoring, Elderly people,
Cognitive impairment

##  1 Introduction

The United Nations has reported (World Population Prospects report111Available
at https://population.un.org/wpp, September 2021.) that 9% of the world
population is over 65 years old, and this percentage will reach 16% in 30
years. The population older than 80 is growing even faster, and it is expected
to reach 450 million by 2050. This urges society to find innovative solutions
to improve the living conditions of our elders, especially of those who live
alone (Callahan et al. 2014; Hancock et al. 2006).

A main issue of the quality of life of elderly people is the severe prevalence
of cognitive impairment disorders. Affected people are mainly over 65 years
old and they were 50 million in 2015, although this segment is expected to
triple by 2050 (Livingston et al. 2017). Regular screening for detecting early
symptoms and monitoring the progression of these disorders have been
considered beneficial for treatment planning and patient autonomy (Borson et
al. 2013). However, it has also been observed that cognitive impairment
assessment in primary care systems is inefficient (Löppönen et al. 2003; Boise
et al. 2004).

As discussed in Section 2, even though existing telecare set-top-boxes and
gateways have increasingly intelligent capabilities, they still do not
communicate autonomously with the elderly using natural language. This is also
the case of the solutions for cognitive evaluation, which mostly rely on sets
of predefined manual tests.

Given the industrial gap, and as demonstrated by the analysis of the state of
the art in Section 2, we propose a novel conversational system for
entertainment and therapeutic monitoring of elderly people that relies on nlp
techniques and Machine Learning for empathetic chatbot behaviour generation
and user-transparent automatic assessment.

From the perspective of the target users, the elderly, the main priority of
any information system should be alleviating loneliness, whether the system
has embedded cognitive monitoring capabilities or not. Accordingly, we want
our solution to be perceived as a friendly intelligent assistant to access
Internet media, that is, a conversational system that reads news. These will
be interspersed with brief dialogues to subtly guide the users through a
series of questions to gather their interests and evaluate their understanding
of the information they have just consumed, which includes word category
understanding and short-term memory, to evaluate cognitive impairment
(Loewenstein et al. 2004; Crocco et al. 2014).

Conversational systems seem an adequate approach for this purpose. These
software programs allow for human interaction with a machine using written or
spoken natural language (Shawar and Atwell 2007). Ideally, the dialogue should
be empathetic (Fung et al. 2018; Rashkin et al. 2019), a still distant goal
even after the recent advances in Artificial Intelligence (ai) and in Natural
Language Generation (nlg) techniques. Before virtual companions (Shum et al.
2018) become a reality, entertainment through interesting information will be
more feasible. In this vein, it is well known that elders feel accompanied by
simply listening to news in their background (Östlund 2010).

More in detail, our system reads recent news items and generates questions
about them. At the same time, in order to automatically evaluate cognitive
capabilities, the system measures the similarity between user answers and a
gold standard (Yang and Powers 2005; Corley and Mihalcea 2005; Li et al. 2006;
Feng et al. 2008) that is automatically generated from the news. To validate
our approach, we have defined an answer similarity metric and we have
performed tests on a sample of 30 patients of Asociación de Familiares de
enfermos de Alzheimer y otras demencias de Galicia (afaga222Available at
https://afaga.com, September 2021., the Galician Association of Relatives of
Patients with Alzheimer’s and other Dementias). This public association seeks
to improve the quality of life of Alzheimer patients, provides guidance and
information to relatives and to the public, and makes society aware of this
reality to achieve a broader and more effective response. It collaborates
actively in research on cognitive impairments.

The rest of this paper is organised as follows. Section 2 reviews related work
and lists our contributions. Section 3 describes our conversational system for
entertainment and user-transparent cognitive assessment. Section 4 presents
our case study and the results of our word similarity approach to
automatically evaluate cognitive impairment. Finally, Section 5 concludes the
paper.

##  2 Related Work

The design of intelligent systems (Wilamowski and Irwin 2015) is a relevant
research field, including for example industrial (Irani and Kamal 2014; Ngai
et al. 2014), military (Ma’sum et al. 2013; Yoo et al. 2014) and social
(Magnisalis et al. 2011; Bernardini et al. 2014; Adamson et al. 2019)
applications. In the telecare domain, personal assistants (Matsuyama et al.
2016; López et al. 2018) and artificial companionship (Chumkamon et al. 2016;
Abdollahi et al. 2017) are relevant.

Regarding the automatic detection of health conditions, there exists a wealth
of work based on Machine Learning, such as Ghoneim et al. (2018), on a smart
healthcare framework to detect medical data tampering; Sedik et al. (2021), on
the analysis of the outbreak of covid-19 disease; Ahmed et al. (2021), on an
unsupervised Machine Learning approach to predict data-types attributes for
optimal processing of telemedicine data, including text and images; and Sarrab
and Alshohoumi (2021), on the real-time detection of abnormalities in streamed
data from IoT sensors (furthermore, in Masud et al. (2021), a mutual
authentication and secret key establishment protocol was proposed to protect
medical IoT networks). Our research contributes to automatic smart telecare
solutions based on Machine Learning.

However, despite the huge advances in ai for artificial reasoning and problem-
solving, interactions are still far from human-like. Most recent ai systems
have partial understanding of natural language and lack cognitive capabilities
to enrich the communication with context-dependent information (Skjuve et al.
2019). Specifically, existing solutions for intelligent conversational systems
are based on retrieval-based methods (Yasuda et al. 2014; Wu et al. 2018),
which select the best candidate among a predefined set of alternative
responses, and generation-based procedures (Oh et al. 2017; Su et al. 2017;
Baby et al. 2017), which rely on nlp techniques to create human-like written
or oral dialogue flows. Note that natural language is especially beneficial
for user interfaces for its spontaneity and friendliness (Liu et al. 2018).

Among the existing intelligent conversational systems, we can mention Google
Duplex (Lindgren and Andersson 2011) and the Neural Responding Machine in
Shang et al. (2015) based on Recurrent Neural Networks. Moreover, in Wen et
al. (2017) the authors presented a dialogue system based on the pipelined
Wizard-of-Oz framework, which, unlike other approaches in the literature, can
make assumptions. Regarding linguistic knowledge, in Wang et al. (2015) and Wu
et al. (2018) syntactic features were used to generate coherent and human-like
texts. Newscasters (Matsumoto et al. 2007), which, as previously said,
generate a feeling of companionship (Östlund 2010), do not sustain dialogues
with end users. Conversational systems have already been considered for
entertainment and healthcare (Noh et al. 2017; Su et al. 2017), although still
at an early stage.

Regarding existing conversational systems for entertainment (Johnson et al.
2016; Correia et al. 2016; Aaltonen et al. 2017), we highlight EduRobot
(Budiharto et al. 2017) which can sing and tell stories, although it is not
oriented to the specific needs of the elderly. It has been suggested to make
conversational systems more appealing by modelling their interfaces as pets or
avatars (Sharkey and Sharkey 2012). Unlike EduRobot, RobAlz (Salichs et al.
2016) has been specifically devised for this audience, but it has no
therapeutic diagnostic capabilities. Few existing intelligent systems for
senior healthcare (Foroughi et al. 2008; Hsu and Chien 2009; Suryadevara et
al. 2012; Tseng et al. 2013; Samanta et al. 2014; Wang et al. 2016) have
human-computer communication capabilities. The system by Yasuda et al. (2014)
for people with dementia is an exception, but its communication capabilities
are very limited: it just selects questions and answers among 120 pre-set
options. Therefore, even though there still is incipient work on digital tools
for therapeutic monitoring of people with dementia and other impairments,
manual tests (written and task-based neurological and neuropsychological
assessments with caretaker supervision) are typically used. For example, the
Mini-Mental State Examination (Ridha and Rossor 2005) is a cognitive test on
orientation, immediate memory, calculation, attention and comprehension, to
cite some tasks, which produces scores about dementia levels. The Mini
Cognitive Assessment Instrument (Milne et al. 2008) includes a verbal memory
task and a clock drawing test. Finally, the camdex test (Ball et al. 2004) is
another standardised manual tool for the diagnosis of mental disorders, which
is especially suitable for the early detection of dementia. It asks questions
related to memory, personality, general mental and intellectual functioning,
and judgement. It also considers specific symptoms and the medical histories
of the users and their families. Note that the white-coat effect is a major
concern in all these manual approaches, apart from the fact that they are
time-consuming and require professional expertise.

Previous research has proved the benefits of combining dichotomous questions
(also named closed questions) with essay questions333Dichotomous or closed
questions have binary yes/no answers. Essay questions allow users to express
themselves freely. to mitigate the white-coat effect (Ridha and Rossor 2005;
Echeburúa et al. 2017). We remark the interest of inserting distracting
questions before attention-demanding questions in cognitive tests (Ridha and
Rossor 2005). Altogether, this combination allows evaluating cognitive
impairment less intrusively (Ball et al. 2004; Ridha and Rossor 2005; Milne et
al. 2008). To the best of our knowledge, our proposal is the first intelligent
system that embeds user-transparent, automatic cognitive assessment into a
newscaster system that sustains dialogues with elderly people, based on nlp
techniques for chatbot behaviour generation and human-machine communication.

We close this section with a review of related industrial initiatives, which
further backs the social relevance of the field under study.

For instance, the Carelife system by Televés444Available at
http://televescorporation.com/areas-de- negocio/sociosanitario, September
2021. analyses personal routines from sensor data for custom home care. Its
home gateway can be extended via peripherals, such as biomedical devices, as
well as via software. Doro launched SmartCare555Available at
https://www.doro.com/es-es/care, September 2021. in 2018. It includes a home
gateway and home sensors to detect behavioural changes. However, neither these
systems nor the sam Robotic Concierge by Luvozo666Available at
https://luvozo.com, September 2021. have built-in intelligence to communicate
with the elderly using natural language.

Regarding general purpose domestic robots, there are examples such as ZenBo by
Asus777Available at https://zenbo.asus.com, September 2021., with video
surveillance, Internet shopping and agenda features. Its interactions are
rather rigid. It can understand vocal orders, but it has no empathetic
capabilities, nor is it tailored to the needs of the elderly.

We believe that a feasible path towards a next generation of intelligent
conversational telecare systems is the augmentation through software of
current platforms such as Carelife and Smartcare by relying on their simple
voice interfaces (which, nowadays, caregivers employ to call the users),
without any additional hardware add-ons. Some platforms are already open in
this aspect. For example, Buddy, by European Blue Frog Robotics888Available at
https://buddytherobot.com/en/buddy-the- emotional-robot, September 2021.,
allows third developers to create new applications and distribute them via its
store. We are neither aware of any application of this robot to entertain and
monitor elderly people, nor of any intelligent functionalities.

Regarding solutions focusing on cognitive evaluation, we must mention three
approaches based on manual tests (written and task-oriented neurological and
neuropsychological assessments), none of which employ automatic nlp
techniques. Neurotrack999Available at https://neurotrack.com, September 2021.
is a set of cognitive tests to evaluate, monitor and strengthen brain health
to reduce the risk of Alzheimer and other dementias. Mezurio101010Available at
https://mezur.io, September 2021. provides support to interactive data
acquisition as a baseline for detecting individuals at risk of developing
Alzheimer’s disease. Altoida111111Available at http://altoida.com, September
2021. tests the functional and cognitive skills of patients with a Machine
Learning algorithm but, as the previous two solutions, it is entirely based on
a set of predefined tests and it does not have any bidirectional communication
capabilities in natural language.

##  3 System Architecture

We present a novel intelligent system specially designed for therapeutic
monitoring of elderly people with different levels of cognitive impairments or
on the verge of suffering them. The users perceive the system as a news
broadcasting service, with which they interact by voice from time to time.
Therapeutic monitoring is embedded as a user-transparent functionality. Figure
1 illustrates the main modules of the system, on which we will elaborate in
the next sections. They include online (news broadcast service and intelligent
dialogue generation system) and local services (Android121212Available at
https://www.android.com, November 2021. application with cognitive attention
assessment service). Online services modules were implemented using the
Eclipse131313Available at https://www.eclipse.org, November 2021. Integrated
Development Environment (ide) tool and Java 1.8 programming
language141414Available at https://www.java.com, November 2021.. They were
deployed on a Tomcat151515Available at http://tomcat.apache.org, November
2021. server to be made available through a rest api, which was programmed
using the Jersey library161616Available at https://eclipse-
ee4j.github.io/jersey, November 2021.. The Android application (for Lollipop
operating system or higher regarding devices compatibility) was developed with
Android Studio171717Available at https://developer.android.com/studio,
November 2021..

The system transforms Spanish speech into text and vice versa as input and
output data (stt/speech-to-text and tts/text-to-speech boxes in Figure 1). For
this purpose, it employs the Google Voice Android Software Development Kit
(sdk) library181818Available at https://developer.android.com/reference/
android/speech/Speech Recognizer, September 2021..

Regarding system activation, we use voice commands and facial recognition. For
implementing the latter we employed the OpenCV library191919Available at
https://opencv.org, September 2021. and an eye-sensing train data
set202020Available at https://github.com/opencv/opencv/blob/
master/data/lbpcascades/lbpcascade_frontalface.xml, September 2021. (note that
previous works have also exploited sophisticated schemes for this purpose,
apart from voice commands (Alsmirat et al. 2019)).

As in Wang et al. (2020), we decided to combine text and images, in our case
to help the users focus during the dialogue stages. Specifically, the dialogue
with the users is guided with basic graphic indicators (see Figure 2): the
screen displays a “muted” or “open” microphone to indicate the system’s or
user’s turn to speak. Moreover, the “facial” expressions of the avatar of the
conversational system provide empathetic feedback to the users based on text
sentiment analysis. Finally, note that the user interface is an animated dog,
simulating a pet as suggested in literature (Sharkey and Sharkey 2012).

Figure 1:  System architecture. Figure 2:  User interface.

To ensure short response times, instead of querying external systems, the
system relies on a MongoDB database212121Available at https://www.mongodb.com,
September 2021. in our own server containing all the necessary linguistic
knowledge.

###  3.1 News Broadcast Service

The news broadcast service requires a varied and updated set of news to engage
the target users. This news is periodically extracted from the Application
Programming Interface (api) of the Spanish National Radio and Television
(RTVE222222Available at https://www.rtve.es/api, September 2021.) channel with
a get query, using the tematicas and noticias api services, to gather news
items on specific topics. This task is performed in the background owing to
the requirements of the api, and it hides news processing to the target users
by generating pre-saved news items on a daily basis. The content retrieved
from the api is saved into a MongoDB structure using a json file. Date and
topic features are exploited for indexing and searching. As a result, the News
Broadcast Service delivers content immediately from the user’s point of view.

News are arranged into five categories: economy, politics, science, society
and sports, all of them at national level except for politics and society,
which focus on Galicia (Spain) in our case, since proximity is appealing to
elderly people.

Table 1 presents an example of a social news item. We provide the user with a
summary of each piece of news by extracting the lead paragraph (for that
purpose, we first split the news items into paragraphs and take the first
paragraph after the title).

Table 1:  Example of news content. Topic | News item
---|---
Society |  |  La Policía Nacional ha desarticulado una peligrosa red de narcotraficantes que operaba en Galicia, Madrid y Alicante en una operación en la que han sido detenidas diez personas con numerosos antecedentes por delitos graves y se han intervenido 1.000 kilos de cocaína, 500.000 euros y diez vehículos de lujo.
---
‘National Police has dismantled a dangerous drug trafficking ring that
operated in Galicia, Madrid, and Alicante in an operation in which ten people
with a wide criminal record have been arrested, and 1,000 kilos of cocaine,
500,000 euros and ten luxury cars have been seized.’

###  3.2 Automatic Question Generation

Our question generation system combines linguistic knowledge from our aLexiS
lexicon (García-Méndez et al. 2018, 2019) with the Name Entity Classification
(nec) functionality from Freeling (Atserias et al. 2006; Padró and Stanilovsky
2012). The former is saved in a MongoDB database to reduce the response time
of the chatbot. Besides, the nec process is executed in the background because
of the complexity of the linguistic analysis of the news.

These two resources allow extracting and identifying personal names,
organisations and locations, which together constitute valuable data for
question generation. More specifically, thanks to the linguistic information
in aLexiS, our conversational system can adjust features such as the gender,
number, person, and tense of the questions it generates.

As previously said, we follow the strategy of combining dichotomous and essay
questions to reduce the white-coat effect and create a more relaxed
atmosphere.

  * •

To generate dichotomous questions, we rely on the nec functionality of
Freeling to extract personal names and locations from the news. This produces
questions such as those in Table 2. The system always generates four similar
options for each question, and one of them is picked at random and presented
to the user. Then, depending on the user’s answer, the system poses the next
question as indicated in Table 3.

Table 2:  Example of dichotomous questions by different nec results. NEC | Example of dichotomous questions
---|---
People | ¿Has oído hablar de ENTIDAD?
‘Have you ever heard about ENTITY?’
Location | ¿Has estado alguna vez en ENTIDAD?
‘Have you ever been to ENTITY?’
Table 3:  Example of dichotomous questions by different nec results depending on the user’s response. Answer | NEC | Example of dichotomous questions
---|---|---
Yes | People | ¿Qué datos conoces de la vida de ENTIDAD?
‘What facts do you know about ENTITY’s life?’
Location | Cuéntame qué es lo que más te ha gustado de ENTIDAD
‘Tell me what you liked the most about ENTITY’
No | People | ¿Por qué ha saltado a los medios de comunicación ENTIDAD?
‘Why has ENTITY jumped into the media?’
Location | ¿Por qué no has estado en ENTIDAD?
‘Why have you never been to ENTITY?’
n/a | People | ¿Qué visión de ENTIDAD nos dan los medios de comunicación?
‘Which is the view of ENTITY in the media?’
Location | ¿Podrías contarme algo relacionado con ENTIDAD?
‘Could you tell me anything about ENTITY?’

Regarding the extraction of the gold standard answers for the aforementioned
types of questions, we obtain these data with Freeling syntactic parsing. Take
the sentence ‘National Police has dismantled a dangerous drug trafficking ring
that operated in Galicia, Madrid, and Alicante’ as an example. The
corresponding essay question using our system is ‘Who has dismantled a
dangerous drug trafficking ring?’, and the correct answer it produces as a
reference is the noun phrase that precedes the verb, ‘National Police’. Note
that the best answer is obtained by extracting the noun phrases that precede
the verb for ‘who’ questions, as in our example. On the other hand, for ‘what’
questions, we use the noun phrase that precedes the verb plus the verb itself.
Take the sentence ‘The Government will automatically extend the social
electric bonds until September 15th’ and its associated question ‘What does
the news say on September 15th will happen?’ as an example of ‘what’ question
handling. The correct produced answer is ‘It will automatically extend social
electric bonds’. Finally, for the question ‘Which places does the news item
mention?’, the correct answer is produced by extracting all location entities
from the news content using the nec functionality by Freeling. Note that in
the first example about the drug trafficking ring, the correct answer is
Galicia, Madrid and Alicante.

Both the generation of questions and the extraction of gold standard answers
are performed in the background, after the daily news-gathering process (see
Section 3.1), using the same indexing scheme.

By combining dichotomous and essay questions, the conversational system
establishes a dialogue with the end users. Each dialogue is composed of the
following three stages:

  * •

News: prior to the dialogue, the conversational system presents the news item.

  * •

User-centred questions: a dichotomous question followed by two essay questions
to distract the user.

  * •

Attention-demanding question: a last essay question on key aspects of the news
item. It allows assessing if the user understood the news piece, if he/she was
focused during the conversation, and his/her short-term memory.

Figure 3 shows an example of a real dialogue according to this structure. To
keep the user engaged, most questions are related to the news item (avatar
marked in yellow).

The resulting user’s utterances are the input to the cognitive attention
assessment service, which calculates the accuracy of the user answers by
comparing them with the gold standard responses. We describe it in the next
section.

Figure 3:  Real conversation example.

###  3.3 Cognitive Attention Assessment Service

We employ the lexical Multilingual Central Repository (mcr232323Available at
http://adimen.si.ehu.es/web/MCR, September 2021.) database (González-Agirre et
al. 2012), which integrates the Spanish WordNet into the EuroWordNet
framework, to obtain the semantic classification of the adjectives, adverbs,
nouns and verbs in the news piece. For that purpose, we extract from mcr three
semantic categories corresponding to Adimen sumo, WordNet Domains and Top
Ontology hierarchies for nouns and verbs, and Top Ontology hierarchies for
adjectives and adverbs, since there is less information available for these
two lexical categories (Pedersen et al. 2004). From mcr we also gather
holonyms, hypernyms, hyponyms, meronyms, synonyms and related data for nouns
and verbs, and only synonyms for adjectives and adverbs. Table 4 shows an
example for noun montaña ‘mountain’.

This linguistic knowledge was added to the aLexiS lexicon within the same json
indexing scheme, which did not affect response time performance.

Table 4:  Semantic data from mcr for noun montaña ‘mountain’. Feature | Value
---|---
Word |  montaña ‘mountain’
WordNet Domain | object
Adimen Sumo | LandArea
Top Ontology | geography, geology
Holonym | -
Hypernym |  elevación ‘elevation’
Hyponym |  Alpes ‘Alps’, volcán ‘volcano’
Meronym |  vertiente ‘slope’, cumbre ‘summit’
Synonym |  monte ‘hill’
Related |  montañero ‘mountaineer’, alpinista ‘alpinist’,
|  montañoso ‘mountainous’, montuoso ‘mountainous’

We obtain the final score for each response as a weighted average, as follows:

$\displaystyle
sim=0.8\frac{\sum_{i=1}^{N_{noun}}{noun^{*}_{i}}+\sum_{i=1}^{N_{verb}}{verb^{*}_{i}}}{N_{noun}+N_{verb}}$
$\displaystyle+0.2\frac{\sum_{i=1}^{N_{adj}}{adj^{*}_{i}}+\sum_{i=1}^{N_{adv}}{adv^{*}_{i}}}{N_{adj}+N_{adv}}$
(1)

Where:

  * •

$N_{x}$ represents the number of words of lexical category $x$ in the ideal
response.

  * •

For the $i$-th word of category $x$ in the ideal answer, we calculate its
similarity with all words within the same lexical category $x$ in the user’s
response, and we take the highest value as $x^{*}_{i}$.

Note that nouns and verbs are considered more relevant than adjectives and
adverbs in expression (3.3). This choice is supported by the fact that the
former generally carry most semantic information in a sentence, whereas the
latter provide nuances (Feng et al. 2008; Corley and Mihalcea 2005).

To obtain $x^{*}_{i}$ we adapted the method by Yang et al. (Yang and Powers
2005) to calculate the similarity $s(word_{1},word_{2})$ between words
$word_{1}$ and $word_{2}$:

$\displaystyle
s{(word_{1},word_{2})}=(1-\gamma)\alpha_{s}\beta^{d(word_{1},word_{2})}+\gamma$
(2)

Where:

  * •

$\alpha_{s}=0.9$ if the words are synonyms and $0.85$ otherwise.

  * •

$d(word_{1},word_{2})=0$ if the words are holonyms, hypernyms, hyponyms,
meronyms or synonyms, related to the same hierarchy category or belonging to
it. Otherwise, $d(word_{1},word_{2})$ is WordNet’s shortest path between
$word_{1}$ and $word_{2}$.

  * •

$\beta$ acts as a depth factor that decreases similarity exponentially, to the
power of the number of hierarchical steps separating the two concepts. In the
tests in Section 4 we set it to $0.7$.

  * •

$\gamma$ is explained below.

Note that certain fairly similar words in the same WordNet domain category
will have a very low similarity value with our method (less than $0.4$).
Consider for example the pair (madera ‘wood’, cartón ‘cardboard’), whose
similarity is $0.27$, a low value taking into account that both terms define
materials and share WordNet domain category ‘substance’; and pair (panadero
‘baker’, maestro ‘teacher’), with a similarity of $0.10$, although these words
represent professions and belong to the same WordNet domain category ‘person’.

To avoid this issue, we define correction factor $\gamma$ in expression (2).
By default $\gamma=0$, except for the following two cases:

  * •

For word pairs that belong to the same WordNet domain category, $\gamma=0.25$.
After applying this to the two previous examples, their similarities grow to
$0.45$ and $0.33$ (from $0.27$ and $0.10$), respectively.

  * •

For all terms with the same stem that have not been already classified as
synonyms, with a similarity of $0.85$ or less, $\gamma=0.5$. This is because
they belong to the same word family, and their similarity should be even
higher for our purposes than in the previous case. For instance, the pair
(flor ‘flower’, florista ‘florist’) would have a similarity value of $0.15$
for $\gamma=0$, but it becomes $0.58$ after applying $\gamma=0.5$.

The goal of these corrections is improving coherence, by defining a ground
truth for word similarity (Li et al. 2006). This was tuned by considering the
value range ($0.3$, $0.6$) for similarity scores in cases such as autógrafo
‘autograph’ versus firma ‘signature’ and cojín ‘cushion’ versus almohada
‘pillow’.

Moreover, we pay special attention to the treatment of numbers. Given the fact
that our goal is to assess the understanding of the news, we must take into
account that even people with healthy minds seldom retain the exact quantities
they have just heard. For example, after listening to the sentence ‘there were
2569 casualties’, a person will likely remember ‘there were over 2500
casualties’. For this reason, we generate all the possible numbers a given
quantity can be rounded to by dividing it by powers of ten, and we assign a
$0.7$ similarity score if any of them produces a match. For instance, if the
correct amount is 2569, the possible right answers that would be assigned a
$0.7$ similarity score are 2000, 2500, 2560, 2570, 2600 and 3000. On top of
that, if the words ‘over’ or ‘under’ are chosen correctly, a similarity score
of $0.9$ is assigned. Going back to the previous example, if the user’s reply
is ‘there were over 2500 casualties’, it receives a $0.9$ similarity score,
whereas if it is ‘there were 2500 casualties’ it gets a $0.7$ similarity
score, since it reflects less understanding of the original information.

As a final example, take the sentence un profesor llevó papel en blanco a su
hogar en la montaña ‘a teacher took blank paper to his home in the mountain’
as the ideal response to a question. The following three user answers produce
different results:

  * •

Un hombre llevó folios blancos a su casa del monte ‘a man took blank paper
sheets to his house on the hill’: this reply uses different words than the
original, but keeps the same meaning. It would obtain a similarity score of
$0.8$.

  * •

Un hombre sacó madera blanca de su apartamento ‘a man took white wood from his
apartment’: this sentence has lost most original meaning, although some common
concepts remain. It would obtain a similarity score of $0.51$.

  * •

Un hombre rompió una silla en una tienda ‘a man broke a chair in a shop’: this
sentence has none of the original meaning left. It would obtain a similarity
score of $0.25$ (note that the subject was correctly inferred).

The $sim$ metric allows the automatic evaluation of comprehension skills from
the questions and their corresponding gold standard answers, and the procedure
reflects the level of concentration during the conversation and the
reliability of short-term memory.

##  4 Experimental Results and Discussion

In this section, we present the validation tests to assess the effectiveness
of our approach to determine cognitive impairment levels.

###  4.1 Hardware

We ran the system on a server with the following characteristics:

  * •

Operating System: Ubuntu 18.04 LTS 64 bits

  * •

Processor: Intel Xeon CPU E5-2620 v2 2.1 GHz

  * •

RAM: 64GB DDR3

  * •

Disk: 3 Tb

###  4.2 Case Study

The case study comprises two experimental scenarios. The first experiment
(Section 4.2.1) studies the $sim$ metric presented in Section 3.3 as a tool to
assess abstraction capabilities, for different user profiles and levels of
cognitive impairment. The second experiment (Section 4.2.2) evaluates Machine
Learning algorithms to automatically detect cognitive impairment in the users
under study.

These experiments were divided in “sessions”, a session being a particular
newscast and its associated dialogue with an elderly person. The profiles of
the participants in these sessions were characterised as follows:

  * •

At least 60 years old.

  * •

Technological skills and hearing problems: existing or not.

  * •

Study levels: basic (high education or less) or superior (bachelor’s, master’s
and doctoral degrees).

  * •

Cognitive impairment level: absent, mild or severe, as established for the
case study by gerontology experts from afaga.

  * •

Stress: yes or no.

  * •

Focus: yes or no.

The experiments lasted for three months and involved 30 users 75.73 $\pm$ 6.60
years old (average $\pm$ standard deviation). All the users involved in the
experiments are patients in the occupational therapy workshops of afaga. The
tests were conducted under the supervision of their caregivers. For annotating
cognitive impairment, afaga applied the Spanish version (Díaz Mardomingo and
Peraita Adrados 2008) of the Global Deterioration Scale standard methodology
(Reisberg et al. 1982). Specifically, 57% of the participants had cognitive
impairment to some extent (40% mild, 17% severe) and the rest were mentally
healthy. We used this methodology as manual baseline for the comparison with
our automatic Machine Learning detection approach in Section 4.2.2.

In each individual experiment, we registered the characteristics of the user.
Tables 5 and 6 shows the session registration sheet, which was filled by the
caregivers, and a real example of a session, with its newscast content and its
associated interview.

Table 5:  Session registration sheet. |  Personal information
---
User ID:
Age:
Are you familiar with the use of new technologies?
$\bigcirc$ Yes $\bigcirc$ No
Do you suffer from hearing problems?
$\bigcirc$ Yes $\bigcirc$ No
Level of education
$\bigcirc$ Basic $\bigcirc$ Superior
Session data (filled by the caregiver)
What is the level of cognitive impairment?
$\bigcirc$ Absent $\bigcirc$ Mild $\bigcirc$ Severe
Frame of mind
$\bigcirc$ Happy $\bigcirc$ Normal $\bigcirc$ Sad
Was the user stressed?
$\bigcirc$ Yes $\bigcirc$ No
Was the user focused?
$\bigcirc$ Yes $\bigcirc$ No
Additional information
Table 6:  News item for session 1. |  News 1: politics
---
User ID:
‘Teresa Ribera has announced this Thursday that the Government will
automatically extend electricity social bonds until September 15th to maintain
the protection of people in a condition of energy vulnerability and to
guarantee basic supplies at home due to the coronavirus crisis.’
Question 1: Does the name Teresa Ribera sound familiar to you?
Answer:
Yes $\rightarrow$ Question 1.1: What facts do you know about the life of
Teresa Ribera?
No $\rightarrow$ Question 1.2: What do you associate the name of Teresa Ribera
to?
Not available $\rightarrow$ Question 1.3: Why do you think people like the
name Teresa?
Question 2: What is the meaning of the word ‘Government’?
Question 3: What happened on September 15th according to the news?
Question 4: Do you consider this news item interesting?

In detail, 17 participants had basic technological skills (e.g., they
regularly used electronic devices such as computers and smartphones), 9
suffered from hearing problems, 16 had a basic education and 14 had a superior
level of education. 18 participants had been diagnosed some cognitive
impairment. Most of the users were in a positive frame of mind (14 were happy
to participate and 16 simply accepted it). 22 users were clearly focused.

Regarding the white-coat effect, it is worth mentioning that 91.67% of the
participants without any cognitive impairment and 61.11% with mild or severe
cognitive impairments were relaxed during the experiments. As it could be
expected beforehand, cognitive impairment level increased with age (Ammal and
Jayashree 2020).

####  4.2.1 Similarity Metric Assessment

Each experiment was composed of five different sessions whose corresponding
news items were related to economy, politics, science, society, and sports.
Each session consisted of a newscast and four questions, as explained in
Section 3.

In the experiments, our system was able to separate users in most cases by sim
scores that were significantly related to their level of cognitive impairment.
Table 7 averages the sim metric for the three groups in the study (absent,
mild and severe impairment).

Table 7:  Average $\pm$ sd sim metric by level of impairment across all sessions. Level of impairment |  Avg. $\pm$ SD sim metric
---|---
Absent | 0.42 $\pm$ 0.17
Mild | 0.29 $\pm$ 0.17
Severe | 0.08 $\pm$ 0.10

By session, Table 8 shows that users with severe cognitive impairments scored
significantly less, while healthy ones or those with mild levels of cognitive
impairment performed significantly better. The effect of mild cognitive
impairment (compared to its absence) in the comprehension skills of the
participants was noticeable in all sessions except for the third. This session
was particularly challenging due to its vocabulary, which included technical
terms such as pymes ‘SMEs’ (acronym for small and medium enterprises).
Furthermore, in sessions 1, 2 and 4 the difference between mild and no
cognitive impairment was noteworthy.

Table 8:  Average $\pm$ sd sim metric at each session by level of impairment. | Session
---|---
| 1 | 2 | 3 | 4 | 5
Absent | 0.39 $\pm$ 0.27 | 0.75 $\pm$ 0.30 | 0.12 $\pm$ 0.10 | 0.48 $\pm$ 0.43 | 0.34 $\pm$ 0.41
Mild | 0.23 $\pm$ 0.21 | 0.65 $\pm$ 0.46 | 0.11 $\pm$ 0.14 | 0.19 $\pm$ 0.38 | 0.27 $\pm$ 0.38
Severe | 0.07 $\pm$ 0.07 | 0.13 $\pm$ 0.14 | 0.01 $\pm$ 0.03 | 0.00 $\pm$ 0.00 | 0.20 $\pm$ 0.45

Session 5 was particularly interesting. In it, users received several clues in
user-centred questions before being presented the attention-demanding
question. Thus, users with mild cognitive impairment performed better in this
session than in session 3, for example. This is coherent with the fact that
the reinforcement of key ideas from the news helped users to accurately answer
attention-demanding questions.

Table 9 shows the average lengths of user responses. They tended to be shorter
for higher impairment levels. In fact, for severe impairment, users tended to
remain silent or answer concisely (e.g., ‘I don’t know’).

Table 9:  Average answer length by level of impairment, in characters. Level of impairment | Avg. length
---|---
Absent | 54.20
Mild | 37.75
Severe | 30.84

Table 10 shows that, for users with cognitive impairments, stress had an
appalling effect on performance, and that high focus had a very positive
outcome, even enhancing $sim$ results from $0.20$ to $0.36$ in relaxed users
and, remarkably, from $0.03$ to $0.26$ in stressed users.

Table 10:  Average $\pm$ sd sim metric for users with cognitive impairments by stress and focus. |  | Stress
---|---|---
|  | No | Yes
Focus | No | 0.20 $\pm$ 0.19 | 0.03 $\pm$ 0.00
Yes | 0.36 $\pm$ 0.14 | 0.26 $\pm$ 0.16

Finally, Table 11 presents sim measurements versus technological skills and
levels of education, for users with cognitive impairments. In view of these
results, more educated users performed better in the experiments than those
with a basic education. Apparently, technological skills only led to higher
$sim$ scores for users with a basic education (note, however, the large
standard deviation of the $sim$ score of skilled users with superior
educational level).

Table 11:  Average $\pm$ sd sim metric for users with cognitive impairments by technological skills and level of education. |  | Technological skills
---|---|---
|  | Agnostic | Skilled
Education | Basic | 0.26 $\pm$ 0.17 | 0.29 $\pm$ 0.17
Superior | 0.34 $\pm$ 0.08 | 0.32 $\pm$ 0.35

####  4.2.2 Automatic Cognitive Impairment Detection

Finally, in order to evaluate the effectiveness of the proposed system, we
trained a set of selected Machine Learning algorithms (Salzberg 1994) for
detecting cognitive impairment: Bayesian Network (bn), Decision Tree (dt),
Random Forest (rf) and linear Support Vector Machine (svm), which have been
widely used in medical applications (Lu et al. 2017; Bratić et al. 2018;
Ghoneim et al. 2018; Rukmawan et al. 2021; Ahmed et al. 2021).

Table 12 shows the training and classification complexity of the Machine
Learning algorithms we selected, for $c$ classes, $d$ features, $k$ instances
of the algorithm (where applicable) and $n$ samples. bn training has linear
training complexity (Lu et al. 2006). dt and rf training have logarithmic
training complexity (Witten et al. 2016; Hassine et al. 2019). svm has the
highest training complexity, but it is very fast in classification time if
trained with a linear kernel, as in our case (Vapnik 2000). We employed the
algorithm implementations from Weka242424Available at
https://www.cs.waikato.ac.nz/ml/weka, September 2021. (Witten et al. 2016).

Table 12:  Training and testing complexity of the Machine Learning algorithms. Classifier | Train complexity | Test complexity
---|---|---
BN | O($n\cdot d$) | O($c\cdot d$)
DT | O($n\cdot\mbox{log}(n)\cdot d$) | O(depth of the tree)
RF | O($n\cdot\mbox{log}(n)\cdot d\cdot k$) | O(depth of the tree$\cdot k$)
SVM | O($n^{2}$) | O($d$)

Firstly, we divided the sample into user classes with and without impairments.
Table 13 shows all the features we considered. Then, we applied the
GainRatioAttributeEval feature selection algorithm, also from Weka, which
evaluates the relevance of the attributes by measuring their gain ratio with
regard to the target class. The most relevant features it selected for the
classification model were, in decreasing importance, length of the response in
characters, focus, sim for question 4 in session 2, age, technology skills,
and sim for question 4 in session 4.

Table 13:  Features for the Machine Learning models training. Type | Feature name | Description
---|---|---
Boolean | Focus |  |  True if the user was focused during the experiments, otherwise false.
---
Stress |  |  True if the user was stressed during the experiments, otherwise false.
---
Studies |  |  True if the user had a superior level of education, otherwise false.
---
Technology |  |  True if the user had technological skills, otherwise false.
---
Nominal | Age |  |  $\\{1,2,3,4\\}$ if the user age was in $\\{[60,70],(70,80],(80,90],(90,100]\\}$, respectively.
---
Numerical | NumChars |  |  Avg. number of characters in the user responses.
---
SimS1Q4 |  |  sim value for question 4 in session 1.
---
SimS2Q4 |  |  sim value for question 4 in session 2.
---
SimS3Q4 |  |  sim value for question 4 in session 3.
---
SimS4Q4 |  |  sim value for question 4 in session 4.
---
SimS5Q4 |  |  sim value for question 4 in session 5.
---

Note that in spite of the results in Table 11, the selection algorithm
preferred technological skills over level of education.

Finally, Table 14 shows the classification results for the selected algorithms
with 10-fold cross validation (Berrar 2019) to avoid overfitting. This
methodology minimises underestimation and overestimation in the results. For
this purpose, the dataset is divided into 10 segments, 9 of which are used for
training and the remaining one for testing. This process is repeated ten times
by avoiding overlapping testing segments in different evaluations. At the end,
the final performance metric is computed as the average of the intermediate
tests. In addition, unlike svm(Jatav and Sharma 2018) and bn(Wood et al. 2019)
algorithms, which are less prone to overfitting issues, in the cases of dt and
rf we limited the folds to 3 and the maximum depth to 5. To avoid bias, the
entries from the same users were grouped to prevent them from being
simultaneously used for training and evaluation.

Note that dt was finally selected due to its better performance, since it
attained a detection accuracy of $86.67$% using the most relevant features.

Table 14: F-measure, recall and response times for the selected algorithms. Classifier | Class | F-measure | Recall | Training (ms) | Testing (ms)
---|---|---|---|---|---
BN | Present | 64.50% | 55.60% | 0.16 | $<$0.01
Absent | 62.10% | 75.00%
DT | Present | 88.20% | 83.30% | 0.15 | $<$0.01
Absent | 84.60% | 91.70%
RF | Present | 74.30% | 72.20% | 0.10 | $<$0.01
Absent | 64.00% | 66.70%
SVM | Present | 78.80% | 72.20% | 2.58 | $<$0.01
Absent | 74.10% | 83.30%

##  5 Conclusions

Even though there exists previous research on intelligent systems for
therapeutic monitoring of cognitive impairment in elderly people, most current
approaches are based on manual tests that rely on human supervision for early
detection.

In this work, to reduce caregivers’ effort and the white-coat effect, we have
proposed a novel conversational system for entertainment and therapeutic
monitoring of elderly people. It relies on nlp techniques for chatbot
behaviour generation and user-transparent automatic assessment, by combining
distracting (user-centred) with attention-demanding questions (embedded
cognitive tests). Thus, our main contribution is a Machine Learning approach
for user-transparent cognitive monitoring that is embedded into a user-centred
entertainment solution. This approach is based on metrics that estimate the
abstraction skills of the users from their answers during the automatic
dialogue stages.

Experimental results with elderly people under the supervision of afaga
gerontologists indicate that our solution is satisfactory and has strong
potential for user-friendly therapeutic monitoring. Preliminary analyses have
obtained a detection accuracy of cognitive impairment close to 90%.

Given these promising results, we plan to enhance the system with empathetic
capabilities through user feedback. At the same time, users will benefit from
real-time encouragement about their performance.

## Acknowledgments

This work was supported by Xunta de Galicia grant grc 2018/053, Spain; and
University of Vigo/CISUG for open access charge. The authors are indebted to
afaga for providing access to real patients and expert knowledge about
monitoring of cognitive impairments, and for validating our results.

## Declarations

### Data Availability

The datasets generated during and/or analysed during the current study are
available from the corresponding author on reasonable request.

### Competing Interests

The authors have no competing interests to declare that are relevant to the
content of this article.

### Ethics approval

All procedures performed in studies involving human participants were in
accordance with the ethical standards of the institutional and/or national
research committee and with the 1964 Helsinki declaration and its later
amendments or comparable ethical standards.

### Consent to participate

Informed consent was obtained from all individual participants in the study.

## References

  * Aaltonen et al. [2017] Iina Aaltonen, Anne Arvola, Päivi Heikkilä, and Hanna Lammi.  Hello pepper, may I tickle you?: Children’s and adults’ responses to an entertainment robot at a shopping mall.  In _Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction_ , pages 53–54. IEEE Computer Society, 2017.  ISBN 9781450348850.  10.1145/3029798.3038362.
  * Abdollahi et al. [2017] Hojjat Abdollahi, Ali Mollahosseini, Josh T. Lane, and Mohammad H. Mahoor.  A pilot study on using an intelligent life-like robot as a companion for elderly individuals with dementia and depression.  In _Proceedings of the International Conference on Humanoid Robotics_ , pages 541–546. IEEE, 2017.  ISBN 978-1-5386-4678-6.  10.1109/HUMANOIDS.2017.8246925.
  * Adamson et al. [2019] Greg Adamson, John C. Havens, and Raja Chatila.  Designing a Value-Driven Future for Ethical Autonomous and Intelligent Systems.  _Proceedings of the IEEE_ , 107(3):518–525, 2019.  ISSN 0018-9219.  10.1109/JPROC.2018.2884923.
  * Ahmed et al. [2021] Syed Thouheed Ahmed, Sharmila Sankar, and M. Sandhya.  Multi-objective optimal medical data informatics standardization and processing technique for telemedicine via machine learning approach.  _Journal of Ambient Intelligence and Humanized Computing_ , 12(5):5349–5358, 2021.  ISSN 1868-5137.  10.1007/s12652-020-02016-9.
  * Alsmirat et al. [2019] Mohammad A. Alsmirat, Fatimah Al-Alem, Mahmoud Al-Ayyoub, Yaser Jararweh, and Brij Gupta.  Impact of digital fingerprint image quality on the fingerprint recognition accuracy.  _Multimedia Tools and Applications_ , 78(3):3649–3688, feb 2019.  ISSN 1380-7501.  10.1007/s11042-017-5537-5.
  * Ammal and Jayashree [2020] S. Meenakshi Ammal and L.S. Jayashree.  Early detection of cognitive impairment of elders using wearable sensors.  In _Systems Simulation and Modeling for Cloud Computing and Big Data Applications_ , pages 147–159. Elsevier, 2020.  10.1016/B978-0-12-819779-0.00010-1.
  * Atserias et al. [2006] Jordi Atserias, Bernardino Casas, Elisabet Comelles, Meritxell González, Lluís Padró, and Muntsa Padró.  Freeling 1.3: Syntactic and semantic services in an open-source NLP library.  In _Proceedings of the International Conference on Language Resources and Evaluation_. European Language Resources Association, 2006.
  * Baby et al. [2017] Cyril Joe Baby, Faizan Ayyub Khan, and J N Swathi.  Home automation using IoT and a chatbot using natural language processing.  In _Proceedings of the Innovations in Power and Advanced Computing Technologies_ , pages 1–6. IEEE, 2017.  ISBN 978-1-5090-5682-8.  10.1109/IPACT.2017.8245185.
  * Ball et al. [2004] Sarah L. Ball, A. J. Holland, F. A. Huppert, P. Treppner, P. Watson, and J. Hon.  The modified CAMDEX informant interview is a valid and reliable tool for use in the diagnosis of dementia in adults with Down’s syndrome.  _Journal of Intellectual Disability Research_ , 48(6):611–620, 2004.  ISSN 0964-2633.  10.1111/j.1365-2788.2004.00630.x.
  * Bernardini et al. [2014] Sara Bernardini, Kaśka Porayska-Pomsta, and Tim J. Smith.  ECHOES: An intelligent serious game for fostering social communication in children with autism.  _Information Sciences_ , 264:41–60, 2014.  ISSN 00200255.  10.1016/j.ins.2013.10.027.
  * Berrar [2019] Daniel Berrar.  Cross-Validation.  In _Encyclopedia of Bioinformatics and Computational Biology_ , pages 542–545. Elsevier, 2019.  ISBN 9780128114322.  10.1016/B978-0-12-809633-8.20349-X.
  * Boise et al. [2004] Linda Boise, Margaret B Neal, and Jeffrey Kaye.  Dementia assessment in primary care: results from a study in three managed care systems.  _The Journals of Gerontology Series A: Biological Sciences and Medical Sciences_ , 59(6):621–626, 2004.  10.1093/gerona/59.6.M621.
  * Borson et al. [2013] Soo Borson, Lori Frank, Peter J Bayley, Malaz Boustani, Marge Dean, Pei-Jung Lin, J Riley McCarten, John C Morris, David P Salmon, Frederick A Schmitt, et al.  Improving dementia care: the role of screening and detection of cognitive impairment.  _Alzheimer’s & Dementia_, 9(2):151–159, 2013.  10.1016/j.jalz.2012.08.008.
  * Bratić et al. [2018] Brankica Bratić, Vladimir Kurbalija, Mirjana Ivanović, Iztok Oder, and Zoran Bosnić.  Machine Learning for Predicting Cognitive Diseases: Methods, Data Sources and Risk Factors.  _Journal of Medical Systems_ , 42(12):1–15, dec 2018.  ISSN 0148-5598.  10.1007/s10916-018-1071-x.
  * Budiharto et al. [2017] Widodo Budiharto, Anggita Dian Cahyani, Pingkan C.B. Rumondor, and Derwin Suhartono.  EduRobot: Intelligent Humanoid Robot with Natural Interaction for Education and Entertainment.  _Procedia Computer Science_ , 116:564–570, 2017.  ISSN 18770509.  10.1016/j.procs.2017.10.064.
  * Callahan et al. [2014] Christopher M Callahan, Tatiana Foroud, Andrew J Saykin, Anantha Shekhar, and Hugh C Hendrie.  Translational research on aging: clinical epidemiology as a bridge between the sciences.  _Translational Research_ , 163(5):439–445, 2014.  10.1016/j.trsl.2013.09.002.
  * Chumkamon et al. [2016] Sakmongkon Chumkamon, Eiji Hayashi, and Masato Koike.  Intelligent emotion and behavior based on topological consciousness and adaptive resonance theory in a companion robot.  _Biologically Inspired Cognitive Architectures_ , 18:51–67, 2016.  ISSN 2212683X.  10.1016/j.bica.2016.09.004.
  * Corley and Mihalcea [2005] Courtney D Corley and Rada Mihalcea.  Measuring the semantic similarity of texts.  In _Proceedings of workshop on empirical modeling of semantic equivalence and entailment_ , pages 13–18, 2005.
  * Correia et al. [2016] Filipa Correia, Tiago Ribeiro, Patricia Alves-Oliveira, Nuno Maia, Francisco S. Melo, and Ana Paiva.  Building a social robot as a game companion in a card game.  In _Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction_ , pages 563–563, 2016.  ISBN 978-1-4673-8370-7.  10.1109/HRI.2016.7451857.
  * Crocco et al. [2014] Elizabeth Crocco, Rosie E. Curiel, Amarilis Acevedo, Sara J. Czaja, and David A. Loewenstein.  An evaluation of deficits in semantic cueing and proactive and retroactive interference as early features of Alzheimer’s disease.  _American Journal of Geriatric Psychiatry_ , 22(9):889–897, 2014.  ISSN 15457214.  10.1016/j.jagp.2013.01.066.
  * Díaz Mardomingo and Peraita Adrados [2008] Carmen Díaz Mardomingo and Herminia Peraita Adrados.  Neuropsychological evaluation and cognitive evolution of a bilingual Alzheimer patient.  _Revista de Psicopatología y Psicología Clínica_ , 13(3):219–228, 2008.  ISSN 2254-6057.  10.5944/rppc.vol.13.num.3.2008.4061.
  * Echeburúa et al. [2017] Enrique Echeburúa, Pedro J. Amor, José Manuel Muñoz, Belén Sarasua, and Irene Zubizarreta.  Escala de Gravedad de Síntomas del Trastorno de Estrés Postraumático según el DSM-5: versión forense (EGS-F).  _Anuario de Psicología Jurídica_ , 27(1):67–77, 2017.  ISSN 11330740.  10.1016/j.apj.2017.02.005.
  * Feng et al. [2008] Jin Feng, Yi-Ming Zhou, and Trevor Martin.  Sentence similarity based on relevance.  In _Proceedings of International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems_ , volume 8, page 833, 2008.
  * Foroughi et al. [2008] Homa Foroughi, Baharak Shakeri Aski, and Hamidreza Pourreza.  Intelligent video surveillance for monitoring fall detection of elderly in home environments.  In _Proceedings of the International Conference on Computer and Information Technology_ , pages 219–224. IEEE, 2008.  ISBN 978-1-4244-2135-0.  10.1109/ICCITECHN.2008.4803020.
  * Fung et al. [2018] Pascale Fung, Dario Bertero, Peng Xu, Ji Ho Park, Chien-Sheng Wu, and Andrea Madotto.  Empathetic dialog systems.  In _Proceedings of the International Conference on Language Resources and Evaluation_ , pages 1–7. European Language Resources Association, 2018.
  * García-Méndez et al. [2018] Silvia García-Méndez, Milagros Fernández-Gavilanes, Enrique Costa-Montenegro, Jonathan Juncal-Martínez, and Francisco J. González-Castaño.  Automatic natural language generation applied to alternative and augmentative communication for online video content services using simpleNLG for Spanish.  In _Proceedings of the Web for All Conference: Internet of Accessible Things_ , pages 1–4. ACM Press, 2018.  ISBN 9781450356510.  10.1145/3192714.3192837.
  * García-Méndez et al. [2019] Silvia García-Méndez, Milagros Fernández-Gavilanes, Enrique Costa-Montenegro, Jonathan Juncal-Martínez, and Francisco J. González-Castaño.  A library for automatic natural language generation of Spanish texts.  _Expert Systems with Applications_ , 120:372–386, 2019.  ISSN 0957-4174.  10.1016/J.ESWA.2018.11.036.
  * Ghoneim et al. [2018] Ahmed Ghoneim, Ghulam Muhammad, Syed Umar Amin, and Brij Gupta.  Medical Image Forgery Detection for Smart Healthcare.  _IEEE Communications Magazine_ , 56(4):33–37, apr 2018.  ISSN 0163-6804.  10.1109/MCOM.2018.1700817.
  * González-Agirre et al. [2012] Aitor González-Agirre, Egoitz Laparra, and German Rigau.  Multilingual central repository version 3.0: upgrading a very large lexical knowledge base.  In _Proceedings of the Global WordNet Conference_ , pages 118–125, 2012.
  * Hancock et al. [2006] Geraldine A. Hancock, Bob Woods, David Challis, and Martin Orrell.  The needs of older people with dementia in residential care.  _International Journal of Geriatric Psychiatry_ , 21(1):43–49, 2006.  ISSN 0885-6230.  10.1002/gps.1421.
  * Hassine et al. [2019] Kawther Hassine, Aiman Erbad, and Ridha Hamila.  Important Complexity Reduction of Random Forest in Multi-Classification Problem.  In _International Wireless Communications & Mobile Computing Conference_, pages 226–231. IEEE, jun 2019.  ISBN 978-1-5386-7747-6.  10.1109/IWCMC.2019.8766544.
  * Hsu and Chien [2009] Chein-Chang Hsu and You Yin Chien.  An Intelligent Fuzzy Affective Computing System for Elderly Living Alone.  In _Proceedings of the International Conference on Hybrid Intelligent Systems_ , pages 293–297. IEEE, 2009.  ISBN 978-0-7695-3745-0.  10.1109/HIS.2009.318.
  * Irani and Kamal [2014] Zahir Irani and Muhammad Mustafa Kamal.  Intelligent Systems Research in the Construction Industry.  _Expert Systems with Applications_ , 41(4):934–950, 2014.  ISSN 09574174.  10.1016/j.eswa.2013.06.061.
  * Jatav and Sharma [2018] Shakuntala Jatav and Vivek Sharma.  An Algorithm for Predictive Data Mining Approach in Medical Diagnosis.  _International Journal of Computer Science and Information Technology_ , 10(1):11–20, oct 2018.  ISSN 09754660.  10.5121/ijcsit.2018.10102.
  * Johnson et al. [2016] David O. Johnson, Raymond H. Cuijpers, Kathrin Pollmann, and Antoine A.J. Van de Ven.  Exploring the Entertainment Value of Playing Games with a Humanoid Robot.  _International Journal of Social Robotics_ , 8(2):247–269, 2016.  ISSN 18754805.  10.1007/s12369-015-0331-x.
  * Li et al. [2006] Y. Li, D. McLean, Z. A. Bandar, J. D. O’Shea, and K. Crockett.  Sentence similarity based on semantic nets and corpus statistics.  _IEEE Transactions on Knowledge and Data Engineering_ , 18(8):1138–1150, 2006.  10.1109/TKDE.2006.130.
  * Lindgren and Andersson [2011] M. Lindgren and I. S. Andersson.  The Karen instruments for measuring quality of nursing care: construct validity and internal consistency.  _International Journal for Quality in Health Care_ , 23(3):292–301, 2011.  ISSN 1353-4505.  10.1093/intqhc/mzq092.
  * Liu et al. [2018] Wei De Liu, Kai Yuan Chuang, and Kuo Yi Chen.  The Design and Implementation of a Chatbot’s Character for Elderly Care.  In _Proceedings of the International Conference on System Science and Engineering_ , pages 1–5, 2018.  ISBN 9781538662854.  10.1109/ICSSE.2018.8520008.
  * Livingston et al. [2017] Gill Livingston, Andrew Sommerlad, Vasiliki Orgeta, Sergi G Costafreda, Jonathan Huntley, David Ames, Clive Ballard, Sube Banerjee, Alistair Burns, Jiska Cohen-Mansfield, Claudia Cooper, Nick Fox, Laura N Gitlin, Robert Howard, Helen C Kales, Eric B Larson, Karen Ritchie, Kenneth Rockwood, Elizabeth L Sampson, Quincy Samus, Lon S Schneider, Geir Selbaek, Linda Teri, Naaheed Mukadam, J Cohen-Mansfield, and N Gitlin.  The Lancet Commissions Dementia prevention, intervention, and care.  _Lancet_ , 390:2673–734, 2017.  10.1016/S0140-6736(17)31363-6.
  * Loewenstein et al. [2004] David A Loewenstein, Amarilis Acevedo, Cheryl Luis, Thomas Crum, Warren W Barker, and Ranjan Duara.  Semantic interference deficits and the detection of mild alzheimer’s disease and mild cognitive impairment without dementia.  _Journal of the International Neuropsychological Society_ , 10(1):91–100, 2004.  10.1017/S1355617704101112.
  * López et al. [2018] Gustavo López, Luis Quesada, and Luis A. Guerrero.  Alexa vs. Siri vs. Cortana vs. Google Assistant: A Comparison of Speech-Based Natural User Interfaces.  In _Advances in Intelligent Systems and Computing_ , pages 241–250. Springer, Cham, 2018.  ISBN 9783319603650.  10.1007/978-3-319-60366-7_23.
  * Löppönen et al. [2003] Minna Löppönen, Ismo Räihä, Raimo Isoaho, Tero Vahlberg, and Sirkka-Liisa Kivelä.  Diagnosing cognitive impairment and dementia in primary health care-a more active approach is needed.  _Age and Ageing_ , 32(6):606–612, 2003.  ISSN 0002-0729.  10.1093/ageing/afg097.
  * Lu et al. [2006] Jingli Lu, Ying Yang, and Geoffrey I. Webb.  Incremental Discretization for Naïve-Bayes Classifier.  In _Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)_ , pages 223–238. Springer, 2006.  ISBN 3540370250.  10.1007/11811305_25.
  * Lu et al. [2017] Shen Lu, Yong Xia, Weidong Cai, Michael Fulham, and David Dagan Feng.  Early identification of mild cognitive impairment using incomplete random forest-robust support vector machine and FDG-PET imaging.  _Computerized Medical Imaging and Graphics_ , 60:35–41, 2017.  ISSN 08956111.  10.1016/j.compmedimag.2017.01.001.
  * Magnisalis et al. [2011] Ioannis Magnisalis, Stavros Demetriadis, and Anastasios Karakostas.  Adaptive and Intelligent Systems for Collaborative Learning Support: A Review of the Field.  _IEEE Transactions on Learning Technologies_ , 4(1):5–20, 2011.  ISSN 1939-1382.  10.1109/TLT.2011.2.
  * Masud et al. [2021] Mehedi Masud, Gurjot Singh Gaba, Salman Alqahtani, Ghulam Muhammad, B. B. Gupta, Pardeep Kumar, and Ahmed Ghoneim.  A Lightweight and Robust Secure Key Establishment Protocol for Internet of Medical Things in COVID-19 Patients Care.  _IEEE Internet of Things Journal_ , pages 1–9, 2021.  ISSN 2327-4662.  10.1109/JIOT.2020.3047662.
  * Ma’sum et al. [2013] M. Anwar Ma’sum, M. Kholid Arrofi, Grafika Jati, Futuhal Arifin, M. Nanda Kurniawan, Petrus Mursanto, and Wisnu Jatmiko.  Simulation of intelligent Unmanned Aerial Vehicle (UAV) for military surveillance.  In _Proceedings of the International Conference on Advanced Computer Science and Information Systems_ , pages 161–166. IEEE, 2013.  ISBN 978-1-4799-4692-1.  10.1109/ICACSIS.2013.6761569.
  * Matsumoto et al. [2007] Rie Matsumoto, Hideki Nakayama, Tatsuya Harada, and Yasuo Kuniyoshi.  Journalist robot: robot system making news articles from real world.  In _Proceedings of the International Conference on Intelligent Robots and Systems_ , pages 1234–1241. IEEE, 2007.  ISBN 978-1-4244-0911-2.  10.1109/IROS.2007.4399598.
  * Matsuyama et al. [2016] Yoichi Matsuyama, Arjun Bhardwaj, Ran Zhao, Oscar Romeo, Sushma Akoju, and Justine Cassell.  Socially-Aware Animated Intelligent Personal Assistant Agent.  In _Proceedings of the Annual Meeting of the Special Interest Group on Discourse and Dialogue_ , pages 224–227. Association for Computational Linguistics, 2016.  10.18653/v1/W16-3628.
  * Milne et al. [2008] A. Milne, A. Culverwell, R. Guss, J. Tuppen, and R. Whelton.  Screening for dementia in primary care: a review of the use, efficacy and quality of measures.  _International Psychogeriatrics_ , 20(5):911–926, 2008.  ISSN 1041-6102.  10.1017/S1041610208007394.
  * Ngai et al. [2014] E.W.T. Ngai, S. Peng, Paul Alexander, and Karen K.L. Moon.  Decision support and intelligent systems in the textile and apparel supply chain: An academic review of research articles.  _Expert Systems with Applications_ , 41(1):81–91, 2014.  ISSN 09574174.  10.1016/j.eswa.2013.07.013.
  * Noh et al. [2017] Sangwoo Noh, Juyeon Han, Jinwoo Jo, and Ahyoung Choi.  Virtual Companion Based Mobile User Interface: An Intelligent and Simplified Mobile User Interface for the Elderly Users.  In _Proceedings of the International Symposium on Ubiquitous Virtual Reality_ , pages 8–9, 2017.  ISBN 9781538630914.  10.1109/ISUVR.2017.10.
  * Oh et al. [2017] Kyo Joong Oh, Dongkun Lee, Byungsoo Ko, and Ho Jin Choi.  A chatbot for psychiatric counseling in mental healthcare service based on emotional dialogue analysis and sentence generation.  In _Proceedings of the IEEE International Conference on Mobile Data Management_ , pages 371–376, 2017.  ISBN 9781538639320.  10.1109/MDM.2017.64.
  * Östlund [2010] Britt Östlund.  Watching television in later life: A deeper understanding of TV viewing in the homes of old people and in geriatric care contexts.  _Scandinavian Journal of Caring Sciences_ , 24(2):233–243, 2010.  ISSN 02839318.  10.1111/j.1471-6712.2009.00711.x.
  * Padró and Stanilovsky [2012] Lluís Padró and Evgeny Stanilovsky.  Freeling 3.0: Towards wider multilinguality.  In _Proceedings of the Language Resources and Evaluation Conference_ , pages 2473–2479. European Language Resources Association, 2012.  ISBN 978-2-9517408-7-7.
  * Pedersen et al. [2004] Ted Pedersen, Siddharth Patwardhan, and Jason Michelizzi.  Wordnet: Similarity-measuring the relatedness of concepts.  In _Proceedings of Conference on Artificial Intelligence_ , volume 4, pages 25–29, 2004.
  * Rashkin et al. [2019] Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau.  Towards empathetic open-domain conversation models: A new benchmark and dataset.  In _Proceedings of the Annual Meeting of the Association for Computational Linguistics_ , pages 5370–5381. Association for Computational Linguistics, 2019.  10.18653/v1/P19-1534.
  * Reisberg et al. [1982] B. Reisberg, S. H. Ferris, M. J. De Leon, and T. Crook.  The Global Deterioration Scale for assessment of primary degenerative dementia.  _American Journal of Psychiatry_ , 139:1136–1139, 1982.  ISSN 0002-953X.  10.1176/ajp.139.9.1136.
  * Ridha and Rossor [2005] Basil Ridha and Martin Rossor.  The Mini Mental State Examination.  _Practical Neurology_ , 5(5):298–303, 2005.  ISSN 1474-7758.  10.1111/j.1474-7766.2005.00333.x.
  * Rukmawan et al. [2021] S. H. Rukmawan, F. R. Aszhari, Z. Rustam, and J. Pandelaki.  Cerebral Infarction Classification Using the K-Nearest Neighbor and Naive Bayes Classifier.  _Journal of Physics: Conference Series_ , 1752:012045, 2021.  ISSN 1742-6588.  10.1088/1742-6596/1752/1/012045.
  * Salichs et al. [2016] Miguel A. Salichs, Irene P Encinar, Esther Salichs, Alvaro Castro-González, and María Malfaz.  Study of Scenarios and Technical Requirements of a Social Assistive Robot for Alzheimer’s Disease Patients and their Caregivers.  _International Journal of Social Robotics_ , 8(1):85–102, 2016.  ISSN 18754805.  10.1007/s12369-015-0319-6.
  * Salzberg [1994] Steven L. Salzberg.  C4.5: Programs for machine learning.  _Machine Learning_ , 16:235–240, 1994.  ISSN 0885-6125.  10.1007/BF00993309.
  * Samanta et al. [2014] N. Samanta, A.K. Chanda, and C. Roy Chaudhuri.  An energy efficient, minimally intrusive multi-sensor intelligent system for health monitoring of elderly people.  _International Journal on Smart Sensing and Intelligent Systems_ , 7(2):762–780, 2014.  ISSN 1178-5608.  10.21307/ijssis-2017-680.
  * Sarrab and Alshohoumi [2021] Mohamed Sarrab and Fatma Alshohoumi.  Assisted-Fog-Based Framework for IoT-Based Healthcare Data Preservation.  _International Journal of Cloud Applications and Computing_ , 11(2):1–16, apr 2021.  ISSN 2156-1834.  10.4018/IJCAC.2021040101.
  * Sedik et al. [2021] Ahmed Sedik, Mohamed Hammad, Fathi E. Abd El-Samie, Brij B. Gupta, and Ahmed A. Abd El-Latif.  Efficient deep learning approach for augmented detection of Coronavirus disease.  _Neural Computing and Applications_ , pages 1–18, jan 2021.  ISSN 0941-0643.  10.1007/s00521-020-05410-8.
  * Shang et al. [2015] Lifeng Shang, Zhengdong Lu, and Hang Li.  Neural Responding Machine for Short-Text Conversation.  In _Proceedings of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing_ , pages 1577–1586. Association for Computational Linguistics, 2015.  ISBN 9781941643723.  10.3115/v1/P15-1152.
  * Sharkey and Sharkey [2012] Amanda Sharkey and Noel Sharkey.  Granny and the robots: Ethical issues in robot care for the elderly.  _Ethics and Information Technology_ , 14(1):27–40, 2012.  ISSN 13881957.  10.1007/s10676-010-9234-6.
  * Shawar and Atwell [2007] Bayan Abu Shawar and Eric Atwell.  Chatbots: Are they Really Useful?  _Ldv forum_ , 22(1):29–49, 2007.
  * Shum et al. [2018] Heung-Yeung Shum, Xiao-Dong He, and Di Li.  From Eliza to XiaoIce: challenges and opportunities with social chatbots.  _Frontiers of Information Technology & Electronic Engineering_, 19(1):10–26, 2018.  10.1631/FITEE.1700826.
  * Skjuve et al. [2019] Marita Skjuve, Ida Maria Haugstveit, Asbjørn Følstad, and Petter Bae Brandtzaeg.  Help! Is my chatbot falling into the uncanny valley? An empirical study of user experience in human-chatbot interaction.  _Human Technology_ , 15(1):30–54, 2019.  ISSN 17956889.  10.17011/ht/urn.201902201607.
  * Su et al. [2017] Ming-Hsiang Su, Chung-Hsien Wu, Kun-Yi Huang, Qian-Bei Hong, and Hsin-Min Wang.  A chatbot using LSTM-based multi-layer embedding for elderly care.  In _International Conference on Orange Technologies_ , pages 70–74. IEEE, 2017.  ISBN 978-1-5386-3276-5.  10.1109/ICOT.2017.8336091.
  * Suryadevara et al. [2012] N.K. Suryadevara, M.T. Quazi, and S.C. Mukhopadhyay.  Intelligent Sensing Systems for Measuring Wellness Indices of the Daily Activities for the Elderly.  In _Proceedings of the International Conference on Intelligent Environments_ , pages 347–350. IEEE, 2012.  ISBN 978-1-4673-2093-1.  10.1109/IE.2012.49.
  * Tseng et al. [2013] Kevin C. Tseng, Chien-Lung Hsu, and Yu-Hao Chuang.  Designing an Intelligent Health Monitoring System and Exploring User Acceptance for the Elderly.  _Journal of Medical Systems_ , 37(6):9967, 2013.  ISSN 0148-5598.  10.1007/s10916-013-9967-y.
  * Vapnik [2000] Vladimir N. Vapnik.  _The Nature of Statistical Learning Theory_.  Springer New York, 2000.  ISBN 978-1-4419-3160-3.  10.1007/978-1-4757-3264-1.
  * Wang et al. [2020] Haoxiang Wang, Zhihui Li, Yang Li, B.B. Gupta, and Chang Choi.  Visual saliency guided complex image retrieval.  _Pattern Recognition Letters_ , 130:64–72, feb 2020.  ISSN 01678655.  10.1016/j.patrec.2018.08.010.
  * Wang et al. [2016] Liang-Hung Wang, Yi-Mao Hsiao, Xue-Qin Xie, and Shuenn-Yuh Lee.  An outdoor intelligent healthcare monitoring device for the elderly.  _IEEE Transactions on Consumer Electronics_ , 62(2):128–135, 2016.  ISSN 0098-3063.  10.1109/TCE.2016.7514671.
  * Wang et al. [2015] Mingxuan Wang, Zhengdong Lu, Hang Li, and Qun Liu.  Syntax-based deep matching of short texts.  In _Proceedings of the International Joint Conference on Artificial Intelligence_ , pages 1354–1361. Association for Computational Linguistics, 2015.  ISBN 9781577357384.
  * Wen et al. [2017] Tsung Hsien Wen, David Vandyke, Nikola Mrkšíc, Milica Gašíc, Lina M. Rojas-Barahona, Pei Hao Su, Stefan Ultes, and Steve Young.  A network-based end-to-end trainable task-oriented dialogue system.  In _Proceedings of the Conference of the European Chapter of the Association for Computational Linguistics_ , pages 437–448. Association for Computational Linguistics, 2017.  ISBN 9781510838604.
  * Wilamowski and Irwin [2015] Bogdan M. Wilamowski and J. David Irwin.  Intelligent Systems.  In _Adaptive Stochastic Optimization Techniques with Applications_ , pages 109–130. CRC Press, 2015.  ISBN 9781439802847.  10.1201/b19256-7.
  * Witten et al. [2016] Ian H. Witten, Eibe Frank, Mark A. Hall, and Christopher J. Pal.  _Data Mining: Practical Machine Learning Tools and Techniques_.  Morgan Kaufmann Publishers, 2016.  ISBN 9780128042915.  10.1016/c2009-0-19715-5.
  * Wood et al. [2019] Alexander Wood, Vladimir Shpilrain, Kayvan Najarian, and Delaram Kahrobaei.  Private naive bayes classification of personal biomedical data: Application in cancer data analysis.  _Computers in Biology and Medicine_ , 105:144–150, 2019.  ISSN 00104825.  10.1016/j.compbiomed.2018.11.018.
  * Wu et al. [2018] Yu Wu, Zhoujun Li, Wei Wu, and Ming Zhou.  Response selection with topic clues for retrieval-based chatbots.  _Neurocomputing_ , 316:251–261, 2018.  ISSN 09252312.  10.1016/j.neucom.2018.07.073.
  * Yang and Powers [2005] Dongqiang Yang and David M. W. Powers.  Measuring semantic similarity in the taxonomy of wordnet.  In _Proceedings of the Australasian Conference on Computer Science_ , pages 315–322. Australian Computer Society, 2005.  ISBN 1920682201.  10.5555/1082161.1082196.
  * Yasuda et al. [2014] K Yasuda, M Fuketa, and J Aoe.  An anime agent system for reminiscence therapy.  _Gerontechnology_ , 13:118–119, 2014.  ISSN 1569111X.  10.4017/gt.2014.13.02.239.00.
  * Yoo et al. [2014] Donghee Yoo, Sungchun No, and Minyoung Ra.  A Practical Military Ontology Construction for the Intelligent Army Tactical Command Information System.  _International Journal of Computers Communications & Control_, 9(1):93–100, 2014.  ISSN 1841-9844.  10.15837/ijccc.2014.1.49.