2021

[1]Juan M. Fernández-Luna

These authors contributed equally to this work.

[1]Departmento de Ciencias de la Computación e Inteligencia Artificial,
Escuela Técnica Superior de Ingenierías Infomática y de Telecomunicación,
CITIC-UGR, Universidad de Granada, C/ Periodista Daniel Saucedo Aranda, S/N,
Granada, 18013, Spain

# Use of Topical and Temporal Profiles and their Hybridisation for Content-
based Recommendation111This paper or a similar version is not currently under
review by a journal or conference. This paper is void of plagiarism or self-
plagiarism as defined by the Committee on Publication Ethics and Springer
Guidelines

Luis M. de Campos<EMAIL_ADDRESS><EMAIL_ADDRESS>Juan F. Huete
<EMAIL_ADDRESS>*

###### Abstract

In the context of content-based recommender systems, the aim of this paper is
to determine how better profiles can be built and how these affect the
recommendation process based on the incorporation of temporality, i.e. the
inclusion of time in the recommendation process, and topicality, i.e. the
representation of texts associated with users and items using topics and their
combination. The main contribution of the paper is to present two different
ways of hybridising these two dimensions and to evaluate and compare them with
other alternatives.

###### keywords:

Content-based recommendation, topical profiles, temporal profiles, publication
venue recommendation

##  1 Introduction

In today’s world, recommender systems (RSs) Bobadilla et al. (2013) play an
extremely important role in our digital life Lu et al. (2015). Many e-commerce
platforms (such as Amazon or eBay, for example), entertainment services (e.g.
Netflix, Spotify), or social media (Instagram, Facebook, Twitter, etc.)
incorporate recommendation into their functionalities. These systems analyse
how users interact with their products and suggest ones that the users might
be interested in.

There are two main types of recommender systems Adomavicius and Tuzhilin
(2005): those based on collaborative filtering (CF) and those based on
content-based recommendation (CBR) Lops et al. (2019). While the first type
generates suggestions based on user ratings for items they have consumed, the
second offers recommendations based on the textual contents of these consumed
items. In both cases, ratings or contents are stored in special structures
called profiles Gauch et al. (2007) and these represent the users’ interests.
RSs exploit these profiles to generate useful suggestions for users.

If we focus on CBR, as long as there is a textual description of the items,
users’ profiles are usually represented by a bag of words which is obtained by
combining the texts for any item the user has shown interest in. As CBR is
primarily based on information retrieval (IR), the recommendation process
consists in carrying out a matching between the active user’s profile (in
terms of IR, a representation of an information need, i.e. a query) and the
textual representation of all the items in the collection (documents in the IR
context). The degree of relevance of each item with respect to the
corresponding profile is therefore computed and a ranking of relevant items to
be recommended is subsequently generated.

This approach is used for most recommendation contexts (books, films, music,
etc.). There are, however, other situations where items are associated with a
group of documents rather than a single text, such as expert finding Lin et
al. (2017), whereby experts on a certain subject are recommended according to
a set of documents that define their expertise to any user who requires them,
or publication venue recommendation Wang et al. (2018), where journals or
conferences are recommended to scientists who wish to know where to publish a
paper. In this case, the venues comprise a series of articles published there.
In these two cases, item profiles might be built to define their specific
informational context and subsequently used in the recommendation stage. This
context in which the items are also described by means of profiles compiled
from the associated documents represents the general premise of this paper.

As we have already mentioned, a large proportion of the user profile consists
of (weighted) terms or keywords, i.e. the words included in the textual
descriptions of items associated to a number that reflects the importance of
each term in the item and/or in the entire collection of terms. Other
alternatives have, however, been explored in the literature in terms of
profile construction such as the inclusion of topical and temporal dimensions.

Regarding such topic-based profiles, in a first approximation, these comprise
topics, concepts or categories rather than terms. They may be easily
constructed by including the topics of interests or categories of an item (for
example, in a recommender system for the digital edition of a journal or
magazine, categories such as sport, local or finances, for example, are
associated with each news item, and these may be directly moved to its
profile). In a second approach, however, the items could be represented by
several subprofiles rather than a single one. In this case, textual
subprofiles could be created for each topic, incorporating the text of all the
associated documents about this topic. Let us imagine a simple situation in
the context of expert finding in academia. Let us suppose that an expert, a
researcher, has published articles about RS, IR, personalisation and
applications. The researcher’s profile would consequently consist of four
subprofiles, each containing the texts of the articles in that area of
expertise. Each item, therefore, would be represented by several subprofiles
that are more topically homogeneous, as documents are not mixed in one profile
(heterogeneous) but grouped according to topic of interest. This method of
organisation would clearly increase the interpretability of these
(sub)profiles. This topical dimension can be automatically incorporated to the
profiles by mining the texts in situations in which these categories are not
clearly available, such as, for example, learning topic models using Latent
Dirichlet al.location (LDA) de Campos et al. (2021), Jelodar et al. (2019) or
by applying clustering algorithms de Campos et al. (2020).

In situations where time is a feature included in those documents that define
items (such as any type of timestamp), this dimension could be incorporated in
different ways into the recommendation Campos et al. (2014). Firstly, temporal
subprofiles may be built by dividing the temporal line of documents into
periods, grouping them into each one of these and thereby building the
corresponding subprofiles. The homogeneity of the profiles (in this case
temporal) is also present in the item profiling process. This approach could
be considered as a generalisation of the well-known profiles based on long and
short-term preferences. Secondly, but unrelated to profiles, another option is
to include time in the recommendation process, by means of the application of
a decay factor that penalises older items.

Bearing in mind these topical and temporal dimensions for building homogeneous
subprofiles, a question immediately arises: would it make sense to combine
both aspects in order to improve the quality of the recommendation in terms of
system effectiveness? Since the state of the art shows that it is suitable,
two innovative methods for carrying out such an integration are presented and
evaluated in this paper and these represent the main contribution of this
article. Our goal is, therefore, to determine whether this mixture is valuable
in comparison with other non-hybrid alternatives. For this purpose, this study
will address the following research questions:

  * •

RQ1: Does a temporal division of the documents associated to items provide a
reliable source for constructing high quality profiles?

  * •

RQ2: Can decay-based techniques which penalise older documents be successfully
incorporated?

  * •

RQ3: To what extent is building item profiles based on latent topics in the
document collection an added valued for the recommendation problem?

  * •

RQ4: Is hybridisation, i.e. the combination of topical and temporal aspects
for creating item profiles, a good alternative for the problem at hand?

  * •

RQ5: Which method is the best form of hybridisation?

Although the profiling proposals presented in this paper may be applied to any
type of item collection represented by text in the context of CBR, evaluation
is performed in the field of publication venue recommendation, i.e. given a
paper which is about to be submitted to a certain venue for revision, the
recommender system would suggest the most suitable journals, conferences or
general scientific events based on the subject of the paper. The CBR system
will build subprofiles for venues, starting from the collection of published
articles, which will be matched against the text of the submitted article.

The remainder of this paper is organised as follows: Section 2 briefly
discusses other related works; Section 3 introduces the different methods for
creating (sub)profiles based on terms, topics and time, and for combining
them; Section 4 focuses on the experimental part of the work, including
results and discussion; and, finally, Section 5 details our conclusions and
outlines our future lines of research.

##  2 Related Work

Starting with the simplest form of representing items or user profiles, the
method usually adopted is to compile a list of weighted terms which are
automatically extracted from the document associated with them Gauch et al.
(2007). These terms are supposed to correctly represent the document subjects
and the weights are responsible for measuring their importance in terms of the
entire document collection and within each document. Some examples of the use
of term-based profiles for recommendation reflect how widely they are used in
CBR and these include TV programme recommendation Wartena et al. (2011),
expert finding de Campos et al. (2020), treatments for patients in a health RS
Bateja et al. (2018), tweet recommendation Benzarti and Faiz (2016), and image
recommendation Karlsen et al. (2018).

An alternative way of representing profiles to keywords is through the use of
tags, concepts, categories or topics. We could say that these are higher-level
features which could symbolise concepts and try to capture the underlying
semantics of the items. Since the profiles comprise more general concepts
rather than just words, certain authors believe that this is beneficial for
the quality of the recommendation Firan et al. (2007), at least in the case of
tags. There are a number of examples of experiences that build and recommend
based on tag profiles Bogers (2018), Aliannejadi and Crestani (2018),
Stakhiyevich and Huang (2019), Yan et al. (2020), Becerra et al. (2017). Tags
are assigned to user profiles either manually or on the basis of a machine
learning-based approach. The use of concepts is discussed in a number of
papers Ren et al. (2015), Narducci et al. (2016), Sharma et al. (2017), Simsek
and Karagoz (2020). In most cases, the concepts are extracted from ontologies
or concept graphs giving the information associated to items or users. With
respect to topic-based profiles Saraswatm et al. (2020), these comprise latent
topics mined directly from document collections, typically using the LDA
algorithm or extensions of it. Starting with user-associated texts, the
profile is fed with the most probable topics associated to the words contained
in them. Certain papers illustrate the use of topic-based profiles to CBR in a
wide variety of problems Chen et al. (2017), Huang and Wu (2019), de Campos et
al. (2021), Fu et al. (2021), Khan et al. (2021). Although the underlying
representation based on topics is very appropriate for representing profiles,
it does, in fact, lack the interpretability offered by terms, tags or
concepts. A translation from topics to human-understandable labels is needed
and this requires an additional effort.

All these previous item or user profiles are monolithic in the sense that all
the possible facets of interests are combined into a single profile. Another
alternative is to consider profiles as comprising different subprofiles, each
associated to a possible facet, thereby capturing the various underlying, non-
explicit topics which are usually extracted by machine learning algorithms
from the associated texts of users and items. These multi-faceted profiles are
no longer flat although they may have different shapes: trees, representing
personal data, expertise and interests Pavan and Luca (2015); graphs of
clusters capturing different facets from different sources Zeng et al. (2018);
two subprofiles to capture user interests and friends’ interests Gulla et al.
(2014); subprofiles comprising subsets of items rated by the user and which
are used to improve the diversity of the recommendations Kaya and Bridge
(2019); different types of subprofiles, each containing keywords, concepts and
tags Narducci et al. (2013), or hierarchies of weighted topics Kook (2005).
Clustering is the usual technique for creating such multi-faceted profiles.
This unsupervised learning is applied to the document collection resulting in
clusters of documents or keywords, which will integrate the profiles as
subprofiles. Each cluster would represent a concept in the entire collection.
There are a number of papers which cover this methodology Somlo and Howe
(2001), McGowan (2002), Yeung et al. (2009), Amini et al. (2014), de Campos et
al. (2020, 2021).

In the research presented in this paper, topic-based profiles will comprise
subprofiles which represent different concepts but rather than containing a
list of topics, they contain terms, i.e. those from the documents associated
with the topics.

Much has been published on temporal dynamics, i.e. the inclusion of time in
recommendation, and this has mainly focused on CF Campos et al. (2014). One of
the most common approaches is to use decay functions to penalise old items and
reward new ones Ding and Li (2005), Yeniterzi et al. (2015). A second
alternative is to include time in the computation of item weights Linda and
Bharadwaj (2019). Another possibility is to integrate time into the rating
matrix in CF and use it to find trusted relationships between users Ngaffo et
al. (2021). Another research line is to consider time frames: in the article
Ramos and Paraboni (2014), the authors propose a CBR system for tweets, where
a specific time frame is learned for each user and only tweets within this
personalised frame are recommended. The same idea has been used by other
authors Si et al. (2017) but for points-of-interest recommendation. One
generalisation is the use of long and short-term profiles as another option to
include time and take into account the users’ most recent interests in
contrast to those which were acquired by interacting with the system some time
ago Li et al. (2011), Xiang et al. (2010). The time domain is included in our
models simply by splitting the documents into time periods of equal size
rather than using long and short-term profiles. Within each time period, the
topic subprofiles are learnt.

Finally, this review of related work will examine the combination of
topicality and temporality in profiles by taking advantage of both dimensions
in order to improve recommendation. This combination is performed by following
a wide range of methods which are outlined in the papers mentioned, but the
most usual way is to apply a latent topic discovery algorithm to the available
text collections, obtain the topics associated to each document and
incorporate time by means of weights associated to topics. Other authors use
decay functions Wangwatcharakul and Wongthanavasu (2021) to mitigate the
impact of old ratings. They also use item reviews in order to obtain the
underlying topics in the collection and associate the rated items to the
corresponding topics in the reviews in order to track how the topics evolve
with time. Finally, they propose an optimisation method to make predictions.
In the article Li et al. (2014) about news recommendation, the authors build
long and short-term profiles. While long-term profiles comprise latent topics
extracted from LDA from a collection of news and weighted by considering a
time decay function to capture how they evolve in time, short-term profiles
comprise topics occurring in documents in the most recent period of time. In
the article Ying et al. (2015), their authors describe a method for the
context of social media to combine interests and temporal context. It is based
on mixing a latent class statistical mixture model to represent topic
distributions not only from users’ interests (user-oriented topics) but also
from a temporal context (time-oriented topics). They also compute the
distribution of topics for items. With all of this information, they are able
to model different users’ interests in different time periods. In Liu (2015),
it is considered the interaction of each user $u$ with each item $i$ in a
given time. LDA is applied to extract topics from the set of textual
representations of all these interactions, which are represented by a topic
distribution. For a given user, once all of their interactions have been
sorted chronologically, the assigned topics are modelled as a time series.
Recommendation takes place when a Gaussian process predicts the value of each
topic at a given time and similarities are computed between the predicted
topic distribution and the distribution associated with each user. In another
article Neshati et al. (2017) about community question answering, the authors
introduce a method for future expert finding which suggests the most suitable
experts for the future. In order to do so, they first apply LDA in order to
extract topics from documents associated to experts, and their corresponding
timestamps, and calculate the probability of a future expert candidate for a
given query. In the context of social media Nishioka and Scherp (2016), users’
interests are extracted from social media streams. Profiles are then built
using weighted concepts, which are the topics obtained by applying LDA to the
collection. Items are also indexed using concepts and matched to user
profiles. Recommendation is carried out by computing a similarity between user
profiles and item profiles. Publication times are also taken into account by
means of decay functions, which penalise the older topics and are included in
the topic weights. In Zeng et al. (2018), it is built temporal user profiles
by directly incorporating time into the LDA algorithm, thereby obtaining topic
distributions for words and times, as in the case of timeSVD++ is Koren
(2010). The last two papers on expert finding use concepts rather than topics
from LDA. In Rybak et al. (2014), the profiles consist of weighted concepts,
where the weights represent the degree of expertise in each concept. The
temporal expertise profile is a set of single profiles which are computed at
different periods of time, and a decay function is incorporated into the
calculation of the concept weights. In Ziaimatin et al. (2012), while short-
term profiles are built by extracting and weighting concepts from an ontology
over given time periods, long-term profiles are built by detecting the
concepts which are uniformly distributed in the short-term profiles.

The way in which temporality and topicality are combined in this paper is a
contribution to the state of the art. In most cases where LDA is used, it has
been applied globally to the entire document collection. In this research, LDA
is applied locally only to those documents belonging to the same period of
time when both dimensions are combined.

##  3 Alternatives to profile construction based on terms, topics and time

###  3.1 Term-based Profiles

In this paper, and in the context of CBR, we assume that each item to be
recommended has an associated set of text documents. For example, in terms of
the context of our experimentation in this paper, we are dealing with journal
recommendation, where the journals are the items and all the articles
published in each one are linked to it. The active user, in this case a
researcher, would be interested in knowing possible journals where they could
publish a recently written paper. A second example might be the so-called
Expert Finding area Balog et al. (2012), Lin et al. (2017). In this case, the
“items” are experts in an area (for instance, researchers, lawyers or
politicians) and their documents are all the possible associated texts (e.g.
scientific articles or web pages relating to their knowledge or expertise,
court cases they were involved in, or their parliamentary interventions,
respectively). The aim would be to recommend experts to people who need their
services according to their needs.

Formally, let $I=\\{i_{1},\ldots,i_{r}\\}$ be the set of items to be
recommended. Linked with each of these, $i\in I$, is a set of $n_{i}$ text
documents $D^{i}=\\{d^{i}_{1},d^{i}_{2},\ldots,d^{i}_{n_{i}}\\}$.

Each item will also be represented by a profile that contains in one way or
another the content of its related documents (the terms appearing in them).
These profiles can basically be organised in one of two ways:

  * •

Monolithic profiles: where all the documents linked to each item $i$ are
concatenated to create a single document, $d^{i}=\cup_{j=1}^{n_{i}}d^{i}_{j}$.
This macro document will act as a unique profile $p_{Mono}^{i}$ for item $i$,
i.e. $p_{Mono}^{i}=\\{d^{i}\\}$.

  * •

Atomic subprofiles: where for item $i$, its profile will comprise as many
subprofiles as documents attached to it but in an isolated, unconcatenated
way, i.e. $p_{Atom}^{i}=\\{d^{i}_{1},d^{i}_{2},\ldots,d^{i}_{n_{i}}\\}$ (each
document is treated as a subprofile in itself).

The collection of items is then represented by a set of profiles,
${\mathcal{P}}=\\{p^{i_{1}},\ldots,p^{i_{r}}\\}$, which will serve as
retrieval units in this context of CBR systems. A graphical representation of
this profile construction process is shown in Figure 1.

Figure 1: Building monolithic and atomic (sub)profiles

These two basic ways of classifying profiles as either monolithic or atomic
correspond in expert finding literature with the so-called profile-based
methods and document-based methods, respectively Balog et al. (2012). There is
no general agreement about which method is preferable, and although document-
based methods tend to be considered better than profile-based methods,
profile-based methods perform better in certain cases de Campos et al. (2015),
Liu et al. (2015)). We will therefore use both approaches as baselines in our
experiments.

We believe that there is room for improvement between the complete atomisation
of isolated and homogeneous subprofiles and the extreme compaction of
monolithic and heterogeneous profiles, and that we can create less extreme
ways of organising the information relating to each item.

###  3.2 Topical profiles

One first alternative for organising an item’s subprofiles (out of the two
basic organisational schemes presented in the previous section) would be to
build more homogeneous subprofiles around the different concepts or topics
which can be identified in the entire collection of text documents associated
to the items. The construction of subprofiles from a topical perspective can
be based on a partition of the document collection by means of a clustering
algorithm which uses the documents’ terms as features. This would identify the
different clusters of documents according to their subjects, placing all the
conceptually-related documents in the same cluster. For an item $i$, each
subprofile will correspond to the concatenation of the documents associated to
$i$ which are assigned to the same cluster, and this results in a set of
topically homogeneous subprofiles. It is apparent that while this clustering
process is global in that it is carried out with the entire document
collection (not with the documents associated to an item), the subprofile
construction is local, as the subprofiles only contain the text of documents
associated to the item $i$.

Although there are many ways of performing this clustering process, in this
paper we will use LDA (Latent Dirichlet Allocation) Blei et al. (2003) for
this purpose. LDA finds $k$ latent topics, $x_{1},x_{2},\ldots,x_{k}$, in a
document collection222The number of topics, $k$, is an input parameter of
LDA., where each topic $x_{l}$ is characterised by a conditional probability
distribution of terms, $p(t\mid x_{l})$, and determines for each document $d$
a probability distribution of topics, $p(x_{l}\mid d)$. For example, in the
context of recommending Computer Science journals, we could highlight the fact
that the articles published in all the journals deal with five different
topics (to clarify the example, these will be called clustering,
classification, regression, association, and feature selection). One document
might deal only with classification (100%), a second one might mainly be about
feature selection (70%) but may also discuss classification as a secondary
topic (30%), a third one might mostly cover regression (90%) but also briefly
touch on feature selection (10%), and a fourth one might cover all the topics
equally (20% each), as it could be an introduction to Machine Learning.

The clustering generated by LDA obtains $k$ clusters, one for each topic
$x_{l}$, where each document is assigned to its most probable topic. For each
item $i$, there are therefore at most333If an item $i$ has no associated
documents about the main topic of $x_{l}$, then the corresponding subprofile
does not exist for that item. $k$ subprofiles, each containing the
concatenated text of those documents associated to $i$ on the main topic of
$x_{l}$. Figure 2 illustrates the entire process for generating the topical
subprofiles.

Figure 2: Building topical profiles

In terms of a more formal description, given the set of all the documents
${\mathcal{D}}=\cup_{i=1}^{r}D^{i}$, each cluster, ${\mathcal{D}}_{l}$,
$l=1,\ldots,k$, consists of the documents of the items which are associated to
the $l$-th topic, $x_{l}$ (documents where the most probable topic is
$x_{l}$), that is to say:

${\mathcal{D}}_{l}=\\{d^{i_{j}}_{m}\,\mid\,x_{l}=\arg\max_{s=1,\ldots,k}p(x_{s}\mid
d^{i_{j}}_{m}),\;j=1,\ldots,r,\,m=1,\ldots,n_{i_{j}}\\}$ (1)

From these sets of documents relating to each of the $k$ topics, the
subprofiles of each item $i$ must be constructed by grouping the documents in
each global cluster that are associated to this item $i$, thereby obtaining a
local cluster, $D^{i}_{l}={\mathcal{D}}_{l}\cap D^{i}$. Each item will
therefore have assigned as many subprofiles as local clusters have been
generated for it. The subprofiles of item $i$ are then documents, $d^{i,l}$,
built by concatenating the documents within each local cluster $D^{i}_{l}$,
$d^{i,l}=\cup_{d^{i}_{j}\in D^{i}_{l}}d^{i}_{j}$. In this case, the topical
profile for an item $i$ is $p_{Top}^{i}=\\{d^{i,1},d^{i,2},\ldots,d^{i,k}\\}$.

###  3.3 Temporal profiles

Another alternative for organising item subprofiles assumes that all the
documents relating to any item have an assigned date (e.g. a publication
date). We could then sort them accordingly and establish certain temporal
divisions to exploit the temporal dimension. From each period, homogeneous
subprofiles (in a temporal sense) could be built to represent an item by
grouping together all the documents associated to that item which belong to
the same period.

For example, in the context of scientific venue recommendation, imagine the
situation of a journal about machine learning with an extensive record in the
publication of articles. As expected, there are times when there are a large
number of papers on the same topic. After a while, new research topics appear
and as these are published, they displace and possibly replace some of the
existing topics. Initially, the majority of work published in journals of this
kind dealt with neural networks but after a few years, researchers became more
interested in support vector machines, and nowadays the focus has shifted
towards deep learning and this is currently one of the most published topics.
Splitting the time line into different intervals and building subprofiles for
the corresponding journal in each period is an alternative and simple way of
endorsing a temporal perspective to subprofiles and reflects how the focus of
the papers has changed over time.

In order to formalise this idea, let us consider the $h+1$ time points
$t_{0}<t_{1}<\ldots<t_{h}$ and the $h$ temporal intervals
$[t_{u-1},t_{u}),\,u=1,\ldots,h$. If $date(d)$ is a function that returns the
date of document $d$, then the $h$ global temporal clusters
${\mathcal{T}}_{u}$ are defined as follows:

${\mathcal{T}}_{u}=\\{d^{i_{j}}_{m}\,\mid\,t_{u-1}\leq
date(d^{i_{j}}_{m})<t_{u},\;j=1,\ldots,r,\,m=1,\ldots,n_{i_{j}}\\}.$ (2)

The local temporal clusters for each item $i$ are built as in the case of
topical clusters by grouping the documents associated to $i$ that belong to
each global cluster ${\mathcal{T}}_{u}$, i.e. $T^{i}_{u}={\mathcal{T}}_{u}\cap
D^{i}$. Each subprofile for item $i$ concatenates the documents in $T^{i}_{u}$
to form a single document $d^{i\mathchar 24635\relax\;u}=\cup_{d^{i}_{j}\in
T^{i}_{u}}d^{i}_{j}$. Each item, therefore, is now represented by at most $h$
temporal subprofiles444This is because it is possible for certain items not to
have any associated documents in a given period.. The temporal profile for
item $i$ in this case is $p_{Temp}^{i}=\\{d^{i\mathchar
24635\relax\;1},d^{i\mathchar 24635\relax\;2},\ldots,d^{i\mathchar
24635\relax\;h}\\}$.

###  3.4 Hybrid profiles: Combining topical and temporal profiles

The next natural step is to combine both previous ways of constructing
profiles by simultaneously exploiting the topical and temporal dimensions in
order to obtain homogeneous subprofiles in terms of these two properties. This
homogeneity could be reached in two different ways:

  * •

By first discovering the underlying global topics within the whole collection
and creating their corresponding clusters (topical division), as explained in
Section 3.2, and secondly by splitting them into temporal units (temporal
division), from which the final subprofiles will be constructed. This approach
is then a topical-temporal one.

  * •

By first splitting the collection into temporal units (temporal division),
secondly by discovering the underlying topics within each single temporal
partition (locally), and finally by building the final subprofiles from them.
In this case, this is a temporal-topical approach. It should be noted that in
each temporal division, the discovered topics would be different.

More specifically, for the topical-temporal approach, let ${\mathcal{D}}_{l}$
be defined as in Equation (1), $l=1,\ldots,k$, and ${\mathcal{T}}_{u}$ be
defined as in Equation (2), $u=1,\ldots,h$. The global topical-temporal
clusters ${\mathcal{DT}}_{lu}$ are then defined as

${\mathcal{DT}}_{lu}={\mathcal{D}}_{l}\cap{\mathcal{T}}_{u},\;l=1,\ldots,k,\;u=1,\ldots,h.$
(3)

Given an item $i$, its local clusters are obtained by joining the documents
found in each global topical-temporal cluster which are associated to $i$:
$DT^{i}_{lu}={\mathcal{DT}}_{lu}\cap D^{i}$. As in the previous cases, the
documents within each local cluster are concatenated to form the corresponding
subprofiles, $d^{i,l\mathchar 24635\relax\;u}=\cup_{d^{i}_{j}\in
DT^{i}_{lu}}d^{i}_{j}$. The topical-temporal profile for item $i$ is
$p^{i}_{TopTemp}=\\{d^{i,1\mathchar 24635\relax\;1},\ldots,d^{i,k\mathchar
24635\relax\;h}\\}$.

In the case of the temporal-topical approach, starting from each
${\mathcal{T}}_{u}$ as defined in Equation (2), $u=1,\ldots,h$, as the
document collection, we use LDA to obtain the $k$ topics
$x_{u1},\ldots,x_{uk}$ corresponding to this collection555It should be noted
that we must apply LDA to every $h$ temporal subcollections of documents,
thereby obtaining specific topics for each time period.. We then proceed in
the same way as with the topical profiles, i.e. we obtain the clusters
${\mathcal{TD}}_{ul}$ as the subset of documents of ${\mathcal{T}}_{u}$ where
the most probable topic is $x_{ul}$:

$\displaystyle{\mathcal{TD}}_{ul}=\\{d^{i_{j}}_{m}\in{\mathcal{T}}_{u}\,\mid\,x_{ul}=\arg\max_{s=1,\ldots,k}p(x_{us}\mid
d^{i_{j}}_{m}),\;j=1,\ldots,r,\,m=1,\ldots,n_{i_{j}}\\},$ (4)
$\displaystyle\;l=1,\ldots,k,\;u=1,\ldots,h.$

We then obtain the clusters associated to each item $i$ as
$TD^{i}_{ul}={\mathcal{TD}}_{ul}\cap D^{i}$. Finally, the documents in
$TD^{i}_{ul}$ are concatenated to build the subprofiles, $d^{i\mathchar
24635\relax\;u,l}=\cup_{d^{i}_{j}\in TD^{i}_{ul}}d^{i}_{j}$. The temporal-
topical profile for item $i$ is $p^{i}_{TempTop}=\\{d^{i\mathchar
24635\relax\;1,1},\ldots,d^{i\mathchar 24635\relax\;h,k}\\}$.

It is worth noting that in both cases in this process, each item $i$ will have
associated at most $k*h$ subprofiles because there might not be any document
associated to $i$ which deals with a specific topic at a given time period.

##  4 Evaluation and results

In this section, we will detail everything relating to the evaluation of the
previously explained alternatives for item profiling and also the results
obtained.

The context on which this experimentation is based is publication venue
recommendation. Given an article (or at least an abstract, title and
keywords), the problem is to recommend to the active user the most suitable
venue for publishing such a paper on account of the suitability of the scope
of the journal. In this context, the items to be recommended are journals and
the text documents associated to the items are the articles published in these
journals.

The following sections will present all the details of the experimental design
and also the results of the experiments.

###  4.1 Test collection

The test collection used in the experimentation is called PMSC-UGR Albusac et
al. (2018) and has been created by the authors from PubMed and Scopus. It
originally contained $762,508$ articles from $12,396$ journals in the
biomedical domain, with a title, abstract, keywords, citations and authors for
each paper. Out of all the authors from this selection of papers, those who
were unequivocally represented by their corresponding ORCID codes were finally
selected, leaving a total of $20,406$ authors.

###  4.2 Recommendation model

Once all the (sub)profiles666All of the methods presented (with the exception
of the one based on monolithic profiles) generate subprofiles, i.e. several
subprofiles per item. The monolithic approach builds exactly one profile per
item and this is why we write (sub)profiles. relating to each item (journal)
have been built, the IR technique is used to make content-based
recommendations, regardless of the method applied. First, all these
(sub)profiles777It should be remembered that each subprofile is a text
document. are indexed by an IR system. In our case, we have developed an
indexing programme based on the Lucene library888https://lucene.apache.org/.
Previously, this piece of software removes stop words and performs stemming,
indexing only the resulting stems.

Given an active user represented by any kind of textual source, the objective
is to obtain a ranking of relevant items. Considering that CBR is based on IR,
an IR model will be responsible for retrieving relevant items in respect of
the user information need and this serves as a query for the IRS. More
specifically, the textual content of a paper to be published by the user
represents a query submitted to the IRS. By using the Lucene implementation of
the Language Model (with Jelinek-Mercer smoothing), a retrieval programme
computes a ranking of journal (sub)profiles which are sorted decreasingly
according to their relevance to the query.

As mentioned in the previous paragraph, the ranking consists of subprofiles.
However, since the active user requires journal recommendations, the
subprofile ranking should then be transformed into a journal ranking. For this
purpose, a final fusion process combines the scores of the subprofiles for
each journal and generates a final journal ranking. This is performed by means
of the CombLgDCS fusion method de Campos et al. (2017), which aggregates the
scores of all of the journal subprofiles, decreasing them proportionally to
the logarithm of their positions in the ranking. This fusion is not necessary
for monolithic profiles since in this case there is only one profile per
journal.

###  4.3 Discovering latent topics

During this process, terms that appear in fewer than $750$ documents are
ignored and also those that appear in more than $90\%$. Finally, only a
maximum of 5000 of the most frequent terms in the corpus have been considered
as LDA input. These figures correspond to previous experiments to set the most
suitable size of the vocabulary. Nevertheless, the subprofiles built with this
technique contain all the terms from the original documents.

###  4.4 Baselines

In this research, two methods of building profiles are considered as
baselines: monolithic profiles and atomic subprofiles. Not only are these the
simplest ways of organising the textual information without taking into
account topical and temporal features, they are also the most extreme
alternatives whereby all the articles in the journal comprise the same profile
(with only one profile per journal) or each individual article forms a
subprofile (there will be as many subprofiles as articles). It is therefore
hoped that these more complex ways of structuring profiles can improve the
quality of CBR.

###  4.5 Experimental design

For evaluation purposes, we have restricted the PMSC-UGR collection to those
papers published between 2007 and 2016 which appear in journals with more than
100 papers in this period999We have removed the journal PlosOne from this set
because it has a much greater number of papers than the others., leaving the
dataset with a total of $1002$ journals. The collection has then been split
into two partitions: the first comprises articles dating from 2007 to 2015 (a
total of $276,679$ papers) and this will be reserved for building the
(sub)profiles and serve as the training set; and the second only contains
articles from 2016 ($32,864$ articles) and this will be used as the test set.
This holdout method is suitable for this evaluation and does not require
cross-validation to obtain reliable results given the large number of articles
in the test set. Each article from the test partition will be considered as a
query to be submitted to the underlying IRS and this query will comprise the
combined text of its title, abstract and keywords.

Starting from the training partition, the following types of (sub)profiles
will be considered for our experiments:

  * •

Monolithic profiles ($Mono$)

  * •

Atomic subprofiles ($Atomic$)

  * •

Topical subprofiles ($Top$)

  * •

Temporal subprofiles ($Temp$): These are built from four temporal partitions
of two years each (2007–2008, 2009–2010, 2011–2012 and 2013–2014) and one of
only one year (2015), i.e. $h=5$.

  * •

Topical + Temporal ($TopTemp$): After applying a global LDA to the entire
article collection, subprofiles are built in the five temporal partitions.

  * •

Temporal + Topical ($TempTop$): Subprofiles are built from the five temporal
partitions after applying local LDA to the article collection in each
partition.

In addition, $10$ instances of randomised subprofiles have been created
($Random1$, $\dots$, $Random10$). Each comprises $5$ random partitions of the
articles from the training set, thereby replacing the 5 temporal subprofiles
with 5 random subprofiles. The retrieval effectiveness of these data sets will
also be measured in order to test whether the temporal divisions differ from
randomness.

For the LDA algorithm on which the $Top$, $TopTemp$ and $TempTop$ approaches
are based, it is necessary to set up the $k$ parameter, i.e. the number of
latent topics to be discovered. This is not an easy task because the quality
of the results could be very high depending on this value. In this research,
we have considered three different values, two of which are related to medical
categories or specialities:

  * •

number of comprehensive medical specialities extracted from the Medical School
blog at St George’s University101010https://www.sgu.edu/blog/medical/ultimate-
list-of-medical-specialities, $k=20$

  * •

number of second-level categories of the MESH
thesaurus111111https://meshb.nlm.nih.gov/treeView, $k=110$

  * •

$k=400$, in order to test a very large number of topics

It is apparent that these three values attempt to cover a wide range of
topics, from a low to a relatively large number of them, in order to evaluate
the performance of the different types of profiles according to the number of
topics discovered by the LDA algorithm. The underlying idea is also to choose
meaningful values of $k$ relating to medical categories.

For the truly temporal approaches, i.e. $Temp$, $TopTemp$ and $TempTop$, and
for $Atomic$, which also supports a temporal treatment, a temporal decay might
also be considered to reduce the influence of older articles or subprofiles.
This is a complementary way of introducing the temporal dimension into this
journal recommendation problem. The underlying idea is that a user would be
more interested in journals where their last published articles are closer to
the paper to be published rather than “older” articles. In order to implement
this idea, once a query has been submitted (the text of the article) and the
score or Relevance Status Value (RSV) for each article or subprofile has been
computed, a decay function is applied in order to modify the corresponding RSV
according to the temporal distance to 2015, which is the year of the “newest”
papers in our collection. Several decay functions might be found in the
literature, for example those presented in Lerrain et al. (2015), but in our
case, two have been implemented to be tested:

  * •

linear: $RSV_{LinearD}=\frac{NRSV}{1+Penalty}$

  * •

doubled squared root: $RSV_{2SqrtD}=\frac{NRSV}{\sqrt{\sqrt{1+Penalty}}}$

where $NRSV$ is the normalised $RSV$, computed by dividing the corresponding
$RSV$ by the maximum of the ranking and $Penalty$ = $2015$ \- $Publication$
$year$. For individual articles in the atomic subprofiles, their publication
year is used directly to compute the new score, whereas for the remaining
temporal subprofiles, their average value is incorporated into the decay
formulas since they contain articles over two years.

Although other decay functions were implemented and tested in this study,
these two were selected in order to show two different penalisation behaviours
as it is evident that the doubled squared root decay penalises older articles
more smoothly than the linear one. The other functions presented behaved in a
similar or even worse way and that is why these two are included as
representative.

A third value of the decay parameter would be ”None”, which means that no
temporal penalisation is considered in the RSV.

###  4.6 Evaluation measures

In order to evaluate the recommendation, and before presenting the evaluation
measures, it is important to determine the ground truth: in this case, only
one journal is relevant for each query (test article) and this is the journal
where the paper has actually been published.

The following evaluation measures found in the literature to determine the
quality of the results obtained by venue recommendation methods are the most
common:

  * •

Recall@X (R@X): This measures the ability of recommending the relevant journal
where the test article has been published in the first $X$ journals in the
ranking (recommended venues). In other words, we compute the average number of
times where the actual venue where a test paper was published is among the
first $X$ recommended venues. In previous work on venue recommendation (e.g.
Luong et al. (2012), Medvet et al. (2014), Wang et al. (2018)), this measure
is also called accuracy@X. Two thresholds $X$ are considered to compute the
values of this measure: $1$ and $5$. $X=1$ is considered because the number of
relevant journals is $1$ and so we would like to know how successful it would
be to recommend only one journal. Since the user does not usually obtain only
one recommendation but a number of alternative journals, $X=5$ is also used as
the threshold.

  * •

Mean Reciprocal Rank (MRR@Y): In this case, the idea is to reflect how high in
the ranking the only relevant journal is recommended. It therefore computes
the average of the inverse of the positions in the ranking for the journal
where each test paper was published. The total number of results (journals) in
the ranking is limited to $Y$ 121212If the relevant journal is not within the
first $Y$ journals in the ranking, then the value is 0.. We have only
considered the top 40 positions in the ranking, i.e. $Y=40$. In our specific
case where only an item (a journal) is relevant, this measure coincides with
mean Average Precision, MAP@Y.

###  4.7 Results

Table 1: Results of the experiments (the best results for each column are shown in bold) Subprofiles | #Topics | Decay | R@1 | R@5 | MRR@40
---|---|---|---|---|---
TempTop | 110 | 2Sqrt | 0.2501 | 0.5655 | 0.3935
TempTop | 20 | 2Sqrt | 0.2490 | 0.5619 | 0.3923
TempTop | 20 | None | 0.2466 | 0.5615 | 0.3903
TempTop | 400 | 2Sqrt | 0.2455 | 0.5596 | 0.3883
TempTop | 110 | None | 0.2451 | 0.5638 | 0.3898
TempTop | 400 | None | 0.2434 | 0.5575 | 0.3860
TopTemp | 400 | 2Sqrt | 0.2373 | 0.5462 | 0.3783
TopTemp | 110 | 2Sqrt | 0.2368 | 0.5463 | 0.3781
Top | 20 | None | 0.2348 | 0.5400 | 0.3760
Top | 110 | None | 0.2346 | 0.5448 | 0.3763
Top | 400 | None | 0.2345 | 0.5462 | 0.3771
Temp |  | 2Sqrt | 0.2341 | 0.5431 | 0.3758
TempTop | 110 | Linear | 0.2335 | 0.5307 | 0.3692
Atomic |  | 2Sqrt | 0.2331 | 0.5403 | 0.3731
TopTemp | 110 | None | 0.2331 | 0.5431 | 0.3747
TopTemp | 20 | 2Sqrt | 0.2329 | 0.5419 | 0.3746
TopTemp | 400 | None | 0.2328 | 0.5430 | 0.3746
TempTop | 400 | Linear | 0.2308 | 0.5266 | 0.3651
TopTemp | 400 | Linear | 0.2298 | 0.5179 | 0.3615
TempTop | 20 | Linear | 0.2294 | 0.5267 | 0.3657
TopTemp | 110 | Linear | 0.2290 | 0.5207 | 0.3618
TopTemp | 20 | Linear | 0.2286 | 0.5230 | 0.3629
TopTemp | 20 | None | 0.2286 | 0.5361 | 0.3699
Atomic |  | None | 0.2282 | 0.5370 | 0.3696
Temp |  | None | 0.2258 | 0.5330 | 0.3671
Atomic |  | Linear | 0.2255 | 0.5118 | 0.3558
Random2 |  | None | 0.2245 | 0.5340 | 0.3656
Random1 |  | None | 0.2243 | 0.5326 | 0.3658
Random6 |  | None | 0.2242 | 0.5330 | 0.3660
Random9 |  | None | 0.2241 | 0.5321 | 0.3653
Random3 |  | None | 0.2240 | 0.5319 | 0.3657
Random5 |  | None | 0.2239 | 0.5330 | 0.3656
Random4 |  | None | 0.2239 | 0.5333 | 0.3655
Random8 |  | None | 0.2238 | 0.5332 | 0.3655
Random0 |  | None | 0.2236 | 0.5335 | 0.3636
Random0 |  | None | 0.2236 | 0.5335 | 0.3653
Monolithic |  | None | 0.2236 | 0.5278 | 0.3653
Random7 |  | None | 0.2227 | 0.5314 | 0.3648
Temp |  | Linear | 0.2147 | 0.5159 | 0.3543
|  |  |  |  |

The results of our experiments are displayed in Table 1. Although the rankings
of methods obtained for the different performance metrics are not identical,
the trends are the same. In fact, if we calculate the Pearson correlation
between these rankings, we always obtain correlation coefficients which are
greater than 0.86 (if we exclude the results obtained by the random
subprofiles and also those which use linear decay, which are both quite poor,
then all the correlation coefficients are greater than 0.98). We can,
therefore, comment on our results without referring to any specific metric.

If we first focus on RQ1, we can conclude that the use of the temporal
dimension alone ($Temp$ subprofiles, without decay) only very slightly
improves the results obtained by the $Mono$ baseline. This implies that the
impact of dividing the monolithic profiles into several parts which are only
based on temporal criteria is limited. In addition, the $Atomic$ approach
performs almost the same as $Temp$. Splitting the collection into years and
building the subprofiles upon them does not, therefore, offer any clear
advantage. The random subprofiles, Random$i$ (with the same number of
subprofiles as $Temp$), perform almost identically to $Mono$ and worse than
$Temp$. This implies that the temporal dimension has a slightly positive
effect on the results which is not attributable merely to the fact of creating
several subprofiles.

When a decay factor is also used, very poor results are obtained for $Temp$
(even worse than the baselines) with the linear version and much better
results with the doubled squared root version (2Sqrt) (which is smoother than
linear). Consequently, the performance of $Temp$ clearly depends on the aid
that decay functions can provide. This behaviour of the three decay methods
(Linear $<$ None $<$ 2Sqrt) also persists when the temporal dimension is
combined with the topical dimension131313Although in this case, the difference
between None and 2Sqrt is smaller. or when it is applied to the other
baseline, $Atomic$, so that the preferred version of decay is always 2Sqrt.
$Temp$ is also better than $Atomic$ (using 2Sqrt decay). By way of conclusion,
a well-designed decay function incorporated into the recommendation process
can boost performance, and this therefore answers RQ2.

In terms of RQ3, the use of topical subprofiles ($Top$) clearly improves the
results of the baselines $Mono$ and $Atomic$, regardless of the number of
topics selected. Since there are no very important differences between the
results of $Top$ with a different number of topics, this parameter does not
seem critical for good behaviour. $Top$ is also better than $Temp$ without
decay and has a similar performance to $Temp$ with 2Sqrt. However, the
contributions of $Top$ and $Temp$ to improved performance seem to be based on
different premises, so that their combination could generate a kind of
synergy. This is indeed the case but it depends on the way the topical and
temporal subprofiles are combined. When $TopTemp$ subprofiles are used, i.e.
first a topical division and then the temporal division (and using the same
topics in every time period), we do not observe any clear improvement in the
results in terms of using only either $Top$ or $Temp$ (and only a miniscule
improvement is obtained). However, when we use the other proposed combination
of $TempTop$ subprofiles, i.e. first a temporal division and then the topical
division (with the topics being specific for each time period), a clear
improvement is apparent. Moreover, the $TempTop$ results are always the best
ones for all the metrics, regardless of the number of topics being considered.
We think that this is due to the fact that topic identification, and the
subsequent construction of subprofiles, is tailored to the set of articles
included in each temporal partition, which is more precise and totally adapted
to the content of such articles. Additionally, the $TopTemp$ approach is more
general and not so well fitted to the texts in each temporal partition.
Hybridising topical and temporal profiles is, therefore, a very interesting
approach but only if temporal divisions are made and subprofiles built based
on topic discovery in each time-based partition (RQ4). The absolutely best
results are obtained using $TempTop$ subprofiles with 110 topics and 2Sqrt
decay (RQ5).

Finally, and in order to try to verify these conclusions, a statistical
significance test has been applied for the measure $R@1$. More specifically,
the McNemar test McNemar (1947) was selected, which is a non-parametric test
for paired data, as recommended in Dietterich (1998) for comparing machine
learning algorithms. It has been run for families of subprofiles (for the
three values of $k$ and the best decay method, when applicable): $TempTop$
with 2Sqrt, $TopTemp$ with 2Sqrt and $Top$ without decay. The idea is to first
determine if there are significant differences in each family. The results of
these tests fulfil the same pattern for $TempTop$ and $TopTemp$ families:
there are no differences between the two top $k$ values and there is with the
worst. In the case of $TempTop$, there are no differences between $110$ and
$20$, but there are with $400$. For $TopTemp$, there are no differences
between $400$ and $110$, but there are for $20$. In terms of $Top$, there are
no differences between the three values of $k$. A final series of tests is run
between the best values of each family, including in this case the baselines:
$TempTop$, 110, 2Sqrt; $TopTemp$, 400, 2Sqrt; $Top$, 20, None; $Temp$, 2Sqrt;
$Atomic$, 2Sqrt; and $Mono$, None. The results show that there are significant
differences between $TempTop$ and the others and also between $Mono$ and the
others, and there are no differences between $TopTemp$, $Top$ and $Temp$. And
there are differences between $TopTemp$ and $Atomic$, but not between
$Atomic$, $Top$ and $Temp$. By way of summary, $TempTop$ is clearly the best
option for combining temporal and topical dimensions and $Mono$ is the worst
alternative.

##  5 Conclusions and Further Research

In this paper, we have focused on testing how useful temporality, topicality
and the combination of these are for the problem of building and using
profiles in content-based recommendation. We have proposed two different ways
of hybridisation. A biomedical journal collection on publication venue
recommendation was used to test our findings and these revealed that the
combination of these two types of approaches is a good alternative although it
is important to note that order matters in terms of performance. From our
experiments, we can conclude that the best option is to create temporal
partitions and discover the latent topics starting from the papers in each
partition using LDA, and then to construct the profiles. It is important to
mention that the number of topics for building the profiles is not a critical
parameter. The application of a decay factor might be a valuable aid but it
clearly depends on the quality of the penalising function. In our context,
2Sqrt helps to improve the performance of the recommendation with hybrid
subprofiles.

In terms of future lines of research, we plan to explore other methods of
combining temporal and topical dimensions to obtain better subprofiles. One
alternative is the use of temporal topic models Blei and Lafferty (2006),
Dieng et al. (2019). Another option is to use methods based on the aggregation
or fusion Wu (2012) of topical and temporal rankings individually obtained by
$Top$ and $Temp$, respectively. Another research line is to design high
quality decay functions that boost the performance of hybridisation. Finally,
we also plan to study the most suitable ways of explaining the recommendations
Tintarev and Masthoff (2012) offered by our models.

#### Acknowledgements

This work has been co-funded by the Spanish Ministerio de Economía y
Competitividad under project PID2019-106758GB-C31, the Junta de Andalucía and
University of Granada under project A‐TIC‐146‐UGR20 (Programa Operativo FEDER
Andalucía 2014-2200), and the European Regional Development Fund (ERDF-FEDER).

##  6 Statements and Declarations

#### Funding

This study was funded by the Spanish Ministerio de Economía y Competitividad
under project PID2019-106758GB-C31, the Junta de Andalucía and University of
Granada under project A‐TIC‐146‐UGR20 (Programa Operativo FEDER Andalucía
2014-2200), and the European Regional Development Fund (ERDF-FEDER).

#### Competing interests

The authors have no relevant financial or non-financial interests to disclose.

#### Author contribution

All authors contributed to the study conception and design. Material
preparation, programming, data collection and analysis were performed by the
three of us. The first draft of the manuscript was written collaboratively and
all authors commented on previous versions of the manuscript. All authors read
and approved the final manuscript.

## References

  * Adomavicius and Tuzhilin (2005) Adomavicius G., Tuzhilin A.: Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering 17(6), 734–749 (2005)
  * Albusac et al. (2018) Albusac C., de Campos, L.M., Fernández-Luna, J.M., Huete, J.F.: PMSC-UGR: A test collection for expert recommendation based on PubMed and Scopus. Lecture Notes in Artificial Intelligence 11160, Advances in Artificial Intelligence, CAEPIA 2018, 34-43 (2018)
  * Aliannejadi and Crestani (2018) Aliannejadi, M., Crestani, F.: Personalized context-aware point of interest recommendation. ACM Transactions on Information Systems 36(4O), 1–28 (2018)
  * Amini et al. (2014) Amini, B., Ibrahim, R., Othman, M.S., Selamat, A.: Capturing scholar’s knowledge from heterogeneous resources for profiling in recommender systems. Expert System with Applications 41(17), 7945–7957 (2014).
  * Yeung et al. (2009) Yeung, C., Gibbins, N. Shadbolt, N.: Multiple interests of users in collaborative tagging systems. In: Weaving services and people on the World Wide Web, Springer, 255–274 (2009)
  * Balog et al. (2012) Balog, K., Fang, Y., de Rijke, M., Serdyukov, P., Si, L.: Expertise retrieval. Foundations and Trends in Information Retrieval 6, 127-256 (2012)
  * Bateja et al. (2018) Bateja, R., Dubey, S., Bhatt, A.: A Patient-centric healthcare model based on health recommender systems. In: Recent Findings in Intelligent Computing Techniques, Springer, 269-276 (2018)
  * Becerra et al. (2017) Becerra, C., Gonzalez, F., Gelbukh, A.: LinearTag Models: Recommendations Using Linear User Profiles Based on Tags. Computación y Sistemas 21(1), 115-12 (2017)
  * Benzarti and Faiz (2016) Benzarti, S., Faiz, R.: EgoTR: Personalized tweets recommendation approach. In: Intelligent Systems in Cybernetics and Automation Theory, 227-238, Springer (2016)
  * Blei et al. (2003) Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet al.location. Journal of Machine Learning Research 3, 993-1022 (2003)
  * Blei and Lafferty (2006) Blei, D.M., Lafferty, J.D.: Dynamic topic models. Proceedings of the 23rd International Conference on Machine Learning, 113-120 (2006)
  * Bobadilla et al. (2013) Bobadilla, J., Ortega, F., Hernando, A., Gutiérrez, A.: Recommender systems survey. Knowledge-Based Systems 46, 109-132 (2013)
  * Bogers (2018) Bogers, T.: Tag-based recommendation. Lecture Notes in Computer Science 10100, 441-479 (2018)
  * Campos et al. (2014) Campos, P., Díez, F., Cantador, I.: Time-aware recommender systems: a comprehensive survey and analysis of existing evaluation protocols. User Modeling and User-Adapted Interaction 24, 67-119, (2014)
  * Chen et al. (2017) Chen, C., Meng, X., Xu, X., Lukasiewicz, T.: Location-aware personalized news recommendation with deep semantic analysis. IEEE Access 5, 1624-1638 (2017)
  * de Campos et al. (2015) de Campos, L.M., Fernández-Luna, J.M., Huete, J.F.: A lazy approach for filtering parliamentary documents. Lecture Notes in Computer Science 9265, 364-378 (2015)
  * de Campos et al. (2017) de Campos, L.M., Fernández-Luna, J.M., Huete, J.F.: Committee-based profiles for politician finding. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 25(Suppl. 2), 21-36 (2017)
  * de Campos et al. (2020) de Campos, L.M., Fernández-Luna, J.M., Huete, J.F., Redondo-Expósito, L.: Automatic construction of multi-faceted user profiles using text clustering and its application to expert recommendation and filtering problems. Knowledge-Based Systems 190, article number 105337 (2020)
  * de Campos et al. (2021) de Campos, L.M., Fernández-Luna, J.M., Huete, J.F., Redondo-Expósito, L.: LDA-based term profiles for expert finding in a political setting. Journal of Intelligent Information Systems 56(3), 529-559 (2021)
  * Dieng et al. (2019) Dieng, A., Ruiz, F., Blei, D.: The dynamic embedded topic model (2019), arXiv:1907.05545v2
  * Dietterich (1998) Dietterich, T.G.: Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation 10(7), 1895-1923 (1998)
  * Ding and Li (2005) Ding, Y., Li, X.: Time weight collaborative filtering. Proceedings of the 14th ACM International Conference on Information and Knowledge Management, 485-492 (2005)
  * Firan et al. (2007) Firan, C.S., Nejdl W., Paiu, R.: The benefit of using tag-based profiles. Proceedings of the LA-WEB conference, 32–41 (2007)
  * Fu et al. (2021) Fu, Y., Shen, B., Chen, Y., Huang, L.: TDMatcher: A topic-based approach to task-developer matching with predictive intelligence for recommendation. Applied Soft Computing 110 (2021)
  * Gauch et al. (2007) Gauch, S., Chandramouli, A., Micarelli, A.: User profiles for personalized information access. In: The Adaptive Web: Methods and Strategies of Web Personalization, 54-89. Springer, (2007)
  * Gulla et al. (2014) Gulla, J.A., Fidjestøl, A.D., Su, X., Castejon, H.: Implicit user profiling in news recommender systems. Proceedings of the 10th International Conference on Web Information Systems and Technologies 1, 185–192 (2014)
  * Huang and Wu (2019) Huang, X., Wu, F.: A novel topic-based framework for recommending long tail products. Computers and Industrial Engineering 137, 1-15 (2019)
  * Jelodar et al. (2019) Jelodar, H., Wang, Y., Yuan, C., Feng, X., Jiang, X., Li, Y., Zhao, L.: Latent Dirichlet al.location (LDA) and topic modeling: models, applications, a survey. Multimedia Tools and Applications 78, 15169–15211 (2019)
  * Karlsen et al. (2018) Karlsen, R., Elahi, N., Andersen, A.: Personalized recommendation of socially relevant images. Proceedings of the 8th International Conference on Web Intelligence, Mining and Semantics, Article number 41 (2018)
  * Kaufman and Rousseuw (1990) Kaufman, L., Rousseeuw, P.J.: Finding groups in data: An introduction to cluster analysis. John Wiley (1990)
  * Kaya and Bridge (2019) Kaya, M., Bridge, D.: Subprofile-aware diversification of recommendations. User Modeling and User-Adapted Interaction 29, 661–700 (2019)
  * Khan et al. (2021) Khan, M., Smyth, B., Coyle, D.: Addressing the complexity of personalized, context-aware and health-aware food recommendations: an ensemble topic modelling based approach. Journal of Intelligent Information Systems 57, 229–269 (2021)
  * Kook (2005) Kook, H.J.: Profiling multiple domains of user interests and using them for personalized web support. Proceedings of the International Conference on Intelligent Computing, 512–520 (2005)
  * Koren (2010) Koren, Y., Collaborative filtering with temporal dynamics. Communications of the ACM 53, 89-97, 2010.
  * Lerrain et al. (2015) Larrain, S., Trattner, C., Parra, D., Graells-Garridoe, E., Nørvåg, K.: Good times bad times: A study on recency effects in collaborative filtering for social ragging. Proceedings of the 9th ACM Conference on Recommender Systems, 269–272 (2015)
  * Li et al. (2011) Li, L., Zheng, L., Li, T., LOGO: a long-short user interest integration in personalized news recommendation. Proceedings of the 5th ACM conference on Recommender systems, 317–320 (2011)
  * Li et al. (2014) Li, L., Zheng, L., Yang, F., Li, T.: Modeling and broadening temporal interest in personalized news recommendation. Expert Systems with Applications 41, 3168-3177 (2014)
  * Lin et al. (2017) Lin, S., Hong, W., Wang, D., Li, T.: A survey on expert finding techniques. Journal of Intelligent Information Systems 49, 255-279 (2017)
  * Linda and Bharadwaj (2019) Linda, S., Bharadwaj, K.: A genetic algorithm approach to context-aware recommendations based on spatio-temporal aspects. In: Integrated Intelligent Computing. Communication and Security, 59-70, Springer (2019)
  * Liu (2015) Liu, X.: Modeling users’ dynamic preference for personalized recommendation. Proceedings of the 24th International Joint Conference on Artificial Intelligence, 1785-179 (2015)
  * Liu et al. (2015) Liu, X., Croft, W.B., Koll, M.: Finding experts in community-based question-answering services. Proceedings of the 14th ACM International Conference on Information and Knowledge Management, 315-316 (2005)
  * Lops et al. (2019) Lops, P., Jannach, D., Musto, C., Bogers, T., Koolen, M.: Trends in content-based recommendation. User Modeling and User-Adapted Interaction 29, 239-249 (2019)
  * Lu et al. (2015) Lu, J., Wu, D., Mao, M., Wang, W., Zhang, G.: Recommender system application developments: A survey. Decision Support Systems 4, 12-32 (2015)
  * Luong et al. (2012) Luong, H., Huynh, T., Gauch, S., Do, L., Hoang, K.: Publication venue recommendation using author networks publication history. Lecture Notes in Computer Science 7198, 426-435 (2012)
  * McGowan (2002) McGowan, J.P., Kushmerick, N., Smyth, B.: Who do you want to be today? Web personae for personalised information access. Web-Based Systems: Second International Conference, 514–517 (2002)
  * McNemar (1947) McNemar, Q.: Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 12(2), 153–157 (1947)
  * Medvet et al. (2014) Medvet, E., Bartoli, A., Piccinin, G.: Publication venue recommendation based on paper abstract. Proceedings of the 26th IEEE International Conference on Tools with Artificial Intelligence, 1004-1010 (2014)
  * Narducci et al. (2016) Narducci, F., Basile, P., Musto, C., Lops, P., Caputo, A., de Gemmis, M., Iaquinta, L., Semeraro, G.: Concept-based item representations for a cross-lingual content-based recommendation process. Information Sciences 374, 15-31 (2016)
  * Narducci et al. (2013) Narducci, F., Musto, C., Semeraro, G., Lops, P., de Gemmis, M.: Exploiting big data for enhanced representations in content-based recommender systems. Proceedings of the 14th International Conference on Electronic Commerce and Web Technologies, 182–193 (2013)
  * Neshati et al. (2017) Neshati, M., Fallahnejad, Z., Beigy, H.: On dynamicity of expert finding in community question answering. Information Processing and Management 53(5), 1026-1042 (2017)
  * Ngaffo et al. (2021) Ngaffo, A., El Ayeb, W., Choukair, Z.: A time‐aware service recommendation based on implicit trust relationships and enhanced user similarities. Journal of Ambient Intelligence and Humanized Computing 12, 3017–3035 (2021)
  * Nishioka and Scherp (2016) Nishioka, C., Scherp, A.: Profiling vs. time vs. content: what does matter for top-k publication recommendation based on Twitter profiles? Proceeding of the Joint Conference on Digital Libraries, 171–180 (2016)
  * Pavan and Luca (2015) M. Pavan, Luca, E.W.D.: Semantic-based expert search in textbook research archives. Proceedings of the 5th International Workshop on Semantic Digital Archives, CEUR Workshop Proceedings, 1529, 18–29 (2015)
  * Ramos and Paraboni (2014) Ramos, C., Paraboni, I.: Temporal aspects of content recommendation on a microblog corpus. Lecture Notes in Computer Science 8775, 189-194 (2014).
  * Ren et al. (2015) Ren, R., Zhang, L., Cui, L., Deng, B., Shil, Y.: Personalized financial news recommendation algorithm based on ontology. Proceedings of 3rd International Conference on Information Technology and Quantitative Management, Procedia Computer Science 55, 843-851 (2015)
  * Rybak et al. (2014) Rybak, J., Balog, K., Nørvåg, K.: Temporal expertise profiling. Lecture Notes in Computer Science 8416, 540-546 (2014)
  * Saraswatm et al. (2020) Saraswatm, M., Chakraverty, S., Sharma, A.: Review-based topic distribution profile for recommender systems. In: Advances in Data Sciences, Security and Applications, Springer, 433-443 (2020)
  * Sharma et al. (2017) Sharma, R., Gopalani, D., Meena, Y.: Concept-based approach for research paper recommendation. Proceedings of the Pattern Recognition and Machine Intelligence Conference, 687-692 (2017)
  * Si et al. (2017) Si, Y., Zhang, F., Liu, W., CTF-ARA: An adaptive method for POI recommendation based on check-in and temporal features. Knowledge-Based Systems 128, 59–70 (2017)
  * Simsek and Karagoz (2020) Simsek, A., Karagoz, P.: Wikipedia enriched advertisement recommendation for microblogs by using sentiment enhanced user profiles. Journal of Intelligent Information Systems 54, 245–269 (2020)
  * Somlo and Howe (2001) Somlo, G.L., Howe, A.E.: Incremental clustering for profile maintenance in information gathering web agents. Proceedings of the 5th International Conference on Autonomous Agents, 262–269 (2001)
  * Stakhiyevich and Huang (2019) Stakhiyevich, P., Huang, Z.: Building user profiles based on user interests and preferences for recommender systems. Proceedings of the IEEE International Conferences on Ubiquitous Computing and Communications (IUCC) and Data Science and Computational Intelligence (DSCI) and Smart Computing, Networking and Services (SmartCNS), 450-455 (2019)
  * Tintarev and Masthoff (2012) Tintarev, N., Masthoff, J.: Evaluating the effectiveness of explanations for recommender systems. User Modeling and User-Adapted Interaction 22, 399–439, (2012)
  * Wang et al. (2018) Wang, D., Liang, Y., Xu, D., Feng, X., Guan, R.: A content-based recommender system for computer science publications. Knowledge-Based Systems 157, 1-9 (2018)
  * Wangwatcharakul and Wongthanavasu (2021) Wangwatcharakul, C., Wongthanavasu, S.: A novel temporal recommender system based on multiple transitions in user preference drift and topic review evolution. Expert Systems with Applications 185 (2021)
  * Wartena et al. (2011) Wartena, C., Slakhorst, W., Wibbels, W., Gantner, Z., Freudenthaler, C., Newell, C., Schmidt-Thieme, L.: Keyword-based TV program recommendation. Proceedings of the 9th Workshop on Intelligent Techniques for Web Personalization and Recommender Systems. Ceurs Workshop Proceedings, 756 (2011).
  * Wu (2012) Wu, S.: Data Fusion in Information Retrieval. In: Adaptation, Learning, and Optimization, vol. 13, Springer (2012)
  * Xiang et al. (2010) Xiang, L., Yuan, Q., Zhao, S., Chen, L., Zhang, X., Yang, Q., Sun, J.: Temporal recommendation on graphs via long- and short-term preference fusion. Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, 723–732 (2010).
  * Yan et al. (2020) Yan, S., Chen, X., Huo, R., Zhang, X., Lin, L.: Learning to build user-tag profile in recommendation system. Proceedings of the 29th ACM International Conference on Information and Knowledge Management, 2877–2884 (2020)
  * Yeniterzi et al. (2015) Yeniterzi, R., Callan, J.: Moving from static to dynamic modelling of expertise for question routing in CQA sites. Proceedings of the 9th International AAAI Conference on Web and Social Media, 702-705 (2015)
  * Ying et al. (2015) Yin, H., Cui, B., Chen, L., Hu, Z., Zhou, X.: Dynamic user modeling in social media system. ACM Transactions on Information Systems 33(3), Article 10 (2015)
  * Zeng et al. (2018) Zeng, H.Z., Chen, Z., Ma, W.: A unified framework for clustering heterogeneous web objects. Proceedings of the 3rd International Conference on Web Information Systems Engineering, 161–172, (2002)
  * Zeng et al. (2018) Zeng, W., Du, Y. Zhang, D., Ye, Z., Dou, Z.: TUP-RS: Temporal User Profile Based Recommender System. Lecture Notes in Artificial Intelligence 10842, 463-474, (2018)
  * Ziaimatin et al. (2012) Ziaimatin, H., Groza, T., Bordea, G., Buitelaar, P., Hunter, J.: Expertise profiling in evolving knowledge-curation platforms. Journal of Computing 2(3), 118-126 (2012).


feels the gust of wind and notices the hardened soil. Question: Does Fatima
believe the soil around the artifact is soft and loose or hardened? Choose one
of the following: a) Fatima believes the soil around the artifact is soft and
loose. b) Fatima believes the soil around the artifact has hardened. Correct
Answer: b) Unintervened: b) Fatima believes the soil around the artifact has
hardened. Intervened: a) Fatima believes the soil around the artifact is soft
and loose.

Story: Rohan is a potter in a small Indian village. Rohan wants to make a
beautiful ceramic vase as a gift for his friend’s housewarming party. Rohan
selects a bag of high-quality clay in his workshop that appears to be soft and
malleable. While Rohan is away for lunch, the hot sun dries the clay, making
it hard and unworkable. Rohan discovers the dried, hard clay. Question: Does
Rohan believe the bag of clay is soft and malleable or hard and unworkable?
Choose one of the following: a) Rohan believes the bag of clay is soft and
malleable. b) Rohan believes the bag of clay is hard and unworkable. Correct
Answer: b) Unintervened: b) Rohan believes the bag of clay is hard and
unworkable. Intervened: a) Rohan believes the bag of clay is soft and
malleable.

Story: Kunal is a talented potter in a small Indian village. Kunal wants to
create a set of earthenware bowls as a gift for his friend’s housewarming.
Kunal inspects a large lump of clay, finding it to be moist and malleable.
However, while Kunal is away for a short break, the sun beats down on the
clay, drying it out and making it hard and brittle. Kunal notices the sun-
dried, brittle clay. Question: Does Kunal believe the clay is moist and
malleable or hard and brittle? Choose one of the following: a) Kunal believes
the clay is moist and malleable. b) Kunal believes the clay is hard and
brittle. Correct Answer: b) Unintervened: b) Kunal believes the clay is hard
and brittle. Intervened: a) Kunal believes the clay is moist and malleable.

Story: Amara, a skilled Nigerian weaver, is preparing to create a traditional
textile for an upcoming cultural exhibition. Amara wants to use a unique
combination of colorful threads that represents her heritage. Amara carefully
chooses a set of vibrant, high-quality threads for her project. A curious
toddler in her home finds the threads and tangles them, making them difficult
to use in the weaving process. Amara notices the tangled threads. Question:
Does Amara believe the threads are strong and perfect for her textile or
tangled and difficult to use? Choose one of the following: a) Amara believes
the threads are strong and perfect for her textile. b) Amara believes the
threads are tangled and difficult to use. Correct Answer: b) Unintervened: b)
Amara believes the threads are tangled and difficult to use. Intervened: a)
Amara believes the threads are strong and perfect for her textile.

Story: Mei is a talented chef in a small Chinese village. She wants to prepare
a special dish using fresh vegetables from her garden. Mei sees that the
vegetables in the garden are ripe and ready to be picked. Later, a hailstorm
damages the garden, bruising and damaging the vegetables. Mei sees the
hailstorm and the damage it caused to the vegetables. Question: Does Mei
believe the vegetables are ripe and ready to be picked or damaged by the
hailstorm? Choose one of the following: a) Mei believes the vegetables are
ripe and ready to be picked. b) Mei believes the vegetables are damaged by the
hailstorm. Correct Answer: b) Unintervened: b) Mei believes the vegetables are
damaged by the hailstorm. Intervened: a) Mei believes the vegetables are ripe
and ready to be picked.

Story: Ngozi is a potter in a small Nigerian village, known for her beautiful
ceramic vases. Ngozi wants to create a vase with a blue glaze to showcase at
the upcoming village art exhibition. She reaches for a container of blue glaze
from her shelf to apply to the vase. Earlier in the day, her apprentice
accidentally switched the labels of the blue and green glaze containers, so
Ngozi unknowingly picks up the container with the green glaze. Ngozi notices
the green glaze. Question: Does Ngozi believe the container holds blue glaze
or green glaze? Choose one of the following: a) Ngozi believes the container
holds blue glaze. b) Ngozi believes the container holds green glaze. Correct
Answer: b) Unintervened: b) Ngozi believes the container holds green glaze.
Intervened: a) Ngozi believes the container holds blue glaze.

Story: Lila is a talented violinist who lives in a bustling city in India.
Today, she has an important audition for a renowned symphony orchestra. Lila
picks up her violin case, which she assumes contains her valuable violin.
However, her younger brother, who was playing around earlier, accidentally
swapped her violin case with his identical-looking case containing a
beginner’s violin. Lila notices the swapped violin cases. Question: Does Lila
believe the violin case contains her valuable violin or a beginner’s violin?
Choose one of the following: a) Lila believes the violin case contains her
valuable violin. b) Lila believes the violin case contains a beginner’s
violin. Correct Answer: b) Unintervened: b) Lila believes the violin case
contains a beginner’s violin. Intervened: a) Lila believes the violin case
contains her valuable violin.