# Train Once, Use Flexibly: A Modular Framework for Multi-Aspect Neural News
Recommendation

Andreea Iana1 Goran Glavaš2 Heiko Paulheim1
1Data and Web Science Group, University of Mannheim, Germany
2CAIDAS, University of Würzburg, Germany
<EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Recent neural news recommenders (NNR) extend content-based recommendation by
(1) aligning additional aspects such as topic or sentiment between the
candidate news and user history or (2) diversifying recommendations w.r.t.
these aspects. This customization is achieved by “hardcoding” additional
constraints into NNR’s architecture and/or training objectives: any change in
the desired recommendation behavior thus requires the model to be retrained
with a modified objective, impeding wide adoption of multi-aspect news
recommenders. In this work, we introduce MANNeR, a modular framework for
flexible multi-aspect (neural) news recommendation that supports ad-hoc
customization over individual aspects at inference time. With metric-based
learning at its core, MANNeR obtains aspect-specialized news encoders and then
flexibly combines aspect-specific similarity scores for final ranking.
Evaluation on two standard news recommendation benchmarks (one in English, one
in Norwegian) shows that MANNeR consistently outperforms state-of-the-art NNRs
on both standard content-based recommendation and single- and multi-aspect
customization. Moreover, with MANNeR we can trivially scale the importance and
find the optimal trade-off between content-based recommendation performance
and aspect-based diversity of recommendations. Finally, we show that both
MANNeR’s content-based recommendation and aspect customization are robust to
domain- and language transfer.

##  1 Introduction

Neural content-based recommenders Li and Wang (2019); Wu et al. (2023),
trained to infer users’ preferences from their click history, represent the
current state of the art in news recommendation. While previously consumed
content clearly points to users’ preferences, aspects other than content
alone, e.g., topical category (e.g., sports) or sentiment, contribute to their
news consumption decisions. Accordingly, some neural news recommendation (NNR)
models leverage information on these aspects in addition to text content, be
it (i) directly as input to the model Wang et al. (2018); Wu et al. (2019a);
Liu et al. (2020); Xun et al. (2021) or (ii) indirectly, as auxiliary training
tasks Wu et al. (2019c, 2020a); Qi et al. (2021b).

Increased personalization, however, is often at odds with diversity Pariser
(2011); Wu et al. (2020a). NNRs, optimized to maximize congruity to the users’
preferences, particularly tend to produce recommendations highly similar in
content to previously consumed news Liu et al. (2021); Wu et al. (2020a).
Another strand of work thus focuses on increasing diversity of recommendations
w.r.t. aspects other than content (e.g., sentiment). To this effect, prior
work either (i) re-ranks the content-based recommendations to decrease the
aspectual similarity between them Rao et al. (2013); Gharahighehi and Vens
(2023), or (ii) trains the NNR model by combining a content-based
personalization objective with an aspect-based diversification objective Wu et
al. (2020a, 2022b); Shi et al. (2022); Choi et al. (2022).

Different users, however, assign different importance to different aspects of
news (e.g., following developing events requires maximization of content-based
overlap with the user’s recent history; in another use-case, a user may prefer
content-wise diversification of recommendations, but within the same topic of
interest). Moreover, with personalization and diversification as mutually
conflicting goals, users should be able to seamlessly define their own optimal
trade-offs between the two. The existing body of work is ill-equipped for such
multi-aspect customization, because each set of preferences – i.e., to
personalize or diversify for each aspect – requires a different NNR model to
be trained from scratch. Put differently, baking global assumptions on
personalization and diversification preferences (i.e., same for all users)
into the NNR model design and training prevents customization at inference
time. We address this critical limitation of existing NNR work.

Contributions. We propose a modular framework for Multi-Aspect (Neural) News
Recommendation (MANNeR) in which we leverage metric-based contrastive learning
to induce a dedicated news encoder for each aspect, starting from a pretrained
language model (PLM). This way, we can obtain aspect-specific similarity
scores for pairs of news, allowing us to define ad-hoc at inference a custom
ranking function for each user, reflecting her preferences across all aspects.
The modular design of MANNeR allows it to be customized for any recommendation
objective (i.e., task) specified over (i) standard (i.e., content-based)
personalization, (ii) aspect-based diversification, and (iii) aspect-based
personalization. The modular design makes MANNeR easily extendable: to support
personalization and diversification over a new aspect (e.g., news outlet), one
only needs to train the aspect-specific encoder for the new aspect.

We evaluate MANNeR on two prominent NNR benchmarks: MIND Wu et al. (2020b)
(English) and Adressa Gulla et al. (2017) (Norwegian), with topical categories
and sentiment as the additional aspects next to content itself. MANNeR
outperforms state-of-the-art NNRs on standard content-based recommendation
across a range of metrics. Thanks to MANNeR’s modular design, we show –
without having to train a multitude of models with different training
objectives – that one can, depending on the recommendation goals, either (i)
vastly increase aspect diversity (i.e., diversity over topics and sentiment)
of recommendations or (ii) improve aspect-based personalization, while
retaining much of the content-based personalization performance. Additionally,
we carry out a nuanced analysis that shows how weighting of aspect relevance
scores affects the trade-off between the content-based recommendation
performance and aspect-based diversity or personalization. Finally, we
demonstrate that MANNeR based on a multilingual PLM is robust to (cross-
lingual) transfer of both content- and aspect-based encoders.

##  2 Background and Related Work

Personalized NNR. Neural content-based models have become the main vehicle of
personalized news recommendation, replacing traditional recommendation systems
relying on manual feature engineering Wu et al. (2023). Most NNR models
consist of a dedicated (i) news encoder (NE) and (ii) user encoder (UE) Wu et
al. (2023). The NE transforms input features into news embeddings Wu et al.
(2023, 2019d, 2019b), whereas UEs create user-level representations by
aggregating and contextualizing the embeddings of clicked news from the user’s
history Okura et al. (2017); An et al. (2019); Wu et al. (2022c). The
candidate’s recommendation score is finally computed by comparing its
embedding against the user embedding Wang et al. (2018); Wu et al. (2019a).
NNR models are primarily trained via standard point-wise classification
objectives with negative sampling Huang et al. (2013); Wu et al. (2021).

Exploiting users’ past behavior as NNR supervision unsurprisingly leads to
recommendations that are content-wise closest to previously consumed news, in
contrast to methods based on non-personalized criteria, e.g., news popularity
Ludmann (2017); Yang (2016) or geographic information Son et al. (2013); Chen
et al. (2017). More recent NNRs seek to augment content-based personalization
by considering other aspects, such as topical categories, sentiment, entities,
outlets, popularity, or recency Wu et al. (2023). These are incorporated in
the NNR either as additional input to the NE Wang et al. (2018); Gao et al.
(2018); Wu et al. (2019a); Liu et al. (2020); Sheu and Li (2020); Lu et al.
(2020); Qi et al. (2021a); Xun et al. (2021), or in the form of an auxiliary
training objective for the NE Wu et al. (2019c, 2020a); Qi et al. (2021b).

Diversification. Optimized for personalization, i.e., to favor matches in news
content (and other aspects like topic or sentiment), personalized NNR reduces
exposure to news dissimilar from those consumed in the past. Recommending
“more of the same”, constrains access to diverse viewpoints and information
Freedman and Sears (1965); Heitz et al. (2022) and leads to homogeneous news
diets and “filter bubbles” Pariser (2011), in turn reinforcing users’ initial
stances Li and Wang (2019). A significant body of work acknowledges this and
attempts to diversify recommendations. One line of work Li et al. (2011); Rao
et al. (2013); Kaminskas and Bridge (2016); Gharahighehi and Vens (2023) re-
ranks the set of personalized recommendations to increase some measure of
diversity (e.g. intra-list distance Zhang and Hurley (2008)). A second line of
work resorts to multi-task training of NNRs Gabriel De Souza et al. (2019); Wu
et al. (2020a); Shi et al. (2022); Wu et al. (2022b); Choi et al. (2022),
coupling the primary content-based personalization objective with auxiliary
objectives that force aspect-based diversification.

Limitations of Current NNRs. Current NNR approaches have an obvious
limitation: by “hardcoding” aspectual requirements (i.e., personalization or
diversification for an aspect) into the NNR’s architecture and/or training
objectives, they cannot be easily adjusted for different (i.e., custom)
recommendation goals. Because even a minor change in the recommendation
objective requires retraining of the NNR model, existing NNRs are ill-equipped
for multi-aspect customization, and generally limited to fixed single-aspect
recommendation scenarios (e.g., content-based personalization with topical
diversification). In this work, we rethink personalized news recommendation
and propose a novel, modular multi-aspect recommendation framework that allows
for ad-hoc creation of recommendation functions over aspects at inference
time. This enables fundamentally different news recommendation: one that lets
each user define their own custom recommendation function, choosing the extent
of personalization or diversification for each aspect.

##  3 MANNeR: Methodology

Figure 1: Illustration of the MANNeR framework. 1 Training: we leverage
metric-based contrastive learning to train aspect-specialized NEs (i.e. CR-
Module for content-based personalization, A-Module for aspect-based
similarity). During 2 inference, we produce aspect-specific similarity scores
between the candidate and clicked news, which we then flexibly aggregate for
final ranking.

The goal of personalized news recommendation is to produce for each candidate
news $n^{c}$ and user $u$, with corresponding click history
$H=\\{n^{u}_{1},n^{u}_{2},...,n^{u}_{N}\\}$, a relevance score $s(n^{c},u)$
that quantifies the candidate’s relevance for the user. In this work, we
define an aspect $A_{p}$ as a categorical variable that encodes a
characteristic of a piece of news (e.g. its topical category or sentiment),
where each news $n_{i}$ can belong only to one value of $A_{p}$ (e.g. if
$A_{p}$ is the sentiment, then $n_{i}$ may take exactly one value from the set
{positive, negative, neutral}). As discussed in §2, aspects are additional
dimensions (next to content) over which to tailor recommendations, whether it
is by (i) personalizing or (ii) diversifying over those dimensions. In line
with earlier work, we define aspect-based personalization as the level of
homogeneity between a user’s recommendations and clicked news w.r.t. the
distribution of aspect $A_{p}$. In contrast, we define aspect-based diversity
as the level of uniformity of aspect $A_{p}$’s distribution among the news in
the recommendation list.

Motivated by the limitations of existing NNR work (see §2), we introduce a
modular framework for multi-aspect news recommendation MANNeR, illustrated in
Figure 1. Starting from a pretrained language model (PLM), during (1)
training, we resort to metric-based contrastive learning to reshape the
representation space of the PLM, independently for each aspect; this results
in one specialized NE for each aspect; at (2) inference, we can arbitrarily
(i.e., depending on the concrete recommendation task/requirements) aggregate
the resulting aspect-specific similarity scores to produce a final ranking
function.

###  3.1 News Encoder

In line with prior work Qi et al. (2021b, c), for the NE we adopt a dual-
component architecture that couples (i) a text and (ii) an entity encoder. The
text encoder, a PLM, transforms the tokenized text input into a text-based
news embedding $\mathbf{n}_{t}$: we concatenate the news title and abstract as
input, and take the PLM’s output representation of the sequence start token
([CLS]) as $\mathbf{n}_{t}$. The entity encoder embeds entities (i.e., all
named entities extracted from the title and abstract) via pretrained entity
embeddings, contextualized in a layer that combines multi-head self-attention
Vaswani et al. (2017) and additive attention Bahdanau et al. (2014), resulting
in an entity-level news embedding $\mathbf{n}_{e}$. The final news embedding
$\mathbf{n}$ is the concatenation of $\mathbf{n}_{t}$ and $\mathbf{n}_{e}$.

###  3.2 Training

MANNeR consists of two types of modules, each with a dedicated NE, responsible
for content-based (CR-Module) and aspect-based (A-Module) recommendation
relevance, respectively. We train both modules with a metric-learning
objective. Concretely, we minimize the supervised contrastive loss (SCL)
Khosla et al. (2020); Gunel et al. (2020) (Eq. 1) which aims to reshape the
encoder’s representation space so that the embeddings of same-class instances
become mutually closer (according to some distance/similarity metric) than
instances of different classes. This is accomplished by contrasting the
distance/similarity score of a positive example (pair of same-class instances)
against the scores of corresponding negative examples (pairs of instances from
different classes):

$\small\mathcal{L}=-\sum_{i=1}^{N}\frac{1}{N_{y_{i}}-1}\sum_{\begin{subarray}{c}j\in[1,N]\\\
i\neq
j,y_{i}=y_{j}\end{subarray}}\log\frac{e^{(\mathbf{n}_{i}\cdot\mathbf{n}_{j}/\tau)}}{\sum_{\begin{subarray}{c}k\in[1,N]\\\
i\neq k\end{subarray}}e^{(\mathbf{n}_{i}\cdot\mathbf{n}_{k}/\tau)}}$ (1)

with $y_{i}$ as the label of the news $n_{i}$, $N$ the batch size, $N_{y_{i}}$
the number of batch instances with the same label $y_{i}$, and $\tau>0$ the
temperature hyperparameter that controls the extent of class separation. We
use the dot product as the similarity metric in SCL for both module types.

CR-Module. Our CR-Module is a modification of the common content-based NNR
architecture Wu et al. (2023). Concretely, we use a dedicated NE to encode
both candidate and clicked news. However, we propose, following Iana et al.
(2023), to replace the widely used UEs (i.e., early fusion of representations
of clicked news) with the much simpler (and non-parameterized) mean-pooling of
dot-product scores between the candidate embedding $\mathbf{n}^{c}$ and
embeddings of clicked news $\mathbf{n}_{i}^{u}$:
$s(\mathbf{n}^{c},u)=\frac{1}{N}\sum_{i=1}^{N}\mathbf{n}^{c}\cdot\mathbf{n}_{i}^{u}$
(i.e., late-fusion of the similarity scores).111Note that this is equivalent
to the dot product between the candidate embedding and the mean of the
embeddings of the user’s clicked news,
$s(\mathbf{n}^{c},u)=\mathbf{n}^{c}\cdot\left(\frac{1}{N}\sum_{i=1}^{N}\mathbf{n}_{i}^{u}\right)$.
This means that we can effortlessly derive user embeddings as averages of
embeddings of their clicked news. This way we reduce the computational
complexity of the standard (i.e., early-fusion) approaches with elaborate
parameterized UEs. We then update the CR-Module’s encoder (i.e., fine-tune the
PLM) by minimizing SCL, with clicked candidate news as positive examples and
non-clicked news as negative examples for the user. Since there are many more
non-clicked news, we resort to negative sampling Wu et al. (2022a).

A-Module. Each A-Module trains a specialized NE for one aspect other than
content. Via the metric-based SCL objective, we reshape the representation
space of the PLM so that it groups news according to the aspect classes. Given
a multi-class aspect $A_{p}$, we first construct the training set from pairs
of candidate and clicked news. Pairs of candidate and clicked news with the
same aspect label make positive pairs for SCL; we obtain the corresponding
negatives by pairing the same candidate with clicked news from other aspect
classes (e.g., for topical category as $A_{p}$, a candidate from sports paired
with the clicked news from politics). For each aspect, we independently fine-
tune (via SCL) a separate copy of the same initial PLM. Note that the
resulting aspect-specific NE encodes no information on user preferences: it
only encodes how similar news are w.r.t. the aspect in question. Importantly,
this means we can seamlessly extend MANNeR to support new aspects by training
additional A-Modules.

###  3.3 Inference

At inference time, the news encoders (NEs) of the CR-Module and of each of the
A-Modules are leveraged in exactly the same way: we encode the candidate news
$n^{c}$ as well as the user’s clicked news $n^{u}_{i}$ with the respective NE,
obtaining their module-specific embeddings $\mathbf{n}^{c}$ and
$\mathbf{n}^{u}_{i}$ – the dot product between the two,
$s=\mathbf{n}^{c}\cdot\mathbf{n}^{u}_{i}$ quantifies their similarity
according to the aspect of the module (or content in case of CR-Module’s NE).
We observed that different NEs produce similarity scores of different
magnitudes. We thus standardize (z-score normalization) each module’s scores
per user. The final similarity score for ranking is then a linear combination
of the aspect-specific similarity scores:

$s_{final}(\mathbf{n}^{c},u)=s_{CR}+\lambda_{A_{p}}s_{A_{p}}$ (2)

where $s_{CR}$ and $s_{A_{p}}$ denote the content and aspect similarity
scores, respectively, and $\lambda_{A_{p}}$ is the scaling weight for the
aspect score. Crucially, this modular formulation of the ranking function
allows different ad-hoc realizations that match custom recommendation goals.
For example, (i) with $\lambda_{A_{p}}=0$, MANNeR will perform standard
content-based personalization, (ii) for $\lambda_{A_{p}}>0$ it will recommend
based on both content- and aspect-based personalization, whereas (iii) for
$\lambda_{A_{p}}<0$ it will personalize by content but diversify for the
aspect at the same time. Note that MANNeR trivially generalizes to
recommendation objectives that include multiple aspects at once:
$s_{final}(\mathbf{n}^{c},u)=s_{CR}+\sum_{A_{p}\in
A}\lambda_{A_{p}}s_{A_{p}}$, with $A$ as the set of all aspects of interest.

##  4 Experimental Setup

We compare MANNeR against state-of-the-art NNRs on a range of single- and
multi-aspect recommendation tasks. We experiment with two aspects: topical
categories and sentiment of news.

Baselines. We evaluate several state-of-the-art NNRs, trained by minimizing
the classification-oriented cross-entropy loss. For fair comparison, following
Wu et al. (2021), we replace the original NEs of all baseline models that do
not use PLMs (which range from aggregations of word embeddings, over
convolutional networks to multi-head self-attention) with the same PLM we
employ in MANNeR.222The only exception is the final text embedding, where Wu
et al. (2021) pool subword tokens with an attention network instead of using
the [CLS] embedding. With that modification, we evaluate six NNRs optimized
purely for content personalization: (1) NAML Wu et al. (2019a), (2) NRMS Wu et
al. (2019d), (3) LSTUR333We evaluate two variants: LSTURini and LSTURcon,
which mutually differ in their user embeddings. We refer the reader to An et
al. (2019) for details. An et al. (2019), (4) MINS Wang et al. (2022), (5)
CAUM Qi et al. (2022), and (6) MINER444We evaluate three variants: MINERmax,
MINERmean, and MINERweighted, which differ in the aggregation strategy for the
final user click score. We refer the reader to Li et al. (2022) for details.
Li et al. (2022); and three NNRs trained with aspect-informed objectives: (7)
TANR Wu et al. (2019c) injects information on topical categories by jointly
optimizing for content-based personalization and topic classification, (8)
SentiRec Wu et al. (2020a) adds regularization for sentiment diversity to its
primary content personalization objective, and (9) SentiDebias Wu et al.
(2022d) uses sentiment-debiasing based on adversarial learning to reduce the
NNR’s sentiment bias (originating from the user data) and generate sentiment-
diverse recommendations.

Data. We carry out the evaluation on two prominent monolingual news
recommendation benchmarks: MINDlarge (denoted MIND) Wu et al. (2020b) with
news in English and Adressa-1 week Gulla et al. (2017) (denoted Adressa) with
Norwegian news. Since Wu et al. (2020b) do not release test labels for MIND,
we use the provided validation portion for testing, and split the respective
training set into temporally disjoint training (first four days of data) and
validation portions (the last day). Following established practices on
splitting the Adressa dataset Hu et al. (2020); Xu et al. (2023), we use the
data of the first five days to construct the user history and the clicks of
the sixth day to build the training dataset. We randomly sample 20% of the
clicks of the last day to create the validation set, and treat the remaining
samples of the last day as the test set. Note that during validation and
testing, we reconstruct the user histories using all the samples of the first
six days of data. Since the Adressa dataset contains only positive samples
(i.e., it includes no data about news seen but not clicked by the user) Gulla
et al. (2017), we randomly sample 20 news as negatives for each clicked
article in order to build impressions following Yi et al. (2021). Table 1
summarizes the statistics of the two datasets.

Regarding aspects, the topical category annotations are already provided in
both datasets. As no sentiment labels exist in neither MIND nor Adressa, we
use a multilingual XLM-RoBERTa Base model Conneau et al. (2020) trained on
tweets and fine-tuned for sentiment analysis Barbieri et al. (2022) to
classify news into one of three classes: positive (pos), neutral, and negative
(neg). We compute real-valued scores using the model’s confidence scores as
follows:

$\small s_{sent}=\begin{cases}(+1)\times s_{pos}\text{, if }\hat{l}=pos\\\
(-1)\times s_{neg}\text{, if }\hat{l}=neg\\\
(1-s_{neutral})\times(s_{pos}-s_{neg})\text{, otherwise}\\\ \end{cases}$ (3)

where $s_{i}$ denotes the model’s score for class $i$, and $\hat{l}$ is the
predicted sentiment class label.

Adressa contains no disambiguated named entities, so we only use news title as
input to NE: this model variant is denoted MANNeRtitle. On MIND, we train and
evaluate both the “full” MANNeR (i.e., with title, abstract and corresponding
entities as NE input) and MANNeRtitle.

| MIND (large) | Adressa (one week)
---|---|---
Statistic | Train | Test | Train | Test
# News | 101,527 | 72,023 | 11,207 | 11,207
# Users | 698,365 | 196,444 | 96,801 | 68,814
# Impressions | 2,186,683 | 365,201 | 218,848 | 146,284
# Categories | 18 | 17 | 18 | 18
Avg. history length | 33.7 | 33.6 | 13.99 | 15.62
Avg. # candidates / user | 37.4 | 37.4 | 21 | 21

Table 1: MIND and Adressa dataset statistics.

Evaluation Metrics. In line with prior work, we report the models’ performance
in terms of four metrics: AUC, MRR, nDCG@5, and nDCG@10. Additionally, we
measure aspect-based diversity of recommendations at position $k$ as the
normalized entropy of the distribution of the aspect $A_{p}$’s values in the
recommendation list:

$D_{A_{p}}@k=-\sum_{j\in A_{p}}\frac{p(j)\log p(j)}{\log(|A_{p}|)}$ (4)

where $A_{p}\in\\{ctg,snt\\}$, $|A_{p}|$ denotes the number of classes of
aspect $A_{p}$, and ctg and snt stand for topical category and sentiment,
respectively.

If aspect-based personalization is successful, the aspect $A_{p}$’s
distribution in the recommendations should be similar to its distribution in
the user history. We thus opt to evaluate personalization with the generalized
Jaccard similarity, a robust measure of similarity between two probability
distributions Bonnici (2020):

$\mathit{PS}_{A_{p}}@k=\frac{\sum_{j=1}^{|A_{p}|}\min(\mathcal{R}_{j},\mathcal{H}_{j})}{\sum_{j=1}^{|A_{p}|}\max(\mathcal{R}_{j},\mathcal{H}_{j})},$
(5)

where $R_{j}$ and $H_{j}$ represent the probability of a news with class $j$
of aspect $A_{p}$ to be contained in the recommendations list $\mathcal{R}$,
and, respectively, in the user history $\mathcal{H}$.

Lastly, since all metrics are bounded to the $[0,1]$ range, we measure the
trade-off between content-based personalization (nDCG@$k$) and either aspect-
based diversity $D_{A_{p}}@k$ or aspect-based personalization
$\mathit{PS}_{A_{p}}@k$ by combining them with the harmonic mean. We denote
this with TAp@$k$ for single-aspect. For multi-aspect evaluation, i.e., when
we rank for content-personalization by diversifying simultaneously over topics
and sentiment, we adopt as the evaluation metric the harmonic mean between
nDCG@$k$, Dctg@$k$ (topical category), and Dsnt@$k$ (sentiment), denoted with
Tall@$k$. Note that, while we use an unweighted harmonic mean to quantify the
trade-off between recommendation and aspect-based customization (i.e. we
equally weigh all the aspects), specific use-cases in which some aspect is
more important than others could, more generally, combine the aspect scores
with a weighted (harmonic) mean.

Implementation and Optimization Details. We use RoBERTa Base Liu et al. (2019)
(a monolingual English PLM) and NB-BERT Base Kummervold et al. (2021); Nielsen
(2023) (a monolingual Norwegian PLM) in experiments on MIND and Adressa,
respectively. In both cases, our training fine-tunes only the last four layers
of the PLM.555In preliminary results, we did not see significant differences
between full fine-tuning of all layers and fine-tuning only the last four
layers. In the interest of computational efficiency, we thus froze the first
eight layers of the transformer. We report the number of parameters for each
model and dataset in Table 2.666For LSTUR, with its user embedding matrix, the
number of parameters depends on the training data size. We use 100-dimensional
TransE embeddings Bordes et al. (2013) pretrained on Wikidata as input to the
entity encoder in NE. We set the maximum history length to 50 and set all
other model-specific hyperparameters for baselines to optimal values reported
in the respective papers. We train all models with mixed precision, and
optimize with the Adam algorithm Kingma and Ba (2014), with the learning rate
set to 1e-5 on MIND and 1e-6 on Adressa.777For the sentiment A-Module:
learning rate = 1e-6 for both datasets. In A-Module training, we sample 20
instances per class,888For $M$ class instances, we obtain $\frac{M^{2}-M}{2}$
positive pairs for that class for SCL. Preliminary experiments with larger $M$
did not yield improvements; we thus kept $M=20$ to reduce the computational
footprint of our experiments. whereas in the CR-Module training we sample four
negatives per positive example. We search for the optimal SCL temperature
$\tau$ using the validation portions of the two dataset variants, sweeping the
intervals $[0.1,0.4]$ with a step of 0.02 for the CR-Module, and $[0.1,0.9]$
with a step of 0.05 for the A-Module, respectively. For fair comparison, we
train all the baseline NNRs under the same training budget as for the CR-
Module in MANNeR: for 5 epochs on MIND and 20 epochs on Adressa, with a batch
size of 8. We train each A-Module for 100 epochs, with the batch size of 60 (3
classes) for sentiment and 360 for topics (18 topics). We run each experiment
five times with different random seeds and report averages and standard
deviations for all metrics. We trained each model on one NVIDIA Tesla A100 GPU
(40GB memory).999Our implementation is publicly available at
https://github.com/andreeaiana/manner

|  | MIND | Adressa
---|---|---|---
Model | Non-trainable | Trainable | Total | Trainable | Total
NAML-PLM | 56.7 | 70.8 | 127 | 124 | 180
NRMS-PLM | 56.7 | 73 | 129 | 126 | 182
LSTURini-PLM | 56.7 | 633 | 690 | 200 | 257
LSTURcon-PLM | 56.7 | 351 | 408 | 161 | 218
MINS-PLM | 56.7 | 73.3 | 130 | 126 | 183
CAUMno entities-PLM | 56.7 | 73.2 | 129 | 126 | 183
CAUM-PLM | 56.7 | 74.9 | 131 | – | –
MINERmax | 56.7 | 68.2 | 124 | 121 | 178
MINERmean | 56.7 | 68.2 | 124 | 121 | 178
MINERweighted | 56.7 | 68.3 | 124 | 121 | 178
TANR-PLM | 56.7 | 70.6 | 127 | 123 | 180
SentiRec-PLM | 56.7 | 73 | 129 | 126 | 182
SentiDebias-PLM | 56.7 | 73.3 | 130 | 126 | 183
MANNeR (CR-Moduletitle) | 56.7 | 67.9 | 124 | 121 | 177
MANNeR (A-Moduletitle) | 56.7 | 67.9 | 124 | 121 | 177
MANNeR (CR-Module) | 56.7 | 70.3 | 126 | – | –
MANNeR (A-Module) | 56.7 | 70.3 | 126 | – | –

Table 2: Number of parameters (in millions) for all models on MIND and
Adressa.

##  5 Results and Discussion

We first discuss the content personalization performance of MANNeR in
comparison to state-of-the-art NNRs. We then analyze its capability for
single- and multi-aspect (i) diversification and (ii) personalization. In the
latter (i.e., aspect customization experiments), we treat MANNeR, and
respectively MANNeRtitle, with just the CR-Module (i.e. $\lambda_{A_{p}}=0$)
as (another) baseline. Lastly, we evaluate MANNeR’s ability to re-use
pretrained aspect-specific modules in cross-lingual transfer experiments.

###  5.1 Content Personalization

| MIND | Adressa
---|---|---
Model | AUC | MRR | nDCG@5 | nDCG@10 | AUC | MRR | nDCG@5 | nDCG@10
NAML-PLM | 61.2$\pm$2.7 | 37.0$\pm$0.2 | 35.1$\pm$0.2 | 41.5$\pm$0.2 | 50.0$\pm$0.0 | 42.2$\pm$2.7 | 43.2$\pm$3.3 | 50.2$\pm$1.9
NRMS-PLM | 61.9$\pm$3.6 | 33.8$\pm$3.0 | 31.8$\pm$3.2 | 38.4$\pm$2.9 | 73.3$\pm$2.7 | 43.4$\pm$2.0 | 45.0$\pm$2.1 | 51.2$\pm$1.6
LSTURini-PLM | 53.0$\pm$1.9 | 34.3$\pm$2.4 | 32.5$\pm$2.5 | 39.0$\pm$2.4 | 67.1$\pm$2.7 | 43.1$\pm$3.1 | 45.0$\pm$3.6 | 51.3$\pm$2.4
LSTURcon-PLM | 50.2$\pm$0.1 | 27.3$\pm$1.4 | 25.4$\pm$1.3 | 31.8$\pm$1.2 | 57.3$\pm$3.3 | 39.0$\pm$3.5 | 40.2$\pm$4.1 | 46.4$\pm$2.9
MINS-PLM | 61.8$\pm$2.0 | 35.7$\pm$1.1 | 33.8$\pm$1.2 | 40.2$\pm$1.1 | 72.5$\pm$4.6 | 43.7$\pm$1.4 | 46.1$\pm$1.8 | 52.6$\pm$1.7
CAUMno entities-PLM | 68.7$\pm$1.7 | 37.5$\pm$1.0 | 35.7$\pm$1.0 | 42.1$\pm$0.9 | 74.2$\pm$3.8 | 41.0$\pm$2.9 | 44.0$\pm$3.5 | 49.5$\pm$3.4
CAUM-PLM | 67.2$\pm$2.3 | 37.1$\pm$1.6 | 35.3$\pm$1.6 | 41.7$\pm$1.5 | – | – | – | –
MINERmax | 65.2$\pm$0.8 | 35.0$\pm$0.9 | 33.1$\pm$0.8 | 39.4$\pm$0.7 | 58.7$\pm$4.5 | 43.2$\pm$1.3 | 45.2$\pm$1.9 | 51.5$\pm$2.6
MINERmean | 63.1$\pm$1.2 | 35.5$\pm$1.1 | 33.7$\pm$1.1 | 40.0$\pm$1.0 | 71.8$\pm$2.2 | 38.5$\pm$4.5 | 40.1$\pm$4.8 | 47.8$\pm$4.0
MINERweighted | 60.2$\pm$3.0 | 29.2$\pm$3.1 | 27.7$\pm$3.1 | 34.0$\pm$3.1 | 69.1$\pm$6.0 | 41.1$\pm$3.7 | 42.2$\pm$4.9 | 47.8$\pm$4.9
TANR-PLM | 58.8$\pm$3.6 | 35.6$\pm$1.3 | 33.7$\pm$1.4 | 40.2$\pm$1.3 | 50.0$\pm$0.0 | 42.5$\pm$4.1 | 44.0$\pm$ 5.1 | 50.3$\pm$4.4
SentiRec-PLM | 60.8$\pm$2.0 | 35.0$\pm$0.7 | 33.2$\pm$0.7 | 39.8$\pm$0.7 | 65.1$\pm$0.9 | 30.4$\pm$0.8 | 28.1$\pm$1.4 | 38.9$\pm$1.1
SentiDebias-PLM | 56.8$\pm$3.4 | 30.3$\pm$2.1 | 28.0$\pm$2.3 | 34.7$\pm$2.2 | 73.4$\pm$1.8 | 43.2$\pm$4.2 | 46.6$\pm$4.1 | 52.0$\pm$2.7
MANNeRtitle (CR-Module) | 68.7$\pm$0.6 | 37.9$\pm$0.4 | 36.1$\pm$0.4 | 42.4$\pm$0.4 | 79.4$\pm$1.7 | 47.0$\pm$2.4 | 48.9$\pm$2.8 | 54.3$\pm$2.5
MANNeR (CR-Module) | 69.7$\pm$0.9 | 38.6$\pm$0.6 | 37.0$\pm$0.6 | 43.2$\pm$0.6 | – | – | – | –
Improvement(%) | \+ 1.6 | \+ 3.1 | \+ 3.8 | \+ 2.6 | \+ 7.0 | \+ 7.6 | \+ 4.9 | \+ 3.2

Table 3: Recommendation performance of MANNeR (CR-Module) and the baselines.
We report averages, standard deviations across five different runs, and the
relative improvement over the best-performing baseline. The best results per
column are highlighted in bold, the second best are underlined.

Table 3 summarizes the results on content personalization for MANNeR and the
baselines. Since the task does not require any aspect-based customization, we
evaluate the MANNeR variant that uses only its CR-Module at inference time
(i.e., $\lambda=0$). MANNeR consistently outperforms all state-of-the-art NNRs
in terms of both classification and ranking metrics on both datasets. Given
that MANNeR’s CR-Module derives the user embedding by merely averaging clicked
news embeddings, these results question the need for complex parameterized
UEs, present in all the baselines, in line with the findings of Iana et al.
(2023).

Ablations. We next ablate the CR-Module’s content personalization performance
along two dimensions: Fig. 2(a) shows MANNeR’s performance on the MIND dataset
for different inputs to the NE; 2(b) shows performance for alternative
architecture designs and training objectives101010For brevity, we report these
results only on MIND; results for Adressa exhibit the same trends.. Fig. 2(a)
reveals that all groups of features – news abstract, as well as entities in
both title and abstract – contribute to the overall performance. We note,
however, that even the CR-Module that was exposed only to raw title text
(i.e., no abstract text nor entity information) outperforms all state-of-the-
art NNRs on content recommendation (cf. Table 3).

Figure 2(b) compares the CR-Module (as detailed in §3) against two alternative
design choices, namely when we replace: (1) the simple averaging of clicked
news embeddings with a parameterized (i.e., trainable) user encoder, namely an
additive attention layer; and (2) our metric-based SCL training objective with
the standard cross-entropy classification loss. Results justify both of our
design decisions: (1) the fact that a trainable UE leads to performance loss
strengthens our skepticism for the need for elaborate UE components Iana et
al. (2023), commonly used in NNR; (2) replacing SCL with the standard cross-
entropy loss results in an even more prominent performance loss: this is in
line with findings from other similarity-oriented NLP tasks in which
contrastive objectives led to better class separation Reimers and Gurevych
(2019); Tunstall et al. .

(a) Input features for the News Encoder (NE).

(b) CR-Module design/training alternatives.

Figure 2: Effect of (a) NE input features and (b) model design/training
choices on MANNeR’s content personalization (results on MIND).

| MIND | Adressa
---|---|---
Model | nDCG@10 | Dctg@10 | Tctg@10 | Dsnt@10 | Tsnt@10 | Tall@10 | nDCG@10 | Dctg@10 | Tctg@10 | Dsnt@10 | Tsnt@10 | Tall@10
NAML-PLM | 41.5$\pm$0.2 | 47.4$\pm$0.7 | 44.3$\pm$0.3 | 65.6$\pm$0.4 | 50.8$\pm$0.2 | 49.7$\pm$0.2 | 50.2$\pm$1.9 | 31.9$\pm$2.3 | 39.0$\pm$2.1 | 61.5$\pm$0.8 | 55.3$\pm$1.5 | 44.4$\pm$1.9
NRMS-PLM | 38.4$\pm$2.9 | 50.2$\pm$2.4 | 43.4$\pm$1.2 | 66.6$\pm$0.9 | 48.7$\pm$2.2 | 49.1$\pm$0.9 | 51.2$\pm$1.6 | 32.4$\pm$0.5 | 39.7$\pm$0.5 | 62.1$\pm$0.3 | 56.1$\pm$1.0 | 45.1$\pm$0.5
LSTURini-PLM | 39.0$\pm$2.4 | 48.4$\pm$2.2 | 43.1$\pm$0.7 | 65.6$\pm$0.5 | 48.9$\pm$1.8 | 48.7$\pm$0.6 | 51.3$\pm$2.4 | 27.6$\pm$5.7 | 35.6$\pm$5.3 | 60.8$\pm$0.8 | 55.6$\pm$1.5 | 41.2$\pm$5.0
LSTURcon-PLM | 31.8$\pm$1.2 | 43.1$\pm$2.0 | 36.6$\pm$1.5 | 66.7$\pm$0.4 | 43.0$\pm$1.1 | 43.0$\pm$1.4 | 46.4$\pm$2.9 | 29.0$\pm$1.3 | 35.7$\pm$1.8 | 61.1$\pm$0.9 | 52.7$\pm$2.2 | 41.4$\pm$1.7
MINS-PLM | 40.2$\pm$1.1 | 48.7$\pm$1.0 | 44.0$\pm$0.4 | 66.7$\pm$0.7 | 50.1$\pm$0.8 | 49.6$\pm$0.4 | 52.6$\pm$1.7 | 34.4$\pm$1.5 | 41.6$\pm$1.5 | 62.0$\pm$0.3 | 56.9$\pm$1.1 | 46.7$\pm$1.2
CAUMno entities-PLM | 42.1$\pm$0.9 | 47.8$\pm$0.6 | 44.8$\pm$0.6 | 65.8$\pm$0.8 | 51.3$\pm$0.6 | 50.1$\pm$0.5 | 49.5$\pm$3.4 | 35.5$\pm$1.0 | 41.3$\pm$1.5 | 62.0$\pm$0.2 | 55.0$\pm$2.2 | 46.5$\pm$1.3
CAUM-PLM | 41.7$\pm$1.5 | 47.8$\pm$1.0 | 44.5$\pm$0.8 | 66.1$\pm$0.4 | 51.1$\pm$1.1 | 50.0$\pm$0.7 | – | – | – | – | – | –
MINERmax | 39.4$\pm$0.7 | 47.5$\pm$0.9 | 43.1$\pm$0.7 | 63.7$\pm$1.3 | 48.7$\pm$0.8 | 48.3$\pm$0.8 | 51.5$\pm$2.6 | 32.6$\pm$0.9 | 39.9$\pm$0.8 | 61.6$\pm$0.3 | 56.1$\pm$1.5 | 45.2$\pm$0.7
MINERmean | 40.0$\pm$1.0 | 49.4$\pm$1.2 | 44.2$\pm$0.4 | 65.7$\pm$0.9 | 4.7$\pm$1.0 | 49.6$\pm$0.5 | 47.8$\pm$4.0 | 31.0$\pm$1.1 | 37.6$\pm$1.8 | 60.9$\pm$0.3 | 53.5$\pm$2.5 | 43.1$\pm$1.6
MINERweighted | 34.0$\pm$3.1 | 49.1$\pm$1.6 | 40.1$\pm$1.9 | 64.6$\pm$1.1 | 44.5$\pm$2.7 | 45.9$\pm$1.7 | 47.8$\pm$4.9 | 32.5$\pm$0.6 | 38.7$\pm$1.8 | 61.0$\pm$0.7 | 53.5$\pm$3.3 | 44.0$\pm$1.7
TANR-PLM | 40.2$\pm$1.3 | 49.9$\pm$1.6 | 44.5$\pm$0.4 | 66.6$\pm$1.0 | 50.2$\pm$0.7 | 50.0$\pm$0.3 | 50.3$\pm$4.4 | 30.3$\pm$3.2 | 37.6$\pm$1.4 | 61.0$\pm$1.5 | 55.0$\pm$2.2 | 43.1$\pm$1.5
SentiRec-PLM | 39.8$\pm$0.7 | 50.4$\pm$0.4 | 44.5$\pm$0.6 | 66.8$\pm$0.7 | 49.8$\pm$0.5 | 50.0$\pm$0.5 | 38.9$\pm$1.1 | 35.7$\pm$1.0 | 37.2$\pm$0.4 | 68.5$\pm$1.0 | 49.6$\pm$1.1 | 43.9$\pm$0.4
SentiDebias-PLM | 34.7$\pm$2.2 | 50.5$\pm$1.8 | 41.1$\pm$1.3 | 68.0$\pm$0.9 | 45.9$\pm$1.9 | 47.3$\pm$1.1 | 52.0$\pm$2.7 | 32.7$\pm$0.6 | 40.2$\pm$1.1 | 62.4$\pm$0.4 | 56.7$\pm$1.1 | 45.6$\pm$1.0
MANNeRtitle (CR-Module) | 42.4$\pm$0.4 | 48.3$\pm$0.3 | 45.2$\pm$0.3 | 65.1$\pm$0.5 | 51.4$\pm$0.4 | 50.3$\pm$0.4 | 54.3$\pm$2.5 | 31.7$\pm$0.2 | 40.0$\pm$0.7 | 61.4$\pm$0.3 | 57.6$\pm$1.5 | 45.3$\pm$0.6
MANNeRtitle ($\lambda_{ctg}=-0.3$, $\lambda_{snt}=0$) | 41.2$\pm$0.5 | 49.8$\pm$0.5 | 45.1$\pm$0.4 | 65.4$\pm$0.6 | 50.6$\pm$0.5 | 50.3$\pm$0.4 | 50.9$\pm$2.5 | 34.1$\pm$0.3 | 40.8$\pm$0.8 | 61.9$\pm$0.3 | 55.8$\pm$1.6 | 46.0$\pm$0.7
MANNeRtitle ($\lambda_{ctg}=0$, $\lambda_{snt}=-0.2$) | 41.8$\pm$0.3 | 48.8$\pm$0.5 | 45.0$\pm$0.4 | 65.4$\pm$0.5 | 51.0$\pm$0.4 | 50.3$\pm$0.4 | 53.8$\pm$2.5 | 32.4$\pm$0.2 | 40.4$\pm$0.7 | 63.0$\pm$0.3 | 58.0$\pm$1.5 | 45.9$\pm$0.6
MANNeR (CR-Module) | 43.2$\pm$0.6 | 49.3$\pm$0.3 | 46.0$\pm$0.3 | 65.4$\pm$0.6 | 52.0$\pm$0.4 | 51.1$\pm$0.2 | – | – | – | – | – | –
MANNeR ($\lambda_{ctg}=-0.2$, $\lambda_{snt}=0$) | 42.0$\pm$0.6 | 51.5$\pm$0.3 | 46.2$\pm$0.3 | 65.6$\pm$0.6 | 51.2$\pm$0.4 | 51.3$\pm$0.3 | – | – | – | – | – | –
MANNeR ($\lambda_{ctg}=0$, $\lambda_{snt}=-0.2$) | 43.1$\pm$0.6 | 49.5$\pm$0.3 | 46.1$\pm$0.3 | 68.1$\pm$0.4 | 52.8$\pm$0.5 | 51.6$\pm$0.2 | – | – | – | – | – | –

Table 4: Diversity (and content personalization) performance on
diversification tasks. For MANNeR, we list the best results (in terms of TAp)
of single-aspect diversification ($\lambda_{ctg}=-0.2$ and $\lambda_{snt}=0$
for topical category diversification; $\lambda_{ctg}=0$ and
$\lambda_{snt}=-0.2$ for sentiment diversification) on MIND. For MANNeRtitle,
we list the best results (in terms of TAp) of single-aspect diversification
($\lambda_{ctg}=-0.3$ and $\lambda_{snt}=0$ for topical category
diversification; $\lambda_{ctg}=0$ and $\lambda_{snt}=-0.2$ for sentiment
diversification) on Adressa. The best results per column are highlighted in
bold, the second best are underlined.

###  5.2 Single-aspect Customization

(a) Single-aspect diversification.

(b) Single-aspect personalization.

Figure 3: Results of single-aspect customization for MANNeR and the best
baseline on MIND.

(a) Single-aspect diversification.

(b) Single-aspect personalization.

Figure 4: Results of single-aspect customization for MANNeRtitle and the best
baseline on Adressa.

#### Diversification.

Table 4 summarizes the results on aspect diversification tasks (i.e. for
topical categories and sentiment). We report the performance for content
personalization (nDCG), single-aspect diversification (Dctg for topical
category, Dsnt for sentiment), single-aspect harmonic mean between content
personalization and aspect diversification (Tctg for topical category, Tsnt
for sentiment, respectively), and the multi-aspect harmonic mean between nDCG,
Dctg, and Dsnt (Tall).111111For brevity, we report metrics only for $k=10$;
results for $k=5$ exhibit the same trends. All baseline models (baseline NNRs
and MANNeR’s CR-Module, without aspect diversification) obtain similar
diversification scores (Dctg and Dsnt). The sentiment-aware SentiRec-PLM, with
an explicit auxiliary sentiment diversification objective, yields the highest
sentiment diversity (Dsnt) on Adressa; this comes at the expense of content
personalization quality (lowest nDCG). On MIND, SentiDebias-PLM – another
sentiment-specific model – achieves the highest sentiment diversity, but also
exhibits the lowest content personalization performance. Overall, the results
in Table 4 point to a trade-off between content personalization and aspectual
diversity: models with higher DAp@$k$ tend to have a lower nDCG@$k$.

Unlike all other models, MANNeR can trade content personalization for
diversity (and vice-versa) with different values of the aspect coefficients
$\lambda$. We analyze this trade-off for different values of $\lambda$ctg and
$\lambda$snt. Figures 3(a) and 4(a) illustrate the MANNeR performance in
single-aspect diversification tasks for different values of $\lambda$ on MIND
and Adressa, respectively. The steady drop in nDCG together with the steady
increase in DAp indeed indicate the existence of a trade-off between content
personalization and aspect diversification. For topical categories we observe
a steeper decline in content personalization quality with improved
diversification than for sentiment. Sentiment diversity reaches peak
performance for $\lambda_{snt}=-0.4$ on MIND and $\lambda_{snt}=-0.6$ on
Adressa, whereas category diversity continues to increase all the way to
$\lambda_{ctg}=-0.9$ on MIND and $\lambda_{ctg}=-1.0$ on Adressa. Intuitively,
content-based personalization is more aligned with the topical than with the
sentiment consistency of recommendations. The best trade-off (i.e., maximal
performance according to TAp@10), is achieved with $\lambda_{ctg}=-0.2$ and
$\lambda_{ctg}=-0.3$ for topics on MIND, and Adressa, respectively, and
$\lambda_{snt}=-0.2$ for sentiment on both datasets.

We attribute these effects to the representation spaces of our A-Modules (for
categories and for sentiment). We thus further investigate the aspect-specific
representation spaces of the two A-Modules. Fig. 5 shows the 2-dimensional
t-SNE visualizations Van der Maaten and Hinton (2008) of the news embeddings
produced with aspect-specialized encoders trained on the two datasets: (a) and
(c) for topical categories; and (b) and (d) for sentiment. The results confirm
our hypothesis: the latent representation space of the encoder was reshaped to
group instances from the same classes. The separation of classes, however, is
less prominent for representation spaces of the encoders trained on Adressa
than for those learned on MIND (e.g., the effect is stronger on the category-
shaped embedding space). We believe that this is because Adressa has 10 times
fewer news than MIND (and contrastive learning, naturally, benefits from more
news pairs), with over half of the topical categories in Adressa represented
with fewer than 100 examples.

(a) Category-shaped embedding space on MIND.

(b) Sentiment-shaped embedding space on MIND.

(c) Category-shaped embedding space on Adressa.

(d) Sentiment-shaped embedding space on Adressa.

Figure 5: t-SNE plots of the news embeddings in the test set of MIND, and
respectively, Adressa.

| MIND | Adreesa
---|---|---
Model | nDCG@10 | PSctg@10 | Tctg@10 | PSsnt@10 | Tsnt@10 | Tall@10 | nDCG@10 | PSctg@10 | Tctg@10 | PSsnt@10 | Tsnt@10 | Tall@10
NAML-PLM | 41.5$\pm$0.2 | 25.6$\pm$0.3 | 31.7$\pm$0.3 | 35.0$\pm$0.2 | 38.0$\pm$0.2 | 32.7$\pm$0.2 | 50.2$\pm$1.9 | 35.6$\pm$0.9 | 41.6$\pm$0.6 | 41.8$\pm$0.1 | 45.6$\pm$0.8 | 41.7$\pm$0.4
NRMS-PLM | 38.4$\pm$2.9 | 23.2$\pm$1.1 | 28.9$\pm$1.7 | 35.0$\pm$0.1 | 28.4$\pm$1.5 | 30.7$\pm$1.3 | 51.2$\pm$1.6 | 34.2$\pm$03 | 41.0$\pm$0.6 | 41.8$\pm$0.0 | 46.0$\pm$0.6 | 41.2$\pm$0.4
LSTURini-PLM | 39.0$\pm$2.4 | 24.1$\pm$0.6 | 29.8$\pm$1.1 | 34.9$\pm$0.4 | 36.8$\pm$1.2 | 31.3$\pm$0.9 | 51.3$\pm$2.4 | 36.2$\pm$2.3 | 42.4$\pm$1.8 | 41.9$\pm$0.1 | 46.1$\pm$1.0 | 42.2$\pm$1.2
LSTURcon-PLM | 31.8$\pm$1.2 | 18.7$\pm$0.9 | 23.5$\pm$1.1 | 33.9$\pm$0.0 | 32.8$\pm$0.7 | 26.2$\pm$0.9 | 46.4$\pm$2.9 | 33.3$\pm$0.2 | 38.8$\pm$1.0 | 41.8$\pm$0.1 | 43.9$\pm$1.3 | 39.7$\pm$0.7
MINS-PLM | 40.2$\pm$1.1 | 24.9$\pm$0.8 | 30.7$\pm$0.8 | 34.8$\pm$0.3 | 37.3$\pm$0.5 | 32.0$\pm$0.7 | 52.6$\pm$1.7 | 33.4$\pm$0.7 | 40.9$\pm$0.5 | 41.8$\pm$0.1 | 46.5$\pm$0.6 | 41.2$\pm$0.3
CAUMno entities-PLM | 42.1$\pm$0.9 | 25.2$\pm$0.2 | 31.5$\pm$0.4 | 35.1$\pm$0.2 | 38.3$\pm$0.4 | 32.6$\pm$0.3 | 49.5$\pm$3.4 | 33.1$\pm$0.5 | 39.6$\pm$1.1 | 41.8$\pm$0.1 | 45.2$\pm$1.4 | 40.3$\pm$0.8
CAUM-PLM | 41.7$\pm$1.5 | 25.1$\pm$0.5 | 31.3$\pm$0.7 | 35.0$\pm$0.1 | 38.1$\pm$0.7 | 32.5$\pm$0.5 | – | – | – | – | – | –
MINERmax | 39.4$\pm$0.7 | 24.3$\pm$0.4 | 30.1$\pm$0.5 | 35.2$\pm$0.2 | 37.2$\pm$0.4 | 31.6$\pm$0.3 | 51.5$\pm$2.6 | 34.0$\pm$0.3 | 41.0$\pm$0.9 | 41.9$\pm$0.0 | 46.2$\pm$1.0 | 41.3$\pm$0.6
MINERmean | 40.0$\pm$1.0 | 23.9$\pm$0.4 | 29.9$\pm$0.5 | 35.0$\pm$0.2 | 37.3$\pm$0.4 | 31.5$\pm$0.4 | 47.8$\pm$4.0 | 34.5$\pm$0.2 | 40.0$\pm$1.3 | 42.0$\pm$0.0 | 44.7$\pm$1.7 | 40.6$\pm$0.9
MINERweighted | 34.0$\pm$3.1 | 22.3$\pm$0.7 | 26.9$\pm$1.4 | 35.0$\pm$0.3 | 34.5$\pm$1.6 | 29.2$\pm$1.2 | 47.8$\pm$4.9 | 33.7$\pm$0.2 | 39.4$\pm$1.8 | 41.9$\pm$0.1 | 44.6$\pm$2.1 | 40.2$\pm$1.2
TANR-PLM | 40.2$\pm$1.3 | 24.6$\pm$0.4 | 30.6$\pm$0.6 | 35.0$\pm$0.3 | 37.5$\pm$0.6 | 31.9$\pm$0.5 | 50.3$\pm$4.4 | 35.0$\pm$1.4 | 41.3$\pm$2.4 | 41.9$\pm$0.2 | 45.6$\pm$1.9 | 41.4$\pm$1.7
SentiRec-PLM | 39.8$\pm$0.7 | 23.8$\pm$0.3 | 29.7$\pm$0.4 | 33.6$\pm$0.5 | 36.5$\pm$0.6 | 30.9$\pm$0.4 | 38.9$\pm$1.1 | 32.2$\pm$0.4 | 35.2$\pm$0.7 | 38.8$\pm$0.2 | 38.8$\pm$0.5 | 36.3$\pm$0.4
SentiDebias-PLM | 34.7$\pm$2.2 | 21.8$\pm$1.2 | 26.8$\pm$1.6 | 34.4$\pm$0.2 | 34.6$\pm$1.2 | 28.9$\pm$1.2 | 52.0$\pm$2.7 | 34.1$\pm$0.2 | 41.2$\pm$0.8 | 41.7$\pm$0.1 | 46.3$\pm$1.0 | 41.4$\pm$0.5
MANNeRtitle (CR-Module) | 42.4$\pm$0.4 | 24.8$\pm$0.1 | 31.3$\pm$0.2 | 35.2$\pm$0.1 | 38.5$\pm$0.1 | 32.5$\pm$0.1 | 54.3$\pm$2.5 | 34.5$\pm$0.1 | 42.2$\pm$0.8 | 42.0$\pm$0.1 | 47.3$\pm$0.9 | 42.1$\pm$0.5
MANNeRtitle ($\lambda_{ctg}=0.4$, $\lambda_{snt}=0$) | 41.9$\pm$0.4 | 25.2$\pm$0.1 | 31.5$\pm$0.1 | 35.1$\pm$0.2 | 38.2$\pm$0.1 | 32.6$\pm$0.1 | 53.6$\pm$1.9 | 36.2$\pm$0.1 | 43.2$\pm$0.7 | 42.1$\pm$0.1 | 47.2$\pm$0.7 | 42.9$\pm$0.4
MANNeRtitle ($\lambda_{ctg}=0$, $\lambda_{snt}=0.1$) | 42.4$\pm$0.4 | 24.9$\pm$0.1 | 31.3$\pm$0.2 | 35.2$\pm$0.2 | 38.5$\pm$0.1 | 32.5$\pm$0.1 | 54.1$\pm$2.4 | 34.7$\pm$0.1 | 42.2$\pm$0.8 | 42.2$\pm$0.1 | 47.4$\pm$0.9 | 42.2$\pm$0.5
MANNeR (CR-Module) | 43.2$\pm$0.6 | 24.7$\pm$0.1 | 31.4$\pm$0.2 | 35.1$\pm$0.1 | 38.7$\pm$0.2 | 32.6$\pm$0.2 | – | – | – | – | – | –
MANNeR ($\lambda_{ctg}=0.7$, $\lambda_{snt}=0$) | 42.9$\pm$0.3 | 27.2$\pm$0.1 | 33.3$\pm$0.1 | 35.2$\pm$0.0 | 38.7$\pm$0.1 | 33.9$\pm$0.1 | – | – | – | – | – | –
MANNeR ($\lambda_{ctg}=0$, $\lambda_{snt}=0.2$) | 42.8$\pm$0.5 | 24.7$\pm$0.1 | 31.3$\pm$0.2 | 35.8$\pm$0.1 | 39.0$\pm$0.2 | 32.7$\pm$0.1 | – | – | – | – | – | –

Table 5: Aspect (and content) personalization performance on personalization
tasks. For MANNeR, we list the best results (in terms of TAp) of single-aspect
customization ($\lambda_{ctg}=0.7$ and $\lambda_{snt}=0$ for topical category
personalization; $\lambda_{ctg}=0$ and $\lambda_{snt}=0.2$ for sentiment
personalization) on MIND. For MANNeRtitle, we list the best results (in terms
of TAp) of single-aspect customization ($\lambda_{ctg}=0.4$ and
$\lambda_{snt}=0$ for topical category personalization; $\lambda_{ctg}=0$ and
$\lambda_{snt}=0.1$ for sentiment personalization) on Adressa. The best
results per column are highlighted in bold, the second best are underlined.

Personalization. Table 5 displays the results on aspect personalization tasks.
Similar to the diversification tasks, we report the performance for content
personalization (nDCG) along with the aspect-personalization scores (PSctg for
topics, PSsnt for sentiment). The results generally mirror those from
diversification. Surprisingly, we find that TANR – a model trained with an
auxiliary topic classification task – underperforms NAML and LSTURini, both of
which add topical categories to the set of NE input features, in category
personalization (PSctg) on MIND and Adressa, respectively. MANNeR’s CR-Module
alone (i.e., without any aspect customization) yields competitive category
personalization performance. We believe that this is because (1) MANNeR’s CR-
Module is best in content personalization and (2) category personalization is
well-aligned with content personalization (i.e., news with similar content
tend to belong to the same category). Figures 3(b) and 4(b) explore the trade-
off between content personalization and aspect personalization, for different
(in this case, positive) values of $\lambda$ on both datasets. The best
topical category personalization (PSctg), obtained for $\lambda_{ctg}>0.7$ on
MIND and $\lambda_{ctg}>0.4$ on Adressa, comes at the (small) expense for
content personalization (small nDCG drop): with too much weight on the
category similarity of news, the impact of content relevance becomes diluted.
Increased sentiment personalization, on the other hand, is much more
detrimental to content personalization. We find this to be intuitive: users do
not really choose what to read based on sentiment. Tailoring recommendations
according to the sentiment of previously clicked news thus leads to many more
content-irrelevant recommendations.

(a) Multi-aspect diversification on MIND.

(b) Multi-aspect personalization on MIND.

Figure 6: Results of multi-aspect customization for MANNeR on MIND.

###  5.3 Multi-aspect Customization

Diversification. Fig. 6(a) explores the trade-off between content
personalization and multi-aspect diversification, i.e. diversifying
simultaneously over both topical categories and sentiments, for different
values of the aspect coefficients $\lambda_{ctg}$ and $\lambda_{snt}$,
respectively.121212For brevity, we report results only on the MIND dataset;
findings on Adressa exhibit the same patterns. We report the multi-aspect
harmonic mean (Tall) between nDCG, Dctg, and Dsnt. We achieve the highest Tall
for $\lambda_{ctg}=-0.2$ and $\lambda_{snt}=-0.25$ on MIND. In line with
results on single-aspect diversification, we observe that improving diversity
in terms of topical categories rather than sentiments has a more negative
effect on content personalization quality, i.e. steeper decline in Tall.
Overall, the results in Fig. 6(a) confirm that MANNeR can generalize to
diversify for multiple aspects at once by weighting individual aspect
relevance scores less than in the single-aspect task. This can be explained by
the fact that weighting several aspects higher at the same time acts as a
double discounting for content personalization, diluting content relevance
disproportionately.

Personalization. Fig. 6(b) shows the results of multi-aspect
personalization.††footnotemark:  Similar to multi-aspect diversification, we
report the trade-off Tall between content personalization (nDCG) and topic and
sentiment personalization, PSctg and PSsnt. On MIND, we achieve the best
multi-aspect trade-off (i.e., the best Tall) for $\lambda_{ctg}=0.5$ and
$\lambda_{snt}=0.2$ on MIND. Stronger enforcing of alignment of candidate news
with user’s history is needed for topical categories than for sentiment (i.e.,
($\lambda_{ctg}$ > $\lambda_{snt}$)). This is because sentiment exhibits low
variance within topical categories (e.g., politics news are mostly negative)
and enforcing categorical personalization thus partly also achieves sentiment
personalization.

(a) Single-aspect diversification.

(b) Single-aspect personalization.

Figure 7: Results of cross-lingual transfer in single-aspect customization for
MANNeRtitle, with modules trained on different (combinations of) source-
language datasets and evaluated on the target-language dataset MIND. The line
style indicates the metric, whereas the line color denotes the source-language
datasets on which we trained the CR-Module and A-Module.

###  5.4 Cross-Lingual Transfer

Lastly, we analyze the transferability of MANNeRtitle across datasets and
languages in single-aspect customization experiments.131313We evaluate only
MANNeRtitle, since the full version of MANNeR which requires news abstracts
and disambiguated named entities cannot be trained on Adressa.

Experimental Setup. Concretely, we compare the MANNeRtitle, with i.e., CR-
Module and A-Module trained on the train portions of the English MIND dataset
against the variants in which CR-Module and/or A-Module are trained on the
Norwegian Adressa dataset and transferred to English for inference on the test
portion of MIND. To enable cross-lingual transfer, in these experiments we
replace the monolingual PLMs used in MANNeRtitle’s NE with a multilingual
DistilBERT Base model Sanh et al. (2019).141414In these cross-lingual transfer
experiments, we fine-tune all 6 PLM’s layers. We report the performance for
content personalization (nDCG), single-aspect diversification (Dctg for topic,
Dsnt for sentiment), and single-aspect personalization (PSctg and PSsnt).

Results and Discussion. Figure 7 summarizes the cross-lingual transfer results
for single-aspect diversification and personalization. Expectedly, MANNeRtitle
trained fully on Adressa (i.e., in Norwegian) suffers from a large drop in
content personalization (ranking) performance, compared to the counterpart
trained on MIND (i.e., in English). In contrast, transferring only the
A-Module, i.e., training the CR-Module on MIND and the A-Module (for topics
and sentiment) on Adressa), yields performance comparable to that of complete
in-language training (i.e., both CR-Module and A-Module trained on MIND). This
is particularly the case for the sentiment A-Module, since the sentiment
labels between the languages/datasets are more aligned than those for topical
categories. This is encouraging, as it suggests that MANNeR has the ability to
“plug-in” A-Modules trained on datasets in languages different that the target
language, when the aspectual labels are not available for the target language.
We highlight that the cross-lingual A-Module transfer is successful despite
the fact that we transfer from a smaller dataset to a larger one (i.e.,
Adressa to MIND) and from a less resourced language to a more resourced one
(i.e., Norwegian to English). This suggests that, coupled with multilingual
PLMs, MANNeR can be used for effective news recommendation in lower-resource
languages, where training data and aspectual labels are scarce. Furthermore,
the results suggest that the A-Modules could be trained on general-purpose
classification datasets (e.g. topic or sentiment classification datasets).

##  6 Conclusion

We have proposed MANNeR, a modular framework for multi-aspect neural news
recommendation, in which we leverage metric-based contrastive learning to
obtain aspect-specialized news encoders, and flexibly combine the
corresponding aspect-specific similarity scores for final ranking. The modular
design of MANNeR allows defining ad-hoc multi-aspect ranking functions (i.e.
diversification or personalization) at inference time, and the seamless
extension of the framework to support new aspects, without the need to train
dedicated NNR models for changes in the recommendation objective. Our
experiments show that MANNeR consistently outperforms state-of-the-art NNRs on
(i) standard content-based recommendation, as well as on single- and multi-
aspect (ii) diversification and (iii) personalization of recommendations. At
the same time, our detailed analyses show that, by weighing the importance of
individual aspects, we can identify optimal trade-offs between content-based
recommendation performance and aspect-based customization. Further, we show
that, if equipped with a multilingual PLM, MANNeR can successfully cross-
lingually transfer aspect-specific news encoders, supporting use cases where
aspect-specific labels (e.g., sentiment) are not available for target-language
data.

## Acknowledgements

The authors acknowledge support by the state of Baden-Württemberg through
bwHPC and the German Research Foundation (DFG) through grant INST 35/1597-1
FUGG. Andreea Iana was supported by the ReNewRS project grant, which is funded
by the Baden-Württemberg Stiftung in the Responsible Artificial Intelligence
program.

## References

  * An et al. (2019) Mingxiao An, Fangzhao Wu, Chuhan Wu, Kun Zhang, Zheng Liu, and Xing Xie. 2019.  Neural news recommendation with long-and short-term user representations.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 336–345.
  * Bahdanau et al. (2014) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014.  Neural machine translation by jointly learning to align and translate.  _ICLR_.
  * Barbieri et al. (2022) Francesco Barbieri, Luis Espinosa Anke, and Jose Camacho-Collados. 2022.  Xlm-t: Multilingual language models in twitter for sentiment analysis and beyond.  In _Proceedings of the Thirteenth Language Resources and Evaluation Conference_ , pages 258–266.
  * Bonnici (2020) Vincenzo Bonnici. 2020.  Kullback-leibler divergence between quantum distributions, and its upper-bound.  _arXiv preprint arXiv:2008.05932_.
  * Bordes et al. (2013) Antoine Bordes, Nicolas Usunier, Alberto Garcia-Durán, Jason Weston, and Oksana Yakhnenko. 2013.  Translating embeddings for modeling multi-relational data.  In _Proceedings of the 26th International Conference on Neural Information Processing Systems-Volume 2_ , pages 2787–2795.
  * Chen et al. (2017) Cheng Chen, Xiangwu Meng, Zhenghua Xu, and Thomas Lukasiewicz. 2017.  Location-aware personalized news recommendation with deep semantic analysis.  _IEEE Access_ , 5:1624–1638.
  * Choi et al. (2022) Seonghwan Choi, Hyeondey Kim, and Manjun Gim. 2022.  Do not read the same news! enhancing diversity and personalization of news recommendation.  In _Companion Proceedings of the Web Conference 2022_ , pages 1211–1215.
  * Conneau et al. (2020) Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Édouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020.  Unsupervised cross-lingual representation learning at scale.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 8440–8451.
  * Freedman and Sears (1965) Jonathan L Freedman and David O Sears. 1965.  Selective exposure.  In _Advances in experimental social psychology_ , volume 2, pages 57–97. Elsevier.
  * Gabriel De Souza et al. (2019) P Moreira Gabriel De Souza, Dietmar Jannach, and Adilson Marques Da Cunha. 2019\.  Contextual hybrid session-based news recommendation with recurrent neural networks.  _IEEE Access_ , 7:169185–169203.
  * Gao et al. (2018) Jie Gao, Xin Xin, Junshuai Liu, Rui Wang, Jing Lu, Biao Li, Xin Fan, and Ping Guo. 2018.  Fine-grained deep knowledge-aware network for news recommendation with self-attention.  In _2018 IEEE/WIC/ACM International Conference on Web Intelligence (WI)_ , pages 81–88. IEEE.
  * Gharahighehi and Vens (2023) Alireza Gharahighehi and Celine Vens. 2023.  Diversification in session-based news recommender systems.  _Personal and Ubiquitous Computing_ , 27(1):5–15.
  * Gulla et al. (2017) Jon Atle Gulla, Lemei Zhang, Peng Liu, Özlem Özgöbek, and Xiaomeng Su. 2017.  The adressa dataset for news recommendation.  In _Proceedings of the international conference on web intelligence_ , pages 1042–1048.
  * Gunel et al. (2020) Beliz Gunel, Jingfei Du, Alexis Conneau, and Veselin Stoyanov. 2020.  Supervised contrastive learning for pre-trained language model fine-tuning.  In _International Conference on Learning Representations_.
  * Heitz et al. (2022) Lucien Heitz, Juliane A Lischka, Alena Birrer, Bibek Paudel, Suzanne Tolmeijer, Laura Laugwitz, and Abraham Bernstein. 2022.  Benefits of diverse news recommendations for democracy: A user study.  _Digital Journalism_ , 10(10):1710–1730.
  * Hu et al. (2020) Linmei Hu, Siyong Xu, Chen Li, Cheng Yang, Chuan Shi, Nan Duan, Xing Xie, and Ming Zhou. 2020.  Graph neural news recommendation with unsupervised preference disentanglement.  In _Proceedings of the 58th annual meeting of the association for computational linguistics_ , pages 4255–4264.
  * Huang et al. (2013) Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. 2013\.  Learning deep structured semantic models for web search using clickthrough data.  In _Proceedings of the 22nd ACM international conference on Information & Knowledge Management_, pages 2333–2338.
  * Iana et al. (2023) Andreea Iana, Goran Glavas, and Heiko Paulheim. 2023.  Simplifying content-based neural news recommendation: On user modeling and training objectives.  In _Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 2384–2388.
  * Kaminskas and Bridge (2016) Marius Kaminskas and Derek Bridge. 2016.  Diversity, serendipity, novelty, and coverage: a survey and empirical analysis of beyond-accuracy objectives in recommender systems.  _ACM Transactions on Interactive Intelligent Systems (TiiS)_ , 7(1):1–42.
  * Khosla et al. (2020) Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020.  Supervised contrastive learning.  In _Proceedings of the 34th International Conference on Neural Information Processing Systems_ , pages 18661–18673.
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization.  _ICLR_.
  * Kummervold et al. (2021) Per E Kummervold, Javier De la Rosa, Freddy Wetjen, and Svein Arne Brygfjeld. 2021\.  Operationalizing a national digital library: The case for a norwegian transformer model.  In _Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa)_ , pages 20–29.
  * Li et al. (2022) Jian Li, Jieming Zhu, Qiwei Bi, Guohao Cai, Lifeng Shang, Zhenhua Dong, Xin Jiang, and Qun Liu. 2022.  Miner: Multi-interest matching network for news recommendation.  In _Findings of the Association for Computational Linguistics: ACL 2022_ , pages 343–352.
  * Li et al. (2011) Lei Li, Dingding Wang, Tao Li, Daniel Knox, and Balaji Padmanabhan. 2011.  SCENE: a scalable two-stage personalized news recommendation system.  In _Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval_ , pages 125–134.
  * Li and Wang (2019) Miaomiao Li and Licheng Wang. 2019.  A survey on personalized news recommendation technology.  _IEEE Access_ , 7:145861–145879.
  * Liu et al. (2020) Danyang Liu, Jianxun Lian, Shiyin Wang, Ying Qiao, Jiun-Hung Chen, Guangzhong Sun, and Xing Xie. 2020.  KRED: Knowledge-aware document representation for news recommendations.  In _Proceedings of the 14th ACM Conference on Recommender Systems_ , pages 200–209.
  * Liu et al. (2021) Ping Liu, Karthik Shivaram, Aron Culotta, Matthew A Shapiro, and Mustafa Bilgic. 2021.  The interaction between political typology and filter bubbles in news recommendation algorithms.  In _Proceedings of the Web Conference 2021_ , pages 3791–3801.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  RoBERTa: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Lu et al. (2020) Feng Lu, Anca Dumitrache, and David Graus. 2020.  Beyond optimizing for clicks: Incorporating editorial values in news recommendation.  In _Proceedings of the 28th ACM conference on user modeling, adaptation and personalization_ , pages 145–153.
  * Ludmann (2017) Cornelius A Ludmann. 2017.  Recommending news articles in the clef news recommendation evaluation lab with the data stream management system odysseus.  In _CLEF (Working Notes)_.
  * Van der Maaten and Hinton (2008) Laurens Van der Maaten and Geoffrey Hinton. 2008.  Visualizing data using t-sne.  _Journal of machine learning research_ , 9(11).
  * Nielsen (2023) Dan Saattrup Nielsen. 2023.  ScandEval: A benchmark for scandinavian natural language processing.  In _The 24rd Nordic Conference on Computational Linguistics_.
  * Okura et al. (2017) Shumpei Okura, Yukihiro Tagami, Shingo Ono, and Akira Tajima. 2017.  Embedding-based news recommendation for millions of users.  In _Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining_ , pages 1933–1942.
  * Pariser (2011) Eli Pariser. 2011.  _The filter bubble: What the Internet is hiding from you_.  penguin UK.
  * Qi et al. (2021a) Tao Qi, Fangzhao Wu, Chuhan Wu, and Yongfeng Huang. 2021a.  Personalized news recommendation with knowledge-aware interactive matching.  In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 61–70.
  * Qi et al. (2021b) Tao Qi, Fangzhao Wu, Chuhan Wu, and Yongfeng Huang. 2021b.  PP-Rec: News recommendation with personalized user interest and time-aware news popularity.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 5457–5467.
  * Qi et al. (2022) Tao Qi, Fangzhao Wu, Chuhan Wu, and Yongfeng Huang. 2022.  News recommendation with candidate-aware user modeling.  In _Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 1917–1921.
  * Qi et al. (2021c) Tao Qi, Fangzhao Wu, Chuhan Wu, Peiru Yang, Yang Yu, Xing Xie, and Yongfeng Huang. 2021c.  HieRec: Hierarchical user interest modeling for personalized news recommendation.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 5446–5456.
  * Rao et al. (2013) Junyang Rao, Aixia Jia, Yansong Feng, and Dongyan Zhao. 2013.  Taxonomy based personalized news recommendation: novelty and diversity.  In _Web Information Systems Engineering–WISE 2013: 14th International Conference, Nanjing, China, October 13-15, 2013, Proceedings, Part I 14_ , pages 209–218. Springer.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-BERT: Sentence embeddings using siamese bert-networks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3982–3992.
  * Sanh et al. (2019) Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019.  Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.  _arXiv preprint arXiv:1910.01108_.
  * Sheu and Li (2020) Heng-Shiou Sheu and Sheng Li. 2020.  Context-aware graph embedding for session-based news recommendation.  In _Proceedings of the 14th ACM Conference on Recommender Systems_ , pages 657–662.
  * Shi et al. (2022) Hao Shi, Zi-Jiao Wang, and Lan-Ru Zhai. 2022.  DCAN: Diversified news recommendation with coverage-attentive networks.  _arXiv preprint arXiv:2206.02627_.
  * Son et al. (2013) Jeong-Woo Son, A-Yeong Kim, and Seong-Bae Park. 2013.  A location-based news article recommendation with explicit localized semantic analysis.  In _Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval_ , pages 293–302.
  * (45) Lewis Tunstall, Nils Reimers, Unso Eun Seo Jo, Luke Bates, Daniel Korat, Moshe Wasserblat, and Oren Pereg.  Efficient few-shot learning without prompts.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Proceedings of the 31st International Conference on Neural Information Processing Systems_ , pages 6000–6010.
  * Wang et al. (2018) Hongwei Wang, Fuzheng Zhang, Xing Xie, and Minyi Guo. 2018.  DKN: Deep knowledge-aware network for news recommendation.  In _Proceedings of the 2018 world wide web conference_ , pages 1835–1844.
  * Wang et al. (2022) Rongyao Wang, Shoujin Wang, Wenpeng Lu, and Xueping Peng. 2022.  News recommendation via multi-interest news sequence modelling.  In _ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 7942–7946. IEEE.
  * Wu et al. (2019a) Chuhan Wu, Fangzhao Wu, Mingxiao An, Jianqiang Huang, Yongfeng Huang, and Xing Xie. 2019a.  Neural news recommendation with attentive multi-view learning.  In _Proceedings of the 28th International Joint Conference on Artificial Intelligence_ , pages 3863–3869.
  * Wu et al. (2019b) Chuhan Wu, Fangzhao Wu, Mingxiao An, Jianqiang Huang, Yongfeng Huang, and Xing Xie. 2019b.  NPA: neural news recommendation with personalized attention.  In _Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining_, pages 2576–2584.
  * Wu et al. (2019c) Chuhan Wu, Fangzhao Wu, Mingxiao An, Yongfeng Huang, and Xing Xie. 2019c.  Neural news recommendation with topic-aware news representation.  In _Proceedings of the 57th Annual meeting of the association for computational linguistics_ , pages 1154–1159.
  * Wu et al. (2019d) Chuhan Wu, Fangzhao Wu, Suyu Ge, Tao Qi, Yongfeng Huang, and Xing Xie. 2019d.  Neural news recommendation with multi-head self-attention.  In _Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP)_ , pages 6389–6394.
  * Wu et al. (2022a) Chuhan Wu, Fangzhao Wu, and Yongfeng Huang. 2022a.  Rethinking InfoNCE: How many negative samples do you need? In _Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22_ , pages 2509–2515. International Joint Conferences on Artificial Intelligence Organization.
  * Wu et al. (2023) Chuhan Wu, Fangzhao Wu, Yongfeng Huang, and Xing Xie. 2023.  Personalized news recommendation: Methods and challenges.  _ACM Transactions on Information Systems_ , 41(1):1–50.
  * Wu et al. (2020a) Chuhan Wu, Fangzhao Wu, Tao Qi, and Yongfeng Huang. 2020a.  SentiRec: Sentiment diversity-aware neural news recommendation.  In _Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing_ , pages 44–53.
  * Wu et al. (2021) Chuhan Wu, Fangzhao Wu, Tao Qi, and Yongfeng Huang. 2021.  Empowering news recommendation with pre-trained language models.  In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 1652–1656.
  * Wu et al. (2022b) Chuhan Wu, Fangzhao Wu, Tao Qi, and Yongfeng Huang. 2022b.  End-to-end learnable diversity-aware news recommendation.  _arXiv preprint arXiv:2204.00539_.
  * Wu et al. (2022c) Chuhan Wu, Fangzhao Wu, Tao Qi, Chenliang Li, and Yongfeng Huang. 2022c.  Is news recommendation a sequential recommendation task? In _Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 2382–2386.
  * Wu et al. (2022d) Chuhan Wu, Fangzhao Wu, Tao Qi, Wei-Qiang Zhang, Xing Xie, and Yongfeng Huang. 2022d.  Removing ai’s sentiment manipulation of personalized news delivery.  _Humanities and Social Sciences Communications_ , 9(1):1–9.
  * Wu et al. (2020b) Fangzhao Wu, Ying Qiao, Jiun-Hung Chen, Chuhan Wu, Tao Qi, Jianxun Lian, Danyang Liu, Xing Xie, Jianfeng Gao, Winnie Wu, and Mind Zhou. 2020b.  MIND: A large-scale dataset for news recommendation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 3597–3606.
  * Xu et al. (2023) Hongyan Xu, Qiyao Peng, Hongtao Liu, Yueheng Sun, and Wenjun Wang. 2023.  Group-based personalized news recommendation with long-and short-term fine-grained matching.  _ACM Transactions on Information Systems_.
  * Xun et al. (2021) Jiahao Xun, Shengyu Zhang, Zhou Zhao, Jieming Zhu, Qi Zhang, Jingjie Li, Xiuqiang He, Xiaofei He, Tat-Seng Chua, and Fei Wu. 2021.  Why do we click: visual impression-aware news recommendation.  In _Proceedings of the 29th ACM International Conference on Multimedia_ , pages 3881–3890.
  * Yang (2016) JungAe Yang. 2016.  Effects of popularity-based news recommendations (“most-viewed”) on users’ exposure to online news.  _Media Psychology_ , 19(2):243–271.
  * Yi et al. (2021) Jingwei Yi, Fangzhao Wu, Chuhan Wu, Ruixuan Liu, Guangzhong Sun, and Xing Xie. 2021\.  Efficient-fedrec: Efficient federated learning framework for privacy-preserving news recommendation.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 2814–2824.
  * Zhang and Hurley (2008) Mi Zhang and Neil Hurley. 2008.  Avoiding monotony: improving the diversity of recommendation lists.  In _Proceedings of the 2008 ACM conference on Recommender systems_ , pages 123–130.