# Optic Fingerprint: Enhancing Security in Visible Light Communication
Networks

Xuanbang Chen§,†, Ziqi Liu§, Xun Zhang§, Yuhao Wang†, Dayu Shi§, Xiaodong
Liu${}^{\dagger}\\!\S$,
§Institut Supérieur D’électronique de Paris, France, †Nanchang University,
China
Email<EMAIL_ADDRESS><EMAIL_ADDRESS>,
<EMAIL_ADDRESS>,
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>This work was supported by a grant of the EU Horizon 2020 program, under the
6G BRAINS project H2020-ICT 101017226. (Corresponding author: Xun ZHANG.)X.
Chen and Z. Liu contributed to the work equally and should be regarded as co-
first authors.

###### Abstract

In addressing physical layer security issues, hardware fingerprinting has been
proven to be a reliable method. Additionally, Visible Light Communication
(VLC) technology offers a solution to the spectrum congestion in next-
generation wireless communications and is noteworthy for its high security.
However, there is currently a lack of a comprehensive and systematic
description of the hardware fingerprints and their extraction mechanisms for
VLC devices. This study aims to bridge this gap by thoroughly analyzing the
hardware fingerprints of VLC devices and proposing an innovative extraction
mechanism, thereby enhancing the security and reliability of the physical
layer. An Optic Fingerprint (OF) model is proposed based on the LED’s inherent
circuit characteristics, capable of extracting and processing unique feature
vectors with high precision. Through extensive experiments, we demonstrate the
model’s efficacy, achieving up to 99.3% accuracy in identifying the same
manufactured white LEDs under variable conditions, marking a significant
improvement in authentication robustness and interference resistance.

###### Index Terms:

Visible light communication, physics-based LED circuit model, optic
fingerprint, machine learning.

##  I Introduction

THE transaction from 5G to 6G networks marks a significant leap in
communication technology but raises emergent and complex security challenges
[1, 2]. The classical cryptographic solutions fall short in these novel
challenges due to their complexity-based paradigms. Consequently, innovative
security approaches, particularly tailored for 6G systems, are desired from
both the industry and academia. Device Fingerprint (DF) is the most potential
technology in Physical Layer Security (PLS) enhancing approaches, owing to its
unique and secure identification capabilities. It leverages the inherent
hardware characteristics of devices, guaranteeing non-replicability and
distinctive identification [3, 4], offers a more robust and adaptable security
framework, well-suited to meet the demands of 6G and IoT applications [5, 6,
7, 8, 9].

Most of the existing research on DF focuses on Radio Frequency (RF)
application scenarios. Researchers proposed the concept of Radio Frequency
Fingerprint (RFF) to achieve the 92.29% accuracy of base station
authentication for Wi-Fi, LTE, ZigBee, and etc. [10, 11, 12, 13, 14, 15].
Whereas in the future 6G era, the available spectrum extended to the optical
band promoting heterogeneous physical layer technologies[16]. Visible Light
Communication (VLC) [17], leveraging signal spectrum from 380nm to 780nm to
transmit signal through the optical wireless channel, possesses attributes
like high capacity, ultra-high data rates, low latency, and inherent security
due to the limited penetration of optical signals.

However, few researches address the issues of DF in the VLC domain. Existing
studies utilize frequency response measurements, notably the S21 parameter, to
develop a DF model[18][19] and employ machine learning or deep learning
approaches[20] to detect DF features, aiming to authenticate device identities
effectively. These studies, however, are largely limited to static
environments, overlooking the impact of environmental and spatial
fluctuations, such as distance, angle, and signal noise interference. Such
neglect results in compromised anti-noise and spatial stability, thereby
undermining the adaptability of these models in complex environments.

This paper proposes a new DF model in the VLC system, which is named the Optic
Fingerprint (OF) model, and the corresponding machine learning-based feature
extraction mechanism. Significant contributions of this study include:

  * •

An OF model is proposed to characterize the unique nonlinearity attributes of
LEDs to form a reliable feature vector.

  * •

The extraction scheme for OF and the security identification framework is
developed, utilizing power spectrum analysis and a machine learning classifier
for precise device identification against an authorized fingerprint database.

  * •

Numerous experiments are performed to validate our proposed OF model. The
proposed OF model shows a high accuracy of 99.3% in identifying commercial
white LEDs. Compared to the traditional S21 fingerprint, the proposed OF model
presents a lower complexity with better performance of anti-environmental
interference.

The rest of this paper is organized as follows. Section II outlines the OF
model and extraction methodology. Section III details the feature extraction
and verification process. Section IV presents the identification accuracy of
the proposed fingerprint. Section V concludes the paper, discussing the
implications of our future directions.

##  II Physics-based Optic Fingerprint Model

Figure 1: The typical VLC link diagram.

A novel OF model is proposed based on the physics variations of LED devices
for security enhancement in this context. Generally, as shown in Fig. 1, an
LED-based VLC system comprises a VLC transmitter, Free Space, and VLC
receiver. Specifically, the power amplifier amplifies the signal to drive LEDs
with lower AC impedance, while the LED driver converts the voltage signal into
a current signal, achieving amplitude modulation of the LED luminous
intensity. The optical modulated signal emitted by the VLC transmitter travels
through the channel and is intensity-detected by the VLC receiver. It is
noteworthy that despite belonging to the same batch, the hardware components
of the VLC transmitter, including the power amplifier, driver, and LED, may
exhibit slight variations due to manufacturing tolerances. These hardware
differences influence the optical signal, which is subsequently transmitted to
the receiver. The received optical signal, denoted as $y(t)$, can be expressed
as

$\displaystyle y(t)=G_{\rm PA}G_{\rm D}H_{\rm C}G_{\rm
Re}\int_{-\infty}^{\infty}x(t)h_{\rm LED}(t-\tau)d\tau,$ (1)

$x(t)$ denotes the electrical signal source. $G_{\rm PA}$, $G_{\rm D}$
represent the gain of the power amplifier and driver, respectively. Moreover,
$h_{\rm LED}(t)$ is the impulse response of the LED. It is worth noting that
the LED, as a bandwidth-limited communication device, is the main source of
hardware nonlinearity and variance. Since the non-line of sight (NLoS)
component of the VLC channel is very weak, the channel response $H_{\rm C}$ is
considered to be a constant loss when only considering the line of sight (LoS)
component, which can be calculated as

$\displaystyle H_{\rm C}=\frac{(m+1)A_{\rm r}}{2\pi
d^{2}}\cos^{m}(\phi)g(\psi)\cos(\psi),$ (2)

where $m=-\rm{ln2/ln(cos\phi_{1/2})}$ is the Lambertian emission order and
$\phi_{1/2}$ is the emission semi-angle of LED. $A_{\rm r}$ is the physical
detection area of the receiver front-end. The channel loss is related to the
distance $d$ (between the LED and the receiver front-end) and the irradiance
angle $\phi$ of the light [21].

Thus, amplifier, driver, and channel impulse responses can all be considered
linear parameters. Therefore, modeling the nonlinearity of LEDs is key to
characterizing the distinctive feature of each device. Numerous generic
mathematical modeling methods have been proposed to characterize the
modulation nonlinearity of LEDs, such as the memory polynomial model, Volterra
model, and Hammerstein model [22]. However, these modeling methods typically
exhibit high implementation complexity and fail to capture the inherent common
modulation nonlinearity of LEDs, which are strongly correlated with the
physical structure of the LED. Our previous work modeled the GaN LED from the
physics aspect, through the analysis of carrier concentrations in each LED
layer, the carrier diffusion, capture, thermal escape, and recombination are
formulated by carrier rate equations [23]. Based on the previous work, the
physics-based LED model is improved to completely characterize the high-
frequency characteristics in this paper, and a novel OF model is proposed.

Figure 2: (a) The physical structure and (b) the equivalent circuit model of
the MQW LED.

As shown in Fig. 2(a), silicon carbide (SiC) and sapphire are the common
substrates used in the LED. Due to the MQW being wrapped by a cladding layer,
it is necessary to consider the capacitance and resistance effects of the
quantum well in the LED model. Thus, $R_{\rm q}$ and $C_{\rm q}$ represent the
equivalent resistance and capacitance of MQW, respectively. Specifically, the
current through $R_{\rm q}$ represents the recombination of electrons, and
$C_{\rm q}$ reflects the charge storage effect in the quantum well [24].
Moreover, $R_{\rm c}$ is the equivalent resistance distributed in p- and
n-Cladding layers. $C_{\rm j}$ is the sum of the barrier capacitance and the
parasitic capacitance. Thus, the corresponding equivalent circuit model is
schematically illustrated in Fig. 2(b). It is worth noting that $h$ represents
the external quantum efficiency. $\eta_{ph}$ is the response of phosphor. The
blue wavelength photons emitted by the LED are excited by the phosphor to
produce yellow light and are mixed into white light to provide white lighting
services to users. The analytical expression of the LED’s impedance $Z_{\rm
LED}(w)$ can be written as follows.

$\displaystyle Z_{\rm LED}(w)=\frac{1}{jwC_{\rm j}}//(R_{\rm c}+R_{\rm
q}//\frac{1}{jwC_{\rm q}})$ (3)
$\displaystyle=\frac{R_{c}+R_{q}+jwR_{c}R_{q}C_{q}}{1-w^{2}R_{c}R_{q}C_{q}C_{j}+jw(R_{q}C_{q}+R_{c}C_{j}+R_{q}C_{j})},$

where $w$ is the angular frequency and $j$ is the imaginary unit. The operator
$//$ denotes the parallel calculation in the circuit. Thus, the parameters of
the LED equivalent model can be extracted from the measured impedance curve.
However, this method is less practical, as it necessitates additional and
complex experiments to obtain device impedance features. In this context, the
frequency response of the LED is exploited to extract model features, allowing
the utilization of existing communication data for obtaining the transmission
response without additional experiments. Specifically, the transfer function
of the LED-based VLC system can be expressed as

$\displaystyle H_{\rm VLC}(w)=G_{\rm PA}G_{\rm D}H_{\rm C}G_{\rm Re}H_{\rm
LED}(w)$ (4) $\displaystyle=\frac{G_{\rm PA}G_{\rm D}H_{\rm C}G_{\rm
Re}h}{1-w^{2}R_{c}R_{q}C_{q}C_{j}+jw(R_{q}C_{q}+R_{c}C_{j}+R_{q}C_{j})}.$

As $G_{\rm PA}$, $G_{\rm D}$, $H_{\rm C}$, $G_{\rm Re}$, and $h$ remain
constant at a specific measurement point, a uniform $\zeta$ is employed to
represent these parameters, simplifying the transmission formulation.
Therefore, the feature parameters of the proposed LED model are summarized as
a five-dimensional vector $R_{c}$, $C_{j}$, $R_{q}$, $C_{q}$, and $\zeta$,
which can be derived from the VLC system response $H_{\rm VLC}$. Additionally,
$H_{\rm C}$ excludes the phosphor response component, as the receiver
typically employs a blue light filter to eliminate signal transmission delays.
Therefore, the feature parameters $R_{c}$, $C_{j}$, $R_{q}$, $C_{q}$, and
$\zeta$ of the proposed LED model can be derived from the VLC system response
$H_{\rm VLC}(w)$.

$\displaystyle\lambda=[R_{c},C_{j},R_{q}].$ (5)

The values of $R_{c}$, $C_{j}$, and $R_{q}$ form the OF model represented by
the three-dimensional feature vector $\lambda$. It is important to note that
the factors $\zeta$ and $C_{q}$ are omitted from the OF model. This exclusion
is justified by the fact that $\zeta$ primarily relies on the testing
equipment types and specific positions, rather than the intrinsic
characteristics of the LED. Additionally, Section IV provides further evidence
that $C_{q}$ lacks distinct characteristics among different LED devices.
$\lambda$ depends on the inherent non-linearity characteristic of each LED,
which indicates its unique fingerprints.

##  III Extraction and Identification

###  III-A Optic Fingerprint Extraction

Fig. 3 illustrates the OF feature extraction process. The LED is the device
for the proposed OF feature extraction, which acts as the access point within
the VLC system. The test signal, encompassing options such as a swept,
baseband, or modulated signal, undergoes LED-induced nonlinearity and is
captured by the VLC receiver. The system response data S21, derived from
processing the received signal, is utilized for parameter extraction in the
proposed OF model. The extraction algorithm employs the nonlinear least
squares method. Specifically, given an initial fingerprint value, the
fingerprint feature value continuously updates to minimize the residual
$\sum_{i=1}^{n}e^{2}(f_{i})$ between the fitted system response data and the
measured data. Note that $n$ denotes the frequency points of S21 data. The
optimal extraction result is then recorded as the LED’s fingerprint feature.
Subsequently, numerous LEDs are incorporated into the OF database after
feature extraction, facilitating device security authentication.

Figure 3: The extraction scheme of the proposed OF model.

###  III-B Security Identification Scheme

Fig. 4 presents an intricate security identification architecture predicated
on our OF identification methodology. This system is engineered to
preemptively negate network security breaches perpetrated by eavesdroppers
(Eve) through identity falsification and unauthorized data acquisition. In the
edge network, OF data for registered devices are meticulously cataloged within
the OF database. The Security Validation Server (SVS) engages in conjunction
with the VLC Access Point to periodically solicit OF data from user equipment
(UE). The SVS rigorously evaluates the OF against the established registry of
the authenticated user (RegU) fingerprints within the database, subsequently
conveying the verification results to the Network Management System (NMS)
housed within the Internet Cloud to complete the security validation process.
The NMS, predicated on the analysis, orchestrates access authorization via the
Access Control Server (ACS) or initiates preemptive alerts through the
Identity Services Engine (ISE). The detection mechanism’s protocol unfolds as
follows:

  1. 1.

The VLC Access Point periodically retrieves OF data from UEs.

  2. 2.

The extracted data are then transmitted to the SVS within the edge network.

  3. 3.

The SVS concurrently retrieves and analyzes corresponding OF data from the
database.

  4. 4.

The SVS’s verdicts are dispatched to the NMS to facilitate security services.

  5. 5.

The NMS adjudicates network access or restricts it through the ACS based on
the analysis.

Figure 4: Security identification scheme based on OF model.

Further, the framework accommodates the registration of new users (NewU),
integrating them seamlessly into the network. The registration process
entails:

  1. 1.

Upon a NewU’s registration request, the VLC Access Point captures pertinent
user data and transmutes it into OF data.

  2. 2.

This data is relayed to the SVS for processing.

  3. 3.

The SVS concurrently retrieves and analyzes corresponding OF data from the
database.

  4. 4.

The SVS updates the OF database with new entries to the OF database

  5. 5.

The SVS concurrently communicates with the NMS.

  6. 6.

The NMS, leveraging the ISE, disseminates new registration details and
provisions services accordingly.

This meticulously orchestrated sequence of operations fortifies the network’s
defense mechanisms against illicit access while contemporaneously ensuring the
OF database remains current, thus fostering a secure and adaptable network
milieu for both extant and nascent users.

##  IV Experiment Setup and Results

###  IV-A Experiment Setup

The experiment test-bed for acquiring the OF is shown in Fig. 5. Specifically,
the vector network analyzer (VNA, Rohde$\&$Schwarz, ZNB20) is employed to
measure the frequency responses of the LED. The measurements utilize a
continuous wave (CW) frequency-sweep technique. Four LED samples (Cree,
XPE2-White) are exploited to establish the OF database. Here, the VNA injects
a CW signal with an electrical power of -5 dBm into the power amplifier (Mini-
circuits, ZHL-6A-S+), sweeping the modulation frequency from 100 KHz to 100
MHz. Bias-T (Mini-Circuits, ZX85-12G-S+) is exploited to drive the LED with
the amplified signal. The VNA then assesses the amplitude and phase variations
between the transmitted and received signals, providing the frequency response
of the test setup, which includes the wireless link and the optical front
ends. The measurements are taken at 15 distinct positions, with each position
measured 10 times, totaling 150 measurements per LED. It is worth noting that
the longest test distance is 60 cm due to the limited power of a single LED.
These measurements form the basis for extracting the normalized feature
vectors of each LED.

Figure 5: Schematic diagram of the experiment setup.

Moreover, for each LED, 50% of the dataset was randomly allocated as the
training set, with the feature vectors serving as signatures for authenticated
devices. A range of machine learning algorithms—Fine Tree (F-Tree), Kernel
Naive Bayes (KNB), Quadratic Support Vector Machine (Q-SVM), Fine K Nearest
Neighbor (F-KNN), Ensemble Bagged Trees (EB-Trees), Ensemble Subspace K
Nearest Neighbor (ES-KNN), and Narrow Neural Network (NN-Network)—was applied
to stratify the training data into four clusters. Following this registration,
the remaining data constituted the test set. Alongside, the original,
unprocessed S21 parameter data were subjected to the same machine learning
classification to provide a baseline for performance comparison.

To rigorously evaluate the identification process, noise disturbances were
superimposed onto the test dataset and the unprocessed S21 data. These
disturbances ranged from 0 dBm to 90 dBm in 5 dBm increments, resulting in a
total of 2550 sets of data, emulating various channel conditions, and testing
the robustness of the machine learning models. The analysis compared the
performance of the models on both the noise-augmented dataset and the original
S21 data, highlighting the efficacy of the OF features in maintaining high
identification accuracy despite the presence of environmental noise.

###  IV-B Results and Analysis

The training OF database comprises 300 samples from 4 LED samples. OF
extraction is accomplished by fitting S21 to the objective function Eq. (4).
Fig. 6 illustrates the fitting results, compared to 300 S21 measured samples.
The frequency response of four LEDs shows different amplitudes due to the
various test locations. Moreover, it can be seen from the subfigure in Fig. 6
that fitted dots show good coincidence with the measured lines, yielding an
average Mean Squared Error (MSE) of less than 1e-4.

Figure 6: The fitting results of four LEDs in different positions.

Furthermore, Fig. 7 presents the five extracted parameters of the equivalent
circuit model based on the aforementioned curve fitting. It can be seen that
the circuit model parameters $C_{q}$ and $\zeta$ of four LED samples are
difficult to distinguish. This is because the impact of different test points
on $\zeta$ blurs its individual, and $C_{q}$ is in the nF range making its
characteristics challenging to distinguish between LEDs. Thus, the feature
vectors [$R_{c}$, $C_{j}$, $R_{q}$] are chosen as the proposed OF model. Fig.
8 shows the clustering result of the proposed OF for four LED samples, which
are clearly clustering at different locations in the space.

((a)) LED1 ((b)) LED2 ((c)) LED3 ((d)) LED4

Figure 7: The extracted features of four LED samples.

Figure 8: The clustering of the proposed OF model [$R_{c}$, $C_{j}$, $R_{q}$].

In the subsequent analysis, various machine-learning algorithms are employed
to assess the classification accuracies and validate the efficacy of the
proposed OF model. As depicted in Fig. 9, the OF model consistently achieves
remarkable accuracy ranging from 99.3% to 95.0% with different algorithms. In
contrast, the S21 fingerprint model [18], under the same machine-learning
evaluations, exhibits a broader spectrum of accuracy rates, peaking at 93.7%
and dropping to as low as 42.0%. Notably, the OF model requires only three
features, whereas the S21 fingerprint model relies on 750 features. This
substantial reduction in feature count, combined with superior accuracy and
stability, highlights the advantages of our proposed OF model over the
conventional S21 model, offering improved efficiency and reduced data
complexity.

Figure 9: The accuracy comparison of different algorithms using traditional
S21 fingerprint and OF models.

Finally, the reliability of the OF model is analyzed by adding Gaussian white
noise of varying powers to both the OF and the traditional S21 fingerprint
models. Fig. 10 illustrates that the proposed OF model consistently achieves
accuracies exceeding 80% across a noise power range from -90 dBm to -20 dBm.
In contrast, the S21 fingerprint model demonstrates inferior noise immunity,
with accuracy declining to below 62%. Furthermore, while the performance of
the OF model begins to degrade for noise levels surpassing -20 dBm, it remains
superior to the S21 fingerprint model. This indicates that fingerprint feature
vectors extracted from LED models inherently capture the LED’s
characteristics, exhibiting robustness against changes in external channel
conditions.

Figure 10: The comparison of anti-noise performance between traditional S21
fingerprint and proposed OF models.

##  V Conclusion

This paper introduces a novel Optic Fingerprint (OF) to apply for enhancing
the physical layer security of the sixth-generation network. Four LED samples
were exploited to verify the mechanism of the proposed OF model. High
classification accuracies were achieved based on the generic machine learning
algorithms, which up to 99.3%. Moreover, the OF model was compared with the
traditional S21 fingerprint model, showcasing a lower complexity with better
performance of anti-environmental interference. It is worth noting that the
LED samples with more different types and manufacturers will be discussed in
future work. The effects introduced by the different receivers will be
considered as well, ensuring its effectiveness in the sixth-generation
security framework.

## References

  * [1] 3rd Generation Partnership Project (3GPP), “Study on 5g security enhancements against false base stations,” Tech. Rep. Tech. Report 33.809, 3GPP, Sophia Antipolis, France, 2018.
  * [2] P. Porambage, G. Gür, D. P. M. Osorio, M. Livanage, and M. Ylianttila, “6g security challenges and potential solutions,” in 2021 Joint European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit), pp. 622–627, IEEE, 2021.
  * [3] M. Liyanage, A. B. Abro, M. Ylianttila, and A. Gurtov, “Opportunities and challenges of software-defined mobile networks in network security,” IEEE security & privacy, vol. 14, no. 4, pp. 34–44, 2016.
  * [4] R. Khan, P. Kumar, D. N. K. Jayakody, and M. Liyanage, “A survey on security and privacy of 5g technologies: Potential solutions, recent advancements, and future directions,” IEEE Communications Surveys & Tutorials, vol. 22, no. 1, pp. 196–248, 2019.
  * [5] M. Bloch and J. Barros, Physical-Layer Security: From Information Theory to Security Engineering.  Cambridge, U.K.: Cambridge University Press, 2011.
  * [6] B. Aazhang et al., “Key drivers and research challenges for 6g ubiquitous wireless intelligence,” tech. rep., White Paper, Sep 2019.
  * [7] V. Ziegler, P. Schneider, H. Viswanathan, M. Montag, S. Kanugovi, and A. Rezaki, “Security and trust in the 6g era,” IEEE Access, vol. 9, pp. 142314–142327, 2021.
  * [8] B. Khorsandi et al., “D1.3, targets and requirements for 6g - initial e2e architecture, hexa-x,” tech. rep., Feb 2022.
  * [9] IEEE International Network Generations Roadmap, “Security and privacy,” 2022.  Available: https://futurenetworks.ieee.org/roadmap.
  * [10] L. Xie, L. Peng, J. Zhang, and A. Hu, “Radio frequency fingerprint identification for internet of things: A survey,” Security and Safety, vol. 3, p. 2023022, 2024.
  * [11] J. Zhang, G. Shen, W. Saad, and K. Chowdhury, “Radio frequency fingerprint identification for device authentication in the internet of things,” IEEE Communications Magazine, 2023.
  * [12] Q. Xu, R. Zheng, W. Saad, and Z. Han, “Device fingerprinting in wireless networks: Challenges and opportunities,” IEEE Communications Surveys & Tutorials, vol. 18, pp. 94–104, 2015.
  * [13] H. Sun, X. Zhu, Y. Liu, and W. Liu, “Construction of hybrid dual radio frequency rssi (hdrf-rssi) fingerprint database and indoor location method,” Sensors, vol. 20, p. 2981, 2020.
  * [14] J. Yu, A. Hu, C. Zhu, L. Peng, and Y. Jiang, “Rf fingerprinting extraction and identification of wireless communication devices,” Journal of Cryptologic Research, vol. 3, pp. 433–446, 2016.
  * [15] K. Merchant, S. Revay, G. Stantchev, and B. Nousain, “Deep learning for rf device fingerprinting in cognitive communication networks,” IEEE Journal of Selected Topics in Signal Processing, vol. 12, pp. 160–167, 2018.
  * [16] W. Jiang, B. Han, M. A. Habibi, and H. D. Schotten, “The road towards 6g: A comprehensive survey,” IEEE Open Journal of the Communications Society, vol. 2, pp. 334–366, 2021.
  * [17] N. Chi, Y. Zhou, Y. Wei, and F. Hu, “Visible light communication in 6g: Advances, challenges, and prospects,” IEEE Vehicular Technology Magazine, vol. 15, no. 4, pp. 93–102, 2020.
  * [18] D. Shi, X. Zhang, A. Vladimirescu, L. Shi, Y. Huang, and Y. Liu, “A device identification method based on led fingerprint for visible light communication system,” in Proceedings of the 15th International Conference on Availability, Reliability and Security, pp. 1–7, 2020.
  * [19] D. Shi, X. Zhang, L. Shi, A. Vladimirescu, W. Mazurczyk, K. Cabaj, B. Meunier, K. Ali, J. Cosmas, and Y. Zhang, “On improving 5g internet of radio light security based on led fingerprint identification method,” Sensors, vol. 21, no. 4, p. 1515, 2021.
  * [20] Z. Liu, D. Shi, S. Oukemeni, and X. Zhang, “Alexnet-based visible light communication devices fingerprint extraction and authentication in broadcast systems,” in 2022 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), pp. 01–05, IEEE, 2022.
  * [21] M. Zhu, Y. Wang, X. Liu, Z. Qi, X. Chen, and J.-Y. Wang, “Physical layer security performance analysis for relay-aided visible light communication system,” IEEE Photonics Journal, vol. 15, no. 3, pp. 1–9, 2023.
  * [22] X. Deng, S. Mardanikorani, Y. Wu, K. Arulandu, B. Chen, A. M. Khalid, and J.-P. M. G. Linnartz, “Mitigating led nonlinearity to enhance visible light communications,” IEEE Transactions on Communications, vol. 66, no. 11, pp. 5593–5607, 2018.
  * [23] D. Shi, X. Zhang, Z. Liu, X. Chen, X. Liu, J. Wang, J. Song, and A. Vladimirescu, “Physics-based modeling of gan mqw led for visible light communication systems,” IEEE Transactions on Electron Devices, vol. 71, no. 1, pp. 337–342, 2024.
  * [24] A. Rashidi, M. Nami, M. Monavarian, A. Aragon, K. DaVico, F. Ayoub, S. Mishkat-Ul-Masabih, A. Rishinaramangalam, and D. Feezell, “Differential carrier lifetime and transport effects in electrically injected iii-nitride light-emitting diodes,” Journal of Applied Physics, vol. 122, no. 3, 2017.

## References

  * [1] 3rd Generation Partnership Project (3GPP), “Study on 5g security enhancements against false base stations,” Tech. Rep. Tech. Report 33.809, 3GPP, Sophia Antipolis, France, 2018.
  * [2] P. Porambage, G. Gür, D. P. M. Osorio, M. Livanage, and M. Ylianttila, “6g security challenges and potential solutions,” in 2021 Joint European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit), pp. 622–627, IEEE, 2021.
  * [3] M. Liyanage, A. B. Abro, M. Ylianttila, and A. Gurtov, “Opportunities and challenges of software-defined mobile networks in network security,” IEEE security & privacy, vol. 14, no. 4, pp. 34–44, 2016.
  * [4] R. Khan, P. Kumar, D. N. K. Jayakody, and M. Liyanage, “A survey on security and privacy of 5g technologies: Potential solutions, recent advancements, and future directions,” IEEE Communications Surveys & Tutorials, vol. 22, no. 1, pp. 196–248, 2019.
  * [5] M. Bloch and J. Barros, Physical-Layer Security: From Information Theory to Security Engineering.  Cambridge, U.K.: Cambridge University Press, 2011.
  * [6] B. Aazhang et al., “Key drivers and research challenges for 6g ubiquitous wireless intelligence,” tech. rep., White Paper, Sep 2019.
  * [7] V. Ziegler, P. Schneider, H. Viswanathan, M. Montag, S. Kanugovi, and A. Rezaki, “Security and trust in the 6g era,” IEEE Access, vol. 9, pp. 142314–142327, 2021.
  * [8] B. Khorsandi et al., “D1.3, targets and requirements for 6g - initial e2e architecture, hexa-x,” tech. rep., Feb 2022.
  * [9] IEEE International Network Generations Roadmap, “Security and privacy,” 2022.  Available: https://futurenetworks.ieee.org/roadmap.
  * [10] L. Xie, L. Peng, J. Zhang, and A. Hu, “Radio frequency fingerprint identification for internet of things: A survey,” Security and Safety, vol. 3, p. 2023022, 2024.
  * [11] J. Zhang, G. Shen, W. Saad, and K. Chowdhury, “Radio frequency fingerprint identification for device authentication in the internet of things,” IEEE Communications Magazine, 2023.
  * [12] Q. Xu, R. Zheng, W. Saad, and Z. Han, “Device fingerprinting in wireless networks: Challenges and opportunities,” IEEE Communications Surveys & Tutorials, vol. 18, pp. 94–104, 2015.
  * [13] H. Sun, X. Zhu, Y. Liu, and W. Liu, “Construction of hybrid dual radio frequency rssi (hdrf-rssi) fingerprint database and indoor location method,” Sensors, vol. 20, p. 2981, 2020.
  * [14] J. Yu, A. Hu, C. Zhu, L. Peng, and Y. Jiang, “Rf fingerprinting extraction and identification of wireless communication devices,” Journal of Cryptologic Research, vol. 3, pp. 433–446, 2016.
  * [15] K. Merchant, S. Revay, G. Stantchev, and B. Nousain, “Deep learning for rf device fingerprinting in cognitive communication networks,” IEEE Journal of Selected Topics in Signal Processing, vol. 12, pp. 160–167, 2018.
  * [16] W. Jiang, B. Han, M. A. Habibi, and H. D. Schotten, “The road towards 6g: A comprehensive survey,” IEEE Open Journal of the Communications Society, vol. 2, pp. 334–366, 2021.
  * [17] N. Chi, Y. Zhou, Y. Wei, and F. Hu, “Visible light communication in 6g: Advances, challenges, and prospects,” IEEE Vehicular Technology Magazine, vol. 15, no. 4, pp. 93–102, 2020.
  * [18] D. Shi, X. Zhang, A. Vladimirescu, L. Shi, Y. Huang, and Y. Liu, “A device identification method based on led fingerprint for visible light communication system,” in Proceedings of the 15th International Conference on Availability, Reliability and Security, pp. 1–7, 2020.
  * [19] D. Shi, X. Zhang, L. Shi, A. Vladimirescu, W. Mazurczyk, K. Cabaj, B. Meunier, K. Ali, J. Cosmas, and Y. Zhang, “On improving 5g internet of radio light security based on led fingerprint identification method,” Sensors, vol. 21, no. 4, p. 1515, 2021.
  * [20] Z. Liu, D. Shi, S. Oukemeni, and X. Zhang, “Alexnet-based visible light communication devices fingerprint extraction and authentication in broadcast systems,” in 2022 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), pp. 01–05, IEEE, 2022.
  * [21] M. Zhu, Y. Wang, X. Liu, Z. Qi, X. Chen, and J.-Y. Wang, “Physical layer security performance analysis for relay-aided visible light communication system,” IEEE Photonics Journal, vol. 15, no. 3, pp. 1–9, 2023.
  * [22] X. Deng, S. Mardanikorani, Y. Wu, K. Arulandu, B. Chen, A. M. Khalid, and J.-P. M. G. Linnartz, “Mitigating led nonlinearity to enhance visible light communications,” IEEE Transactions on Communications, vol. 66, no. 11, pp. 5593–5607, 2018.
  * [23] D. Shi, X. Zhang, Z. Liu, X. Chen, X. Liu, J. Wang, J. Song, and A. Vladimirescu, “Physics-based modeling of gan mqw led for visible light communication systems,” IEEE Transactions on Electron Devices, vol. 71, no. 1, pp. 337–342, 2024.
  * [24] A. Rashidi, M. Nami, M. Monavarian, A. Aragon, K. DaVico, F. Ayoub, S. Mishkat-Ul-Masabih, A. Rishinaramangalam, and D. Feezell, “Differential carrier lifetime and transport effects in electrically injected iii-nitride light-emitting diodes,” Journal of Applied Physics, vol. 122, no. 3, 2017.


# Towards Analyzing the Bias of News Recommender Systems Using Sentiment and
Stance Detection

Mehwish Alam<EMAIL_ADDRESS>FIZ Karlsruhe, Karlsruhe Institute of
TechnologyGermany ,  Andreea Iana<EMAIL_ADDRESS>University of MannheimGermany ,  Alexander Grote<EMAIL_ADDRESS>FZI
Forschungszentrum Informatik, Karlsruhe Institute of TechnologyGermany ,
Katharina Ludwig<EMAIL_ADDRESS>University of MannheimGermany
,  Philipp Müller<EMAIL_ADDRESS>University of MannheimGermany and
Heiko Paulheim<EMAIL_ADDRESS>University of MannheimGermany

(2022)

###### Abstract.

News recommender systems are used by online news providers to alleviate
information overload and to provide personalized content to users. However,
algorithmic news curation has been hypothesized to create filter bubbles and
to intensify users’ selective exposure, potentially increasing their
vulnerability to polarized opinions and fake news. In this paper, we show how
information on news items’ stance and sentiment can be utilized to analyze and
quantify the extent to which recommender systems suffer from biases. To that
end, we have annotated a German news corpus on the topic of migration using
stance detection and sentiment analysis. In an experimental evaluation with
four different recommender systems, our results show a slight tendency of all
four models for recommending articles with negative sentiments and stances
against the topic of refugees and migration. Moreover, we observed a positive
correlation between the sentiment and stance bias of the text-based
recommenders and the preexisting user bias, which indicates that these systems
amplify users’ opinions and decrease the diversity of recommended news. The
knowledge-aware model appears to be the least prone to such biases, at the
cost of predictive accuracy.

news recommendation, filter bubbles, echo chambers, polarization, stance
detection, sentiment analysis, German news articles

††journalyear: 2022††copyright: rightsretained††conference: Companion
Proceedings of the Web Conference 2022; April 25–29, 2022; Virtual Event,
Lyon, France††booktitle: Companion Proceedings of the Web Conference 2022 (WWW
’22 Companion), April 25–29, 2022, Virtual Event, Lyon, France††doi:
10.1145/3487553.3524674††isbn: 978-1-4503-9130-6/22/04††ccs: Information
systems Recommender systems††ccs: Information systems Sentiment analysis††ccs:
Information systems Personalization

##  1\. Introduction

Every day, large volumes of news articles are being published online, leading
to an information load that exceeds the consumptive capacities of Internet
users. To alleviate this information overload and to provide customized
content to users based on their past interests, recommender systems are widely
employed by news providers. However, this algorithmic news curation influences
users’ exposure to diverse content, by selectively filtering out articles that
seem irrelevant (in order to maximize user engagement) (Pariser, 2011) or
inconsistent with the readers’ preexisting beliefs and attitudes, as users are
more likely to accept information that reinforces their opinions (Freedman and
Sears, 1965). The power of recommender systems to shape users’ perception of
the world has over time lead to concerns that individuals are being isolated
from diverse perspectives through algorithmically created ”filter bubbles”
(Pariser, 2011) and self-selected ”echo chambers” (Sunstein, 2001; Jamieson
and Cappella, 2008), in which they only interact with individuals with similar
ideological viewpoints. In the context of (political) news, an over-exposure
to less diverse viewpoints may determine an attitude-reinforcing spiral
(Stroud, 2017; Donsbach and Mothes, 2013) which, in the long run, can lead to
opinion polarization, or even radicalization of individuals with extreme
political or ideological viewpoints (Bakshy et al., 2015; Boutyline and
Willer, 2017; Liu et al., 2021).

Diversity represents a paramount characteristic not only for news quality
(McQuail, 1992), but also for ensuring a balanced and broad variety of
information to create a well-informed public in any democratic society
(Helberger, 2019). Nonetheless, news diversity is a multi-faceted concept
(Haim et al., 2018) which can refer, simultaneously, to pluralism of sources
(source diversity) (Voakes et al., 1996), of discussed topics (content
diversity) (Haim et al., 2018), and of stances taken on a given topic
(viewpoint diversity) (Baden and Springer, 2017). Some previous works have
focused on constructing datasets for detecting and analyzing the more general
notion of media bias in news articles (Spinde et al., 2020; Lim et al., 2020;
Färber et al., 2020), and on establishing a connection between the research on
media bias in the social science and computer science fields (Hamborg et al.,
2019).

From a recommender-system point of view, sentiment analysis and stance
detection have been used to measure and control news diversity. In sentiment
analysis, systems determine whether a piece of text is positive, negative, or
neutral (Liu et al., 2010; Ravi and Ravi, 2015). However, in stance detection,
systems are to determine an article’s or author’s viewpoints towards a given
target issue, which may not even be explicitly mentioned in the text (Küçük
and Can, 2020; Reuver et al., 2021). Stance and sentiment annotations can be
used to identify whether news recommender systems intensify selective exposure
and increase users’ vulnerability to polarized opinions and fake news. At the
same time, stance and sentiment labels can constitute cues for controlling the
level of polarization in personalized recommender systems, and for generating
suggestions with more diverse sentiment orientations and viewpoints.

In this paper, we use sentiment and stance annotations to analyze whether
different kinds of recommender systems suffer from an underlying bias towards
a certain sentiment or viewpoint, and whether this decreases diversity of
recommendations and intensifies users’ selective exposure (Pitoura et al.,
2021). To this end, we use transfer learning to annotate GeNeG, a dataset of
German news articles, with sentiment scores and stance labels. These
annotations are then used to examine the results of four recommender systems
with regards to the sentiment and stances identified in the suggested
articles, as well as those from the user’s reading history.

The rest of the paper is structured as follows. Firstly, we discuss related
work on stance detection in various languages (Section 2). Secondly, we
introduce GeNeG and describe its curation and annotation process (Section 3).
Thirdly, in Section 4 we evaluate a set of news recommender systems, and
examine their sentiment and stance bias. We conclude with a discussion of the
findings and an outlook of the next steps planned for this study in Section 5.

##  2\. Related Work

In the following, various studies are discussed which take into account the
creation of datasets for stance detection in different languages since the
multilinguality aspect of stance detection is fairly understudied. We then
more particularly focus on the multilinguality aspect in stance detection. For
an exhaustive survey on techniques for stance detection please refer to (Küçük
and Can, 2020). In the current study, we do not discuss knowledge-aware news
recommender systems, however, for more details on knowledge-aware news
recommender systems please refer to (Iana et al., 2022a).

###  2.1. Existing Datasets for Stance Detection

STANDER (Conforti et al., 2020) is a dataset for stance detection and fine-
grained evidence retrieval task for future research in stance detection, as
well as multitask learning. It contains 3,291 annotated articles by experts
(professional journalists). This corpus is also aligned with the Twitter
dataset WT-WT (Conforti et al., 2020) corpus. The topic of the news articles
in STANDER is “mergers of US companies in the healthcare industry”. The stance
annotations are ‘support’, ‘refute’, ‘comment’, and ‘unrelated’. Moreover, the
corpus also includes ‘evidences’ as an annotation which indicates the part of
text used to identify the stance of the article. 2 to 4 annotators were
employed, and majority voting was used to decide on the final annotation.

In (Mohammad et al., 2016), the authors present a dataset of tweets annotated
with the tweeter’s stance (in favor or against) regarding an already chosen
target. The dataset contains six targets of interest commonly debated in the
United States, i.e, ‘Atheism’, ‘Climate Change is a Real Concern’, ‘Feminist
Movement’, ‘Hillary Clinton’, ‘Legalization of Abortion’, and ‘Donald Trump’.
For each target, 1000 tweets were randomly sampled from the initially
collected 1.7 million tweets. The tweets were annotated by 8 annotators. In
total, the corpus contains 4,870 annotated tweets. This dataset has been used
for the stance detection task in SemEval
2016111https://alt.qcri.org/semeval2016/task6/. This stance dataset, which was
subsequently also annotated for sentiment, can be used to better understand
the relationship between stance, sentiment, entity relationships, and textual
inference.

In the Fake News detection Challenge, FNC-1 (Hanselowski et al., 2018), the
organizers focus on performing stance detection on document level. The
documents are to be classified into four classes: ‘agree’, ‘discuss’,
‘disagree’, ‘unrelated’. The data contains 2,587 headlines and documents. Out
of these documents, 7.4% are labeled ‘agree’, 2.0% ‘disagree’, 17.7%
‘discuss’, and 72.8% ‘unrelated’, respectively. All the above datasets for
stance detection are for English language, and thus, not usable for performing
stance detection on GeNeG.

###  2.2. Multilingual Stance Detection

In (Mascarell et al., 2021) the authors create Cheese, a new dataset
containing 3,693 pairs of debate questions and the associated Swiss news
articles in German, annotated with stances and emotions. The news articles
were published between 2004 and 2020, and cover 24 topics, including science,
environment, politics, religion, society, etc. The stance annotation include
‘in favor’, ‘against’, ‘discussion’, and ‘unrelated’, while the emotion
annotations are ‘joy’, ‘trust’, ‘fear’, ‘anticipation’, ‘sadness’, ‘disgust’,
‘anger’, ‘surprise’, and ’no emotion’. Along with the dataset, the authors
provide a supervised classification task targeting the stance of the news
article with respect to the question. The classification algorithm uses a
contextual language model in German (German BERT).

SardiStance (Cignarella et al., 2020) is a stance detection task for tweets in
Italian. The shared task was created for two different settings: (i) textual
stance detection, exploiting only the information provided by the tweet, and
(ii) contextual stance detection, with the addition of information on the
tweet itself, such as the number of retweets, the number of favors or the date
of posting, contextual information about the author, such as follower count,
location, user’s biography, and additional knowledge extracted from the user’s
network of friends, followers, retweets, quotes and replies. The dataset
constructed for the task contains 700K tweets written in Italian about
“Movimento delle Sardine“. The tweets were collected from 11-2019 to 01-2020,
and annotated with the labels ‘against’, ‘in favor’, ‘neutral’, and
‘irrelevant’. The dataset also contains annotations indicating whether the
tweet is ‘ironic’, ‘non-ironic’, or ‘undefined’.

In (Xu et al., 2016), the authors present the task of stance detection on
Chinese Microblogs. This is further divided into two sub-tasks, where the
first is a supervised classification task which detects the stance towards
five targets of interest with given labeled data. The second is an optional
unsupervised task which requires only unlabeled data.

Previously discussed datasets focus on only one language. As a step further,
in x-stance (Vamvas and Sennrich, 2020) the authors facilitate multilingual,
multi-target stance detection, which also helps in performing cross-lingual
stance detection. The authors use Multilingual BERT (Devlin et al., 2019) for
performing stance classification. The dataset contains question and news
article pairs in English, French, German, and Italian.

##  3\. Enriching German News Knowledge Graph with Sentiments and Stance

This section discusses necessary details about GeNeG, along with the methods
adopted for performing sentiment analysis as well as stance detection.

###  3.1. GeNeG

GeNeG (Iana et al., 2022b) is a news knowledge graph constructed from a
dataset of 4,557 German news articles on the topic of refugees and migration,
collected from 39 media outlets that cover a wide political spectrum,
including far-right and far-left online publications. The corpus consists of
news published between 01-01-2019 and 20-10-2020, selected based on keyword
stems representative of the topic (e.g., ‘flüchtl’, ‘migrant’, ‘asyl’), and
collected from the news outlets via a Web
crawler222https://github.com/andreeaiana/german-news. The dataset contains
rich textual and metadata information, such as content, provenance, publishing
dates, authors, or tagged keywords. Furthermore, named entities have been
extracted from the articles’ content (e.g. persons, locations) and metadata
(e.g. publisher, author) and disambiguated using Wikidata (Vrandečić and
Krötzsch, 2014).

The news knowledge graph built from this dataset represents a heterogeneous
network, where the news content and real-world entities are represented as
nodes, while different relations between these items constitute the graph’s
edges. The nodes are divided into literals, denoting textual content, dates or
polarization scores, and entities, indicating identified named entities. In
turn, entity nodes can be linked to Wikidata, or non-disambiguated (if they
are not found in Wikidata). GeNeG is provided in three flavors: a base graph –
containing textual information, metadata, and entities extracted from the
articles, an entities graph – derived from the base version by removing all
literal nodes, it contains only entities extracted from the articles and it is
enriched with their three-hop Wikidata neighbors, and a complete graph –
combining the previous two graphs, and incorporating both literals and
entities. The base GeNeG contains 54,327 nodes and 186,584 edges denoting 16
properties. The entities GeNeG consists of 844,935 nodes, and 6,615,972 edges
representing 1,263 properties. Lastly, the complete GeNeG comprises of 868,159
nodes and 6,656,779 edges denoting 1,271 properties.

###  3.2. Sentiment Annotations

To annotate each news article with a sentiment score, we used a pre-trained
BERT-based (Devlin et al., 2019) sentiment classification model for German
language texts (Guhr et al., 2020). The model uses a softmax function to
calculate a probability estimate, which classifies each document as either
positive, neutral or negative. To transform these probabilities into a
sentiment score in the range of -1 to 1, we take the negative sentiment
probability and subtract it from the positive sentiment probability. More
formally, this is expressed by $s=p_{p}-p_{n}$, where $s$ equals the sentiment
score, while $p_{p}$ and $p_{n}$ represent the positive and negative sentiment
probability, respectively. We ignore the neutrality score, since it is
implicitly encoded as $1-p_{p}-p_{n}$. On the GeNeG corpus we observed a
slight skewness towards articles with negative sentiment, as indicated by the
mean and median sentiment scores of $s_{mean}=-0.154$, and
$s_{median}=-0.019$, respectively.

###  3.3. Transfer Learning for Stance Detection on German News Articles

In order to perform stance detection on the news articles collected on the
topic of refugees and migration, transfer learning was performed. In order to
do so, the first step is to select a proper training dataset for articles in
German, and the second step is to classify the articles into two classes,
namely ’in favor ’ or ’against’.

#### Training Data Selection

There are two datasets which are introduced for German language, i.e., Cheese
(Mascarell et al., 2021) which specifically focuses on German language, and
the German part of x-stance (Vamvas and Sennrich, 2020). The Cheese dataset
was excluded from consideration for training since the number of instances in
each of the classes, i.e., ‘in favor’, ‘against’, ‘unrelated’, ‘discussion’,
were highly unbalanced (i.e., 702, 286, 1428, and 774), leading to unbalanced
classes predicted for our corpus. This issue has been discussed in (Roy et
al., 2022) in more detail, where the authors propose a cascade of binary
classifiers to consider the hierarchy of classes in stances. More
specifically, the authors first classify the text into relevant or irrelevant,
and then label the relevant texts as either ’neutral’ or ’stance’. The final
binary classification is performed using ’in favor’ or ’against’ class labels
for the texts which were previously classified as ’stance’.

In our case, we use the x-stance (Vamvas and Sennrich, 2020) dataset for
training the classifier, which contains a balanced number of articles for both
classes, i.e., ‘in favor’ and ‘against’. The training data was extracted from
x-stance with the language label ’de’, i.e., only the German subset. The
statistics of the corpus extracted from x-stance are given in Table 1.

Table 1. Statistics of the the German part of the x-stance dataset. Dataset | Against | Favor | Total
---|---|---|---
Training | 17130 | 16720 | 33850
Validation | 1451 | 1420 | 2871
Test | 5882 | 6009 | 11891

#### Stance Detection in GeNeG

Stance detection takes into account a question and article pair, and
classifies the stance of the article as being in favor or against the question
at hand. The questions used to build the question and news article pairs for
classification are given in Table 2. GermanBert333https://huggingface.co/bert-
base-german-cased (Chan et al., 2020) was fine-tuned on the training dataset
provided in x-stance. The hyper-parameters for the best performing model on
the test set are: $learning~{}rate=3e-5$ and $number~{}of~{}epochs=4$.
Afterwards, transfer learning was performed for classifying the news articles
in GeNeG (as test dataset) into one of the two classes, i.e., ‘in favor’ or
‘against’, according to one of the predefined questions. In future studies we
would like to perform human evaluation on the annotated GeNeG articles. Table
3 show the results of the stance detection for each of the questions, as well
as the average score, computed as $\frac{favor-against}{favor+against}$.

Table 2. Questions for the question-news article pairs. German Question | English Translation (for understandability)
---|---
(Q1) Befürworten Sie, dass Flüchtlinge nach Deutschland kommen? | Are you in favor of refugees coming to Germany?
(Q2) Befürworten Sie, dass Flüchtlinge in Deutschland leben? | Are you in favor of refugees living in Germany?
(Q3) Befürworten Sie, dass Flüchtlinge in Deutschland arbeiten? | Are you in favor of refugees working in Germany?
(Q4) Sollte Deutschland Flüchtlinge aufnehmen? | Should Germany take in refugees?
(Q5) Sollte Deutschland Flüchtlingen helfen? | Should Germany help refugees?
Table 3. Articles ‘in favor’ of or ‘against’ the pairing question. Question | Articles in Favor | Articles Against | Avg. Score
---|---|---|---
(Q1) | 2165 | 2392 | -0.050
(Q2) | 2193 | 2364 | -0.038
(Q3) | 2210 | 2347 | -0.030
(Q4) | 2120 | 2437 | -0.070
(Q5) | 2192 | 2365 | -0.038

Finally, GeNeG was additionally populated with the output of the stance
detection with the help of two newly created properties, namely geneg:in_favor
and geneg:against. The resulting triples have the form $\langle
article,stance,question\rangle$, where $stance\in\\{{\tt
geneg:in\\_favor},{\tt geneg:against}\\}$, and
$question\in\\{Q_{n},n=\overline{1,5}\\}$. The dataset is available through
Zenodo444https://doi.org/10.5281/zenodo.6039372%****␣main.tex␣Line␣250␣****
with restricted access.

##  4\. Sentiment and Stance Bias of News Recommenders

The stance and sentiment of news articles can be used to determine whether
news recommenders are biased towards a certain sentiment or stance of the
articles, and if this, in turn, correlates with and reinforces users’ existing
viewpoints on a given topic. In this section, we firstly introduce the four
news recommender systems analyzed and describe the user data. Afterwards, we
evaluate the performance of the models and investigate their sentiment and
stance bias.

###  4.1. Recommendation models

Collaborative filtering is the most adopted recommendation method in fields
such as music or movie recommendation. Nonetheless, content-based approaches
are the most widely used in the field of news recommendation mostly due to the
fact that users generally do not have long-term profiles on news websites,
with reading history being limited to a single session, and feedback being
collected almost exclusively implicitly from click logs (Karimi et al., 2018;
Iana et al., 2022a). We compared four different content-based recommender
systems in our analysis. The first three are _textual_ , i.e., they are solely
based on the articles’ texts, whereas the last one is _knowledge-based_ ,
using the graph representation and the information from Wikidata instead of
the articles’ texts. All text-based models use cosine similarity to determine
the similarity between a candidate news article and the user’s history of read
news.

  * •

The Term Frequency-Inverse Document Frequency (TF-IDF) recommender uses a TF-
IDF (Schütze et al., 2008) vector representation for each article.

  * •

The Word2vec recommender encodes articles using word embeddings learned from a
large text corpus (Mikolov et al., 2013). More specifically, we use a pre-
trained German word2vec model trained on the Common Crawl and Wikipedia
dataset (Grave et al., 2018) to learn latent representations of the words in
an article. The article’s vector representation is computed as the average of
its words’ embeddings.

  * •

The Transfomer architecture uses an attention mechanism to incorporate context
in text embeddings (Vaswani et al., 2017). We use a pre-trained cross-lingual
model for sentence embeddings in English and German (Reimers and Gurevych,
2019; May, 2020) to encode the sentences in the news articles. The final
representation of the article is obtained by averaging the embedding vectors
of all of its sentences.

  * •

RippleNet (Wang et al., 2018) is a knowledge-aware recommender which
propagates a user’s potential preferences along the edges of the knowledge
graph. The model generates ripple sets (i.e. sets of multi-hop entity
neighbors encoding potential user interests) based on the entities extracted
from the user’s read articles. These ripple sets are used to explore higher-
order preferences, where the strength of the user’s preference diminishes
proportional to the distance from the original seed in the knowledge graph.
The final click probability is predicted using the preference distribution of
the user for a candidate news, obtained by superposing multiple ripple sets.
For RippleNet, we use the entities GeNeG as the knowledge graph for the
recommender. In particular, the sentiment and stance information was _not_
included in the graph used to compute the recommendations.

###  4.2. User Data

We collected the user data through an online study aimed at measuring the
political polarization effect of recommender systems on users. The study was
based on the news dataset described in Section 3.1. However, 732 articles
longer than 1,500 words were removed from the experiment in order to limit the
reading time required for the participants, and to ensure a strong response to
the stimulus. Each of the participants in the study was randomly assigned to
one of four recommenders, namely TF-IDF, Word2vec, Transformer, or a random
recommendation baseline. Each participant was asked to choose an article from
a preview of six articles, and then to read it. The user’s choices were
included in his or her reading history. This process was repeated four times,
resulting in four interactions per participant.

We split the user data into training and test sets using a 80:20 ratio. Since
85% of the user ratings were generated using recommendations of the text-based
recommenders, these baselines are prone to overfit. Therefore, we generate a
test subset containing only user ratings for randomly suggested articles from
the complete test set. The user data statistics can be found in Table 4. For
more details on the online study and data collection, please refer to Appendix
A.

Table 4. User data statistics. Dataset | Items | Users
---|---|---
Total | 3,825 | 1,417
Training | 3,365 | 1,414
Complete test | 1,633 | 1,174
Random test | 316 | 177

###  4.3. Evaluation of Recommender Systems

In the following, we describe the experimental setup and discuss the
evaluation results of the proposed recommenders.

####  4.3.1. Experimental Setup

We evaluated the four recommender models on click-though-rate (CTR)
prediction. In this scenario, each recommender is applied on every user-
article pair from the test set to predict the user’s likelihood of clicking
the candidate article. Furthermore, we applied a min-max scaling to the
similarity measures outputted by the text-based recommenders as an
approximation of probability scores. We use Accuracy (ACC), Area Under the
Curve (AUC), and the F1 score to evaluate the performance of the models.

The key parameters settings for the analyzed recommender systems 555The code
and data are available at https://github.com/andreeaiana/geneg_benchmarking.
are as follows. For TF-IDF, we use an n-gram range of 1 and 2, and l2-norm
regularization. The Word2vec and the Transformer recommenders use embedding
vectors of dimension 300, and respectively, 768. In RippleNet, we empirically
set the number of hops to $H=1$, the size of the user’s ripple set to 16, the
dimension of the item and knowledge graph embeddings to $d=48$, and the
training weight of the knowledge graph embedding to $\lambda_{2}=0.03$. We use
the default values for news recommendation from (Wang et al., 2018) for the
other parameters.

####  4.3.2. Results

The experimental results of the benchmarked recommenders are summarized in
Table 5. The text-based recommenders achieve the best scores in terms of AUC
and F1 on the complete test set, whereas RippleNet outperforms the Word2vec
and Transformer recommenders in terms of accuracy. However, the text-based
recommendation models overfit on the complete test data, due to the user
ratings provenance. More specifically, the ratings collected from users
assigned to a TF-IDF recommender account for more than 55% of the cases (for
more in-depth statistics regarding the ratings’ provenance, please refer to
Appendix A).

Table 5. CTR prediction results. Model | Complete | Random
---|---|---
ACC | AUC | F1 | ACC | AUC | F1
TF-IDF | 0.732 | 0.873 | 0.647 | 0.487 | 0.499 | 0
Word2vec | 0.514 | 0.794 | 0.674 | 0.499 | 0.474 | 0.663
Transformer | 0.505 | 0.779 | 0.671 | 0.499 | 0.515 | 0.665
RippleNet | 0.553 | 0.574 | 0.523 | 0.559 | 0.578 | 0.531

Consequently, we investigated the performance of the recommender systems on
the random test set, in order to reduce the bias of the text-based models. In
this evaluation setting, RippleNet outperforms all other recommenders in terms
of accuracy and AUC. Moreover, it obtains a higher F1 score than the TF-IDF
model,which is unable to make any correct predictions for the articles in this
test set, resulting in a F1 score of 0.

Overall, we conclude that, by using only entities in GeNeG for computing
recommendations, RippleNet is able to achieve a decent performance, while
being more robust to changes in the structure of the data compared to the
purely text-based recommenders.

###  4.4. Bias Analysis

In addition to the recommenders’ predictive accuracy, we analyzed their
recommendations to identify whether the recommenders are prone to stance or
sentiment bias. In this study, each of the text-based models generated an
output containing the top $k$ most similar articles to the ones in the user
history, whereas RippleNet recommended the $k$ articles with the highest
probability of being clicked by the user. In our subsequent bias analysis, we
set the number of recommended articles for all recommenders to $k=5$, as we
observed that the majority of investigated outlets suggest, on average, five
other articles related to the one currently read by the user.

In order to quantify the stance of the news articles and to calculate an
overall bias score for the users and recommenders, we employ the following
transformation function for an article’s stance label:

$stance\\_score=\begin{cases}+1&\text{if }stance\\_label=\text{Favor}\\\
-1&\text{if }stance\\_label=\text{Against}\\\ \end{cases}$

For each user, we compute the average sentiment bias score as the mean of the
sentiment scores of the articles included in his or her reading history.
Similarly, the recommender sentiment bias score per user was obtained as the
average of all sentiment scores of the recommended articles. Lastly, the mean
of all recommender sentiment bias scores over all users constitutes the
recommender’s average sentiment bias score. The average user and recommender
stance bias scores are calculated analogously. The average bias scores for
both sentiment and stances fall in the interval $[-1,1]$, where -1 represents
a user’s or recommender’s tendency for articles with negative sentiments or
stances against the given topic, whereas +1 denotes the opposite situation.

####  4.4.1. Recommender Bias

Table 6. Average user and recommender sentiment scores. The statistical significance with a Student’s $t$-test is denoted with * ($p-value<0.01$) and ** ($p-value<0.05$). A star in the case of user sentiments denotes statistical significance between the average user and corpus scores. In the case of recommender sentiment, (*/*) denotes statistical significance, firstly with the average user score, and secondly with the average corpus score. A ‘-’ sign shows that no such significance was found. Test set |  Avg. user sentiment score | Avg. recommender sentiment score
---|---|---
TF-IDF | Word2vec | Transformer | RippleNet
Complete | -0.171* | -0.162 (-/-) | -0.169 (-/**) | -0.157 (-/-) | -0.148 (*/-)
Random | -0.169 | -0.141 (-/-) | -0.170 (-/-) | -0.160 (-/-) | -0.150 (**/-)
Table 7. Average user and recommender stance scores. The statistical significance with a Student’s $t$-test is denoted with * ($p-value<0.01$) and ** ($p-value<0.05$). A star in the case of user stance denotes statistical significance between the average user and corpus scores. In the case of recommender stance, (*/*) denotes statistical significance, firstly with the average user score, and secondly with the average corpus score. A ‘-’ sign shows that no such significance was found. Question |  Avg. user stance score | Avg. recommender stance score (complete / random test set)
---|---|---
TF-IDF | Word2vec | Transformer | RippleNet
(Q1) | -0.109 / -0.093 | -0.140 (-/-) / -0.227 (**/–) | -0.165 (**/-) / -0.219 (**/-) | -0.136 (-/-) / -0.172 (-/-) | -0.082 (-/-) / -0.054 (-/-)
(Q2) | -0.102** / -0.093 | -0.132 (-/-) / -0.220 (**/–) | -0.158 (**/-) / -0.207 (**/-) | -0.131 (-/-) / -0.169 (-/-) | -0.074 (-/-) / -0.038 (-/-)
(Q3) | -0.092** / -0.081 | -0.127 (-/**) / -0.215 (**/–) | -0.149 (**/**) / -0.205 (**/-) | -0.116 (-/**) / -0.164 (-/-) | -0.062 (-/**) / -0.024 (-/-)
(Q4) | -0.117 / -0.106 | -0.167 (**/-) / -0.255 (**/–) | -0.178 (**/-) / -0.268 (*/-) | -0.157 (-/-) / -0.179 (-/-) | -0.095 (-/-) / -0.084 (-/-)
(Q5) | -0.079 / -0.081 | -0.130 (**/-) / -0.237 (*/–) | -0.135 (**/-) / -0.199 (**/-) | -0.124 (-/-) / -0.143 (-/-) | -0.060 (-/-) / -0.055 (-/-)

Firstly, we used the average recommender bias scores to answer the following
question: do the recommender systems have a tendency to recommend articles
with a certain sentiment or stance?

Table 6 shows the results in terms of sentiment bias. As it can be seen here,
all recommenders are more likely to suggest articles with negative sentiments.
However, this skewness towards negative sentiments should be interpreted by
taking into account that the news articles in GeNeG have, on average, a
negative sentiment score, as shown in Section 3.2. Nonetheless, we observe
that textual recommenders, in 5 out of 6 cases, are more prone towards news
with negative sentiments, as indicated by the average recommender sentiment
bias, which is larger than the average sentiment score of the articles in the
dataset, namely $s_{mean}=-0.154$. In contrast, the knowledge-aware model
recommends, in both cases, articles with slightly less negative sentiment than
the average sentiment score of the news in the corpus. However, these
observations are statistically significant only for the Word2vec recommender
on the complete test set, for $p-value<0.05$ , as indicated in Table 6.

Similarly, Table 7 illustrates the average recommender stance bias on both
test sets. All recommenders show a bias towards news articles that take a
stance against the topic of refugees and migration, for all questions used to
represent this topic. As shown in Table 3, in GeNeG there are more articles
against the topic, than in favor of it, for all of the five questions used in
the process of stance detection. While this can partly explain the
recommenders’ preferences for the more negative articles, the average scores
for the articles selected by the recommenders are still more negative than the
average scores in the entire corpus depicted above. Again, as for the
sentiment bias, the knowledge-based recommender has the weakest negative bias,
showing only little deviation from the overall average scores in the dataset.
Nonetheless, the difference between the average sentiment scores of the
recommenders and the corpus is statistically significant only for question
(Q3).

####  4.4.2. Correlation between the Recommender and User Biases

We then pose the question: how does the recommenders’ sentiment and stance
bias correlate with the existing user sentiment and stance bias? This analysis
aims to identify whether the recommenders tend to generate suggestions that
agree with or reinforce the user’s existing beliefs. Tables 6 and 7 show the
average sentiment and stance user biases.

We observed that readers have a slight preference for news articles with
negative sentiments (Table 6), which is not fully explained by the overall
skewness towards negative sentiment in the dataset. Nevertheless, in seven out
of eight cases, the recommender’s average sentiment bias is marginally less
negative than the user bias, indicating that the recommenders include articles
with less negative sentiment in their results list. Here again, one should
take into consideration that the articles in our dataset have, on average, a
negative sentiment. Even in this context, we notice that users are more prone
to reading articles with negative sentiments, with the average user bias on
the complete test set being -0.171, and respectively, -0.169 on the random
test set, slightly larger than the average sentiment score of the news
articles in GeNeG, $s_{mean}=-0.154$. According to the Student’s $t$-test,
this is a statistically significant difference. However, there does not seem
to be a large difference between the recommender and the users’ bias scores,
with the former following the same pattern. This could indicate that the
recommenders indeed learn from the user’s history and preferences and generate
suggestions that amplify the readers’ existing preferences for a given
sentiment. However, we observed a statistically significant difference between
the user and the recommender average sentiment scores in the case of RippleNet
on the complete test set, which shows that the knowledge-based recommender
might be less prone to amplifying existing sentiment preferences.

A similar pattern can be observed in the case of stance bias. On the one hand,
the text-based recommenders appear to exacerbate the user’s preference towards
news against the topic of refugees and migration, as indicated by the larger
negative average stance bias scores of the recommenders, on both test sets.
Additionally, this behavior of the recommenders appears to be more pronounced
on the random test set. The difference between the average stance scores of
Word2vec recommendations and the average user stance scores are significant on
both test sets, while in the case of TF-IDF this observation holds true mostly
on the random test set. On the other hand, although the knowledge-aware
recommender also seems biased towards viewpoints against the given topic, it
also appears to suggest, on average, more articles which also have favorable
stances towards the topic. This is shown by the average stance bias score of
RippleNet, which is lower than the average user stance bias score. We observed
no statistically significant differences between the user and recommender
stance bias in the case of the Transformer and RippleNet recommenders.

Through a closer look at the correlation between the recommender and the user
sentiment and stance biases, we identified five possible cases of bias
correlation, as follows:

  * •

(C1) Bias in the same direction: both the user and the recommender have a
tendency towards negative (positive) sentiments and/or against (in favor)
stances.

  * •

(C2) Bias in opposite directions: the user has a preference for negative
(positive) sentiments and/or against (in favor) stances, while the recommender
tends to suggest articles with an opposite sentiment and/or stance.

  * •

(C3) Skewed towards the user: the user has a preference for articles with
negative (positive) sentiments and/or against stances, while the recommender
is balanced with regards to the sentiment and/or stance of the recommended
articles.

  * •

(C4) Skewed towards the recommender: the user is balanced with regards to the
sentiment and/or stance of the read articles, while the recommender tends to
suggest articles with negative (positive) sentiment and/or against (in favor)
stance.

  * •

(C5) No bias: neither the user, nor the recommender show any sentiment and/or
stance bias.

It should be noted that in the case of sentiment bias, only the first two
cases, (C1) and (C2), are possible, given the nature of the sentiment scores,
and the fact that there are no perfectly neutral articles in the corpus.

Table 8 shows the number of users that fall in each of the first two cases
with regards to sentiment bias. The results show that for both test sets,
almost always the recommender will suggest articles with the same sentiments
as the ones found in the user’s history.

Table 8. Recommender-user sentiment correlation counts. Bias case | Counts (complete / random test set)
---|---
TF-IDF | Word2vec | Transformer | RippleNet
(C1) | 1166 / 177 | 1166 / 177 | 1165 / 177 | 1160 / 177
(C2) | 8 / 0 | 8 / 0 | 9 / 0 | 14 / 0

Furthermore, Table 9 shows the number of users for each of the five bias cases
and questions in terms of stance bias. We observe that all recommenders
generate recommendations that most often fall in one of the first two cases,
with the same-direction bias being more likely. The third most probable case
is the one in which the overall stance bias score of the user is close to 0,
denoting a balanced news consumption with articles both in favor and against
the topic, whereas the recommender is prone to suggesting articles with one
particular viewpoint. The least likely cases are the ones in which the user is
biased towards a certain stance, while the recommender is neutral, followed by
the scenario in which both the user and the recommender are neutral with
regards to the stance of the news. Overall, the prevalence of the the first
bias case is indicative of the fact that recommenders not only learn from
user’s reading history, but also tend to amplify existing opinions by
recommending news with the same stances as the ones previously read by the
user.

Table 9. Recommender-user stance correlation counts.

Question | Bias case | Counts (complete / random test set)
---|---|---
TF-IDF | Word2vec | Transformer | RippleNet
Q1 | (C1) | 600 / 81 | 575 / 68 | 575 / 88 | 470 / 83
(C2) | 358 / 63 | 367 / 70 | 374 / 52 | 471 / 61
(C3) | 14 / 1 | 30 / 7 | 23 / 5 | 31/ 1
(C4) | 198 / 31 | 198 / 31 | 194 / 29 | 193/ 31
(C5) | 4 / 1 | 4 / 1 | 8 / 3 | 9 / 1
Q2 | (C1) | 595 / 79 | 575 / 69 | 572 / 86 | 479 / 82
(C2) | 362 / 65 | 367 / 69 | 378 / 54 | 462 / 62
(C3) | 15 / 1 | 30 / 7 | 22 / 5 | 31 / 1
(C4) | 199 / 31 | 198 / 31 | 193 / 29 | 194 / 31
(C5) | 3 / 1 | 4 / 1 | 9 / 3 | 8 / 1
Q3 | (C1) | 600 / 78 | 574 / 68 | 574 / 86 | 477 / 81
(C2) | 360 / 66 | 370 / 70 | 375 / 54 | 467 / 63
(C3) | 15 / 1 | 31 / 7 | 26 / 5 | 31 / 1
(C4) | 196 / 31 | 195 / 31 | 193 / 30 | 191 / 31
(C5) | 3 / 1 | 4 / 1 | 6 / 2 | 8 / 1
Q4 | (C1) | 611 / 82 | 590 / 76 | 538 / 86 | 495 / 87
(C2) | 342 / 54 | 356 / 61 | 402 / 48 | 448 / 50
(C3) | 13 / 2 | 20 / 1 | 26 / 4 | 23 / 1
(C4) | 206 / 38 | 198 / 39 | 204 / 37 | 200 / 39
(C5) | 2 / 1 | 10 / 0 | 4 / 2 | 8 / 0
Q5 | (C1) | 624 / 89 | 592 / 77 | 542 / 88 | 498 / 84
(C2) | 337 / 51 | 357 / 62 | 403 / 49 | 451 / 57
(C3) | 10 / 2 | 22 / 3 | 26 / 5 | 22 / 1
(C4) | 200 / 34 | 194 / 35 | 201 / 34 | 195 / 35
(C5) | 3 / 1 | 9 / 0 | 2 / 1 | 8 / 0

Lastly, we compute the Pearson correlation coefficient between the recommender
and the user average bias scores, to further investigate whether our initial
observations are statistically significant. Tables 10 and 11 illustrate the
results of recommender-user sentiment, and respectively, stance bias
correlation.

Table 10. Recommender-user sentiment bias correlation. Statistical significance is denoted with * ($p-value<0.01$) and ** ($p-value<0.05$ ). Test set | Pearson Correlation (p-value)
---|---
TF-IDF | Word2vec | Transformer | RippleNet
Complete | 0.325* | 0.347* | 0.216* | 0.045**
Random | 0.253* | 0.310* | 0.129 | -0.033
Table 11. Recommender-user stance bias correlation. Statistical significance
is denoted with * ($p-value<0.01$) and ** ($p-value<0.05$ ).

Question | Pearson Correlation (p-value) (complete / random test set)
---|---
TF-IDF | Word2vec | Transformer | RippleNet
(Q1) | 0.248* / 0.078 | 0.196* / 0.029 | 0.187* / 0.162** | -0.022 / 0.111
(Q2) | 0.241* / 0.078 | 0.196* / 0.019 | 0.176* / 0.148** | -0.008 / 0.115
(Q3) | 0.252* / 0.072 | 0.199* / 0.003 | 0.195* / 0.143 | -0.009 / 0.091
(Q4) | 0.280* / 0.135 | 0.249* / 0.002 | 0.143* / 0.230* | -0.019 / 0.148
(Q5) | 0.320* / 0.192** | 0.269* / 0.042 | 0.168* / 0.230* | -0.011 / 0.112

In the case of sentiment bias, the results indicate a statistically
significant positive correlation between the recommender and the user bias for
the TF-IDF and Word2vec models on both test sets. In the case of the
Transformer recommender, we notice a statistically significant positive
correlation only on the complete test set. This means that if a user prefers
only articles with a negative sentiment towards the topic of, for example,
refugees living in Germany, these recommenders will continue recommending news
with a negative sentiment. In this case, the user’s exposure to articles with
positive sentiment will be constrained by the recommender, and the user will
not have access to news that might not agree with his existing views. In
contrast, in the case of RippleNet, we found a statistically significant
correlation between the recommender and the user bias for the complete test
set, for $p-value<0.05$, and no statistical significance on the random test.
The latter could potentially indicate that the recommender is less prone to
amplifying the user’s existing preference for a particular sentiment.

As far as stance bias is concerned, text-based recommenders are also
statistically significant positively correlated with the existing user bias on
the complete test set, for all questions, as shown in Table 11. However, this
pattern does not hold true in the case of the random test set. In this case,
we noticed that only the Transformer recommender shows a positive correlation
with the user bias, which is statistically significant for four out of five
questions, whereas the Word2vec model has no statistically significant bias
correlation with the user history. In comparison, our analysis shows that the
stance bias of the TF-IDF model is positively correlated with the user’s bias
only on question (Q5) for $p-value<0.05$. Lastly, as in the case of sentiment
bias, RippleNet shows no statistically significant correlation with the
existing stance bias of users on either of the two tests sets, again
demonstrating that the knowledge-based recommender is less prone to amplifying
user bias.

##  5\. Discussion

In this paper, we have introduced GeNeG, a German news corpus on the topic of
refugees and migration, including polarity annotations with respect to
sentiment and stance. We have demonstrated that those annotations can be
utilized to quantify the sentiment and stance bias of different recommender
algorithms.

Our experiments show that purely text-based recommender systems expose
amplification of user attitudes with respect to sentiment and stance, both
constituting cues for the creation of filter bubbles in the process of
algorithmic news curation, and the decrease of news diversity in terms of
sentiments and viewpoints. Similar results have been found by Gao et al. (Gao
et al., 2018), who conducted experiments to understand the effects of stance
and credibility labels on online news selection and consumption. The results
of their study show that these labels intensify the tendency of the people to
look for opinions which are in-line with their own beliefs, and may lead the
people being more vulnerable to polarized opinions and fake news.

In contrast, the knowledge-aware recommender appears to be less prone to both
types of biases, but also less powerful in terms of accuracy on click-through
rate prediction when computing recommendations solely based on entities. On
the one hand, text-based recommenders are predisposed to identifying sentiment
and stance cues in the text, as their recommendations are based only on
textual information. On the other hand, using knowledge bases as sources of
side information and taking into account solely named entities identified in
the text (regardless of the textual context in which they are mentioned)
diminishes such underlying bias, while capturing deeper knowledge-level
connections between articles. Therefore, we argue that future research should
find a balance between two obviously conflicting goals: performance (as
understood by prediction accuracy) and bias (in terms of the sentiments and
stances expressed in the recommended news articles). The results of this paper
indicate that knowledge-based approaches are a good candidate when it comes to
develop less biased recommenders. However, future work should investigate also
the behavior of collaborative-filtering approaches in terms of sentiment and
stance bias. Moreover, the analysis could be extended to other datasets in
order to examine if the same patterns can be observed for a larger variety of
topics.

As a future perspective, we would like to enable the recommender systems to
take into account the sentiments, as well as the stance of an article. The
news recommender system should be able to recommend topics which allow the
user to consider differing opinions as compared to their own beliefs, hence
reducing the effect of filter bubbles. Since in this paper we have shown how
bias can be quantified, it would also be possible to introduce a combined
score as a new optimization goal for algorithmic recommenders, which trades
off accuracy and bias in a single score (just like, e.g., F1 score trades off
recall and precision). Optimizing news recommender systems towards such a
target function could be one possible approach towards developing less biased
news recommender systems.

###### Acknowledgements.

The work presented in this paper has been conducted in the ReNewRS project,
which is funded by the Baden-Württemberg Stiftung in the Responsible
Artificial Intelligence program.

## References

  * (1)
  * Baden and Springer (2017) Christian Baden and Nina Springer. 2017.  Conceptualizing viewpoint diversity in news discourse.  _Journalism_ 18, 2 (2017), 176–194.
  * Bakshy et al. (2015) Eytan Bakshy, Solomon Messing, and Lada A Adamic. 2015\.  Exposure to ideologically diverse news and opinion on Facebook.  _Science_ 348, 6239 (2015), 1130–1132.
  * Boutyline and Willer (2017) Andrei Boutyline and Robb Willer. 2017.  The social structure of political echo chambers: Variation in ideological homophily in online networks.  _Political psychology_ 38, 3 (2017), 551–569.
  * Chan et al. (2020) Branden Chan, Stefan Schweter, and Timo Möller. 2020\.  German’s Next Language Model. In _Proceedings of the 28th International Conference on Computational Linguistics_. International Committee on Computational Linguistics, Barcelona, Spain (Online), 6788–6796.  https://doi.org/10.18653/v1/2020.coling-main.598
  * Cignarella et al. (2020) Alessandra Teresa Cignarella, Mirko Lai, Cristina Bosco, Viviana Patti, and Paolo Rosso. 2020\.  SardiStance @ EVALITA2020: Overview of the Task on Stance Detection in Italian Tweets. In _Proceedings of the Seventh Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2020), Online event, December 17th, 2020_ _(CEUR Workshop Proceedings, Vol. 2765)_. CEUR-WS.org.  http://ceur-ws.org/Vol-2765/paper159.pdf
  * Conforti et al. (2020) Costanza Conforti, Jakob Berndt, Mohammad Taher Pilehvar, Chryssi Giannitsarou, Flavio Toxvaerd, and Nigel Collier. 2020.  STANDER: An Expert-Annotated Dataset for News Stance Detection and Evidence Retrieval. In _Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020_ _(Findings of ACL, Vol. EMNLP 2020)_ , Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, 4086–4101.  https://doi.org/10.18653/v1/2020.findings-emnlp.365
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers)_ , Jill Burstein, Christy Doran, and Thamar Solorio (Eds.). Association for Computational Linguistics, 4171–4186.  https://doi.org/10.18653/v1/n19-1423
  * Donsbach and Mothes (2013) Wolfgang Donsbach and Cornelia Mothes. 2013.  The dissonant self: Contributions from dissonance theory to a new agenda for studying political communication.  _Annals of the International Communication Association_ 36, 1 (2013), 3–44.
  * Färber et al. (2020) Michael Färber, Victoria Burkard, Adam Jatowt, and Sora Lim. 2020.  A multidimensional dataset based on crowdsourcing for analyzing and detecting news bias. In _Proceedings of the 29th ACM International Conference on Information & Knowledge Management_. 3007–3014.
  * Freedman and Sears (1965) Jonathan L Freedman and David O Sears. 1965.  Selective exposure.  In _Advances in experimental social psychology_. Vol. 2. Elsevier, 57–97.
  * Gao et al. (2018) Mingkun Gao, Ziang Xiao, Karrie Karahalios, and Wai-Tat Fu. 2018\.  To Label or Not to Label: The Effect of Stance and Credibility Labels on Readers’ Selection and Perception of News Articles.  _Proceedings of the ACM on Human-Computer Interaction_ 2, CSCW (2018), 55:1–55:16.  https://doi.org/10.1145/3274324
  * Grave et al. (2018) Edouard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, and Tomas Mikolov. 2018\.  Learning word vectors for 157 languages. In _Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)_.
  * Guhr et al. (2020) Oliver Guhr, Anne-Kathrin Schumann, Frank Bahrmann, and Hans Joachim Böhme. 2020.  Training a Broad-Coverage German Sentiment Classification Model for Dialog Systems. In _Proceedings of the 12th Language Resources and Evaluation Conference_. European Language Resources Association, Marseille, France, 1627–1632.
  * Haim et al. (2018) Mario Haim, Andreas Graefe, and Hans-Bernd Brosius. 2018\.  Burst of the filter bubble? Effects of personalization on the diversity of Google News.  _Digital journalism_ 6, 3 (2018), 330–343.
  * Hamborg et al. (2019) Felix Hamborg, Karsten Donnay, and Bela Gipp. 2019.  Automated identification of media bias in news articles: an interdisciplinary literature review.  _International Journal on Digital Libraries_ 20, 4 (2019), 391–415.
  * Hanselowski et al. (2018) Andreas Hanselowski, Avinesh P. V. S., Benjamin Schiller, Felix Caspelherr, Debanjan Chaudhuri, Christian M. Meyer, and Iryna Gurevych. 2018.  A Retrospective Analysis of the Fake News Challenge Stance-Detection Task. In _Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, USA, August 20-26, 2018_. Association for Computational Linguistics, 1859–1874.  https://aclanthology.org/C18-1158/
  * Helberger (2019) Natali Helberger. 2019\.  On the democratic role of news recommenders.  _Digital Journalism_ 7, 8 (2019), 993–1012.
  * Iana et al. (2022a) Andreea Iana, Mehwish Alam, and Heiko Paulheim. 2022a.  A Survey On Knowledge-Aware News Recommender Systems.  _Semantic Web Journal_ (2022).
  * Iana et al. (2022b) Andreea Iana, Alexander Grote, Kaharina Ludwig, Mehwish Alam, Philipp Müller, Christof Weinhardt, Harald Sack, and Heiko Paulheim. 2022b.  _GeNeG: German News Knowledge Graph_.  https://doi.org/10.5281/zenodo.5913171
  * Jamieson and Cappella (2008) Kathleen Hall Jamieson and Joseph N Cappella. 2008.  _Echo chamber: Rush Limbaugh and the conservative media establishment_.  Oxford University Press.
  * Karimi et al. (2018) Mozhgan Karimi, Dietmar Jannach, and Michael Jugovac. 2018\.  News recommender systems–Survey and roads ahead.  _Information Processing & Management_ 54, 6 (2018), 1203–1227.
  * Küçük and Can (2020) Dilek Küçük and Fazli Can. 2020.  Stance Detection: A Survey.  _ACM Computing Surveys (CSUR)_ 53, 1 (2020), 12:1–12:37.  https://doi.org/10.1145/3369026
  * Lim et al. (2020) Sora Lim, Adam Jatowt, Michael Färber, and Masatoshi Yoshikawa. 2020.  Annotating and analyzing biased sentences in news articles using crowdsourcing. In _Proceedings of the 12th Language Resources and Evaluation Conference_. 1478–1484.
  * Liu et al. (2010) Bing Liu et al. 2010\.  Sentiment analysis and subjectivity.  _Handbook of natural language processing_ 2, 2010 (2010), 627–666.
  * Liu et al. (2021) Ping Liu, Karthik Shivaram, Aron Culotta, Matthew A Shapiro, and Mustafa Bilgic. 2021\.  The Interaction between Political Typology and Filter Bubbles in News Recommendation Algorithms. In _Proceedings of the Web Conference 2021_. 3791–3801.
  * Ludwig et al. (2022) Katharina Ludwig, Alexander Grote, Andreea Iana, Mehwish Alam, Heiko Paulheim, Harald Sack, Christof Weinhardt, and Philipp Müller. 2022.  Recommended Polarization? The (limited) effects of content- and sentiment-based news recommendation on affective, ideological, and perceived polarization.  _72nd Annual ICA Conference, One World, One Network?_ (2022).
  * Mascarell et al. (2021) Laura Mascarell, Tatyana Ruzsics, Christian Schneebeli, Philippe Schlattner, Luca Campanella, Severin Klingler, and Cristina Kadar. 2021\.  Stance Detection in German News Articles. In _Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER)_. Association for Computational Linguistics, Dominican Republic, 66–77.  https://doi.org/10.18653/v1/2021.fever-1.8
  * May (2020) Philip May. 2020\.  _Cross English & German RoBERTa for Sentence Embeddings_.  https://huggingface.co/T-Systems-onsite/cross-en-de-roberta-sentence-transformer
  * McQuail (1992) Denis McQuail. 1992\.  _Media performance: Mass communication and the public interest_. Vol. 144.  Sage London.
  * Mikolov et al. (2013) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013\.  Efficient estimation of word representations in vector space.  _arXiv preprint arXiv:1301.3781_ (2013).
  * Mohammad et al. (2016) Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiao-Dan Zhu, and Colin Cherry. 2016.  A Dataset for Detecting Stance in Tweets. In _Proceedings of the Tenth International Conference on Language Resources and Evaluation LREC 2016, Portorož, Slovenia, May 23-28, 2016_. European Language Resources Association (ELRA).
  * Pariser (2011) Eli Pariser. 2011\.  _The filter bubble: What the Internet is hiding from you_.  Penguin UK.
  * Pitoura et al. (2021) Evaggelia Pitoura, Kostas Stefanidis, and Georgia Koutrika. 2021\.  Fairness in Rankings and Recommendations: An Overview.  _The VLDB Journal_ (2021), 1–28.
  * Ravi and Ravi (2015) Kumar Ravi and Vadlamani Ravi. 2015.  A survey on opinion mining and sentiment analysis: tasks, approaches and applications.  _Knowledge-based systems_ 89 (2015), 14–46.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-bert: Sentence embeddings using siamese bert-networks. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_. 3982–3992.
  * Reuver et al. (2021) Myrthe Reuver, Antske Fokkens, Suzan Verberne, H Toivonen, and M Boggia. 2021.  No NLP task should be an island: multi-disciplinarity for diversity in news recommender systems.  _Proceedings of the EACL Hackashop on news media content analysis and automated report generation_ (2021), 45–55.
  * Roy et al. (2022) Arjun Roy, Pavlos Fafalios, Asif Ekbal, Xiaofei Zhu, and Stefan Dietze. 2022.  Exploiting stance hierarchies for cost-sensitive stance detection of Web documents.  _Journal of Intelligent Information Systems_ 58, 1 (2022), 1–19.
  * Schütze et al. (2008) Hinrich Schütze, Christopher D Manning, and Prabhakar Raghavan. 2008.  _Introduction to information retrieval_. Vol. 39.  Cambridge University Press Cambridge.
  * Spinde et al. (2020) Timo Spinde, Felix Hamborg, and Bela Gipp. 2020\.  An integrated approach to detect media bias in german news articles. In _Proceedings of the ACM/IEEE joint conference on digital libraries in 2020_. 505–506.
  * Stroud (2017) Natalie Jomini Stroud. 2017\.  Selective exposure theories.  In _The Oxford handbook of political communication_.
  * Sunstein (2001) Cass R Sunstein. 2001\.  _Echo chambers: Bush v. Gore, impeachment, and beyond_.  Princeton University Press Princeton, NJ.
  * Vamvas and Sennrich (2020) Jannis Vamvas and Rico Sennrich. 2020.  X-Stance: A Multilingual Multi-Target Dataset for Stance Detection.  _CoRR_ abs/2003.08385 (2020).  arXiv:2003.08385 https://arxiv.org/abs/2003.08385
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _Advances in neural information processing systems_ 30 (2017).
  * Voakes et al. (1996) Paul S Voakes, Jack Kapfer, David Kurpius, and David Shano-yeon Chern. 1996.  Diversity in the news: A conceptual and methodological framework.  _Journalism & Mass Communication Quarterly_ 73, 3 (1996), 582–593.
  * Vrandečić and Krötzsch (2014) Denny Vrandečić and Markus Krötzsch. 2014.  Wikidata: a free collaborative knowledgebase.  _Commun. ACM_ 57, 10 (2014), 78–85.
  * Wang et al. (2018) Hongwei Wang, Fuzheng Zhang, Jialin Wang, Miao Zhao, Wenjie Li, Xing Xie, and Minyi Guo. 2018.  Ripplenet: Propagating user preferences on the knowledge graph for recommender systems. In _Proceedings of the 27th ACM international conference on information and knowledge management_. 417–426.
  * Xu et al. (2016) Ruifeng Xu, Yu Zhou, Dongyin Wu, Lin Gui, Jiachen Du, and Yun Xue. 2016\.  Overview of NLPCC Shared Task 4: Stance Detection in Chinese Microblogs. In _Natural Language Understanding and Intelligent Applications - 5th CCF Conference on Natural Language Processing and Chinese Computing, NLPCC 2016, and 24th International Conference on Computer Processing of Oriental Languages, ICCPOL 2016, Kunming, China, December 2-6, 2016, Proceedings_ _(Lecture Notes in Computer Science, Vol. 10102)_. Springer, 907–916.  https://doi.org/10.1007/978-3-319-50496-4_85

##  Appendix A User Study and Data Collection

The experimental setup is extensively described in (Ludwig et al., 2022). The
participants were recruited through the online-access panel of Respondi AG,
and selected using a quote procedure to match German Internet users aged 18 to
74. The mean average age was 46.56, with a standard deviation of 15.68, 47.7%
of the participants were females, and 38.5% of the respondents had an _Abitur_
(German university entrance certificate) or higher qualification. Only
participants with desktop-based devices were included given the display of the
experimental conditions. Moreover, some participants were removed from the
final user dataset if these did not meet the age range of the quota sample,
spent less than 120 seconds or more than 2.5 hours with the recommender
system, or provided very consistent answering patterns across all constructs,
resulting in a final sample of 1,417 participants out of 1,801 initial ones.

The study consisted in a two-factorial experiment with an incomplete design.
The first factor consisted in using one of three types of text-based news
recommender systems (i.e. TF-IDF, Word2vec, Transformer) or a random
recommendation. The second factor concerned the inclusion of sentiment in the
recommender. Only the TF-IDF recommender was enriched with sentiment scores,
and thus, ratings gathered based on it are over-represented in the data. More
specifically, in the test data, there are 786 unique users with TF-IDF
generated ratings, 211 with Word2vec-generated ratings, 209 with Transformer-
generated ratings, and 211 with randomly generated ratings.

Each participant was firstly asked to complete a questionnaire including
questions regarding polarization, political interests, attitudes towards
refugees, and demographics. Afterwards, the subjects were asked to select the
news that they found the most interesting from a list of six news previews,
consisting of their titles and first lines of text. The participants would
then read the news, and the process would be repeated four times. At the end
of the experiment, the participants would be asked again to complete a follow-
up questionnaire, in order to examine how the recommended news influenced
their political polarization measured by the initial questionnaire.