# Signed Latent Factors for Spamming Activity Detection

Yuli LIU  Australian National University & Data61, CSIROCanberraAustralia
<EMAIL_ADDRESS>
(2021)

###### Abstract.

Due to the increasing trend of performing spamming activities (e.g., Web spam,
deceptive reviews, fake followers, etc.) on various online platforms to gain
undeserved benefits, spam detection has emerged as a hot research issue.
Previous attempts to combat spam mainly employ features related to metadata,
user behaviors, or relational ties. These works have made considerable
progress in understanding and filtering spamming campaigns. However, this
problem remains far from fully solved. Almost all the proposed features focus
on a limited number of observed attributes or explainable phenomena, making it
difficult for existing methods to achieve further improvement.

To broaden the vision about solving the spam problem and address long-standing
challenges (class imbalance and graph incompleteness) in the spam detection
area, we propose a new attempt of utilizing signed latent factors to filter
fraudulent activities. The spam-contaminated relational datasets of multiple
online applications in this scenario are interpreted by the unified signed
network. Two competitive and highly dissimilar algorithms of latent factors
mining (LFM) models are designed based on multi-relational likelihoods
estimation (LFM-MRLE) and signed pairwise ranking (LFM-SPR), respectively. We
then explore how to apply the mined latent factors to spam detection tasks.
Experiments on real-world datasets of different kinds of Web applications
(social media and Web forum) indicate that LFM models outperform state-of-the-
art baselines in detecting spamming activities. By specifically manipulating
experimental data, the effectiveness of our methods in dealing with incomplete
and imbalanced challenges is validated.

latent factors, spamming activities, pairwise ranking

††journalyear: 2021††copyright: rightsretained††conference: 22 Talks; August
22 - August 27, 2022; San Diego, California, USA††booktitle: 22 Talks, August
22 - August 27, 2022, 2022, San Diego, California††ccs: Information systems
Social networks††ccs: Information systems Data mining

##  1\. Introduction

A wide variety of spamming activities have exerted enormous negative impacts
on Web platforms. We list some examples to introduce these fraudulent
campaigns: (i) fake following relationships on social networking sites. To
gain undeserved fans (or excessive influence) for promoting products or
services (Almaatouq et al., 2014; Liu et al., 2016a), many malicious users
turn to the underground follower market to buy traded followers, disrupting
the fair following mechanism; (ii) opinion spam on review websites (Jindal and
Liu, 2008; Mukherjee et al., 2013). If spammers unfairly evaluate targeted
(cheating) items, the manipulated sentiment of user-generated contents would
likely be instilled to prospective consumers, affecting their final decisions
(Luca, 2016); (iii) Web spam refers to activities that use various techniques
to make spam Web pages achieve higher-than-deserved rankings in a search
engine’s result list (Gyongyi et al., 2004; Liu et al., 2008).

Existing anti-spam approaches mainly fall into three categories: (i) Behavior
Analysis, leveraging patterns of conducts (e.g., rating patterns of reviews
(Kumar et al., 2018; Jindal et al., 2010) and clicking attributes of search
results (Liu et al., 2008)), behavioral properties (e.g., synchronicity and
normality (Jiang et al., 2014)), etc. to catch abnormal behavior signals; (ii)
Content Processing, extracting distinctive characteristics by profile analysis
(Aggarwal and Kumaraguru, 2015; Shah and Pomerantz, 2010), semantic
representation (Sandulescu and Ester, 2015; Kim et al., 2015; Ren and Zhang,
2016) and syntactic structure (Fairbanks et al., 2018; Kennedy et al., 2019)
for filtering spam; (iii) Relational Structure, label propagation methods
(e.g., bipartite propagation with attribute-related weights (Zhang et al.,
2015), Markov Random Fields with intuitive parameters (Rayana and Akoglu,
2015)) and graph properties (Noekhah et al., 2020; Li et al., 2019) have been
fully utilized.

These works lead us to know that existing detection methods are mainly
implemented based on noticeable or explainable clues/ties that produce a
limited number of features for dealing with a specific type of spamming
activity. Consequently, ideas of designing detection models are hard to
stimulate and performances of combating spam are restricted (Liu, 2020; Zhang
et al., 2015). To overcome the limits of mainstream methods, we intend to
combat spamming activities from a fresh perspective, i.e., applying latent
factors to spam detection tasks. In the recommendation area, using matrix
factorization (MF) methods to acquire latent factors for predicting users’
preferences has become a common practice (He et al., 2017). This is because
latent factors represent potential interests (features) of users (items), and
expand known information by providing implicit indications. Given the fact
that networks used for spam detection tasks consist of normal (positive) and
spam (negative) instances, we treat these contaminated relational datasets as
signed networks. And then the signed latent factor mining (LFM) model is
utilized to extract two types (normal and spam) of potential representations
for each node. In this scenario, no explicit feature is required, but implicit
factors are mined to filter malicious campaigns.

We develop two competitive algorithms to mine spam-detection oriented signed
latent factors: (i) Pointwise Learning — multi-relational likelihoods
estimation (LFM-MRLE), and (ii) Pairwise Learning — signed pairwise ranking
(LFM-SPR). These two methods (MRLE and SPR) consider null relationships (non-
linked node pairs) and label unknown activities (i.e., unlabeled data or
labeled instances in test dataset), respectively. Three benefits are brought
by designing different LFM models: (i) It validates that this new perspective
of LFM possesses considerable potentials and capabilities for combating
spamming campaigns (evidences in Section 5.3); (ii) We can analyze the
influence of null and label unknown information based on experimental results;
(iii) Combining signed latent factors of MRLE and SPR can boost the final
detection performance.

The other intuitions of applying the LFM model derive from the intention of
tackling long-standing challenges in the spam detection area: (i) (Annotation
Difficulty) Because spamming campaigns in recent years are usually
crowdsourced form (Liu et al., 2017), the deliberate camouflage of spammers
superficially increases the difficulty of labeling malicious instances. This
causes two intractable problems: the lack of training labels (Mukherjee et
al., 2013; Rayana and Akoglu, 2015) and class imbalance (i.e., the imbalanced
ratio between known spam instances and normal/unlabeled data) (Mukherjee et
al., 2013; Fayazi et al., 2015). In this work, LFM models make full use of
different kinds of relationships (e.g., non-linked pairs and label unknown
links), and extract expanded potential factors from known information,
providing feasibility of tackling these problems; (ii) (Incomplete Graph
Structure) It is unlikely for researchers to crawl and analyze the whole
online network and metadata in a website, due to insufficient permissions and
computations. In this scenario, Web users or other instances in experimental
datasets inevitably lose some relational ties or signals that may be useful
for identifying spamming activities (Rout et al., 2017). Furthermore, numerous
collected users may only be related to a few numbers of items/targets in the
experimental graph, generating a highly sparse relational matrix, which
further aggravates the degree of graph incompleteness. Another motivation to
introduce the thought of latent factors to the anti-spam area is evoked, since
it is well known that the latent factors of MF are mainly used to learn
incomplete or potential interactions (Krishna et al., 2018; Zhang et al.,
2018).

It has been reported that designing a specific detection model to deal with a
certain kind of spamming activity makes researchers exhausted to tackle the
continuously emerging spamming phenomena (Liu, 2020). Besides, feature-based
detection models suffer from limited robustness and capability, since even the
same type of spamming campaign frequently changes its cheating strategies (Liu
et al., 2016a; Zhang et al., 2015). To cope with these predicaments,
relational datasets of different kinds of fraudulent phenomena in this paper
are interpreted by the unified signed network, and then LFM models that
involve no specific features can be generalized to detect disparate spamming
activities.

We summarize the contributions of this work as follows:

  * •

We uniformly use the signed network to represent various spam-contaminated
datasets and introduce a new perspective (i.e., LFM) to address the spam
detection problem.

  * •

Two competitive and highly dissimilar algorithms, SPR and MRLE, are proposed
to mine spam-detection oriented signed latent factors, tackling the long-
standing challenges.

  * •

To explore the capability of signed latent factors for filtering spamming
activities, five types of application strategies of signed latent
representations are proposed and evaluated. We will release the code upon
acceptance.

##  2\. SIGNED LATENT FACTOR MINING MODELS

To increase the generality and usability of signed latent factors for spam
detection (i.e., detecting multiple kinds of spamming activities), we use the
unified relational structure to represent disparate contaminated networks. Due
to the interactivity between users and other targets of different online
platforms (Liu, 2020), spam-contaminated networks can be uniformly treated as
the signed user-target relational structure. In this scenario, targets can
represent products, Web pages, videos, etc. that users interact with on Web
applications.

Existing detection methods mainly focus on filtering three types of spamming
objectives: spammers (Wang et al., 2011), malicious activities (Li et al.,
2019) or cheating targets (Liu et al., 2016a). In this work, we aim to filter
the second type, because detecting fraudulent activities facilitates timely
eliminating and alleviating negative influences of malicious phenomena (Kim et
al., 2015; Liu et al., 2017). In the signed user-target structure,
edges/entries are mainly constructed by activities, e.g., the user-review-
product structure on review websites, user-click-page relationships of search
engines, and user-like-post networks on social media platforms. Therefore,
detecting spamming activities in this paper means identifying abnormal edges
(entries) in signed networks (matrix). Besides, LFM models only rely on
ties/relations between users and targets to mine signed latent factors,
without using any crafted features. It means that spammers can not easily
avoid LFM models’ detection by changing disguise strategies, because inherent
connections between spammers and malicious targets are hard to conceal (Liu et
al., 2016a; Gong et al., 2014).

###  2.1. Application Strategies

Table 1. Operators for applying latent factors. Notation | Description | Dimensionality
---|---|---
Average ($Avg$) | $f(u,t)=\frac{1}{2}(W_{u}+H_{t})$ | $\ \ \ d^{+}+d^{-}$
Concatenate ($Con$) | $f(u,t)=(W_{u},H_{t})$ | $2*(d^{+}+d^{-})$
Subtraction ($Sub$) | $f(u,t)=(W_{u}^{-}-W_{u}^{+},H_{t}^{-}-H_{t}^{+})$ |  $d^{+}+d^{+}$
Spam Inner product ($IP^{-}$) | $f(u,t)=W_{u}^{-}H_{t}^{-T}$ | $\ \ \ \ \ \ \ \ \ 1$
Normal Inner product ($IP^{+}$) | $f(u,t)=W_{u}^{+}H_{t}^{+T}$ | $\ \ \ \ \ \ \ \ \ 1$

In the recommendation area, using low-dimensional latent factors obtained from
matrix factorization to represent potential preferences (features) of users
(items) is a popular method. As we know, the recommender system aims to
predict satisfying items for users, regardless of the moral label of a
user/item (i.e., without considering whether a user/item is benign or
malicious). Therefore, the latent factors mined for recommendation have no
normal or spamming sign (i.e., each user/item is usually represented by one
vector). However, in the area of spam detection, all instances are divided
into two classes, normal (positive) and spamming (negative). Consequently, to
capture two classes of latent factors corresponding to the normal and spamming
indications of each instance, all users and targets in the unified structure
are characterized by two vectors, i.e., positive factors $W_{u}^{+}$ and
$H_{t}^{+}$ for all users and targets (dimensions of positive latent factors
are $d^{+}$), and negative forms $W_{u}^{-}$ and $H_{t}^{-}$ of users and
targets (dimensions are both $d^{-}$).

As mentioned above, we focus on detecting spamming activities, i.e., edges
(entries) of a graph (matrix). Referring to previous work (Xu et al., 2019; Su
et al., 2018), the class of an activity on Web applications is determined by
the linked user and target. Therefore, after obtaining the signed latent
factors of nodes (entities), we need to explore how to comprehensively apply
them to filtering malicious activities. In this work, five kinds of
application modes are proposed to utilize the signed latent factors of linked
nodes (user and target) to represent a link (activity), which are listed in
Table 1: (i) Average ($Ave$), calculating the average results after adding
$W_{u}$ and $H_{t}$, where $W_{u}$ and $H_{t}$ denote the concatenation of
user latent factors ($W_{u}^{+}$ and $W_{u}^{-}$) and target latent factors
($H_{t}^{+}$ and $H_{t}^{-}$), respectively; (ii) Concatenate ($Con$), all
kinds of latent factors are directly and integrally concatenated. This type of
operator produces the combined factor vectors with the widest dimensionality —
$2*(d^{+}+d^{-})$; (iii) Subtraction ($Sub$), using the spamming indications
of user and target ($W_{u}^{-}$ and $H_{t}^{-}$) to subtract the normal
indications ($W_{u}^{+}$ and $H_{t}^{+}$), with the intention of avoiding
misjudging normal activities as spam ones. We need to note that the dimensions
of positive and negative factors should be the same when we use $Ave$ and
$Sub$ operators.

The above-mentioned three application modes utilize the combined factors as
features and feed them into a classification model to filter spamming
activities. The other two operators based on calculating the inner product are
inspired by the common practice of MF recommendation. The fourth operator
$IP^{-}$ denotes the inner product of $W_{u}^{-}$ and $H_{t}^{-}$,
representing the possibility of the edge ($u,t$) being deceptive (negative).
Similarly, the result of $IP^{+}$ denotes the normal possibility of the
corresponding edge. The negative and positive inner product results of all the
activities in datasets can be ranked in descending order, and we hold that
spamming activities will rank high (low) in the $IP^{-}$ ($IP^{+}$) result
list.

##  3\. LFM-MRLE MODEL

We propose the first LFM model — multi-relational likelihoods estimation
(MRLE) on the basis of defining three types of naturally existing relations in
spam-related networks inspired by the link prediction work (Xu et al., 2019):
(i) normal/positive relations ($nor$), i.e., benign activities; (ii)
spam/negative ($sp$) ties; (iii) non-linked ($non$) node pairs, i.e., there is
no interaction between users and targets. The null relationships are often
ignored by existing detection models.

After defining three types of spam-related relationships, the next key step is
to quantify them through three types of likelihood formulations based on
analyzing the interplay between spam and normal indications of each node pair.
To quantify the indicative meanings of likelihoods and obtain nonlinear latent
factors, we utilize the activation function
$f_{a}(x)=\frac{p_{0}exp(x)}{1+p_{0}(exp(x)-1)}$, which is similar to the
common sigmoid function in neural networks. Given a node pair $(u,t)$, we use
$F_{ut}^{-}=f_{a}(W^{-}_{u}H^{-}_{t}$) to denote the calculated spamming
signal, where $W^{-}_{u}H^{-}_{t}$ is the inner product of spam latent
representations of user $u$ and target $t$ in user spam latent factor matrix
$W^{-}$ and target spam latent factor matrix $H^{-}$. Similarly, we can also
capture the quantified normal signal, using
$F_{ut}^{+}=f_{a}(W^{+}_{u}H^{+}_{t}$).

We now describe how to formulate three types of relations and explain
corresponding indicative meanings: (i) (normal link) If an edge $(u,t)$ (i.e.,
activity performed by $u$ to $t$) is normal, the user $u$ and target $t$ are
connected by a benign tie, and they are both inclined to be normal. It means
that the value of normal indication $F_{ut}^{+}$ should be large and
$F_{ut}^{-}$ is assumed to be small, so we combine them with the goal of
making the score of $F_{ut}^{+}(1-F_{ut}^{-})$ large; (ii) (spam link)
Oppositely, in this case, the $u$ and $t$ are associated with fraudulent
activity, i.e., the possibility for corresponding link being spam are high. So
we use $(1-F_{ut}^{+})F_{ut}^{-}$ to represent the spam likelihood; (iii)
(non-linked pair) Different from most previous works, the unconnected
relationship is also considered in this work. We tend to believe that null
relationships capture the possible discrepancy between corresponding node
pairs, and signify that there is neither the positive nor negative relation
between $u$ and $t$, which can be represented by
$(1-F_{ut}^{+})(1-F_{ut}^{-})$. Besides, three types of edge sets are
represented by $E^{nor}$, $E^{sp}$ and $E^{non}$, respectively.

###  3.1. Estimating Multi-Relational Likelihoods

We integrate multi-relational likelihoods using the maximum likelihood
formulation, and employ negative log-likelihood objective function to estimate
signed latent factors, defined as follows:

(1) $\displaystyle L=$ $\displaystyle-\sum\nolimits_{(u,t)\in
E^{nor}}\log(F_{ut}^{+}(1-F_{ut}^{-}))$ $\displaystyle-\sum\nolimits_{(u,t)\in
E^{sp}}\log((1-F_{ut}^{+})F_{ut}^{-})$ $\displaystyle-\sum\nolimits_{(u,t)\in
E^{non}}\log((1-F_{ut}^{+})(1-F_{ut}^{-})).$

This objective function is composed of three components, corresponding to
three sets of relationships.

To calculate signed latent factors of each node in the unified signed network,
the components based on edge form need to be transformed into the node-based
format. As there are two types of entities in relational data: user and
target, the integrated formulation is then divided into two parts, i.e., user
and target forms. The user-perspective one is:

(2) $\displaystyle L_{(u)}=$ $\displaystyle-\sum\nolimits_{t\in
T^{nor}(u)}\log(F_{ut}^{+}(1-F_{ut}^{-}))$ $\displaystyle-\sum\nolimits_{t\in
T^{sp}(u)}\log((1-F_{ut}^{+})F_{ut}^{-})$ $\displaystyle-\sum\nolimits_{t\in
T^{non}(u)}\log((1-F_{ut}^{+})(1-F_{ut}^{-}))$

where $T$ denotes the set of targets, and $T^{nor}(u)$, $T^{sp}(u)$, and
$T^{non}(u)$ represent the sets of targets that respectively receive normal,
spam, and null connections from user $u$, i.e., three types of relations. We
can see that a user’s likelihood is calculated based on all targets in the
signed network that have different relationships with it. In real-world
networks, the majority of targets have no link to a specific user. If all of
them are considered when we estimate the user’s $non$ likelihood, a huge
amount of computational resources will be consumed. Therefore, in the process
of calculating the null relational likelihood, we only sample $n$ targets that
are not associated with the user. Through experiments of fixing other
parameters but vary $n$, the effect of considering null relationships in
detecting spamming activities is investigated (shown in Section 5.3).
Similarly, the other objective function focusing on targets can be formulated
by replacing $u$ to $t$, or conversely. To save space, the corresponding
formulation is not presented.

To minimize the objective functions, the coordinate descent method (Lin, 2007;
Xu et al., 2019) is used, where signed latent vectors of an instance are
updated while fixing all the other vectors in each iteration. The minimization
task then becomes a convex optimization problem. The derivatives of a user’s
signed latent factors are:

(3) $\displaystyle\frac{\partial L_{(u)}}{\partial W_{u}^{+}}=$
$\displaystyle-\sum\nolimits_{t\in T^{nor}(u)}((1-F_{ut}^{+})H_{t}^{+})$
$\displaystyle+\sum\nolimits_{t\in T^{sp}(u)\cup
T^{non}(u)}F_{ut}^{+}H^{+}_{t}$

(4) $\displaystyle\frac{\partial L_{(u)}}{\partial W_{u}^{-}}=$
$\displaystyle-\sum\nolimits_{t\in T^{sp}(u)}((1-F_{ut}^{-})H_{t}^{-})$
$\displaystyle+\sum\nolimits_{t\in T^{nor}(u)\cup
T^{non}(u)}F_{ut}^{-}H^{-}_{t}.$

The other two derivatives, i.e., normal and spam target latent factors
($H^{+}$, $H^{-}$), can be easily deduced based on the equations above.

Additionally, to validate the convergence of our MRLE model, we implement the
L-BFGS-B algorithm (Zhu et al., 1997). By performing experiments on a real-
world dataset used by Liu et al. (Liu et al., 2016a) described in Section 5.1,
we find that the optimization algorithm successfully achieves convergence
after only 72 times of iterations.

In the calculating process, we find that LFM-MRLE estimates the single
predictor (i.e., individual latent vector as is shown in Equation 3 and 4),
without calculating the difference between two predictors. Besides, although
the MRLE method considers non-linked relationships in the learning process,
the label unknown activities (unlabeled instances and labeled data used for
testing in signed networks) are omitted. In the spam detection area, all
existing connections (including label unknown ones) are useful, because they
keep the integrity of networks and facilitate extracting distinctive structure
properties for detecting spam (Li et al., 2017; Jiang et al., 2014). In view
of these two problems, we try to take comparative ranking information into
account (i.e., to estimate the difference of two predictors) on the integral
network. From this side, the commonly used objective function — pairwise loss
in BPR method (Rendle et al., 2009b) comes to mind, which is still a highly
competitive approach in item recommendation (He et al., 2017).

##  4\. LFM-SPR MODEL

Figure 1. Motivating Example of Signed Pairwise Ranking

.

We now start to introduce the preliminaries of LFM-SPR model based on signed
pairwise relations. The signed user-target matrix $Y$: $U\times T$, where $U$
and $T$ represent user and target sets, is defined according to the signed
network as,

(5) $y_{ut}=\left\\{\begin{aligned} +,&\ \ \ \ \ \ \text{ if user }u\text{
perform a normal activity to target }t\\\ -,&\ \ \ \ \ \ \text{ if user
}u\text{ perform a spam activity to target }t\\\ ?,&\ \ \ \ \ \ \text{ if
label of interaction between }u\text{ and }t\text{ is unknown }\\\ 0,&\ \ \ \
\ \ \text{ no link between }u\text{ and }t.\end{aligned}\right.$

Here the matrix is quite different from the common interpretation of
recommendation works called 0/1 scheme (Rendle et al., 2009a) that uses value
$1$ to indicate the observed interaction and $0$ to denote the unobserved one.
The reason of defining four types of entries mainly lies in: (i) The spam-
contaminated matrix is signed, with normal ($+$) and spam ($-$) entries; (ii)
To consider all existing connections into signed latent factors mining for
combating spam activities, the label unknown links ($?$) are also used to
provide additional clues for comparing signed ranking pairs.

In order to clarify the SPR model, we use Figure 1 as a motivating example.
The symbols $+$ and $-$ denote known normal and spamming activities, i.e.,
labeled edges used as training data. A value of $0$ indicates the null
relationship. The entries with the symbol of $?$ are composed of two types of
instances: (i) unlabeled activities, and (ii) labeled activities used as test
data, i.e., entries with $?$ are unknown for SPR learning. Usually, in
recommendation tasks, the observed feedbacks divided into the test dataset are
directly treated as unobserved ones in the learning process, which will be
used to evaluate performances of the recommender system (He et al., 2017).
However, it is necessary to set a new type of entries (i.e., $?$) to represent
the unique instances in spam detection tasks: (i) There are unlabeled datasets
in many works of filtering spam due to the difficulty of labeling instances
(i.e., Annotation Difficulty), which are unable to be treated as positive nor
negative ones; (ii) If we use the common practice of recommendation tasks to
directly handle the unknown connections as unobserved feedbacks ($0$), these
existing but label unknown links will be blended into null relations. It means
that an important type of relational ties that plays a distinctive role in
detecting spam (Gong et al., 2014) will be ignored.

In real-world spamming activity detection experiments, to fairly evaluate the
model, randomly dividing training and test datasets is often used. In this
paper, we aim to detect spamming activities rather than malicious users or
targets, so all or part of the labeled activities of some users may be used as
test data after dividing the dataset, just like $u_{3}$ (all entries are
unknown) and $u_{4}$ (one entry with $?$) in the matrix of Figure 1. However,
to train and test the MF model for a personalized recommendation, it is quite
common for researchers to randomly select a part of each user’s feedbacks as
test data (Rendle et al., 2009b; He et al., 2017), i.e., each user contains
observed feedbacks. This makes predicting and testing recommendation results
possible. As for spamming activity detection tasks, each user has unconcealed
activities in the training process is unfair for evaluation. Because spammers
(benign users) mainly perform spam (normal) behaviors (Lu et al., 2013; Liu et
al., 2016a), the label of a user’s known activities is a strong clue for a
detection model to give the same class for the user’s other unknown activities
as the label of known ones.

The recommendation problem usually can be abstracted as learning
$\hat{y_{ui}}=f(u,i|\Theta)$, where $\hat{y_{ui}}$ indicates the predicted
score of an interaction between user $u$ and item $i$ (He et al., 2017), and
$\Theta$ represents the parameter vector of an arbitrary models class (e.g.,
matrix factorization). And two types of objective function are most widely
used to estimate $\Theta$ in existing literature — pointwise loss (Hu et al.,
2008; He et al., 2017) and pairwise loss (Rendle et al., 2009b). To formulate
a pairwise loss function for an item recommendation, the first step is to
construct pairwise relations which are usually based on specific user’s
preferences, i.e., a user prefers observed feedbacks over unobserved ones
(Rendle et al., 2009b). Therefore, the matrix factorization of BPR only needs
to learn one type of latent factors according to user preferences. As for the
contaminated matrix, however, to mine signed latent factors, pairwise
relations are built from two perspectives: normal ($+$) level and spam ($-$)
level. Through this way, we can not only know spamming pairwise rankings to
learn negative latent factors but also can learn positive ones based on normal
ranking relations. Consequently, each user/target is represented by two low-
dimensional factors.

Overall, the signed pairwise learning method for spam detection is quite
different from BPR for the recommendation, not only in the objective but also
in data processing and ranking pair comparison (summarised in the Related Work
section). The design of SPR shows that the thought of mining spam detection-
oriented signed latent factors is not restricted to a certain direction. Two
implementations of LFM signify that more possibilities and insights for using
latent factors to solve the spam problem are provided.

###  4.1. Constructing Ranking Pairs

On the left and right sides of Figure 1, two levels of ranking relations are
extracted for specific users, i.e., spam ($>_{u}^{-}$) and normal
($>_{u}^{+}$) ranking relations. From spam level, if target $t$ has received a
spam ($-$) action from user $u$, i.e., a negative ($u$, $t$) entry in signed
matrix, we can assume that user $u$ brings more spamming indications to target
$t$ than other targets that have normal or null interactions with $u$. For
example, on the top left of Figure 1, user $u_{1}$ performs spamming
activities to $t_{1}$ and $t_{3}$, but no feedback is provide to $t_{2}$,
$t_{4}$ and $t_{5}$. So we tend to believe that targets $t_{1}$ and $t_{3}$
are more likely to be malicious entities compared to other targets from the
perspective of user $u_{1}$. That is, using signed feedbacks of $u_{1}$, six
spam ranking relations can be found: $t_{1}>_{u_{1}^{-}}t_{2}$,
$t_{1}>_{u_{1}^{-}}t_{4}$, $t_{1}>_{u_{1}^{-}}t_{5}$, and $t_{3}$ also ranks
higher than $t_{2}$, $t_{4}$ and $t_{5}$ based on $u_{1}$’s perspective.

On the other hand, if there are label unknown activities ($?$) in a user’s
feedbacks, it means that clues that facilitate obtaining ranking pairs are
unclear. And the corresponding user is named ranking-unclear user. To tackle
the clue unclear problem, we attempt to seek complementary evidences from
existing interactions of related targets that are linked to the ranking-
unclear user, i.e., comparing ranking relations based on the interaction
distribution of corresponding target columns. Taking $u_{3}$ for example, we
can see that all of its interactions ($u_{3}$ row) are unknown ($?$) for
learning process. In the motivating matrix, target $t_{1}$ receives a class
unknown ($?$) activity from $u_{3}$, but there are the other two activities
from $u_{1}$ and $u_{4}$ (in $t_{1}$ column), i.e., ($u_{1}$, $t_{1}$) and
($u_{4}$, $t_{1}$), which are known spamming links ($-$). Based on this
evidence, we can say that $t_{1}$ is suspicious due to its involvement in
cheating activities. It means that the label unknown feedback ($u_{3}$,
$t_{1}$) of ranking-unclear user $u_{3}$ has higher possibility of being
malicious than $u_{3}$’s non-linked relationships ($u_{3}$, $t_{2}$) and
($u_{3}$, $t_{5}$), i.e., $t_{1}>_{u_{3}^{-}}t_{2}$,
$t_{1}>_{u_{3}^{-}}t_{5}$. However, from the perspective of $u_{3}$, we cannot
deduce the ranking relations between $t_{1}$, $t_{3}$ and $t_{4}$ (targets
that receive $?$ interactions from $u_{3}$), since the columns of $t_{3}$ and
$t_{4}$ are also related to spamming interactions, i.e., ($u_{1}$, $t_{3}$)
and ($u_{4}$, $t_{4}$). The spam ranking relations of $u_{3}$ is shown on the
bottom left of Figure 1.

In summary, the process of acquiring signed ranking relations can be divided
into two parts: (i) analyzing class known entries ($+$ or $-$), and (ii) in
face of unknown interactions ($?$). Complementary evidence from the linked
targets should be taken into account when dealing with the label unknown case
of a specific user. Besides, it is important to mention that to cheat
detection models, spammers occasionally interact with normal targets to
disguise themselves (i.e., disguised-activities). To alleviate the
perturbations of this disguise strategy, we set an identification threshold
$\xi$ (Liu, 2020). When we assign $\xi=1$, for example, the entry ($u_{3}$,
$t_{3}$) will rank lower than ($u_{3}$, $t_{1}$) from the perspective of
$u_{3}$, because there is only one spamming interaction in $t_{3}$ column,
i.e., the number of related spamming links ($-$) of $t_{3}$ is not bigger than
$\xi$. It means that the evidence of $t_{3}$ being fraudulent is not enough.
However, $t_{1}$ receives two spam feedbacks ($>1$), showing severe
fraudulence. Therefore, in the process of obtaining ranking pairs, we can
regard the label unknown entries (e.g., ($u_{3}$, $t_{1}$)) that are related
to fraudulent-enough targets (e.g., $t_{1}$) as auxiliary-higher-ranking
interactions.

Similarly, the pairwise ranking relations from the normal level can also be
constructed (shown on the right side of Figure 1).

###  4.2. Optimizing Signed Pairwise Ranking

The widely used pairwise ranking method BPR (Rendle et al., 2009b) uses
parameter $\Theta$ to determine personalized rankings, and applies Bayesian
formulation to maximize the following posterior probability,

(6) $\displaystyle p(\Theta|i>_{u}j))\propto p(i>_{u}j|\Theta)p(\Theta).$

Here, $p(i>_{u}j|\Theta)$ denotes the likelihood function, which captures the
individual probability that user $u$ prefers target $i$ over target $j$, i.e.,
$i$ represents the higher-ranking target and $j$ is the lower one from the
view of $u$. And it is defined as:
$p(i>_{u}j|\Theta):=\sigma(\hat{x}_{uij}(\Theta))$, where $\sigma$ denotes the
common logistic sigmoid: $\sigma(x)=\frac{1}{1+e^{-x}}$. Further, the
$\hat{x}_{uij}(\Theta)$ captures the relationship between target $i$ and
target $j$ based on the preferences of user $u$. It can be decomposed into:
$\hat{x}_{uij}=\hat{x}_{ui}-\hat{x}_{uj}$. In the pairwise loss paradigm,
researchers try to predict the difference between two predictions
$\hat{x}_{ui}$ and $\hat{x}_{uj}$, rather than a single predictor
$\hat{x}_{ui}$ or $\hat{x}_{uj}$.

To mine the SPR signed latent factors, signed trainable parameter vectors
$\Theta^{+}$ and $\Theta^{-}$ are defined, corresponding to two types of
ranking relations: $\hat{x}_{uij}^{+}$ and $\hat{x}_{uij}^{-}$, i.e., pairwise
ranking relations from normal and spam levels. To sum it up, the integrated
form of posterior probability optimization used in this paper can be
formulated as:

(7) $\displaystyle\ln\prod_{(u,i,j)\in
Y_{S}^{+}}p\left(\Theta^{+}\mid\hat{x}_{uij}^{+}\right)\prod_{(u,i,j)\in
Y_{S}^{-}}\left(\Theta^{-}\mid\hat{x}_{uij}^{-}\right)$
$\displaystyle=\sum_{(u,i,j)\in
Y_{S}^{+}}\ln\sigma\left(\hat{x}_{uij}^{+}\right)+\sum_{(u,i,j)\in
Y_{S}^{-}}\ln\sigma\left(\hat{x}_{uij}^{-}\right)+\ln\left(\Theta^{+}\right)+\ln\left(\Theta^{-}\right)$
$\displaystyle=\sum_{(u,i,j)\in
Y_{S}^{+}}\ln\sigma\left(\hat{x}_{uij}^{+}\right)+\sum_{(u,i,j)\in
Y_{S}^{-}}\ln\sigma\left(\hat{x}_{uij}^{-}\right)-\lambda_{\Theta^{+}}\left\|\Theta^{+}\right\|^{2}-\lambda_{\Theta^{-}}\left\|\Theta^{-}\right\|^{2}$

where the triple $(u,i,j)\in{Y_{S}^{+}}$ means the normal ranking pair
$\hat{x}^{+}_{ui}-\hat{x}^{+}_{uj}$ in the training dataset of signed matrix
$Y$. To complete the Bayesian modeling approach, a general prior density
$p(\Theta)$ with zero mean and variance matrix:
$\sum(\Theta)=\lambda_{\Theta}I$ is introduced (Rendle et al., 2009b). In the
practical calculation, $\lambda_{\Theta}$ can be seen as the model specific
regularization parameter. To mine signed latent factor matrices, two specific
regularization parameters are defined $\lambda_{\Theta^{+}}$ and
$\lambda_{\Theta^{-}}$. After calculating the logarithm of Equation 7, the
parameters $\Theta^{+}$ and $\Theta^{-}$ are separated into two accumulation
parts. It means that we can separately and flexibly calculate the positive and
negative latent factors. By flipping the signs of all components in Equation
7, the maximum optimization is transformed into the commonly used minimizing
task.

The LFM-SPR model aims to mine positive and negative latent factors for each
user and target. In the optimization equation, the user and target are
incorporated into pairwise relations, so we first need to decompose the
pairwise estimator $\hat{x}_{uij}^{+}$ into:
$W_{u}^{+}H_{i}^{+T}-W_{u}^{+}H_{j}^{+T}$ where $W_{u}^{+}$, $H_{i}^{+}$ and
$H_{j}^{+}$ denote the normal latent factors of user $u$, target $i$ and
target $j$, respectively. Because the calculation of optimizing signed latent
factors can be separated, we only present the formulation of normal part. To
learn positive signed latent factors from spam-contaminated matrix, the
gradient for three parameters $W_{u}^{+}$, $H_{i}^{+}$ and $H_{j}^{+}$ are
needed to derive, using the stochastic gradient descent algorithm:

(8) $\displaystyle\frac{\partial}{\partial
W_{u}^{+}}=\frac{1}{1+e^{W_{u}^{+}H_{i}^{+T}-W_{u}^{+}H_{j}^{+T}}}\cdot(H_{j}^{+}-H_{i}^{+})+\lambda_{\Theta^{+}}W_{u}^{+}$

(9) $\displaystyle\frac{\partial}{\partial
H_{i}^{+}}=\frac{1}{1+e^{W_{u}^{+}H_{i}^{+T}-W_{u}^{+}H_{j}^{+T}}}\cdot-
W_{u}^{+}+\lambda_{\Theta^{+}}H_{i}^{+}$

(10) $\displaystyle\frac{\partial}{\partial
H_{j}^{+}}=\frac{1}{1+e^{W_{u}^{+}H_{i}^{+T}-W_{u}^{+}H_{j}^{+T}}}\cdot
W_{u}^{+}+\lambda_{\Theta^{+}}H_{j}^{+}$

Finally, an update for these three parameters can be performed in each
iteration:

(11)
$\displaystyle\Theta^{+}\Leftarrow\Theta^{+}-\alpha\frac{\partial}{\partial\Theta^{+}}$

where $\alpha$ denotes the learning rate. The spam latent factors can also be
calculated via the similar approach.

Up to this point, two LFM models (MRLE and SPR) have been introduced. We can
see that both models aim to capture signed latent representations of each
instance, but the basic thoughts are quite different. By utilizing multi-
relational likelihoods and signed pairwise rankings, we try to potentially
represent each user and target as comprehensively as possible.

##  5\. EXPERIMENTS

Table 2. AUC performances of compared methods in two datasets with varying size (%) of labeled data ($n=20$, $d=30$, $\xi=2$). Methods | Social | Forum
---|---|---
20% |  25% |  30% |  40% |  50% |  20% |  25% |  30% |  40% |  50%
MRLEAvg | 0.708 | 0.733 | 0.765 | 0.770 | 0.792 | 0.639 | 0.665 | 0.690 | 0.709 | 0.716
MRLECon | 0.809 | 0.832 | 0.857 | 0.870 | 0.876 | 0.718 | 0.733 | 0.755 | 0.767 | 0.779
MRLESub | 0.716 | 0.758 | 0.778 | 0.784 | 0.803 | 0.640 | 0.663 | 0.695 | 0.715 | 0.721
MRLE${}_{IP^{-}}$ | 0.704 | 0.715 | 0.741 | 0.757 | 0.763 | 0.639 | 0.642 | 0.681 | 0.695 | 0.709
MRLE${}_{IP^{+}}$ | 0.651 | 0.664 | 0.685 | 0.688 | 0.696 | 0.584 | 0.595 | 0.617 | 0.629 | 0.635
MRLE$-non$ | 0.734 | 0.756 | 0.774 | 0.792 | 0.813 | 0.661 | 0.674 | 0.692 | 0.706 | 0.728
MRLE$+au$ | 0.812 | 0.847 | 0.872 | 0.879 | 0.886 | 0.725 | 0.741 | 0.764 | 0.775 | 0.781
SPRAvg | 0.710 | 0.742 | 0.750 | 0.762 | 0.764 | 0.656 | 0.674 | 0.693 | 0.707 | 0.712
SPRCon | 0.813 | 0.842 | 0.859 | 0.864 | 0.871 | 0.732 | 0.745 | 0.752 | 0.764 | 0.771
SPRSub | 0.725 | 0.754 | 0.771 | 0.783 | 0.794 | 0.660 | 0.684 | 0.708 | 0.714 | 0.726
SPR${}_{IP^{-}}$ | 0.709 | 0.720 | 0.731 | 0.742 | 0.757 | 0.642 | 0.659 | 0.671 | 0.683 | 0.691
SPR${}_{IP^{+}}$ | 0.642 | 0.655 | 0.673 | 0.689 | 0.693 | 0.562 | 0.576 | 0.584 | 0.607 | 0.615
GraphHITS | 0.690 | 0.745 | 0.762 | 0.775 | 0.782 | 0.620 | 0.646 | 0.671 | 0.680 | 0.687
GraphBi | 0.698 | 0.750 | 0.774 | 0.783 | 0.791 | 0.637 | 0.659 | 0.679 | 0.685 | 0.690
FeatureSocial | 0.616 | 0.678 | 0.703 | 0.729 | 0.742 | - | - | - | - | -
FeatureForum | - | - | - | - | - | 0.491 | 0.543 | 0.563 | 0.581 | 0.595
FactorGraph | 0.703 | 0.765 | 0.782 | 0.796 | 0.806 | 0.643 | 0.679 | 0.692 | 0.702 | 0.710
LatentSVD | 0.648 | 0.685 | 0.742 | 0.769 | 0.783 | 0.594 | 0.602 | 0.645 | 0.664 | 0.671
LatentGAUC | 0.714 | 0.735 | 0.775 | 0.798 | 0.819 | 0.641 | 0.658 | 0.697 | 0.710 | 0.735
Latentn-aware | 0.737 | 0.750 | 0.785 | 0.804 | 0.821 | 0.652 | 0.661 | 0.698 | 0.736 | 0.740

In this section, we comprehensively evaluate three aspects of our LFM models:
effectiveness, stability of dealing with class imbalance, and tolerance in the
face of the incomplete network.

###  5.1. Datasets

We start by introducing two real-world datasets used in this work. As we
mentioned before, LFM models are capable of filtering various spamming
activities resorting to the unified signed networks transformed from original
networks of online platforms. To evaluate this generality, we select two
disparate datasets from two types of Web applications — the microblogging
service and Web forum. These datasets are contaminated by different malicious
activities, i.e., fake following relationships and deceptive opinions. If our
methods (MRLE and SPR) achieve considerable effectiveness in dealing with
these two categories of spamming campaigns, the usability and generality of
signed latent factors in dealing with spam problems can be validated.

The first dataset Social is collected by (Liu et al., 2016a), focusing on
combating the fake follower problem (Almaatouq et al., 2014) on social
networking sites. The authors buy about 3K fake followers from the underground
market, which are directly treated as a ground-truth spammer dataset. They
then use empirical evidences to collect a non-spam user set. In order to
obtain a relatively complete network, all the neighbours (followers and
followees) of two kinds of user sets (spammers and non-spam users) are
crawled. On microblogging platforms, each user has two social identities,
i.e., follower and followee. To construct the unified signed network, we
regard the followee identity of an account as the target node. It means that
targets (followees) are linked to social user (follower) nodes by following
relationships. Our objective is to detect fake following links. As following
activities (links) are not directly labeled in Social, we regard the links
between labeled spammers (fake followers) and suspicious followees (followed
by more than one fake followers) as spam ones.

The second dataset Forum is crawled for studying the opinion spam problem in
Web forum (Chen and Chen, 2015), i.e., filtering deceptive contents that are
used to promote or demote malicious products/services. A real case: Samsung
probed over ‘fake web reviews’ in a popular Web forum is revealed. The authors
gather all the certain deceptive posts on the revealed list and non-spam posts
in the same time window. The dataset includes 10K first posts (5% spams), 148K
replies, and 60K user accounts (0.52% spammers). The first post in the Web
forum is like a topic/question, and other users can reply to it with user-
generated contents. In this work, we focus on filtering deceptive replies
(posted by labeled spammers) that link responders (users) to the first posts
(targets).

###  5.2. Compared Methods

In Section 2.1, five types of application strategies of signed latent factors
are described, which can be generally summarized into two types: (i) using
latent factors to construct feature sets that are fed into a popular
classification model — logistic regression; (ii) referring to recommendation
methods based on matrix factorization, we rank the inner product results of
spam latent factors ($IP^{-}$ operator) and select top-ranked activities as
spam ones. If the $IP^{+}$ operator is used, the low ranked links are regarded
as spamming ones.

To show that LFM models outperform existing mainstream methods, multiple
baselines are used to compare and analyze: (i) pruned MRLE model — MRLE$-non$
(i.e., ignoring non-linked relations), and improved MRLE model — MRLE$+au$
that regards high ranking interactions identified by SPR as auxiliary known
spamming links; (ii) graph propagation-based approaches applied to both Social
and Forum, HITS-like detection method — GraphHits proposed by (Liu et al.,
2016a) and bipartite graph propagation model with polished edge weight —
GraphBi (Zhang et al., 2015); (iii) feature-based models, utilizing powerful
logistic regression model to classify spamming activities based on common used
features for fake follower detection (Aggarwal and Kumaraguru, 2015; Liu et
al., 2016a) (FeatureSocial) and specific features extracted to combat opinion
spam in Web forum (Chen and Chen, 2015; Jindal et al., 2010) (FeatureForum);
(iv) graph and feature combined baseline — FactorGraph is developed based on
the probabilistic factor graph model (Liu et al., 2017), integrating
correlation factors and attribute factors.

Three state-of-the-art latent-factor methods are selected as baselines to
indicate that addressing the spam detection problem using latent factors is a
unique and innovative work, and our LFM models have distinct advantages on
combating spam over these baselines. (i) LatentSVD (unsigned factors, $d=30$)
is conducted by the widely adopted Singular Value Decomposition (SVD) method
(Abdi, 2007) on the original user-target matrix. It means that the sign of
each entry is not considered, and latent factors of all the instances are
mined based on the network structure; (ii) LatentGAUC (unsigned factors,
$d=30$) quantify the ranking performance in signed networks by focusing on the
head (spam) and tail (normal) of a ranking list (Song and Meyer, 2015). A low-
rank model is employed to obtain ranking scores; (iii) A negative-aware MF
recommendation approach(Lin et al., 2019) explicitly addresses the uncertainty
of unseen user-item associations, i.e., quantifying the degree of uncertainty
for unseen associations by leveraging user preference similarity. To estimate
the signed latent factors of each entity, we develop Latentn-aware (signed
factors, $d^{-}=d^{+}=30$) by separately utilizing spam and normal
associations to calculate spam- and normal-oriented user preference
similarities.

###  5.3. Effectiveness

Because we use a new perspective to solve the spam problem, it is important to
detail how to set the training and test datasets. For example, 20% labeled
data in an experiment means that twenty percent of labeled spamming activities
in the whole dataset are randomly selected, i.e., making their labels known to
SPR and MRLE models in the learning process. To keep class balance, the same
number of normal instances as the amount of selected 20% spamming activities
are selected. As for the remaining instances (not in the training dataset),
they are label unknown for the SPR model (i.e., with a symbol of $?$ in the
signed matrix). And MRLE model knows that these links are existed in the
signed network but does not divide them into any of the three types of
relational pairs. To fairly compare the mentioned methods above, they all
utilize the same size of prior knowledge (i.e., the same training dataset).

Table 2 reports the AUC performances of all the compared methods over two
datasets. The useful observations derived from this table can be summarized
into following aspects: (i) Two types of signed latent factors mined from MRLE
and SPR both surely provide effective potential indications for detecting spam
campaigns, because five operators for applying LFM all play relatively
positive detection roles. Among designed operators, the $Con$ obtains the best
detection performance on both datasets with different sizes of varying labeled
data compared to the other operators. In the following contents, SPR and MRLE
represent the LFM models using the $Con$ operator; (ii) Without using null
relationships (MRLE$-non$), the detection performances will be negatively
affected, which indicates the indispensable effects of seldom-mentioned $non$
relations. It’s important to note that the improved MRLE model (MRLE$+au$)
achieves the best performance in most cases by taking full advantages of non-
linked relations and auxiliary ranking information of SPR; (iii) Bipartite
propagation (GraphBi) and HITS-like (GraphHITS) methods achieve better
performances than the other feature-based compared approaches, especially in
the case of using the small size of prior knowledge (e.g., 20%). This is
mainly because propagation model makes use of the inherent connections between
spammers and targets (Zhang et al., 2015; Rayana and Akoglu, 2015) on the
whole graph; (iv) Even if a small proportion of labeled data is used as prior
knowledge, our LFM models achieve relatively high AUC values compared with
other methods. It demonstrates that the signed latent factors have the ability
to deal with the insufficient labeled dataset by mining more attributes that
are hidden. Besides, with the increase of training data, the performances of
LFM models improve smoothly, which means that our methods are relatively
stable; (v) The SPR model that considers label unknown activities performs
better than MRLE when a few labeled datasets (20% or 30%) are used in Social
and Forum. Two factors contribute to this result, 1) label unknown instances
bring additional evidence for comparing ranking relations which are especially
useful when the labeled data is limited, and 2) SPR model makes full use of
the labeled data through repeatedly comparing them with others; (vi) Three
latent-factors baselines use the $con$ operator to represent links. The
LatentSVD does not perform well, as SVD latent factors are mined based on the
network structure ignoring the sign of each entry. Although Latentn-aware
provides added information for the unseen interaction by quantifying the
degree of uncertainty and LatentGAUC considers signed links, they both are not
properly tailored to make full use of the special label unknown and null
relations in spam detection tasks.

We also perform experiments using the combined method — MRLE-SPR, which
concatenate all signed latent factors mined from MRLE and SPR. The detection
results of MRLE-SPR are better than the individual models, increasing by 2.4%
and 3.1% compared to MRLE and SPR, respectively, using Social (30% labeled
data, $n=20$, $d=30$, $\xi=2$). It shows that the combined model MRLE-SPR
learns from each other’s strong points and provides more potentials.

As is introduced in the formulations of MRLE and SPR, several parameters are
used, such as $n$ (size of unconnected pairs for MRLE method), $d^{-}$ and
$d^{+}$ (dimensions of positive and negative latent factors), $p_{0}$ in
activation function of MRLE, learning rate $\alpha$ of SPR, regularization
parameters $\lambda_{\Theta^{-}}$ and $\lambda_{\Theta^{+}}$ of SPR, and $\xi$
using as the identification threshold for SPR. By multiple experiments and
empirical evidences, we initialize $p_{0}=0.01$, $\alpha=0.005$, and
$\lambda_{\Theta^{-}}=\lambda_{\Theta^{+}}=0.01$ for two experimental
datasets.

To gain financial benefit, spammers mainly perform activities that are linked
to malicious targets. It means that the disguised-activities which are used by
spammers to cheat detection models through building connections to normal
targets are scattered with a relatively small number (Liu, 2020). Therefore,
it is not feasible to assign a big value to the identification threshold $\xi$
(mentioned in Section 4.1). Through multiple experiments, we find that setting
the value of $\xi$ smaller than 3 is advisable. If the value of $\xi$ is too
large, the improvement of LFM models will be impeded. To save space, the
influence of parameter $\xi$ for detection performance is not demonstrated,
but we find that our experiments on two datasets achieve the best performance
when $\xi=2$ (shown in Table 2). On the other hand, the sizes of non-linked
pairs ($n$) and dimensions ($d$) of latent factors have important indicative
influence for detecting spamming activities. We therefore use Figure 2 to
investigate the effects of $n$ and $d$. In Table 2, the results of LFM models
are obtained in the case of setting $n=20$ and $d^{-}=d^{+}=30$. As we can
separately calculate normal and spam latent factors, it is feasible to assign
$d^{-}$ and $d^{+}$ with different values. However, due to the insignificant
effects of setting different dimensions for positive and negative latent
factors in our experiments, we assign the same value to $d^{-}$ and $d^{+}$
(i.e., $d$ ). As Figure 2 shows, the detection performances are sensitive to
the size of $n$ and $d$ when the size values are relatively small, but
corresponding influences on detection performances are gradually reduced with
the increase of the values of $n$ and $d$.

(a) Size of non-linked relations

(b) Dimensions

Figure 2. The impact of parameters $n$ and $d$.

(a) Fake followers

(b) Opinion spam

Figure 3. Precision@k of compared methods.

To further evaluate and verify the effectiveness of our models, another common
metric precision@k for spam detection tasks is used and the related results
are shown in Figure 3. The major findings learned from Figure 3 are: (i) LFM
models outperform other competitive methods; (ii) more than 90% of activities
in the top 500 ranking results are deceptive, so we can say that solely
relying on the signed latent factors to select high-ranking instances as
spamming results achieves very low error. This type of precision is quite
important for spam detection tasks, because it means that normal instances are
unlikely to be identified as spam ones.

###  5.4. Dealing with Long-standing Challenges

Table 3. F-measure performances in dealing with class imbalance ( % labeled data, $n=20$, $d=30$). % | Social | Forum
---|---|---
MRLE |  SPR | Bigraph |  HITS | Feature | Factor | n-aware |  MRLE |  SPR | Bigraph |  HITS | Feature | Factor | n-aware
20 | 0.620 | 0.629 | 0.492 | 0.440 | 0.334 | 0.529 | 0.565 | 0.533 | 0.535 | 0.434 | 0.447 | 0.301 | 0.468 | 0.507
25 | 0.635 | 0.640 | 0.535 | 0.485 | 0.398 | 0.551 | 0.570 | 0.550 | 0.543 | 0.501 | 0.472 | 0.332 | 0.503 | 0.516
30 | 0.649 | 0.646 | 0.576 | 0.527 | 0.417 | 0.580 | 0.594 | 0.582 | 0.579 | 0.548 | 0.493 | 0.367 | 0.534 | 0.556
40 | 0.664 | 0.658 | 0.593 | 0.549 | 0.437 | 0.602 | 0.615 | 0.597 | 0.591 | 0.570 | 0.531 | 0.378 | 0.574 | 0.589
50 | 0.670 | 0.665 | 0.601 | 0.564 | 0.460 | 0.615 | 0.637 | 0.608 | 0.602 | 0.584 | 0.556 | 0.403 | 0.590 | 0.601
Table 4. F-measure in face of varying degrees of network incompleteness (40% labeled data, $n=20$, $d=30$). $Degree$ | Social | Forum
---|---|---
MRLE |  SPR | Bigraph |  HITS | Feature | Factor | n-aware |  MRLE |  SPR | Bigraph |  HITS | Feature | Factor | n-aware
0% | 0.664 | 0.658 | 0.593 | 0.549 | 0.437 | 0.602 | 0.615 | 0.597 | 0.591 | 0.570 | 0.531 | 0.378 | 0.574 | 0.589
3% | 0.659 | 0.635 | 0.574 | 0.541 | 0.431 | 0.585 | 0.609 | 0.590 | 0.586 | 0.561 | 0.516 | 0.370 | 0.567 | 0.570
5% | 0.647 | 0.630 | 0.556 | 0.527 | 0.422 | 0.569 | 0.587 | 0.584 | 0.582 | 0.537 | 0.508 | 0.361 | 0.541 | 0.562
7% | 0.641 | 0.622 | 0.536 | 0.503 | 0.414 | 0.550 | 0.564 | 0.570 | 0.563 | 0.509 | 0.482 | 0.355 | 0.518 | 0.537
10% | 0.627 | 0.608 | 0.502 | 0.462 | 0.393 | 0.527 | 0.531 | 0.548 | 0.540 | 0.476 | 0.460 | 0.340 | 0.483 | 0.505

Three long-standing challenges — the lack of training labels, class imbalance,
and network incompleteness are mentioned in the Introduction section. By
varying labeled data used in the training process, the experimental results in
Table 2 have indicated that LFM models perform well in coping with limited
spam instances. When only 20% labeled instances in Social are selected as
training dataset, the MRLECon still achieves 0.809 AUC. To evaluate our
models’ ability in confronting the other two challenges, specialized
experiments are designed.

It is important to notice that in the spam detection area, the widely used
evaluation method of area under the ROC curve (AUC) is less sensitive to
imbalanced data compared to the other metrics for spam ranking tasks. To
comprehensively evaluate our models in the face of class imbalance, we further
apply the F-measure that synthesizes precision and recall measures, providing
more precise and rounded illustrations of a detection method.

In this work, both the training and test class imbalances are considered by
employing specific experimental setups: (i) To simulate different degrees of
training class imbalance, we fix the number of normal activities in the
training dataset which is the same as the number of 50% spam instances. In
this way, reducing the percentage of selected spam labeled data (less than
50%) will cause training class imbalance, i.e., the fewer amount (%) of spam
instances are selected the more severe of class imbalance; (ii) As for test
class imbalance, the ratio of spam to normal activities is fixed to 1:10. For
example, in an experiment, 80% of spamming activities (2K) are not selected as
training data but are used for testing. We then randomly choose 20K instances
that are not labeled spamming activities and are not used in the training
process as normal ones for evaluation. To save space, only the results of the
negative-aware method Latentn-aware are listed in the following tables, which
achieves the best performance among three latent-factors baselines.

Table 3 shows detection results of all compared methods in dealing with
different levels of class imbalance. The indications from Table 3 are
summarised as: (i) Even if the F-measure is used for evaluation, LFM models
still outperform other baselines in the face of different degrees of class
imbalance; (ii) Compared with structure-based approaches, feature-based
methods are not strong enough to deal with severe class imbalance; (iii) By
analyzing the changing trend of each method’s performance using different
amounts of seeds, the F-measure performances of our models are more stable
than the other ones. We use Table 4 to report compared methods’ performances
of tolerating varying degrees of network incompleteness. To evaluate this, we
randomly remove different proportions (%) of links (i.e., activities) from the
original networks. We can find that compared to the results on the whole
complete dataset (i.e., 0%), all the experiments on more “incomplete” networks
obtain inferior performances, which means that the incompleteness of graph
poses an obstacle to detection models. However, the effectiveness drops of LFM
methods are lower than other models when increasing the degree of
incompleteness, i.e., our models achieve better tolerance in face of the
incomplete graph.

##  6\. RELATED WORK

Three lines of researches are closely related to this paper: spam detection,
signed social networks, and pairwise learning.
Spam Detection. The most commonly used spam detection method is to utilize
machine learning to train a classifier to distinguish spam instances from
normal ones. The differences among the classifiers mainly lie in the features
used to represent the data (Liu et al., 2017). It’s generally recognized that
there are clues for spotting spam in user-generated contents. Kim et al. (Kim
et al., 2015) deploy a deep framework to identify linguistic features for
detecting opinion spam. The other content-based features (e.g., semantic
representation and syntactic structure) are also exploited to combat deceptive
reviews (Kim et al., 2015; Sandulescu and Ester, 2015) or promotion campaigns
(Liu et al., 2017, 2016b). Besides, employing user rating patterns (Lim et
al., 2010; Kumar et al., 2018) and clicking behaviors (Shah, 2017; Su et al.,
2018) for spam detection is also quite common. To combat well-disguised
crowdturfing activities, many existing works attempt to mine collusive
signals, which can be implemented from two aspects: (i) capturing synchronized
behaviors (Li et al., 2016; Jiang et al., 2014); (ii) grouping spammers or
spamming activities (Xu et al., 2013; Fayazi et al., 2015). By utilizing
inherent relationships between spammers and malicious targets, structure-based
approaches achieve relatively good performances in detecting spam, which are
mainly divided into two types: probabilistic graphical models (Su et al.,
2018; Lu et al., 2013) and label propagation methods (Zhang et al., 2015; Liu
et al., 2016a). In this work, the signed latent factors are mined based on
relational ties, which means that our method can be seen as a type of
structure-based approach. However, the signed latent factor models are capable
of uncovering plentiful potential factors from various online networks, which
are fed into a classification algorithm to achieve spam detection. On the
other hand, MRLE and SPR utilize the null relationships and label unknown
links, respectively. And the process of comparing pairs in SPR is relatively
random (different from other structure-based methods whose inference process
is guided by edges). We can construct enough comparing ranking pairs based on
existing entries ($+$, $-$ and $?$) to ensure acceptable detection
performance.
Signed Social Networks. Many real-world relations can be represented by signed
networks with positive and negative links, e.g., the positive/negative
election of Wikipedia admins (Cabunducan et al., 2011) and trust/distrust
(Leskovec et al., 2010) in Epinions. Comparing unsigned networks, two unique
properties are presented in signed networks distinct properties and collective
properties (Tang et al., 2016), which involve more information and facilitate
mining tasks (e.g., node classification, link prediction recommendation,
etc.). Although there are many similar tasks for mining signed and unsigned
networks, the task of sign prediction (inferring the signs of existing links)
is unique to signed networks. This work is among the first to treat spam
contaminated relational datasets as signed social networks with spam and
normal links between users and targets. Based on that treatment, we design the
spam detection-oriented signed latent factor models.
Pairwise Learning. Rendle et al. (Rendle et al., 2009b) propose the pairwise
learning algorithm — BPR which is optimized based on the relative preference
of a specific user over pairs of items. BPR is a highly competitive approach
in the item recommendation area and has been utilized in many papers as the
state-of-the-art baseline up until recently (He et al., 2017). In this paper,
our SPR model is inspired by the core thought of BPR method in the
recommendation area. However, there are significant differences between SPR
and BPR: (i) SPR makes full use of the existing relations between users and
targets (including unlabeled and test instances in our dataset) and designs
three types of entries according to the signed network; (ii) Comparing pairs
are constructed from two levels (spam and normal) based on user-oriented
perspective by analyzing potential indications; (iii) Each user and item
acquire two types of latent factors, i.e., spam (negative) and normal
(positive) potential representations.

##  7\. CONCLUSIONS

Detecting online spamming activities has been extensively studied. In this
work, we present a new thought to tackle this popular and intractable problem
by introducing the signed latent factors mining into the spam detection area.
To provide more potentials and possibilities, two signed latent factor mining
models from two quite different perspectives (pointwise and pairwise) are
proposed, which are both effective and perform well in coping with the long-
standing challenges in combating spam. We comprehensively explore how to
represent activities (links) using signed user and target latent factors, by
proposing and evaluating five types of application operators. In order to
improve the generality, we design the unified network form to represent
different kinds of contaminated relational data on various Web platforms. By
performing experiments on two disparate datasets, the respectable usability
and generality for spam detection of LFM models are validated. Besides, we
believe that the LFM thought can also be extended to other spam detection
tasks, e.g., fraud view problem in online video platforms (user-video network)
(Shah, 2017) and deceptive review detection in online shopping sites (user-
product matrix) (Jindal and Liu, 2008). In the future, we will evaluate our
models on the other tricky spam detection tasks.

## References

  * (1)
  * Abdi (2007) Hervé Abdi. 2007\.  Singular value decomposition (SVD) and generalized singular value decomposition.  _Encyclopedia of measurement and statistics_ (2007), 907–912.
  * Aggarwal and Kumaraguru (2015) Anupama Aggarwal and Ponnurangam Kumaraguru. 2015.  What they do in shadows: Twitter underground follower market. In _2015 13th Annual Conference on Privacy, Security and Trust (PST)_. IEEE, 93–100.
  * Almaatouq et al. (2014) Abdullah Almaatouq, Ahmad Alabdulkareem, Mariam Nouh, Erez Shmueli, Mansour Alsaleh, Vivek K Singh, Abdulrahman Alarifi, Anas Alfaris, and Alex Pentland. 2014.  Twitter: who gets caught? observed trends in social micro-blogging spam. In _Proceedings of the 2014 ACM conference on Web science_. 33–41.
  * Cabunducan et al. (2011) Gerard Cabunducan, Ralph Castillo, and John Boaz Lee. 2011\.  Voting behavior analysis in the election of Wikipedia admins. In _2011 International Conference on Advances in Social Networks Analysis and Mining_. IEEE, 545–547.
  * Chen and Chen (2015) Yu-Ren Chen and Hsin-Hsi Chen. 2015.  Opinion spam detection in web forum: a real case study. In _WWW_. 173–183.
  * Fairbanks et al. (2018) James Fairbanks, Natalie Fitch, Nathan Knauf, and Erica Briscoe. 2018.  Credibility Assessment in the News: Do we need to read. In _Proc. of the MIS2 Workshop held in conjuction with 11th Int’l Conf. on Web Search and Data Mining_. 799–800.
  * Fayazi et al. (2015) Amir Fayazi, Kyumin Lee, James Caverlee, and Anna Squicciarini. 2015\.  Uncovering crowdsourced manipulation of online reviews. In _SIGIR_. ACM, 233–242.
  * Gong et al. (2014) Neil Zhenqiang Gong, Mario Frank, and Prateek Mittal. 2014\.  Sybilbelief: A semi-supervised learning approach for structure-based sybil detection.  _IEEE Transactions on Information Forensics and Security_ 9, 6 (2014), 976–987.
  * Gyongyi et al. (2004) Zoltan Gyongyi, Hector Garcia-Molina, and Jan Pedersen. 2004\.  Combating web spam with trustrank. In _Proceedings of the 30th international conference on very large data bases (VLDB)_.
  * He et al. (2017) Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017\.  Neural collaborative filtering. In _WWW_. International World Wide Web Conferences Steering Committee, 173–182.
  * Hu et al. (2008) Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008.  Collaborative filtering for implicit feedback datasets. In _2008 Eighth IEEE International Conference on Data Mining_. Ieee, 263–272.
  * Jiang et al. (2014) Meng Jiang, Peng Cui, Alex Beutel, Christos Faloutsos, and Shiqiang Yang. 2014.  Catchsync: catching synchronized behavior in large directed graphs. In _Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining_. 941–950.
  * Jindal and Liu (2008) Nitin Jindal and Bing Liu. 2008.  Opinion spam and analysis. In _Proceedings of the 2008 international conference on web search and data mining_. ACM, 219–230.
  * Jindal et al. (2010) Nitin Jindal, Bing Liu, and Ee-Peng Lim. 2010.  Finding unusual review patterns using unexpected rules. In _Proceedings of the 19th ACM international conference on Information and knowledge management_. ACM, 1549–1552.
  * Kennedy et al. (2019) Stefan Kennedy, Niall Walsh, Kirils Sloka, Andrew McCarren, and Jennifer Foster. 2019.  Fact or Factitious? Contextualized Opinion Spam Detection. In _ACL_. 344–350.
  * Kim et al. (2015) Seongsoon Kim, Hyeokyoon Chang, Seongwoon Lee, Minhwan Yu, and Jaewoo Kang. 2015.  Deep semantic frame-based deceptive opinion spam analysis. In _CIKM_. ACM, 1131–1140.
  * Krishna et al. (2018) Vaibhav Krishna, Tian Guo, and Nino Antulov-Fantulin. 2018\.  Is Simple Better? Revisiting Non-linear Matrix Factorization for Learning Incomplete Ratings. In _2018 IEEE International Conference on Data Mining Workshops (ICDMW)_. IEEE.
  * Kumar et al. (2018) Srijan Kumar, Bryan Hooi, Disha Makhija, Mohit Kumar, Christos Faloutsos, and VS Subrahmanian. 2018.  Rev2: Fraudulent user prediction in rating platforms. In _Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining_. ACM, 333–341.
  * Leskovec et al. (2010) Jure Leskovec, Daniel Huttenlocher, and Jon Kleinberg. 2010\.  Predicting positive and negative links in online social networks. In _WWW_. 641–650.
  * Li et al. (2016) Huayi Li, Geli Fei, Shuai Wang, Bing Liu, Weixiang Shao, Arjun Mukherjee, and Jidong Shao. 2016.  Modeling review spam using temporal patterns and co-bursting behaviors.  _arXiv preprint arXiv:1611.06625_ (2016).
  * Li et al. (2017) Huayi Li, Geli Fei, Shuai Wang, Bing Liu, Weixiang Shao, Arjun Mukherjee, and Jidong Shao. 2017.  Bimodal distribution and co-bursting in review spam detection. In _WWW_. International World Wide Web Conferences Steering Committee, 1063–1072.
  * Li et al. (2019) Ke Li, Bin Guo, Qiuyun Zhang, Jianping Yuan, and Zhiwen Yu. 2019.  CrowdGuard: Characterization and Early Detection of Collective Content Polluters in Online Social Networks. In _Companion Proceedings of The 2019 World Wide Web Conference_. ACM, 1063–1070.
  * Lim et al. (2010) Ee-Peng Lim, Viet-An Nguyen, Nitin Jindal, Bing Liu, and Hady Wirawan Lauw. 2010.  Detecting product review spammers using rating behaviors. In _Proceedings of the 19th ACM international conference on Information and knowledge management_. ACM, 939–948.
  * Lin (2007) Chih-Jen Lin. 2007\.  Projected gradient methods for nonnegative matrix factorization.  _Neural computation_ 19, 10 (2007), 2756–2779.
  * Lin et al. (2019) Sheng-Chieh Lin, Yu-Neng Chuang, Sheng-Fang Yang, Ming-Feng Tsai, and Chuan-Ju Wang. 2019.  Negative-Aware Collaborative Filtering.. In _RecSys (Late-Breaking Results)_. 41–45.
  * Liu (2020) Yuli Liu. 2020.  Recommending Inferior Results: A General and Feature-Free Model for Spam Detection. In _Proceedings of the 29th ACM International on Conference on Information and Knowledge Management_.
  * Liu et al. (2008) Yiqun Liu, Rongwei Cen, Min Zhang, Shaoping Ma, and Liyun Ru. 2008.  Identifying web spam with user behavior analysis. In _Proceedings of the 4th international workshop on Adversarial information retrieval on the web_. 9–16.
  * Liu et al. (2016a) Yuli Liu, Yiqun Liu, Min Zhang, and Shaoping Ma. 2016a.  Pay Me and I’ll Follow You: Detection of Crowdturfing Following Activities in Microblog Environment.. In _IJCAI_. 3789–3796.
  * Liu et al. (2017) Yuli Liu, Yiqun Liu, Ke Zhou, Min Zhang, and Shaoping Ma. 2017.  Detecting collusive spamming activities in community question answering. In _WWW_. International World Wide Web Conferences Steering Committee, 1073–1082.
  * Liu et al. (2016b) Yuli Liu, Yiqun Liu, Ke Zhou, Min Zhang, Shaoping Ma, Yue Yin, and Hengliang Luo. 2016b.  Detecting promotion campaigns in query auto completion. In _CIKM_. 125–134.
  * Lu et al. (2013) Yuqing Lu, Lei Zhang, Yudong Xiao, and Yangguang Li. 2013\.  Simultaneously detecting fake reviews and review spammers using factor graph model. In _Proceedings of the 5th annual ACM web science conference_. ACM, 225–233.
  * Luca (2016) Michael Luca. 2016\.  Reviews, reputation, and revenue: The case of Yelp. com.  _Com (March 15, 2016). Harvard Business School NOM Unit Working Paper_ 12-016 (2016).
  * Mukherjee et al. (2013) Arjun Mukherjee, Vivek Venkataraman, Bing Liu, and Natalie Glance. 2013.  What yelp fake review filter might be doing?. In _Seventh international AAAI conference on weblogs and social media_.
  * Noekhah et al. (2020) Shirin Noekhah, Naomie binti Salim, and Nor Hawaniah Zakaria. 2020\.  Opinion spam detection: Using multi-iterative graph-based model.  _Information Processing & Management_ 57, 1 (2020), 102140\.
  * Rayana and Akoglu (2015) Shebuti Rayana and Leman Akoglu. 2015.  Collective opinion spam detection: Bridging review networks and metadata. In _Proceedings of the 21th acm sigkdd international conference on knowledge discovery and data mining_. ACM, 985–994.
  * Ren and Zhang (2016) Yafeng Ren and Yue Zhang. 2016.  Deceptive opinion spam detection using neural network. In _Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers_. 140–150.
  * Rendle et al. (2009a) Steffen Rendle, Leandro Balby Marinho, Alexandros Nanopoulos, and Lars Schmidt-Thieme. 2009a.  Learning optimal ranking with tensor factorization for tag recommendation. In _Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining_. 727–736.
  * Rendle et al. (2009b) Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009b.  BPR: Bayesian personalized ranking from implicit feedback. In _Proceedings of the twenty-fifth conference on uncertainty in artificial intelligence_. AUAI Press.
  * Rout et al. (2017) Jitendra Kumar Rout, Smriti Singh, Sanjay Kumar Jena, and Sambit Bakshi. 2017.  Deceptive review detection using labeled and unlabeled data.  _Multimedia Tools and Applications_ 76, 3 (2017), 3187–3211.
  * Sandulescu and Ester (2015) Vlad Sandulescu and Martin Ester. 2015.  Detecting singleton review spammers using semantic similarity. In _WWW_. ACM, 971–976.
  * Shah and Pomerantz (2010) Chirag Shah and Jefferey Pomerantz. 2010.  Evaluating and predicting answer quality in community QA. In _Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval_. 411–418.
  * Shah (2017) Neil Shah. 2017\.  FLOCK: Combating astroturfing on livestreaming platforms. In _WWW_. International World Wide Web Conferences Steering Committee, 1083–1091.
  * Song and Meyer (2015) Dongjin Song and David Meyer. 2015.  Recommending positive links in signed social networks by optimizing a generalized AUC. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 29.
  * Su et al. (2018) Ning Su, Yiqun Liu, Zhao Li, Yuli Liu, Min Zhang, and Shaoping Ma. 2018\.  Detecting Crowdturfing” Add to Favorites” Activities in Online Shopping. In _WWW_. 1673–1682.
  * Tang et al. (2016) Jiliang Tang, Yi Chang, Charu Aggarwal, and Huan Liu. 2016\.  A survey of signed network mining in social media.  _ACM Computing Surveys (CSUR)_ 49, 3 (2016).
  * Wang et al. (2011) Guan Wang, Sihong Xie, Bing Liu, and S Yu Philip. 2011\.  Review graph based online store review spammer detection. In _2011 IEEE 11th International Conference on Data Mining_. IEEE, 1242–1247.
  * Xu et al. (2013) Chang Xu, Jie Zhang, Kuiyu Chang, and Chong Long. 2013\.  Uncovering collusive spammers in Chinese review websites. In _CIKM_. ACM.
  * Xu et al. (2019) Pinghua Xu, Wenbin Hu, Jia Wu, and Bo Du. 2019\.  Link Prediction with Signed Latent Factors in Signed Social Networks. In _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. ACM.
  * Zhang et al. (2018) Daokun Zhang, Jie Yin, Xingquan Zhu, and Chengqi Zhang. 2018\.  SINE: Scalable Incomplete Network Embedding. In _2018 IEEE International Conference on Data Mining (ICDM)_. IEEE, 737–746.
  * Zhang et al. (2015) Yongfeng Zhang, Yunzhi Tan, Min Zhang, Yiqun Liu, Tat-Seng Chua, and Shaoping Ma. 2015\.  Catch the black sheep: unified framework for shilling attack detection based on fraudulent action propagation. In _Twenty-Fourth International Joint Conference on Artificial Intelligence_.
  * Zhu et al. (1997) Ciyou Zhu, Richard H Byrd, Peihuang Lu, and Jorge Nocedal. 1997\.  Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization.  _ACM Transactions on Mathematical Software (TOMS)_ 23, 4 (1997), 550–560.