# Unbiased Pairwise Learning from Implicit Feedback for Recommender Systems
without Biased Variance Control

Yi Ren  Tencent BeijingChina<EMAIL_ADDRESS>,  Hongyan Tang
TencentBeijingChina<EMAIL_ADDRESS>,  Jiangpeng Rong  Tencent
BeijingChina<EMAIL_ADDRESS>and  Siwen Zhu  TencentBeijingChina
<EMAIL_ADDRESS>
(2023)

###### Abstract.

Generally speaking, the model training for recommender systems can be based on
two types of data, namely explicit feedback and implicit feedback. Moreover,
because of its general availability, we see wide adoption of implicit feedback
data, such as click signal. There are mainly two challenges for the
application of implicit feedback. First, implicit data just includes positive
feedback. Therefore, we are not sure whether the non-interacted items are
really negative or positive but not displayed to the corresponding user.
Moreover, the relevance of rare items is usually underestimated since much
fewer positive feedback of rare items is collected compared with popular ones.
To tackle such difficulties, both pointwise and pairwise solutions are
proposed before for unbiased relevance learning. As pairwise learning suits
well for the ranking tasks, the previously proposed unbiased pairwise learning
algorithm already achieves state-of-the-art performance. Nonetheless, the
existing unbiased pairwise learning method suffers from high variance. To get
satisfactory performance, non-negative estimator is utilized for practical
variance control but introduces additional bias. In this work, we propose an
unbiased pairwise learning method, named UPL, with much lower variance to
learn a truly unbiased recommender model. Extensive offline experiments on
real world datasets and online A/B testing demonstrate the superior
performance of our proposed method.

Recommender Systems; Unbiased Learning; Implicit Feedback; Missing-Not-At-
Random; Inverse Propensity Score

††journalyear: 2023††copyright: acmlicensed††conference: Proceedings of the
46th International ACM SIGIR Conference on Research and Development in
Information Retrieval; July 23–27, 2023; Taipei, Taiwan††booktitle:
Proceedings of the 46th International ACM SIGIR Conference on Research and
Development in Information Retrieval (SIGIR ’23), July 23–27, 2023, Taipei,
Taiwan††price: 15.00††doi: 10.1145/3539618.3592077††isbn:
978-1-4503-9408-6/23/07††ccs: Information systems Recommender systems

##  1\. Introduction

Recommender systems usually rely on implicit user feedback for model training
owning to the cheap cost of collecting such data (Liu et al., 2010). For this
scenario, the typical model learning techniques (Koren et al., 2009; Rendle et
al., 2012; Johnson, 2014), recognize interacted items as positive and all the
other items as potential negative examples. There are mainly two difficulties
to learn unbiased user preference based on implicit feedback data. First, we
are not sure whether a non-interacted item is irrelevant to the user or
positive but not yet recommended (Bekker et al., 2019; Elkan and Noto, 2008).
The second challenge is that the positive feedback is missing-not-at-random
(MNAR) (Yang et al., 2018; Steck, 2010) since users are more likely to
interact with the popular items because of the exposure bias and selection
bias (Chen et al., 2020).

To handle the aforementioned challenges, multiple methods (Hu et al., 2008;
Liang et al., 2016; Saito et al., 2020; Saito, 2020; Lee et al., 2021) are
proposed to learn unbiased models based on the Missing-Not-At-Random implicit
feedback data. WMF (Hu et al., 2008) addresses the first problem of positive-
unlabeled feedback by upweighting the training loss of the interacted ones.
ExpoMF (Liang et al., 2016) models the items’ exposure probability for more
accurate training weight of different items. Rel-MF (Saito et al., 2020)
adopts inverse propensity weighting (Schnabel et al., 2016; Swaminathan and
Joachims, 2015; Rosenbaum and Rubin, 1983) and derives the unbiased point-wise
loss to achieve better recommendation quality. And MF-DU (Lee et al., 2021)
separately estimate the exposure probability for interacted and non-interacted
data for better bias correction. Unlike the point-wise algorithms of (Hu et
al., 2008; Liang et al., 2016; Saito et al., 2020; Lee et al., 2021), Unbiased
Bayesian Personalized Ranking (Saito, 2020) extends the pair-wise learning
algorithm of (Rendle et al., 2012) and formulates an unbiased objective
function. As pairwise approach is more suitable than the pointwise methods for
the ranking task of recommender systems (Song et al., 2018; Wang et al.,
2018b), UBPR (Saito, 2020) achieves state-of-the-art recommendation
performance. Nevertheless, UBPR (Saito, 2020) suffers from high variance. As a
result, non-negative estimator is applied to practically control the variance
at the cost of introducing additional bias.

One related but different line of research (Hu et al., 2019; Joachims et al.,
2017; Ai et al., 2018; Wang et al., 2018a; Agarwal et al., 2019; Ren et al.,
2022) focuses on learning unbiased learning to rank models by modelling
position bias (Chen et al., 2020) as a counterfactual effect. In contrast,
exposure bias and selection bias (Chen et al., 2020) are much more important
in our setting.

In this paper, we design an unbiased pairwise learning algorithm with lower
variance so as to circumvent the step of variance control and perform truly
unbiased learning. We summarize our main contributions below.

  * •

We design an effective unbiased pairwise learning algorithm for implicit
feedback with theoretically lower variance than the existing approach.

  * •

Thanks to the proposed low variance algorithm, its unbiased pairwise loss
function can be directly used without further bias-variance trade-off.

  * •

We conduct offline experiments and online A/B testing to evaluate and
understand the effectiveness of our method.

##  2\. Preliminaries And Notations

Let $u\in U$ represent a user and $i,j\in I$ denote an item. In addition, let
$D_{point}=U\times I$ be the set of all possible interaction data. $c_{u,i}$
denotes the binary random variable of implicit interaction between $u$ and
$i$. And $r_{u,i}$ is the binary random variable representing the relevance
between $u$ and $i$. Furthermore, $o_{u,i}$ denotes the status of exposure $i$
to $u$. Only the interaction variables are observed for implicit feedback. The
implicit feedback $c_{u,i}$ can be modeled below.

$\displaystyle P(c_{u,i}=1)$ $\displaystyle=P(o_{u,i}=1)\times P(r_{u,i}=1)$
(1) $\displaystyle=\theta_{u,i}\times\gamma_{u,i},\forall(u,i)\in D_{point}$

where $\theta_{u,i}=P(o_{u,i}=1)$ and $\gamma_{u,i}=P(r_{u,i}=1)$ represent
the exposure and relevance probability respectively. Moreover, we use
$\theta_{u,i}^{-}$ to represent the posterior exposure probability with
negative implicit feedback and can be computed below.

(2)
$\displaystyle\\!\\!\\!\\!\theta_{u,i}^{-}=P(o_{u,i}=1|c_{u,i}=0)=\frac{\theta_{u,i}(1-\gamma_{u,i})}{\\!1-\theta_{u,i}\gamma_{u,i}}$

And $P(r_{u,i}\\!\\!>\\!\\!r_{u,j})$, which equals $P(r_{u,i}=1,r_{u,j}=0)$,
denotes the probability distribution that user $u$ prefers item $i$ over item
$j$. Furthermore, given independent observation probability of different
items, it is intuitive to derive $P(c_{u,i}=1,c_{u,j}=0,o_{u,i}=1,o_{u,j}=1)$
as follows.

$\displaystyle\\!\\!\\!\\!P(c_{u,i}=1,c_{u,j}=0,o_{u,i}=1,o_{u,j}=1)$
$\displaystyle\\!\\!\\!\\!\\!\\!=P(o_{u,i}=1)P(o_{u,j}=1)P(r_{u,i}=1,r_{u,j}=0))$
(3)
$\displaystyle\\!\\!\\!\\!\\!\\!=\theta_{u,i}\theta_{u,j}P(r_{u,i}=1,r_{u,j}=0))$

##  3\. Methodology

In this section, we propose an unbiased pairwise learning algorithm, named
UPL, with lower variance than the state-of-the-art methods. It performs truly
unbiased pairwise learning based on implicit feedback by circumventing the
step of external variance control.

###  3.1. Existing Unbiased Pairwise Learning Method for Biased Implicit
Feedback

In the pairwise setting, the loss function is defined on the pair of positive
item $i$ and negative item $j$ for user $u$. For this scenario, we let
$f(u,i)$ denotes the ranker function to be learned to reflect the relevance
between $u$ and $i$. And $L(f(u,i),f(u,j))$ denotes the pairwise loss
function. UBPR (Saito, 2020) utilizes an unbiased estimator for the ideal
pairwise loss and achieves the state-of-the-art results. The unbiased loss
function is defined as follows, where $Reg(f)$ and $\lambda$ are the
regularization loss and loss weight respectively. Unlike BPR (Rendle et al.,
2012) with strict pair defined on the implicit feedback, $c_{u,j}$ below can
be positive or negative.

(4)
$R_{ubpr}(f)=\sum_{u,i,j}{\frac{c_{u,i}}{\theta_{u,i}}(1-\frac{c_{u,j}}{\theta_{u,j}})L(f(u,i),f(u,j))}+\lambda
Reg(f)$

Nonetheless, its variance depends on the inverse of the product of two
propensity scores, namely $\theta_{u,i}$ and $\theta_{u,j}$. Thus, it tends to
produce suboptimal results, especially for the tail items with low exposure
probability. Furthermore, the unbiased estimator can take negative values by
definition, thereby causing ever severe variance issues and hindering model
convergence. Therefore, inspired by the work of positive-unlabeled learning
(Kiryo et al., 2017), non-negative estimator, which clips large negative
values, is applied for practical variance control but introduces additional
bias.

DataSets | Methods | DCG@K | Recall@K | MAP@K
---|---|---|---|---
K=3 | K=5 | K=8 | K=3 | K=5 | K=8 | K=3 | K=5 | K=8
Yahoo! R3 | WMF | 0.08312 | 0.09643 | 0.10809 | 0.08962 | 0.11807 | 0.15041 | 0.06809 | 0.07649 | 0.08343
Rel-MF | 0.08586 | 0.09894 | 0.11019 | 0.09243 | 0.12040 | 0.15165 | 0.07052 | 0.07896 | 0.08574
MF-DU | 0.08493 | 0.09828 | 0.10967 | 0.09150 | 0.12002 | 0.15160 | 0.06995 | 0.07854 | 0.08538
BPR | 0.09077 | 0.10415 | 0.11405 | 0.09745 | 0.12577 | 0.15327 | 0.07455 | 0.08313 | 0.08917
UBPR | 0.09418 | 0.10745 | 0.11693 | 0.10127 | 0.12941 | 0.15555 | 0.07792 | 0.08667 | 0.09263
$UBPR_{NClip}$ | $0.09061^{-}$ | $0.10389^{-}$ | $0.11425^{-}$ | $0.09745^{-}$ | $0.12572^{-}$ | $0.15442^{-}$ | $0.07485^{-}$ | $0.08350^{-}$ | $0.08988^{-}$
$UPL$ | 0.09698 | 0.11005 | 0.11906 | 0.10403 | 0.13175 | 0.15673 | 0.08072 | 0.08945 | 0.09511
Coat | $WMF$ | 0.10320 | 0.12415 | 0.14580 | 0.11322 | 0.15815 | 0.21844 | 0.08616 | 0.10017 | 0.11392
Rel-MF | 0.10516 | 0.12474 | 0.14815 | 0.11513 | 0.15648 | 0.22141 | 0.08728 | 0.10088 | 0.11539
MF-DU | 0.10529 | 0.12596 | 0.14955 | 0.11511 | 0.15923 | 0.22419 | 0.08795 | 0.10212 | 0.11628
$BPR$ | 0.10048 | 0.12127 | 0.14451 | 0.11023 | 0.15477 | 0.21932 | 0.08357 | 0.09717 | 0.11158
$UBPR$ | 0.10727 | 0.12735 | 0.14990 | 0.11769 | 0.16082 | 0.22359 | 0.08897 | 0.10229 | 0.11641
$UBPR_{NClip}$ | 0.10597 | 0.12649 | 0.14900 | $0.11598^{-}$ | 0.16013 | 0.22279 | 0.08808 | 0.10142 | 0.11520
$UPL$ | 0.10730 | 0.12886 | 0.15073 | 0.11787 | 0.16416 | 0.22510 | 0.08923 | 0.10375 | 0.11751
Table 1. Ranking performance on Yahoo! R3 and Coat datasets. (The results of
UPL with underline indicate p < 0.05 for one-tailed t-test with the best
competitor. And $UBPR_{NClip}$’ results with ’-’ indicate significant worse
than UBPR (p<0.05).)

###  3.2. Proposed Unbiased Pairwise Estimator

First, the ideal pairwise risk function and the ranker learned through
empirical risk minimization are defined in Equation 5 and Equation 6
respectively. With the ideal pairwise risk of Equation 5, we try to minimize
the overall loss for valid tuples of $(u,i,j)$.

(5) $R_{ideal}(f)=\int{}L(f(u,i),f(u,j))dP(r_{u,i}=1,r_{u,j}=0)$

(6)
$\hat{f}_{ideal}=\arg\min_{f}(\sum_{r_{u,i}=1,r_{u,j}=0}{L(f(u,i),f(u,j))}+\lambda
Reg(f))$

Then, according to equation 2, we can prove the unbiased risk function defined
on the implicit feedback pairs:

$\displaystyle\\!\\!\\!\\!R_{UPL}(f)$
$\displaystyle\\!\\!\\!\\!\\!\\!=\\!\\!\int{}\frac{L(f(u,i),f(u,j))}{\theta_{u,i}\theta_{u,j}}\\!P(o_{u,j}\\!=\\!1|c_{u,i}\\!=\\!1,c_{u,j}\\!=\\!0)dP(c_{u,i}\\!\\!=\\!\\!1,\\!c_{u,j}\\!\\!=\\!\\!0)\\!$
$\displaystyle\\!\\!\\!\\!\\!\\!=\\!\\!\int{}\frac{L(f(u,i),f(u,j))}{\theta_{u,i}\theta_{u,j}}dP(c_{u,i}\\!\\!=\\!\\!1,\\!c_{u,j}\\!\\!=\\!\\!0,\\!o_{u,i}\\!\\!=\\!\\!1,\\!o_{u,j}\\!\\!=\\!\\!1)\\!$
$\displaystyle\\!\\!\\!\\!\\!\\!=\\!\\!\int{}\frac{L(f(u,i),f(u,j))}{\theta_{u,i}\theta_{u,j}}dP(r_{u,i}\\!\\!=\\!\\!1,\\!r_{u,j}\\!\\!=\\!\\!0)P(o_{u,i}\\!\\!=\\!\\!1)P(o_{u,j}\\!\\!=\\!\\!1)\\!$
(7)
$\displaystyle\\!\\!\\!\\!\\!\\!=\\!\\!\int{}{L(f(u,i),f(u,j))}dP(r_{u,i}=1,r_{u,j}=0)\\!=R_{ideal}(f)$

As $R_{UPL}$ is an unbiased estimator of $R_{ideal}$, we can learn an unbiased
ranker through minimizing the corresponding empirical loss function. Moreover,
given independent exposure probabilities between different items, we can see
$P(o_{u,j}\\!=\\!1|c_{u,i}\\!=\\!1,c_{u,j}\\!=\\!0)=P(o_{u,j}\\!=\\!1|c_{u,j}\\!=\\!0)=\theta_{u,j}^{-}$.
All in all, the unbiased ranker can be learned based on Equation 3.2, where
$Reg(f)$ and $\lambda$ are the regularization loss and loss weight
respectively. In contrast to UBPR, the empirical risk is only dependent on the
valid pairs of implicit feedback, for which $c_{u,i}$ is greater than
$c_{u,j}$.

$\displaystyle\\!\\!\\!\\!\\!\hat{f}_{UPL}=\arg\min_{f}(\sum_{c_{u,i}=1,c_{u,j}=0}{\\!\frac{L(f(u,i)f(u,j))\theta_{u,j}^{-}}{\theta_{u,i}\theta_{u,j}}}+\lambda
Reg(f))\\!\\!$ (8)
$\displaystyle\\!\\!\\!\\!\\!=\arg\min_{f}(\sum_{c_{u,i}=1,c_{u,j}=0}{\\!\frac{L(f(u,i),f(u,j))(1-\gamma_{u,j})}{\theta_{u,i}(1-\theta_{u,j}\gamma_{u,j})}}+\lambda
Reg(f))\\!\\!$

First, we can see the risk function cannot take negative values. Furthermore,
we can derive its variance as below. Because of contrained space, we omit the
detailed proof here.

$\displaystyle\\!\\!\\!\\!\\!\\!VAR_{UPL}=\sum_{\begin{subarray}{c}c_{u,i}=1\\\
c_{u,j}=0\end{subarray}}\frac{(\frac{1}{\theta_{u,i}}-\gamma_{u,i})\gamma_{u,i}(1-\gamma_{u,j})^{2}L_{u,i,j}^{2}}{(1-\theta_{u,j}\gamma_{u,j})^{2}}$
(9) $\displaystyle+\sum_{\begin{subarray}{c}c_{u,i}=1,c_{u,j}=0\\\
c_{u,k}=0,k\neq
j\end{subarray}}\frac{(\frac{1}{\theta_{u,i}}-\gamma_{u,i})\gamma_{u,i}(1-\gamma_{u,j})(1-\gamma_{u,k})L_{u,i,j}L_{u,i,k}}{(1-\theta_{u,j}\gamma_{u,j})(1-\theta_{u,k}\gamma_{u,k})}$

where $L_{u,i,j}$ stands for $L(f(u,i)f(u,j))$. Since
$(1-\theta_{u,j}\gamma_{u,j})$ is usually far away from 0 even for the most
popular items and other factors, including $\gamma_{u,i}$ and
$(1-\gamma_{u,j})$, are bounded between 0 and 1, the variance of Equation 3.2
largely depends on the inverse of $\theta_{u,i}$. In comparison, the variance
of UBPR (Saito, 2020) is impacted by the inverse of the product of
$\theta_{u,i}$ and $\theta_{u,j}$, thereby producing a much greater variance.
As a result, we do not adopt any further variance reduction techniques on top
of the unbiased pairwise learning estimator.

Interestingly, if $\theta_{u,j}^{-}$ is approximated with $\theta_{u,j}$ in
equation 3.2, we can derive exactly the same estimator practically applied in
UBPR after clipping all of the negative values. From this perspective, it is
pretty obvious that the UBPR algorithm is not really unbiased in practice. To
verify the necessity of variance control in UBPR, we will also remove the
variance-bias trade-off step to test its performance in our experiments.

###  3.3. Learning Algorithm

1

Data: implicit feedback data $\\{c_{u,i}\\};$ learned relevance from Rel-MF
$\\{\gamma_{u,i}\\}$

Input: mini-batch size $M;$ regularization parameter $\lambda;$ learning_rate
$\alpha$

Output: learned ranker $f$’s parameters $W$

2

3Initialize parameters $W$ of Ranker $f$

4 Estimate propensity scores of $\\{\theta_{u,i}\\}$

5 while  _Not Convergence_ do

6 Sample one mini-batch data of size $M$

7
$Loss=\sum_{c_{u,i}=1,c_{u,j}=0}{\\!\frac{L(f(u,i),f(u,j))(1-\gamma_{u,j})}{\theta_{u,i}(1-\theta_{u,j}\gamma_{u,j})}}+\lambda
Reg(f)$

8 Compute the gradient of $W$ based on $Loss$

9 Update $W$ according to $\alpha$

Return $W$

Algorithm 1 Unbiased Pairwise Learning (UPL)

In this section, we formally describe the learning algorithm. First, the
dependence on $\gamma_{u,j}$ is one problem of the above learning algorithm.
Thanks to the work of Rel-MF (Saito et al., 2020), theoretically, the Rel-MF
model can predict unbiased relevance after convergence, which is also used as
our input. Though the prediction results of Rel-MF may deviate, intuitively,
we are likely to get better performance than UBPR since it uses a rather loose
approximation for $\theta_{u,j}^{-}$. Then, the ranker $f$ is trained by
optimizing the empirical risk function of Equation 3.2. The whole learning
procedure is illustrated in Algorithm 1. As the model training is of a general
process, it can be combined with any pairwise algorithms, such as (Rendle et
al., 2012; Song et al., 2018; Kim et al., 2019),

##  4\. Experiments

Subsets | Methods | DCG@K | Recall@K | MAP@K
---|---|---|---|---
K=3 | K=5 | K=8 | K=3 | K=5 | K=8 | K=3 | K=5 | K=8
Cold-start users | WMF | 0.08406 | 0.09618 | 0.10773 | 0.08984 | 0.11584 | 0.14790 | 0.07002 | 0.07791 | 0.08465
Rel-MF | 0.08729 | 0.09944 | 0.11058 | 0.09337 | 0.11943 | 0.15038 | 0.07290 | 0.08077 | 0.08737
MF-DU | 0.08701 | 0.09958 | 0.11058 | 0.09306 | 0.11992 | 0.15046 | 0.07305 | 0.08108 | 0.08766
BPR | 0.09209 | 0.10442 | 0.11450 | 0.09892 | 0.12501 | 0.15307 | 0.07647 | 0.08442 | 0.09055
UBPR | 0.09560 | 0.10880 | 0.11792 | 0.10230 | 0.13035 | 0.15537 | 0.08061 | 0.08942 | 0.09507
UPL | 0.09689 | 0.11005 | 0.11905 | 0.10345 | 0.13139 | 0.15626 | 0.08278 | 0.09159 | 0.09711
Rare items | WMF | 0.04227 | 0.05324 | 0.06011 | 0.04711 | 0.07057 | 0.08935 | 0.03214 | 0.03890 | 0.04291
Rel-MF | 0.04454 | 0.05484 | 0.06141 | 0.04940 | 0.07147 | 0.08943 | 0.03429 | 0.04064 | 0.04456
MF-DU | 0.04545 | 0.05565 | 0.06199 | 0.05032 | 0.07210 | 0.08940 | 0.03520 | 0.04156 | 0.04532
BPR | 0.04720 | 0.05690 | 0.06287 | 0.05208 | 0.07283 | 0.08917 | 0.03713 | 0.04311 | 0.04672
UBPR | 0.05268 | 0.06160 | 0.06639 | 0.05797 | 0.07694 | 0.08995 | 0.04122 | 0.04691 | 0.04998
UPL | 0.05483 | 0.06336 | 0.06761 | 0.06016 | 0.07826 | 0.08981 | 0.04366 | 0.04923 | 0.05198
Table 2. Ranking performance for cold-start users and rare items on Yahoo! R3
(The results of UPL with underline indicate p < 0.05 for one-tailed t-test
with the best competitor.)

In this section, we demonstrate the effectiveness of the proposed UPL
algorithm. We open all of the related source code on
Github111https://github.com/renyi533/unbiased-pairwise-rec/tree/main.

###  4.1. Experimental Setup

####  4.1.1. Datasets

We used Yahoo! R3222http://webscope.sandbox.yahoo.com/ and
Coat333https://www.cs.cornell.edu/schnabts/mnar/. They are selected because we
can leverage the explicit feedback data with five-star ratings for relevance
evaluation. Morevoer, they include both Missing-Not-At-Random training data
and Missing-Completely-At-Random test data. First, we transform five-star
ratings into relevance probability with
$\gamma_{u,i}=\epsilon+(1-\epsilon)\frac{2^{r}-1}{2^{r_{max}}-1}$. We apply
$\epsilon=0.1$ for training while $\epsilon=0$ for testing. Second, we sample
binary relevance variable with $r_{u,i}=Bern(\gamma_{u,i})$. Third, we define
the exposure variable $o_{u,i}$ based on whether user $u$ rated item $i$.
Finally, we compute $c_{u,i}=o_{u,i}r_{u,i}$, which is the observed implicit
feedback.

####  4.1.2. Evaluation Metrics

We adopt three widely used implicit recommendation evaluation metrics – DCG@k
(Discounted Cumulative Gain), Recall@k and MAP@k (Mean Average Precision). The
detailed definition can be found in (Saito et al., 2020; Saito, 2020). We
report results with k = 3, 5, 8.

####  4.1.3. Models

We tested pointwise algorithms of WMF (Hu et al., 2008), Rel-MF (Saito et al.,
2020), MF-DU (Lee et al., 2021) and pairwise algorithms of BPR (Rendle et al.,
2012) and UBPR (Saito, 2020) as baselines. For UPL, we combined with the BPR
(Rendle et al., 2012) algorithm. For UBPR, we also included the variant of
removing the variance-bias trade-off step, named $UBPR_{NClip}$, to study the
impact of its variance. We followed the definition of (Saito, 2020) for rare
items (less than 100 clicks in the training set) and cold-start users (less
than six clicks in the training set) to study the performance of different
algorithms in these two harder tasks.

Following (Lee et al., 2021), for MF-DU, we separately computed the propensity
scores for clicked and non-clicked items respectively as:

(10) $\theta_{*,i}^{Click}=(\frac{n_{i}}{max_{i\in
I}n_{i}})^{0.5}\;\;\;\;\theta_{*,i}^{NClick}=(1-\frac{n_{i}}{max_{i\in
I}n_{i}})^{0.5}$

where $n_{i}$ means the number of interactions to the item $i$ by all users.
While for Rel-MF, UBPR and UPL, following (Saito et al., 2020; Saito, 2020),
we used $\theta_{*,i}^{Click}$ as the propensity scores for all of the items.

To be Consistent with (Saito, 2020), we randomly sampled 10% of the training
data as validation data to tune the user-item latent factors within [100,
300], L2 regularization hyperparameter within [$10^{-7},10^{-3}$], and the
non-negative estimator’s clipping threshold for UBPR in [-10, 0]. All models
are implemented with tensorflow (Abadi et al., 2016) and optimized using the
Adam optimizer (Kingma and Ba, 2014) with an initial learning rate of 0.001
and a mini-batch size of 256.

For each model, we performed 50 runs on both datasets to report the experiment
results.

###  4.2. Experiment Results

####  4.2.1. Overall Offline Results

For the overall ranking performance on both the Yahoo! R3 and Coat datasets,
the results of all methods are presented in Table 1. For both datasets, our
UPL algorithm outperforms the other baseline methods under all settings. For
the results of UPL, the underline symbol indicates a significant gain (p<0.05
for one tailed t-test) over the best competitor algorithm. For all the results
of Yahoo! R3, UPL achieves significant performance gain. Though the results of
Coat are of more variance, UPL still showes significant performance gain for
the metrics of DCG@5, Recall@5 and MAP@5. These results support that our
strategy of performing true unbiased pairwise learning is effective in
improving the ranking performances.

In addition, the pairwise algorithms show better ranking performance than the
pointwise ones. For instance, BPR achieves even better results than the
debiased pointwise algorithms in Yahoo! R3 datasets. And UBPR is the second
best method for almost all settings.

Finally, $UBPR_{NClip}$ is of worse performance than UBPR, especially for
Yahoo R3 dataset, which verifies the necessity to perform external variance
control for UBPR. Moreover, though the loss clipping threshold for UBPR is
tuned between [-10, 0], the best performance is always derived with a value
very near to 0, which is exactly one approximation of UPL by approximating
$\theta_{u,j}^{-}$ with $\theta_{u,j}$ as described in section 3.2.

In summary, the proposed approach in this paper is helpful for addressing the
positive-unlabeled and Missing-Not-At-Random issues, which constitute the main
challenges for recommender model learning from implicit feedback data.

####  4.2.2. Offline Results for Rare Items and Cold-start Users

Considering the importance of recommendation for rare items and cold-start
users in real-world scenarios, we added the experiment results on Yahoo R3!
datasets for rare items and cold-start users in table 2. Except the metric of
Recall@8 for rare items, UPL achieves significant gain over the most
competitive baselines. And for Recall@8 of rare items, UBPR does not show
significant gain over UPL. Overall, we can draw the conclusion that the
proposed method is a promising choice for constructing recommender systems
based on implicit feedback.

####  4.2.3. Online A/B Testing

We verified the effectiveness of our proposed algorithm by replacing the
existing BPR (Rendle et al., 2012) model with the UPL model in the matching
stage of a large-scale content recommender system at Tencent. There were tens
of recall models deployed in the matching stage. The union of the outputs of
these models was used as the input to the ranking stage. And the BPR model
used to be the most important recall model by accounting for about 30%
displayed items. The seven days A/B testing showed a significant gain of 0.7%
for our main online metric of app stay time per person. Moreover, we
classified the items to long-tail (<3,000 impressions), hot (500,000+) and
others based on their exposure count before the A/B tesing. For long-tail and
other contents, we observed more exposure by 1.31% and 1.74% respectively.
While the impressions for hot items decreased 2.43%.

##  5\. Conclusion

In this paper, we propose an unbiased pairwise learning method, named UPL, for
recommender systems based on implicit feedback. We theoretically prove the
unbiasedness of UPL and its advantage of lower variance over the existing
method of UBPR (Saito, 2020). Extensive offline experiments and online A/B
testing help to verify UPL’s effectiveness at tackling the positive-unlabeled
and Missing-Not-At-Random challenge for recommender model learning from the
biased implicit feedback. Both the theoretical and empirical results suggest
the proposed method is a competent candidate.

## References

  * (1)
  * Abadi et al. (2016) Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016\.  $\\{$TensorFlow$\\}$: A System for $\\{$Large-Scale$\\}$ Machine Learning. In _12th USENIX symposium on operating systems design and implementation (OSDI 16)_. 265–283.
  * Agarwal et al. (2019) Aman Agarwal, Kenta Takatsu, Ivan Zaitsev, and Thorsten Joachims. 2019.  A general framework for counterfactual learning-to-rank. In _Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval_. 5–14.
  * Ai et al. (2018) Qingyao Ai, Keping Bi, Cheng Luo, Jiafeng Guo, and W Bruce Croft. 2018.  Unbiased learning to rank with unbiased propensity estimation. In _The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval_. 385–394.
  * Bekker et al. (2019) Jessa Bekker, Pieter Robberechts, and Jesse Davis. 2019\.  Beyond the selected completely at random assumption for learning from positive and unlabeled data. In _Joint European Conference on Machine Learning and Knowledge Discovery in Databases_. Springer, 71–85.
  * Chen et al. (2020) Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. 2020\.  Bias and debias in recommender system: A survey and future directions.  _arXiv preprint arXiv:2010.03240_ (2020).
  * Elkan and Noto (2008) Charles Elkan and Keith Noto. 2008.  Learning classifiers from only positive and unlabeled data. In _Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining_. 213–220.
  * Hu et al. (2008) Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008.  Collaborative filtering for implicit feedback datasets. In _2008 Eighth IEEE international conference on data mining_. Ieee, 263–272.
  * Hu et al. (2019) Ziniu Hu, Yang Wang, Qu Peng, and Hang Li. 2019\.  Unbiased lambdamart: an unbiased pairwise learning-to-rank algorithm. In _The World Wide Web Conference_. 2830–2836.
  * Joachims et al. (2017) Thorsten Joachims, Adith Swaminathan, and Tobias Schnabel. 2017\.  Unbiased learning-to-rank with biased feedback. In _Proceedings of the Tenth ACM International Conference on Web Search and Data Mining_. 781–789.
  * Johnson (2014) Christopher C Johnson. 2014\.  Logistic matrix factorization for implicit feedback data.  _Advances in Neural Information Processing Systems_ 27, 78 (2014), 1–9.
  * Kim et al. (2019) Seunghyeon Kim, Jongwuk Lee, and Hyunjung Shim. 2019\.  Dual neural personalized ranking. In _The World Wide Web Conference_. 863–873.
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ (2014).
  * Kiryo et al. (2017) Ryuichi Kiryo, Gang Niu, Marthinus C Du Plessis, and Masashi Sugiyama. 2017.  Positive-unlabeled learning with non-negative risk estimator.  _Advances in neural information processing systems_ 30 (2017).
  * Koren et al. (2009) Yehuda Koren, Robert Bell, and Chris Volinsky. 2009\.  Matrix factorization techniques for recommender systems.  _Computer_ 42, 8 (2009), 30–37.
  * Lee et al. (2021) Jae-woong Lee, Seongmin Park, and Jongwuk Lee. 2021.  Dual Unbiased Recommender Learning for Implicit Feedback. In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_. 1647–1651.
  * Liang et al. (2016) Dawen Liang, Laurent Charlin, James McInerney, and David M Blei. 2016.  Modeling user exposure in recommendation. In _Proceedings of the 25th international conference on World Wide Web_. 951–961.
  * Liu et al. (2010) Jiahui Liu, Peter Dolan, and Elin Rønby Pedersen. 2010\.  Personalized news recommendation based on click behavior. In _Proceedings of the 15th international conference on Intelligent user interfaces_. 31–40.
  * Ren et al. (2022) Yi Ren, Hongyan Tang, and Siwen Zhu. 2022.  Unbiased Learning to Rank with Biased Continuous Feedback. In _Proceedings of the 31st ACM International Conference on Information & Knowledge Management_. 1716–1725.
  * Rendle et al. (2012) Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2012.  BPR: Bayesian personalized ranking from implicit feedback.  _arXiv preprint arXiv:1205.2618_ (2012).
  * Rosenbaum and Rubin (1983) Paul R Rosenbaum and Donald B Rubin. 1983.  The central role of the propensity score in observational studies for causal effects.  _Biometrika_ 70, 1 (1983), 41–55.
  * Saito (2020) Yuta Saito. 2020\.  Unbiased Pairwise Learning from Biased Implicit Feedback. In _Proceedings of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval_. 5–12.
  * Saito et al. (2020) Yuta Saito, Suguru Yaginuma, Yuta Nishino, Hayato Sakata, and Kazuhide Nakata. 2020.  Unbiased recommender learning from missing-not-at-random implicit feedback. In _Proceedings of the 13th International Conference on Web Search and Data Mining_. 501–509.
  * Schnabel et al. (2016) Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims. 2016\.  Recommendations as treatments: Debiasing learning and evaluation. In _international conference on machine learning_. PMLR, 1670–1679.
  * Song et al. (2018) Bo Song, Xin Yang, Yi Cao, and Congfu Xu. 2018\.  Neural collaborative ranking. In _Proceedings of the 27th ACM International Conference on Information and Knowledge Management_. 1353–1362.
  * Steck (2010) Harald Steck. 2010\.  Training and testing of recommender systems on data missing not at random. In _Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining_. 713–722.
  * Swaminathan and Joachims (2015) Adith Swaminathan and Thorsten Joachims. 2015.  The self-normalized estimator for counterfactual learning.  _advances in neural information processing systems_ 28 (2015).
  * Wang et al. (2018b) Menghan Wang, Xiaolin Zheng, Yang Yang, and Kun Zhang. 2018b.  Collaborative filtering with social exposure: A modular approach to social recommendation. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 32.
  * Wang et al. (2018a) Xuanhui Wang, Nadav Golbandi, Michael Bendersky, Donald Metzler, and Marc Najork. 2018a.  Position bias estimation for unbiased learning to rank in personal search. In _Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining_. 610–618.
  * Yang et al. (2018) Longqi Yang, Yin Cui, Yuan Xuan, Chenyang Wang, Serge Belongie, and Deborah Estrin. 2018\.  Unbiased offline recommender evaluation for missing-not-at-random implicit feedback. In _Proceedings of the 12th ACM conference on recommender systems_. 279–287.


$(u,\mathop{\textrm{{tr}}}u)\not\in L^{\infty}(\Omega)\times
L^{\infty}(\Gamma)$, so $\mathcal{u}$ cannot be Hölder continuous.

###  4.3. Asymptotic behavior and eventual positivity

In this section, we derive asymptotic properties of our solution. We are going
to skip the proofs whenever neither variable coefficients nor extra Robin-term
are relevant and the ideas can be carried over directly from [DKP21, Chapter
6], as is the case for the next two results.

######  Lemma 4.9.

The operator ${\mathcal{A}}_{\delta}$ has compact resolvent.

######  Corollary 4.10.

There exists an orthonormal basis $(\mathcal{e}_{n})_{n}$ of ${\mathcal{H}}$
consisting of eigenfunctions of ${\mathcal{A}}_{\delta}$, say
${\mathcal{A}}_{\delta}\mathcal{e}_{n}=\lambda_{n}\mathcal{e}_{n}$, where the
sequence $\lambda_{n}$ is increasing to $\infty$, allowing the representation

${\mathcal{A}}_{\delta}\mathcal{f}=\sum_{k=1}^{\infty}\lambda_{k}\left\langle\mathcal{f},\mathcal{e}_{k}\right\rangle_{{\mathcal{H}}}\mathcal{e}_{k}$

for all $\mathcal{f}\in D({\mathcal{A}}_{\delta})$. Moreover, as
$\mathcal{e}_{n}\in D({\mathcal{A}}_{\delta}^{\infty})$, it has a Hölder
continuous representative in the sense that there exists a function $e_{n}\in
C^{0,\vartheta}(\Omega)$ such that
$\mathcal{e}_{n}=(e_{n}|_{\Omega},e_{n}|_{\Gamma})$.

Finally, for all $\mathcal{f}\in{\mathcal{H}}$, the semigroup ${\mathcal{T}}$
can be represented as

$\displaystyle\mathcal{u}(t)=(u_{1}(t),u_{2}(t))={\mathcal{T}}(t)\mathcal{f}=\sum_{k=1}^{\infty}e^{-\lambda_{k}t}\left\langle\mathcal{f},\mathcal{e}_{k}\right\rangle_{{\mathcal{H}}}\mathcal{e}_{k}.$
(4.15)

######  Lemma 4.11.

  1. n(i)

If $\gamma=0$ almost everywhere, then we have
$\ker({\mathcal{A}}_{\delta})\subseteq{\rm
span}({\mathbbm{1}_{\Omega}},{\mathbbm{1}_{\Gamma}})$ and
$\int_{\Gamma}\delta|\mathop{\textrm{{tr}}}u_{1}|^{2}\textrm{d}S=0$.

  2. n(ii)

If $\gamma=0$, $\delta=0$ almost everywhere, then $\lambda_{1}=0$ and
$\ker({\mathcal{A}})={\rm
span}({\mathbbm{1}}_{\Omega},{\mathbbm{1}}_{\Gamma}).$

  3. n(iii)

If $\gamma,\delta\geq 0$ and either $\gamma>0$ or $\delta>0$ on a set of
positive surface measure, then $\lambda_{1}>0$ and we have
$\ker({\mathcal{A}}_{\delta})=\\{0\\}$.

  4. n(iv)

If $\int_{\Gamma}\gamma\,{\mathrm{d}}S<0$, then $\lambda_{1}<0$.

###### Proof.

In cases (i)–(iii), we have $\gamma,\delta\geq 0$. Hence,
${\mathfrak{a}}_{\delta}$ is accretive, so we have $\lambda_{1}\geq 0$. Thus,
whether $\lambda_{1}=0$ or $\lambda_{1}>0$ depends only on
$\ker({\mathcal{A}}_{\delta})$.

(i) Suppose $\gamma=0$ almost everywhere, then
$\mathcal{u}\in\ker({\mathcal{A}}_{\delta})$ implies
$u_{1}\in\ker(B_{N,\delta})$. More precisely, let
$\mathcal{u}\in\mathrm{ker}({\mathcal{A}}_{\delta})\subseteq
D({\mathcal{A}}_{\delta})\subseteq D({\mathfrak{a}}_{\delta})$. Then

$0=\left\langle{\mathcal{A}}_{\delta}\mathcal{u},\mathcal{u}\right\rangle_{\mathscr{H}}={\mathfrak{a}}_{\delta}(\mathcal{u},\mathcal{u})=\int_{\Omega}\alpha|B_{N,\delta}u_{1}|^{2}\mathrm{d}x.$

It follows that $\alpha|B_{N,\delta}u_{1}|^{2}=0$ and hence, since
$\alpha(x)\geq\eta$, $u_{1}\in\ker(B_{N,\delta})$. This means

$0=\left\langle Q\nabla u_{1},\nabla
u_{1}\right\rangle_{\Omega}+\left\langle\delta
u_{1},u_{1}\right\rangle_{\Gamma}=\|\sqrt{Q}\nabla
u_{1}\|_{\Omega}^{2}+\|\sqrt{\delta}\mathop{\textrm{{tr}}}u_{1}\|^{2}$

and shows $\nabla u_{1}=0$, whence $u_{1}$ is constant (and, therefore, also
$u_{2}=\mathop{\textrm{{tr}}}u_{1}$). Moreover,
$\int_{\Gamma}\delta|\mathop{\textrm{{tr}}}u_{1}|^{2}\mathrm{d}S=0$.

(ii) If $\delta=0$, naturally also the converse holds as
$B_{N}{\mathbbm{1}}_{\Omega}=-\mathop{\textrm{{div}}}Q\nabla{\mathbbm{1}}_{\Omega}=0$
and the constant functions satisfy the Neumann boundary condition, which shows
$\lambda_{1}=0.$

(iii) If $\gamma=0$, $\delta>0$ on a set of positive surface measure
$\Gamma_{0}$, due to (i), $u_{1}$ is still constant. But now, we also find a
set of positive measure $\Gamma_{\varepsilon}\subseteq\Gamma$ where
$\delta>\varepsilon$. However, if we had $u=c\neq 0$ by (i), this would yield

$0=\int_{\Gamma}\delta|\mathop{\textrm{{tr}}}u_{1}|^{2}\mathrm{d}S\geq\int_{\Gamma_{\varepsilon}}\delta|\mathop{\textrm{{tr}}}u_{1}|^{2}\mathrm{d}S\geq\varepsilon
c^{2}S(\Gamma_{\varepsilon})>0.$

Analogously, if $\gamma>0$ on a set of positive measure and
$\mathcal{u}\in\ker({\mathcal{A}}_{\delta})$, we calculate

$\displaystyle
0=\left\langle{\mathcal{A}}_{\delta}\mathcal{u},\mathcal{u}\right\rangle_{\mathscr{H}}={\mathfrak{a}}(\mathcal{u},\mathcal{u})$
$\displaystyle=\int_{\Omega}\alpha|\Delta
u_{1}|^{2}\mathrm{d}x+\int_{\Gamma}\beta^{-1}\gamma|u_{2}|^{2}{\mathrm{d}}S$
$\displaystyle\geq\int_{\Gamma_{\varepsilon}}\beta^{-1}\gamma
c^{2}\,\mathrm{d}S\geq\|\beta\|^{-1}_{\infty}\varepsilon
c^{2}S(\Gamma_{\varepsilon})>0,$

another contradiction.

(iv) Plugging $({\mathbbm{1}}_{\Omega},{\mathbbm{1}}_{\Gamma})\in
D({\mathfrak{a}}_{\delta})$ into the usual Rayleigh quotient, we obtain a
negative value as $\int_{\Gamma}\beta^{-1}\gamma\,{\mathrm{d}}S<0$, and thus
$\lambda_{1}<0$. ∎

This yields the following asymptotic behavior of the semigroup
${\mathcal{T}}$.

######  Theorem 4.12.

  1. n(i)

If $\gamma=0$, $\delta=0$ almost everywhere, then
$\|{\mathcal{T}}(t)\mathcal{f}-\bar{\mathcal{f}}\|_{\mathcal{H}}\leq
e^{-\lambda_{2}t}\|\mathcal{f}\|_{{\mathcal{H}}}$ for all
$\mathcal{f}\in{\mathcal{H}}$, where

$\bar{\mathcal{f}}\coloneqq\frac{1}{\lambda_{d}(\Omega)+\int_{\Gamma}\beta^{-1}\mathrm{d}S}\left(\int_{\Omega}f_{1}\mathrm{d}x+\int_{\Gamma}\beta^{-1}f_{2}\mathrm{d}S\right)({\mathbbm{1}}_{\Omega},{\mathbbm{1}}_{\Gamma}),$

and $\lambda_{2}>0$ is the second eigenvalue of ${\mathcal{A}}$.

  2. n(ii)

If $\gamma,\delta\geq 0$ and $\gamma>0$ or $\delta>0$ on a set of positive
measure, then $\|{\mathcal{T}}(t)\mathcal{f}\|_{\mathcal{H}}\leq
e^{-\lambda_{1}t}\|\mathcal{f}\|_{{\mathcal{H}}}$ holds for all
$\mathcal{f}\in{\mathcal{H}}$. Thus, in this case, the semigroup
${\mathcal{T}}$ is exponentially stable.

  3. n(iii)

If $\int_{\Gamma}\gamma\,{\mathrm{d}}S<0$, then
$\|{\mathcal{T}}(t)\|=e^{-\lambda_{1}t}\to\infty$ as $t\to\infty$.

###### Proof.

For (i) observe that in this case $\lambda_{1}=0$ and
$\bar{\mathcal{f}}=e^{-\lambda_{1}t}\langle\mathcal{f},\mathcal{e}_{1}\rangle_{{\mathcal{H}}}\mathcal{e}_{1}$
in view of Lemma 4.11. Thus (4.15) and Parseval’s identity yield

$\|{\mathcal{T}}(t)\mathcal{f}-\bar{\mathcal{f}}\|_{\mathcal{H}}^{2}=\sum_{k=2}^{\infty}e^{-\lambda_{k}t}|\langle\mathcal{f},\mathcal{e}_{k}\rangle_{{\mathcal{H}}}|^{2}\leq\sum_{k=2}^{\infty}e^{-\lambda_{2}t}|\langle\mathcal{f},\mathcal{e}_{k}\rangle_{{\mathcal{H}}}|^{2}\leq
e^{-\lambda_{2}t}\|f\|^{2}_{{\mathcal{H}}}.$

This proves (i). In case (ii), we have $\lambda_{1}>0$ (see again Lemma 4.11),
and (ii) follows by a similar computation. (iii) follows by considering an
eigenvalue corresponding to the eigenvalue $\lambda_{1}$. ∎

######  Remark 4.13.

For our following positivity investigations, we consider the Hilbert lattice
$H=L^{2}(\Omega\,\dot{\cup}\,\Gamma,\mu)$, where $\mu$ is given by
$\mu(A)=\int_{A\cap\Omega}{\mathbbm{1}}\,\mathrm{d}\lambda+\int_{A\cap\Gamma}\beta^{-1}\mathrm{d}S$,
with positivity cone

$H^{+}\coloneqq\\{u\in H\leavevmode\nobreak\ |\leavevmode\nobreak\ u(x)\geq
0\leavevmode\nobreak\ \mu\text{-a.e}\\}.$ (4.16)

Once can easily show that $H$ can be identified with our Hilbert space
${\mathcal{H}}=L^{2}(\Omega)\times L^{2}(\Gamma,\beta^{-1}\mathrm{d}S)$ from
Definition 2.10 (i), and $u$ with $\mathcal{u}=(u_{1},u_{2}).$

######  Proposition 4.14.

Let $\gamma\geq 0$, then the semigroup ${\mathcal{T}}$ generated by
$-{\mathcal{A}}$ is real, but neither positive nor $L^{\infty}$-contractive.

###### Proof.

The same arguments as in [DKP21, Lemma 3.5] work as Robin and Neumann boundary
conditions are equal for test functions. ∎

Again however, we may show that the semigroup ${\mathcal{T}}$ is _eventually_
positive in the sense of [DGK16a], [DGK16b] and [DG18], as the critical
ingredient is the Hölder continuity established in the previous section also
for the variable coefficient case. We recall a simplified version of the
definition from [DG18, Section 1] and the used criterion.

######  Definition 4.15.

Let $(\Omega,\mu)$ be a $\sigma$-finite measure space and $T$ a real
$C_{0}$-semigroup on the space $H=L^{2}(\Omega,\mu)$ with positivity cone
$H^{+}$ given in the sense of (4.16). Then, $T$ is called _eventually
positive_ if there is some time $t_{0}>0$ such that for all $f\in
H_{+}\setminus\\{0\\}$ and $t\geq t_{0}$ there is some $\varepsilon>0$ for
which $T(t)f\geq\varepsilon$ holds $\mu$-almost everywhere.

######  Theorem 4.16.

Let $(\Omega,\mu)$ be a $\sigma$-finite measure space and $T$ a real
$C_{0}$-semigroup generated by a self-adjoint operator $A$ on
$H=L^{2}(\Omega,\mu)$. If $D(A^{\infty})\subseteq L^{\infty}(\Omega,\mu)$,
then the following assertions are equivalent:

  1. n(i)

$T$ is eventually positive.

  2. n(ii)

$\ker(s(A)-A)$ is one-dimensional and contains a vector $v$ such that
$v\geq\varepsilon$ holds $\mu$-almost everywhere for some $\varepsilon>0$.

######  Theorem 4.17.

Let $\gamma=\delta=0$. Then the semigroup ${\mathcal{T}}$ is eventually
positive in the sense of Definition 4.15.

###### Proof.

We apply Theorem 4.16 for $H$ defined as in Remark 4.13 and
$A=-{\mathcal{A}}$. As $\beta,\beta^{-1}$ are bounded,
$L^{\infty}(\Omega\,\dot{\cup}\,\Gamma,\mu)$ can be identified with
$L^{\infty}(\Omega)\times L^{\infty}(\Gamma).$ ${\mathcal{T}}$ is real as a
consequence of Proposition 4.14, and the operator ${\mathcal{A}}$ is self-
adjoint due to the symmetry of the form (see Theorem 2.12). Finally, we have
that $D({\mathcal{A}}^{\infty})$ embeds into $L^{\infty}(\Omega)\times
L^{\infty}(\Gamma)$ by Theorem 4.7. Now, to deduce eventual positivity, we
only have to verify assertion (ii) of Theorem 4.16. But this a direct
consequence of Lemma 4.11 (ii) that yields $s(-{\mathcal{A}})=\lambda_{1}=0$
as $\gamma=0$, $\delta=0$, and
$({\mathbbm{1}}_{\Omega},{\mathbbm{1}}_{\Gamma})\in\ker({\mathcal{A}}).$ ∎

For a counterexample for eventual positivity in the case $\gamma>0$,
$\delta=0$, $Q=I_{\mathrm{d}}$ we refer the reader to [DKP21, Section 7].

##  5\. Application to systems of higher order

In this short excursion, we point out that the abstract theory established in
Chapter 2 is not necessarily restricted to problems of order 4, though
identification of the abstract operators and regularity characterizations can
be more difficult.

######  Proposition 5.1.

Let $\Omega\subset{\mathbb{R}}^{d}$ be a bounded Lipschitz domain,
$H=L^{2}(\Omega)$, and let $\Delta_{N}$ denote the Neumann Laplacian. For
$k\in{\mathbb{N}}$ consider the form
${\mathfrak{b}}_{k}=\left\langle\Delta^{k}u,\Delta^{k}v\right\rangle_{\Omega}$
on $D({\mathfrak{b}}_{k})=D(\Delta_{N}^{k})$. Then ${\mathfrak{b}}_{k}$ is
admissible and $B_{N}=\Delta_{N}^{2k},$ whence also
$C_{c}^{\infty}(\Omega)\subset D(B_{N})\cap D(B_{D})$ is satisfied, and we are
in the situation of Theorem 2.12 and Remark 2.13.

###### Proof.

The form ${\mathfrak{b}}_{k}$ is symmetric and accretive by default. We have
$\|u\|^{2}_{{\mathfrak{b}}_{k}}=\|u\|^{2}_{\Omega}+\|\Delta^{k}u\|_{\Omega}^{2}.$
As $\Delta_{N}^{k}$ is a closed operator. $D(\Delta_{N}^{k})$ is a Hilbert
space with respect to the graph norm and hence the form is closed. The
continuity follows from Cauchy–Schwarz’s inequality. Thus ${\mathfrak{b}}_{k}$
is generating and its associated operator $B_{N}$ is self-adjoint. We have
$C_{c}^{\infty}(\Omega)\subseteq D(\Delta_{N}^{k})\subseteq
D(\Delta_{N})\subseteq H^{1}(\Omega)$ and may choose $\rho=1/2$ in (2.1). For
the last embedding we calculate $\|u\|^{2}_{H^{1}(\Omega)}=\left\langle\Delta
u,u\right\rangle_{\Omega}\leq C(\|u\|_{\Omega}^{2}+\|\Delta
u\|_{\Omega}^{2}).$ That $D(\Delta^{k}_{N})$ is continuously embedded in
$D(\Delta^{N})$ follows by the closed graph theorem applied to the identity
id: $D(\Delta^{N}_{k})\rightarrow D(\Delta_{N})$. $D(\Delta_{N}^{k})$ is also
dense in $H^{1}(\Omega)$ due to [Ouh05, Proof of Lemma 1.25] and the
analyticity of the semigroup. We begin by showing that $B_{N}$ coincides with
$\Delta^{2k}$ distributionally. Let $u\in D(B_{N})$ and $v\in
C_{c}^{\infty}(\Omega)\subset D(\Delta_{N}^{2k})$, then $\left\langle
f,\varphi\right\rangle=\left\langle\Delta_{N}^{k}u,\Delta_{N}^{k}\varphi\right\rangle=\left\langle(\Delta_{N}^{k})^{*}u,\Delta_{N}^{k}\varphi\right\rangle=\left\langle
u,\Delta^{2k}\varphi\right\rangle$ as $\Delta^{k}\varphi\in
D(\Delta_{N}^{k}).$ Hence $f=\Delta^{2k}u$ as $\Delta_{N}$ and thus
$\Delta_{N}^{k}$ is self-adjoint. So $D(B_{N})\subset\\{u\in
D(\Delta_{N}^{k})\leavevmode\nobreak\ |\leavevmode\nobreak\
\Delta_{N}^{2k}u\in L^{2}(\Omega)\\}.$ On the other hand
$D(\Delta_{N}^{2k})\subset D(B_{N})$ as for the same reason for $u\in
D(\Delta_{N}^{2k})$ the relation
$\left\langle\Delta_{N}^{2k}u,v\right\rangle=\left\langle\Delta_{N}^{k}u,\Delta_{N}^{k}v\right\rangle$
holds for all $v\in D(\Delta_{N}^{k})$. So $\Delta^{2k}_{N}\subset
B_{N}=B_{N}^{*}\subset(\Delta_{N}^{2k})^{*}=\Delta_{N}^{2k}.$ ∎

Now we obtain $D(B_{0})=D(B_{D})\cap D(B_{N})=D(B_{N})\cap H_{0}^{1}(\Omega)$
by Proposition 2.4. Thus $D(B_{0})=\\{u\in
D(\Delta_{N}^{2k})\leavevmode\nobreak\ |\leavevmode\nobreak\
\mathop{\textrm{{tr}}}u=0\\}.$ Now for $u\in D(B_{0}^{*})$, there is a $f\in
L^{2}(\Omega)$ such that for all $v\in D(B_{0})$ we have $\left\langle
u,B_{0}v\right\rangle=\left\langle u,\Delta^{2k}v\right\rangle=\left\langle
f,v\right\rangle_{\Omega}$, so $B_{0}^{*}$ is a restriction of the maximal
$L^{2}$-realization of the distribution $\Delta^{2k}$.

Hence (taking $\alpha=\beta=1,\gamma=\delta=0$ in Remark 2.13) we can find a
solution of the system

$\displaystyle\partial_{t}u+\Delta^{4k}u$ $\displaystyle=0$
$\displaystyle\textnormal{ in }(0,\infty)\times\Omega,$ (5.1)
$\displaystyle\mathop{\textrm{{tr}}}\Delta^{4k}u+{\mathscr{N}}^{{\mathfrak{b}}}(\Delta^{2k})u$
$\displaystyle=0$ $\displaystyle\textnormal{ on }(0,\infty)\times\Gamma,$
(5.2) $\displaystyle{\mathscr{N}}^{{\mathfrak{b}}}u$ $\displaystyle=0$
$\displaystyle\textnormal{ on }(0,\infty)\times\Gamma,$ (5.3) $\displaystyle
u|_{t=0}$ $\displaystyle=u_{0}$ $\displaystyle\textnormal{ in }\Omega,$ (5.4)

where $N_{\mathfrak{b}}$ is given by the abstract definition

$\displaystyle D({\mathscr{N}}^{{\mathfrak{b}}})\coloneqq$
$\displaystyle\\{u\in D(B_{0}^{*})\leavevmode\nobreak\ |\leavevmode\nobreak\ $
$\displaystyle\hphantom{spacesp}\exists g\in L^{2}(\Gamma)\forall v\in
D(\Delta_{N}^{2k}):\left\langle\Delta^{2k}u,v\right\rangle_{\Omega}-\left\langle
u,\Delta^{2k}v\right\rangle_{\Omega}=\left\langle
g,\mathop{\textrm{{tr}}}v\right\rangle_{\Gamma}\\}$

and ${\mathscr{N}}^{\mathfrak{b}}=g$. The identification of ${\mathscr{N}}$ is
more difficult here, but some progress can be made using the theory of quasi-
boundary triples (cf. [BM14], [BHdS20, Chapter 8.6]).

For all $u\in H^{0}_{\Delta}(\Omega)$ and $v\in D(\Delta_{N})$ we have

$\displaystyle\left\langle\Delta u,v\right\rangle_{\Omega}-\left\langle
u,\Delta
v\right\rangle_{\Omega}=\left\langle\tilde{\tau}_{N}u,\mathop{\textrm{{tr}}}v\right\rangle_{{\mathcal{G}}_{0}^{\prime}\times{\mathcal{G}}_{0}},$
(5.5)

where $\tilde{\tau}_{N}$ is the extension of the Neumann trace from
$H^{0}_{\Delta}(\Omega)$ to the space ${\mathcal{G}}_{0}^{\prime}$ introduce
there, which is the dual space of $\mathop{\textrm{{tr}}}D(\Delta_{N})$
equipped with a Hilbert space structure. For this version of Green’s formula
also see [DKP21, Proposition 2.4]. This allows us to characterize
${\mathscr{N}}^{\mathfrak{b}}$ at least on a subset of $D(B_{0}^{*})$. Let

$V=\\{u\in L^{2}(\Omega)\leavevmode\nobreak\ |\leavevmode\nobreak\
\Delta^{2k}u,\Delta^{2k-1}u\in
L^{2}(\Omega),\Delta^{j}u\in\ker{\tilde{\tau}_{N}},\textrm{\leavevmode\nobreak\
for\leavevmode\nobreak\ }j=0,...,2k-2\\}.$

######  Lemma 5.2.

Let $u\in D({\mathscr{N}}^{\mathfrak{b}})\cap V$, then $\Delta^{j}u\in
H^{3/2}_{\Delta}(\Omega)$ for $j=0,...,2k-1$ and
${\mathscr{N}}^{\mathfrak{b}}u=\partial_{\nu}\Delta^{2k-1}u.$

###### Proof.

As we have $\Delta^{j}u\in L^{2}(\Omega)$ for $j=0,...,2k$ and
$D(B_{0})\subset D(B_{N})=D(\Delta_{N}^{2k})$ for $u\in
D({\mathscr{N}}_{b})\cap V$ and $v\in D(\Delta_{N}^{2k})$ we have

$\displaystyle\left\langle g,\mathop{\textrm{{tr}}}v\right\rangle$
$\displaystyle=\left\langle\Delta^{2k}u,v\right\rangle_{\Omega}-\left\langle
u,\Delta^{2k}v\right\rangle_{\Omega}=\sum_{j=0}^{2k-1}\left\langle\tilde{\tau}_{N}\Delta^{j}u,\mathop{\textrm{{tr}}}\Delta^{2k-1-j}v\right\rangle_{{\mathcal{G}}_{0}^{\prime}\times{\mathcal{G}}_{0}}$
$\displaystyle=\left\langle\tilde{\tau}_{N}\Delta^{2k-1}u,\mathop{\textrm{{tr}}}v\right\rangle_{{\mathcal{G}}_{0}^{\prime}\times{\mathcal{G}}_{0}}.$

Now as $D(\Delta_{N}^{2k})$ is dense in $H^{1}(\Omega)$,
$\mathop{\textrm{{tr}}}D(\Delta_{N}^{2k})$ is dense in $L^{2}(\Gamma)$. Thus
we have $g=\tilde{\tau}_{N}\Delta^{2k-1}u\in L^{2}(\Gamma)$ and

$\displaystyle\left\langle\Delta^{2k}u,v\right\rangle_{\Omega}-\left\langle\Delta^{2k-1}u,\Delta
v\right\rangle_{\Omega}$
$\displaystyle=\left\langle\tilde{\tau}_{N}\Delta^{2k-1}u,\mathop{\textrm{{tr}}}v\right\rangle_{{\mathcal{G}}_{0}^{\prime}\times{\mathcal{G}}_{0}}=\left\langle
g,\mathop{\textrm{{tr}}}v\right\rangle_{\Gamma}$

for all $v\in D(\Delta_{N}^{2k})$, and by approximation also for all $v\in
D(\Delta_{N})$ (as $D(\Delta_{N}^{2k})$ is a core of $\Delta_{N}$). Finally,
we obtain $\Delta^{2k-1}u\in H^{3/2}_{\Delta}(\Omega)$ and
${\mathscr{N}}_{b}u=\partial_{\nu}\Delta^{2k-1}u$ for $u\in
D({\mathscr{N}}_{b})\cap V$ by [DKP21, Proposition 2.4 (ii)] and similarly
$\Delta^{j}u\in H^{3/2}_{\Delta}(\Omega)$ for all $j=0,...,2k-1$. ∎

## References

  * [AE96] Herbert Amann and Joachim Escher.  Strongly continuous dual semigroups.  Annali di Matematica Pura ed Applicata, 171:41–62, 1996.
  * [AF03] Robert A. Adams and John J. F. Fournier.  Sobolev spaces, volume 140 of Pure and Applied Mathematics (Amsterdam).  Elsevier/Academic Press, Amsterdam, second edition, 2003.
  * [AMPR03] Wolfgang Arendt, Giorgio Metafune, Diego Pallara, and Silvia Romanelli.  The Laplacian with Wentzell-Robin boundary conditions on spaces of continuous functions.  Semigroup Forum, 67(2):247–261, 2003.
  * [BE19] Tim Binz and Klaus-Jochen Engel.  Operators with Wentzell boundary conditions and the Dirichlet-to-Neumann operator.  Math. Nachr., 292(4):733–746, 2019.
  * [BGM22] Jussi Behrndt, Fritz Gesztesy, and Marius Mitrea.  Sharp boundary trace theory and Schrödinger Operators on Bounded Lipschitz Domains.  Preprint arXiv:2209.09230v1 [math.FA], 2022.
  * [BHdS20] Jussi Behrndt, Seppo Hassi, and Henk de Snoo.  Boundary value problems, Weyl functions, and differential operators, volume 108 of Monographs in Mathematics.  Birkhäuser/Springer, Cham, 2020.
  * [BM14] Jussi Behrndt and Till Micheler.  Elliptic differential operators on Lipschitz domains and abstract boundary value problems.  Journal of Functional Analysis, 267(10):3657–3709, 2014.
  * [DG18] Daniel Daners and Jochen Glück.  A criterion for the uniform eventual positivity of operator semigroups.  Integral Equations Operator Theory, 90(4):Paper No. 46, 19, 2018\.
  * [DGK16a] Daniel Daners, Jochen Glück, and James B. Kennedy.  Eventually positive semigroups of linear operators.  J. Math. Anal. Appl., 433(2):1561–1593, 2016.
  * [DGK16b] Daniel Daners, Jochen Glück, and James B. Kennedy.  Eventually and asymptotically positive semigroups on Banach lattices.  J. Differential Equations, 261(5):2607 – 2649, 2016.
  * [DKP21] Robert Denk, Markus Kunze, and David Ploß.  The Bi-Laplacian with Wentzell boundary conditions on Lipschitz domains.  Integral Equations Operator Theory, 93: Paper No. 13, 26pp., 2021\.
  * [DPRS23] Robert Denk, David Ploß, Sophia Rau, and Jörg Seiler.  Boundary value problems with rough boundary data.  Journal of Differential Equations, 366:85–131, 2023.
  * [DT08] Jesús Ildefonso Díaz and Lourdes Tello.  On a climate model with a dynamic nonlinear diffusive boundary condition.  Discrete Contin. Dyn. Syst. Ser. S, 1(2):253–262, 2008.
  * [EF05] Klaus-Jochen Engel and Genni Fragnelli.  Analyticity of semigroups generated by operators with generalized Wentzell boundary conditions.  Adv. Differential Equations, 10(11):1301–1320, 2005.
  * [Eng03] Klaus-Jochen Engel.  Second order differential operators on $C[0,1]$ with Wentzell-Robin boundary conditions.  In Evolution equations, volume 234 of Lecture Notes in Pure and Appl. Math., pages 159–165. Dekker, New York, 2003.
  * [EPS03] Joachim Escher, Jan Prüss, and Gieri Simonett.  Analytic solutions for a Stefan problem with Gibbs-Thomson correction.  J. Reine Angew. Math., 563:1–52, 2003.
  * [FGGR08] Angelo Favini, Gisèle Ruiz Goldstein, Jerome A. Goldstein, and Silvia Romanelli.  Fourth order operators with general Wentzell boundary conditions.  Rocky Mountain J. Math., 38(2):445–460, 2008.
  * [GM08] Fritz Gesztesy and Marius Mitrea.  Generalized Robin boundary conditions, Robin-to-Dirichlet maps, and Krein-type resolvent formulas for Schrödinger operators on bounded Lipschitz domains.  In Perspectives in partial differential equations, harmonic analysis and applications, volume 79 of Proc. Sympos. Pure Math., pages 105–173. Amer. Math. Soc., Providence, RI, 2008.
  * [Gol06] Gisèle Ruiz Goldstein.  Derivation and physical interpretation of general boundary conditions.  Adv. Differential Equations, 11(4):457–480, 2006.
  * [Kon67] VA Kondrat’eV.  Boundary problems for elliptic equations in domains with conical or angular points.  Trans. Moscow Math. Soc, 16(227-313):129–133, 1967.
  * [LM72] Jacques-Louis Lions and Enrico Magenes.  Non-homogeneous boundary value problems and applications. Vol. I.  Springer-Verlag, New York-Heidelberg, 1972.
  * [Nit10] Robin Nittka.  Elliptic and parabolic problems with Robin boundary conditions on Lipschitz domains.  PhD thesis, Universität Ulm, 2010.  _http://dx.doi.org/10.18725/OPARU-1790_.
  * [Nit11] Robin Nittka.  Regularity of solutions of linear second order elliptic and parabolic boundary value problems on Lipschitz domains.  J. Differential Equations, 251(4-5):860–880, 2011.
  * [Ouh05] El Maati Ouhabaz.  Analysis of heat equations on domains, volume 31 of London Mathematical Society Monographs Series.  Princeton University Press, Princeton, NJ, 2005.
  * [Plo24] David Ploß.  Wentzell boundary conditions for elliptic fourth-order operators.  PhD thesis, Universität Konstanz, Konstanz, 2024.
  * [PRZ06] Jan Prüss, Reinhard Racke, and Songmu Zheng.  Maximal regularity and asymptotic behavior of solutions for the Cahn-Hilliard equation with dynamic boundary conditions.  Ann. Mat. Pura Appl. (4), 185(4):627–648, 2006.
  * [RZ03] Reinhard Racke and Songmu Zheng.  The Cahn-Hilliard equation with dynamic boundary conditions.  Adv. Differential Equations, 8(1):83–110, 2003.
  * [Tri95] Hans Triebel.  Interpolation theory, function spaces, differential operators.  Johann Ambrosius Barth, Heidelberg, second edition, 1995.
  * [War13] Mahamadi Warma.  Parabolic and elliptic problems with general Wentzell boundary condition on Lipschitz domains.  Commun. Pure Appl. Anal., 12(5):1881–1905, 2013.
  * [Wlo87] J. Wloka.  Partial differential equations.  Cambridge University Press, Cambridge, 1987.  Translated from the German by C. B. Thomas and M. J. Thomas.