# An IPW-based Unbiased Ranking Metric in Two-sided Markets

Keisho Oh  Recruit Co., Ltd.TokyoJapan<EMAIL_ADDRESS>,  Naoki
Nishimura  Recruit Co., Ltd.TokyoJapan<EMAIL_ADDRESS>,  Minje Sung
Tokyo Institute of TechnologyTokyoJapan<EMAIL_ADDRESS>,  Ken
Koboyashi  Tokyo Institute of TechnologyTokyoJapan
<EMAIL_ADDRESS>and  Kazuhide Nakata  Tokyo Institute of
TechnologyTokyoJapan<EMAIL_ADDRESS>
(2023)

###### Abstract.

In modern recommendation systems, unbiased learning-to-rank (LTR) is crucial
for prioritizing items from biased implicit user feedback, such as click data.
Several techniques, such as Inverse Propensity Weighting (IPW), have been
proposed for single-sided markets. However, less attention has been paid to
two-sided markets, such as job platforms or dating services, where successful
conversions require matching preferences from both users. This paper addresses
the complex interaction of biases between users in two-sided markets and
proposes a tailored LTR approach. We first present a formulation of feedback
mechanisms in two-sided matching platforms and point out that their implicit
feedback may include position bias from both user groups. On the basis of this
observation, we extend the IPW estimator and propose a new estimator, named
_two-sided IPW_ , to address the position bases in two-sided markets. We prove
that the proposed estimator satisfies the unbiasedness for the ground-truth
ranking metric. We conducted numerical experiments on real-world two-sided
platforms and demonstrated the effectiveness of our proposed method in terms
of both precision and robustness. Our experiments showed that our method
outperformed baselines especially when handling rare items, which are less
frequently observed in the training data.

learning-to-rank, unbiased learning, recommender system

††copyright: acmcopyright††journalyear: 2023††doi:
XXXXXXX.XXXXXXX††conference: ; August 7, 2023; Long Beach, CA

##  1\. Introduction

Information retrieval and recommendation systems have become increasingly
prevalent. One of the key features of these systems is that they present users
with items ranked in accordance with their preferences. To this end, such
systems often use learning-to-rank (LTR) algorithms that capture users’
preferences, prioritizing items that are more likely to be relevant or
appealing to the user. As the number of available items continues to grow, LTR
has become an essential component of effective recommendation systems,
attracting considerable interest from both industry and academic fields.

One of the key challenges of LTR is _unbiased learning_ , which aims to
accurately estimate users’ preferences without being influenced by biases in
training data. Usually, LTR methods rely on users’ implicit feedback, such as
clicks, because this type of data can be easily collected during users’
natural behaviors in the system. However, such implicit feedback sometimes
reflects not only users’ true preferences but also other factors like their
first impressions of items or the item’s position in the list (Wen et al.,
2019). As a result, learning users’ preferences from implicit feedback can be
biased and vulnerable, which leads to suboptimal recommendations and a
degraded user experience. To address this issue, the unbiased LTR framework
has attracted much attention (Chen et al., 2023). This research topic was
first introduced by Joachims et al. (2017), and several unbiased LTR methods
have been proposed (Wang et al., 2016; Ai et al., 2018; Wang et al., 2018; Hu
et al., 2019) to improve the quality of recommendation systems.

While several unbiased learning techniques have been proposed in the
literature on LTR, existing methods mainly focus on single-sided markets, and
less attention has been paid to two-sided markets. In two-sided markets, the
final conversion is determined only when preferences from both users match. In
such a situation, position biases are included in the implicit feedback not
only from one-side users but also from the opposite users. Let us consider the
case of a two-sided job-matching platform for example. A job seeker first
inputs a query to search for job postings matching his/her preference. Then,
the job seeker browses the list of job postings as a result and clicks on
several of them to confirm the more detailed information or submit an
application (Chen et al., 2019). On the other hand, a recruiter browses a list
of applicants to choose some promising ones. Finally, the recruiter makes a
final decision on which applicant to hire. In such a scenario, both the job
seeker and recruiter tend to recognize items presented at higher positions. As
a result, there exists not only a position bias against job seekers but also
one that recruiters are more likely to hire the candidates who are displayed
at the top of the list.

Against this background, we propose an unbiased LTR framework for two-sided
matching platforms. In this work, we first present an estimator called the
two-sided inverse probability weighting (IPW) estimator by extending the (IPW)
estimator (Joachims et al., 2017). We prove that the proposed metric is
unbiased where biased is included in implicit feedback from the two sides in
matching platforms. Also, we present a practical LTR algorithm with the two-
sided IPW estimator. To the best of our knowledge, we are the first to propose
an unbiased LTR method in two-sided matching platforms.

The contributions of our study can be summarized as follows:

  * •

We are the first to give a formulation of feedback mechanisms in two-sided
matching platforms and point out that their implicit feedback may include
position bias from both user groups.

  * •

To address these biases, we propose a new estimator, named _two-sided IPW_ by
extending the IPW estimator.

  * •

We also proved that the proposed estimator is unbiased against position biases
in two-sided markets.

  * •

We conducted numerical experiments on both synthetic and real datasets and
demonstrated that the LTR method with our estimator significantly outperformed
existing methods.

##  2\. Related Work

###  2.1. Learning-to-Rank

LTR refers to a framework for constructing ranking models where items are
sorted in accordance with their degrees of relevance or preferences of users.
LTR was first introduced by Fuhr (1989) and due to its practical importance,
this topic has attracted much attention. The approaches of existing LTR
methods are categorized into three types: pointwise, pairwise, and listwise
approaches. The pointwise approaches (Crammer and Singer, 2001; Cooper et al.,
1992; Li et al., 2007; Nallapati, 2004; Chu and Keerthi, 2007; Cossock and
Zhang, 2006) focus on determining whether a single item is relevant or not by
itself. They train a classification or regression model to predict whether the
individual item is relevant to the input query. On the other hand, the
pairwise approaches (Burges, 2010; Freund et al., 2003; Joachims, 2006; Xu and
Li, 2007; Shen and Joshi, 2005) use pairs of items as a training instance and
train a ranking function from the relative order of relevance between the
items in each pair. The listwise methods (Cao et al., 2007; Guiver and
Snelson, 2008; Qin et al., 2009; Taylor et al., 2008; Volkovs and Zemel, 2009;
Wu et al., 2009; Xia et al., 2008; Xu and Li, 2007; Yue et al., 2007; Burges,
2010; Bruch, 2021) aim to train a ranking function where the ranked lists of
items are given. Contrary to the pointwise and pairwise approaches, the
listwise approaches can capture the nature of the ranking problems, and the
previous studies showed that the listwise approaches delivered better
performance than the others (Cao et al., 2007; Qin et al., 2008, 2009).

###  2.2. Unbiased Learning-to-rank Algorithms

The Unbiased LTR framework has attracted much attention as an effective
approach to train a ranking model from biased feedback for position bias (Chen
et al., 2023). Joachims et al. (2017) first dealt with the unbiased ranking
problems and presented the counterfactual framework to learn an unbiased
ranking function by using the IPW estimator. For the propensity weights,
intervention-based and result randomization-based estimation methods have been
proposed (Joachims et al., 2017; Wang et al., 2016). Ai et al. (2018) devised
a method called DLA that simultaneously trains a ranking function and
estimates propensity weights. Wang et al. (2018) proposed a method for
position bias estimation with a regression-based expectation-maximization
algorithm. Furthermore, Hu et al. (2019) proposed a listwise unbiased LTR
method called LambdaMart.

As previously mentioned, several methods have been proposed for training an
unbiased ranking function. However, these existing studies focus on cases
where the bias is derived from the implicit feedback from only one-side users.
In contrast, our proposed method addresses the cases where the implicit
feedback includes not only the bias derived from one side but also from the
opposite side.

###  2.3. Recommendation in Matching Platforms

As matching platforms have been emerging in various fields such as job
recruiting and dating, two-sided recommendation methods have begun to be
actively studied. Yang et al. (2022) used Bayesian personalized ranking (BPR)
loss function to predict the two-sided preferences taking into account the
expectation from both sides. Tu et al. (2014) proposed a Latent dirichlet
allocation (LDA) probabilistic model to learn the preferences in two-sided
dating platforms. Su et al. (2022) focused on the ranking problem in two-sided
matching and proposed an optimization problem to maximize the overall social
welfare. In addition, Saito and Joachims (2022) defined a ranking optimization
problem to maximize the Nash Social Welfare and provided its tractable
formulation. However, these existing studies do not address the issue that the
implicit feedback includes bias from both sides and thus, these methods are
expected not to perform well under a biased dataset.

##  3\. Proposed Method

We first present a formulation of feedback mechanisms in matching platforms
and point out it potentially has two types of position bias. Then, we address
how to debias these two types of position biases in ranking metrics. Finally,
we derive an unbiased LTR method for two-sided matching platforms.

###  3.1. Notation and Problem Setting

We consider a platform that has two sets of users, a proactive set $U$ and a
reactive set $V$. We assume two types of users that interact with each other
as follows: the platform first presents each proactive user $u\in U$ with a
personalized ranking list $V_{u}\subseteq V$ that includes reactive users who
the user $u$ might be interested in. In accordance with the ranking list, each
proactive user $u$ selects specific reactive users on the basis of his/her
preferences. Next, for each reactive side user $v\in V$, the platform shows a
new ranking list $U_{v}\subseteq U$ including proactive users who have already
selected $v$. We define $\mathrm{rank}_{u}(v)$ as the ranking of user $v$ in
the list shown to $u$ and $\mathrm{rank}_{v}(u)$ as the ranking of user $u$
conversely.

To express implicit interactions from the proactive user $u\in U$ to the
reactive user $v\in V$, we use a binary random variable $Y_{u\rightarrow v}$.
If $Y_{u\rightarrow v}=1$, the proactive user $u$ gives positive feedback to
the reactive user $v$; otherwise $Y_{u\rightarrow v}=0$. We also introduce a
binary random variable $Y_{u\leftarrow v}$ to express the implicit feedback
from the reactive user $v$ to the proactive user $u$. To denote the relevance
from the proactive user $u$ to the reactive user $v$, we use a binary label
$R_{u\rightarrow v}\in\\{0,1\\}$. Conversely, $R_{u\leftarrow v}$ represents
the relevance from $v$ to $u$. If $R_{u\rightarrow v}=1$ (resp.
$R_{u\leftarrow v}=1$), the reactive user $v$ (resp. the proactive user $u$)
is relevant to the proactive user $u$ (resp. the reactive user $v$). We denote
the exposure variable as $O_{u\rightarrow v}$ (resp. $O_{u\leftarrow v}$)
whether the reactive user $v$ (resp. the proactive user $u$) is exposed to the
proactive user $u$ (resp. the reactive user $v$). Let us denote
$\theta_{u\rightarrow v},\theta_{u\leftarrow v}$ the probability of
$O_{u\rightarrow v},O_{u\leftarrow v}$ get $1$ respectively. We summarize the
notation in Table 1.

Table 1. Summary of notation Notation | Explanation
---|---
$u,u_{i}$ | proactive user
$v,v_{j}$ | reactive user
$U$ | set of proactive users
$V$ | set of reactive users
$U_{v}$ | set of proactive users shown to $v$
$V_{u}$ | set of reactive users shown to $u$
$\mathrm{rank}_{u}(v)$ | the rank of $v$ at the list shown to $u$
$Y_{u\rightarrow v}$ / $Y_{u\leftarrow v}$ | proactive/reactive user’s feedback
$R_{u\rightarrow v}$ / $R_{u\leftarrow v}$ | true proactive/reactive side-specific relevance
$O_{u\rightarrow v}$ / $O_{u\leftarrow v}$ | proactive/reactive exposure indicator
$\theta_{u\rightarrow v}$ / $\theta_{u\leftarrow v}$ | proactive/reactive exposure probability

In this study, we rely on the following two technical assumptions, which are
often used in causal inference:

###### Assumption 1.

For all proactive user $u\in V$, the exposure probability
$\theta_{u\rightarrow v}$ only depends on the position of $v\in V$ in the
list, i.e., $O_{u\rightarrow v}\sim P(\cdot\mid\mathrm{rank}_{u}(v))$.

###### Assumption 2.

For any proactive-reactive user pair $(u,v)\in U\times V$,
$\theta_{u\rightarrow v},\theta_{u\leftarrow v}\in(0,1]$.

We can now model implicit feedback mechanisms in the two-sided matching
platform. For proactive users, $u\in U$ selects $v\in V$ only if $u$ examined
$v$ ($O_{u\rightarrow v}=1$) and found it relevant ($R_{u\rightarrow v}=1$).
For reactive users, $v\in V$ selects $u\in U$ only if the following conditions
are all satisfied:

  1. (1)

The proactive user $u$ has already selected $v$ ($Y_{u\rightarrow v}=1$);

  2. (2)

The proactive user $u$ has been exposed to $u$ ($O_{u\leftarrow v}=1$);

  3. (3)

The proactive user $u$ is relevant to $v$ ($R_{u\rightarrow v}=1$).

In this setting, the implicit feedback model can be expressed as follows:

(1) $\displaystyle Y_{u\rightarrow v}$ $\displaystyle=O_{u\rightarrow v}\cdot
R_{u\rightarrow v},$ (2) $\displaystyle Y_{u\leftarrow v}$
$\displaystyle=Y_{u\rightarrow v}\cdot O_{u\leftarrow v}\cdot R_{u\leftarrow
v}.$

###  3.2. Ranking Metric and Naive Estimator

To evaluate the ranking $\mathrm{rank}_{u}(v)$ in the two-sided matching
platform, we define a _two-sided relevance_ as $R_{uv}\coloneq R_{u\rightarrow
v}(1+R_{u\leftarrow v})$. The two-sided relevance $R_{uv}$ represents the
following three scenarios:

$\displaystyle R_{uv}=\begin{cases}2&\text{if~{}$R_{u\rightarrow v}=1$
and~{}$R_{u\leftarrow v}=1$},\\\ 1&\text{if~{}$R_{u\rightarrow v}=1$
and~{}$R_{u\leftarrow v}=0$},\\\ \ 0&\text{otherwise}.\end{cases}$

Then, we can define the ground-truth performance of a given ranking
$\mathrm{rank}_{u}(v)$ as follows:

$\displaystyle\mathcal{R}\left(\mathrm{rank}_{u}(v),R_{u\rightarrow
v},R_{u\leftarrow v}\right)$ (3)
$\displaystyle\quad\coloneq\frac{1}{\left|U\right|}\sum_{u\in U}\sum_{v\in
V_{u}}\lambda(\mathrm{rank}_{u}(v))\cdot g(R_{u\rightarrow v},R_{u\leftarrow
v}).$

Here, $g(R_{u\rightarrow v},R_{u\leftarrow v})$ is the gain function defined
as

$g(R_{u\rightarrow v},R_{u\leftarrow v})\coloneq
2^{R_{uv}}-1=2^{R_{u\rightarrow v}(1+R_{u\leftarrow v})}-1,$

and $\lambda\left(\mathrm{rank}_{u}(v)\right)$ is a weight function
determining the ranking metric. For instance, when we set

$\lambda\left(\mathrm{rank}_{u}(v)\right)=\frac{\mathbf{1}(\mathrm{rank}_{u}(v)\leq
K)}{\log_{2}\left(\mathrm{rank}_{u}(v)+1\right)},$

where $\mathbf{1}(\cdot)$ is the indicator function and $K$ is a user-defined
positive integer, Section 3.2 corresponds to $\mathrm{DCG}@K$.

Since we cannot directly observe the relevance $R_{u\rightarrow v}$ and
$R_{u\leftarrow v}$, we consider designing an estimator for the ground-truth
metric with the observable implicit feedback $Y_{u\rightarrow v}$ and
$Y_{u\leftarrow v}$. Let us define a surrogate gain function with
$Y_{u\rightarrow v}$ and $Y_{u\leftarrow v}$ as

(4) $\displaystyle h(Y_{u\rightarrow v},Y_{u\leftarrow v})$
$\displaystyle:=2^{Y_{u\rightarrow v}+Y_{u\leftarrow v}}-1.$

We note that from Equations 1 and 2, the following holds

$\displaystyle h(Y_{u\rightarrow v},Y_{u\leftarrow v})$
$\displaystyle=2^{O_{u\rightarrow v}R_{u\rightarrow v}(1+O_{u\leftarrow
v}R_{u\leftarrow v})}$ $\displaystyle=g(O_{u\rightarrow v}R_{u\rightarrow
v},O_{u\leftarrow v}R_{u\leftarrow v}).$

Thus, we can regard $h(Y_{u\rightarrow v},Y_{u\leftarrow v})$ as an
alternative to the gain function $g$ based on the observable implicit feedback
$Y_{u\rightarrow v}$ and $Y_{u\leftarrow v}$.

Using this surrogate gain function $h$, we can define a naive estimator of
Section 3.2 as follows:

$\displaystyle\hat{\mathcal{R}}\left(\mathrm{rank}_{u}(v),Y_{u\rightarrow
v},Y_{u\leftarrow v}\right)$ (5)
$\displaystyle\quad\coloneq\frac{1}{\left|U\right|}\sum_{u\in U}\sum_{v\in
V_{u}}\lambda(\mathrm{rank}_{u}(v))\cdot h(Y_{u\rightarrow v},Y_{u\leftarrow
v}).$

However, the naive estimator $\hat{\mathcal{R}}$ is biased as shown below:

###### Proposition 3.1.

Suppose that $O_{u\rightarrow v}$ and $O_{u\leftarrow v}$ are independent of
each other and $\theta_{u\rightarrow v},\theta_{u\leftarrow v}\neq 1$. Then,
the following holds

$\displaystyle\mathbb{E}_{O_{u\rightarrow v},O_{u\leftarrow
v}}\left[\hat{\mathcal{R}}\left(\mathrm{rank}_{u}(v),Y_{u\rightarrow
v},Y_{u\leftarrow v}\right)\right]$
$\displaystyle\quad\not\propto\mathcal{R}\left(\mathrm{rank}_{u}(v),R_{u\rightarrow
v},R_{u\leftarrow v}\right).$

###### Proof.

It is sufficient to show that

$\displaystyle\mathbb{E}_{O_{u\rightarrow v},O_{u\leftarrow
v}}\left[h(Y_{u\rightarrow v},Y_{u\leftarrow v})\right]\not\propto
g(R_{u\rightarrow v},R_{u\leftarrow v}).$

Since $O_{u\rightarrow v},O_{u\leftarrow v}$ are binary random variables, it
holds that

$\displaystyle 2^{Y_{u\rightarrow v}}$ $\displaystyle=2^{O_{u\rightarrow
v}R_{u\rightarrow v}}$ (6) $\displaystyle=(1-O_{u\rightarrow
v})+O_{u\rightarrow v}2^{R_{u\rightarrow v}},$ $\displaystyle
2^{Y_{u\leftarrow v}}$ $\displaystyle=2^{O_{u\rightarrow v}O_{u\leftarrow
v}R_{u\rightarrow v}R_{u\leftarrow v}}$ (7) $\displaystyle=(1-O_{u\rightarrow
v}O_{u\leftarrow v})+O_{u\rightarrow v}O_{u\leftarrow v}2^{R_{u\rightarrow
v}R_{u\leftarrow v}}.$

Then, we have

(8) $\displaystyle 2^{Y_{u\rightarrow v}}\cdot 2^{Y_{u\leftarrow v}}$
$\displaystyle=(1-O_{u\rightarrow v})+O_{u\rightarrow v}(1-O_{u\leftarrow
v})2^{R_{u\rightarrow v}}$ (9) $\displaystyle\quad+O_{u\rightarrow
v}O_{u\leftarrow v}2^{R_{u\rightarrow v}(1+R_{u\leftarrow v})}.$

Since $O_{u\rightarrow v},O_{u\leftarrow v}$ are independent and
$\theta_{u\rightarrow v},\theta_{u\leftarrow v}\neq 1$, we obtain

$\displaystyle\mathbb{E}_{O_{u\rightarrow v},O_{u\leftarrow
v}}\left[h(Y_{u\rightarrow v},Y_{u\leftarrow v})\right]$
$\displaystyle\quad=\mathbb{E}_{O_{u\rightarrow v},O_{u\leftarrow
v}}\left[2^{Y_{u\rightarrow v}}\cdot 2^{Y_{u\leftarrow v}}\right]$
$\displaystyle\quad=(1-\theta_{u\rightarrow v})+\theta_{u\rightarrow
v}\left(1-\theta_{u\leftarrow v}\right)2^{R_{u\rightarrow v}}$ (10)
$\displaystyle\quad\quad+\theta_{u\rightarrow v}\theta_{u\leftarrow
v}\underbrace{2^{R_{u\rightarrow v}(1+R_{u\leftarrow v})}}_{g(R_{u\rightarrow
v},~{}R_{u\leftarrow v})+1},$ $\displaystyle\quad\not\propto g(R_{u\rightarrow
v},R_{u\leftarrow v}),$

which completes the proof. ∎

###  3.3. Unbiased Ranking Metric Estimator

We have seen that the naive estimator (3.2) can be biased in estimating the
ground-truth ranking metric with implicit feedback. To remedy this issue, we
extend the IPW estimator (Joachims, 2006) and propose a new estimator, called
_two-sided IPW estimator_ , for the ranking metric (3.2).

To take into account the position bias in the two-sided matching platform, we
first introduce the following weighted gain function:

$\displaystyle h_{\mathrm{IPW}}(Y_{u\rightarrow v},Y_{u\leftarrow v})$
$\displaystyle=\frac{1}{\theta_{u\rightarrow v}\theta_{u\leftarrow
v}}\left(2^{Y_{u\rightarrow v}}\left(2^{Y_{u\leftarrow v}}-1\right)\right)$
$\displaystyle\quad+\frac{1}{\theta_{u\rightarrow v}}\left(2^{Y_{u\rightarrow
v}}-1\right).$

With $h_{\mathrm{IPW}}(Y_{u\rightarrow v},Y_{u\leftarrow v})$, we define the
_two-sided IPW estimator_ as follows:

(11) $\begin{split}&\hat{\mathcal{R}}_{\mathrm{IPW}}(Y_{u\rightarrow
v},Y_{u\leftarrow v})\\\ &\quad\coloneq\frac{1}{\left|U\right|}\sum_{u\in
U}\sum_{v\in V_{u}}\lambda(\mathrm{rank}_{u}(v))\cdot
h_{\mathrm{IPW}}(Y_{u\rightarrow v},Y_{u\leftarrow v}).\end{split}$

We show that $\hat{\mathcal{R}}_{\mathrm{IPW}}$ is an unbiased estimator of
the ground-truth metric (3.2).

###### Theorem 3.2.

Suppose that $O_{u\rightarrow v}$ and $O_{u\leftarrow v}$ are independent of
each other. Then it follows that

$\begin{split}&\mathbb{E}_{O_{u\rightarrow v},O_{u\leftarrow
v}}\left[\hat{\mathcal{R}}_{\mathrm{IPW}}(Y_{u\rightarrow v},Y_{u\leftarrow
v})\right]\\\ &\quad=\mathcal{R}\left(\mathrm{rank}_{u}(v),R_{u\rightarrow
v},R_{u\leftarrow v}\right).\end{split}$

###### Proof.

From Equation 6, the expectation of $2^{Y_{u\rightarrow v}}$ can be calculated
as

$\displaystyle\mathbb{E}\left[2^{Y_{u\rightarrow v}}\right]$
$\displaystyle=\mathbb{E}\left[(1-O_{u\rightarrow v})+O_{u\rightarrow
v}2^{R_{u\rightarrow v}}\right]$ $\displaystyle=(1-\theta_{u\rightarrow
v})+\theta_{u\rightarrow v}2^{R_{u\rightarrow v}}.$

Combined with Equation 10, the expectation of
$h_{\mathrm{IPW}}\left(Y_{u\rightarrow v},Y_{u\leftarrow v}\right)$ is
calculated as

$\displaystyle\mathbb{E}_{O_{u\rightarrow v},O_{u\leftarrow
v}}\left[h_{\mathrm{IPW}}\left(Y_{u\rightarrow v},Y_{u\leftarrow
v}\right)\right]$ (12) $\displaystyle~{}=\mathbb{E}_{O_{u\rightarrow
v},O_{u\leftarrow v}}\left[\frac{1}{\theta_{u\rightarrow v}\theta_{u\leftarrow
v}}2^{Y_{u\rightarrow v}}2^{Y_{u\leftarrow v}}\right.$
$\displaystyle~{}\quad+\left.\frac{1}{\theta_{u\rightarrow
v}}\left(1-\frac{1}{\theta_{u\leftarrow v}}\right)2^{Y_{u\rightarrow
v}}-\frac{1}{\theta_{u\rightarrow v}}\right]$
$\displaystyle~{}=\frac{1}{\theta_{u\rightarrow v}\theta_{u\leftarrow
v}}\mathbb{E}_{O_{u\rightarrow v},O_{u\leftarrow v}}\left[2^{Y_{u\rightarrow
v}}2^{Y_{u\leftarrow v}}\right]$
$\displaystyle~{}\quad+\frac{1}{\theta_{u\rightarrow
v}}\left(1-\frac{1}{\theta_{u\leftarrow v}}\right)\mathbb{E}_{O_{u\rightarrow
v},O_{u\leftarrow v}}\left[2^{Y_{u\rightarrow
v}}\right]-\frac{1}{\theta_{u\rightarrow v}}$
$\displaystyle~{}=\frac{1}{\theta_{u\rightarrow v}\theta_{u\leftarrow
v}}\biggl{(}(1-\theta_{u\rightarrow v})+\theta_{u\rightarrow
v}\left(1-\theta_{u\leftarrow v}\right)2^{R_{u\rightarrow v}}$
$\displaystyle~{}\quad+\theta_{u\rightarrow v}\theta_{u\leftarrow
v}\left(g\left(R_{u\rightarrow v},~{}R_{u\leftarrow
v}\right)+1\right)\biggr{)}$
$\displaystyle~{}\quad+\frac{1}{\theta_{u\rightarrow
v}}\left(1-\frac{1}{\theta_{u\leftarrow
v}}\right)\left((1-\theta_{u\rightarrow v})+\theta_{u\rightarrow
v}2^{R_{u\rightarrow v}}\right)-\frac{1}{\theta_{u\rightarrow v}}$ (13)
$\displaystyle~{}=g\left(R_{u\rightarrow v},~{}R_{u\leftarrow v}\right).$

From Sections 3.2, 11 and 13, we obtain the desired result. ∎

##  4\. Experiment

###  4.1. Experimental Setup

####  4.1.1. Data Preparation

We conducted numerical experiments on the networking recommendation data set
collected by Su et al. (2022). The data set provides a matrix expressing the
bidirectional preferences within the range $[0,1]$ of 925 users. We denote
this matrix as $M\in[0,1]^{925\times 925}$ , where $M_{uv}$ represents the
preference from $u$ to $v$. Then we preprocessed the data as follows.

  1. (1)

To simulate a two-sided market, we randomly divided the user group into a
proactive side $U$ and a reactive side $V$.

  2. (2)

For the purpose of cross-validation, we split each $U$ and $V$ into 5-fold,
which is denoted by $U_{1},\cdots,U_{5}$ and $V_{1}\cdots,V_{5}$.

  3. (3)

We picked the $k$-th fold elements of $\left\\{M_{ij},i\in U_{k},j\in
V_{k}\right\\}$ as the test data, and split the rest elements as train and
validation data set.

  4. (4)

We defined the exposure probabilities $\theta_{u\rightarrow
v},\theta_{u\leftarrow v}$ as follows:

$\displaystyle\theta_{\cdot\rightarrow v}=\left(\frac{\sum_{k\in
U}M_{kv}}{\underset{v^{\prime}\in V}{\max}\sum_{k\in
U}M_{kv^{\prime}}}\right)^{\eta},$
$\displaystyle\theta_{u\leftarrow\cdot}=\left(\frac{\sum_{k\in
V}M_{ku}}{\underset{u^{\prime}\in U}{\max}\sum_{k\in
V}M_{ku^{\prime}}}\right)^{\eta},$

where $\eta$ is a hyperparameter. Here, the exposure probability for each user
is determined by the proportion of interactions of the user compared to the
one with the most interactions.

  5. (5)

We randomly generated feedback variables as follows:

$\displaystyle R_{u\rightarrow v}\sim\mathrm{Bern}(M_{u\rightarrow
v}),O_{u\rightarrow v}\sim\mathrm{Bern}(\theta_{u\rightarrow v}),$
$\displaystyle R_{u\leftarrow v}\sim\mathrm{Bern}(M_{u\leftarrow
v}),O_{u\leftarrow v}\sim\mathrm{Bern}(\theta_{u\leftarrow v}),$
$\displaystyle Y_{u\rightarrow v}=O_{u\rightarrow v}\cdot R_{u\rightarrow v},$
$\displaystyle Y_{u\leftarrow v}=Y_{u\rightarrow v}\cdot O_{u\leftarrow
v}\cdot R_{u\leftarrow v}.$

Table 2. Methods comparison Methods | Training Loss | Validation Metric
---|---|---
Conventional | $l$ | $\sum_{v\in V_{u}}\lambda(\mathrm{rank}_{u}(v))\cdot\left(2^{Y_{u\rightarrow v}+Y_{u\leftarrow v}}-1\right)$
IPW1 | $l_{\mathrm{IPW1}}$ | $\sum_{v\in V_{u}}\lambda(\mathrm{rank}_{u}(v))\cdot\frac{\left(2^{Y_{u\rightarrow v}+Y_{u\leftarrow v}}-1\right)}{\theta_{u\rightarrow v}}$
IPW2 (proposed) | $l_{\mathrm{IPW2}}$ | $\sum_{v\in V_{u}}\lambda(\mathrm{rank}_{u}(v))\cdot h_{\mathrm{IPW}}(Y_{u\rightarrow v},Y_{u\leftarrow v})$
Table 3. Test score of all models for each fold. For each fold and setting, the best and worst results are denoted in bold and underlined, respectively. DCG | @3 | @10 | @20 | @30
---|---|---|---|---
fold | baseline | IPW1 | IPW2 | baseline | IPW1 | IPW2 | baseline | IPW1 | IPW2 | baseline | IPW1 | IPW2
1 | 0.9270 | 0.7900 | 0.9790 | 2.3565 | 1.8320 | 2.3729 | 3.5849 | 2.7561 | 3.6886 | 5.1954 | 4.8274 | 5.2256
2 | 0.6972 | 0.6705 | 0.8885 | 1.8785 | 1.9064 | 2.0376 | 3.0666 | 3.1751 | 3.3240 | 4.9080 | 4.9642 | 5.0639
3 | 0.6144 | 0.6587 | 0.8769 | 1.6822 | 1.6650 | 1.9998 | 2.7682 | 2.7702 | 2.9078 | 4.3774 | 4.3975 | 4.5385
4 | 1.2125 | 1.0749 | 1.0014 | 2.7112 | 2.7789 | 2.2095 | 4.1889 | 4.1606 | 3.3233 | 6.4263 | 6.4055 | 6.0450
5 | 0.9372 | 0.8957 | 0.9127 | 2.3912 | 2.4353 | 2.3590 | 3.6510 | 3.6809 | 3.6798 | 5.5462 | 5.5516 | 5.5205
Table 4. Test score for each hyperparameter $\eta$. The results are highlighted in the same way as Table 3. DCG | @3 | @10 | @20 | @30
---|---|---|---|---
$\eta$ | baseline | IPW1 | IPW2 | baseline | IPW1 | IPW2 | baseline | IPW1 | IPW2 | baseline | IPW1 | IPW2
0.5 | 0.9270 | 0.7900 | 0.9790 | 2.3565 | 1.8320 | 2.3729 | 3.5849 | 2.7561 | 3.6886 | 5.1954 | 4.8274 | 5.2256
0.6 | 1.0835 | 1.1663 | 0.9171 | 2.4787 | 2.5708 | 2.0815 | 3.7181 | 3.8070 | 3.2703 | 5.3004 | 5.3813 | 5.0614
0.8 | 1.0004 | 0.8008 | 1.0619 | 2.2892 | 1.7519 | 2.4417 | 3.4547 | 2.6889 | 3.7024 | 5.1839 | 4.8023 | 5.2809
1.0 | 0.9048 | 0.8916 | 1.0209 | 2.2697 | 1.9884 | 2.4283 | 3.5770 | 2.9147 | 3.6907 | 5.1688 | 4.9179 | 5.2619

####  4.1.2. Methods

We used Matrix Factorization-based models to predict mutual preferences. To
represent each slide of preferences, we prepared two kinds of embedding space
$W_{pro}$ and $W_{rea}$. For a pair $(u,v),u\in U,v\in V$, we define a mutual
preference score $s_{uv}$ as follows:

  1. (1)

$s_{u\rightarrow v}=\sigma(w_{u}^{\top}w_{v})$, where $w_{u},w_{v}\in
W_{pro}$,

  2. (2)

$s_{u\leftarrow v}=\sigma(\tilde{w}_{u}^{\top}\tilde{w}_{v})$, where
$\tilde{w}_{u},\tilde{w}_{v}\in W_{rea}$,

  3. (3)

$s_{uv}=s_{u\rightarrow v}\cdot s_{u\leftarrow v}$,

where $\sigma(\cdot)$ is the sigmoid function. As an intuitive explanation,
$s_{u\rightarrow v}$ represents the preference score from $u$ to $v$, and
$s_{u\leftarrow v}$ represents the preference score from $v$ to $u$. In
conducting the experiment, we fixed the dimension of the embedding space at
64.

We formulate the training loss related to a certain user $u$’s instances as

$l=-\sum_{v\in V_{u}}y_{u\rightarrow v}\log\frac{s_{u\rightarrow
v}}{\sum_{v^{\prime}\in V_{u}}s_{u\rightarrow v^{\prime}}}-\sum_{v\in
V_{u}}y_{u\leftarrow v}\log\frac{s_{u\leftarrow v}}{\sum_{v^{\prime}\in
V_{u}}s_{u\leftarrow v^{\prime}}},$

where the first term corresponds to the learning of the proactive side’s
preferences, and the second term corresponds to the reactive side’s. Also, two
IPW versions of the list-wise loss (one for the conventional IPW loss and the
other for two-sided IPW) can be expressed as follows:

$\displaystyle l_{\mathrm{IPW1}}$
$\displaystyle=-\sum_{v}\frac{y_{u\rightarrow v}}{\theta_{u\rightarrow
v}}\log\frac{s_{u\rightarrow v}}{\sum_{v^{\prime}}s_{u\rightarrow
v^{\prime}}}-\sum_{v}\frac{y_{u\leftarrow v}}{\theta_{u\rightarrow
v}}\log\frac{s_{u\leftarrow v}}{\sum_{v^{\prime}}s_{u\leftarrow v^{\prime}}},$
$\displaystyle l_{\mathrm{IPW2}}$
$\displaystyle=-\sum_{v}\frac{y_{u\rightarrow v}}{\theta_{u\rightarrow
v}}\log\frac{s_{u\rightarrow v}}{\sum_{v^{\prime}}s_{u\rightarrow
v^{\prime}}}-\sum_{v}\frac{y_{u\leftarrow v}}{\theta_{u\rightarrow
v}\theta_{u\leftarrow v}}\log\frac{s_{u\leftarrow
v}}{\sum_{v^{\prime}}s_{u\leftarrow v^{\prime}}}.$

During the training phase, we saved the best model with the best corresponding
ranking metric best for evaluating the test data. Table 2 summarises the
training loss and validation metric used in each method.

For simplicity, we used the propensity score $\theta_{u\rightarrow
v},\theta_{u\leftarrow v}$ that was employed for the data preparation phase in
the training and validation phases as well.

####  4.1.3. Evaluation

Since we are able to access the ground-truth relevance of each pair, we
evaluated the models by the discounted cumulative gain (DCG) using the true
relevance and the ranking deduced by the score $s_{uv}$. For a given positive
integer $K$, the DCG score for the user $u$ can be calculated as

$DCG@K(u)=\sum_{i=1}^{K}\frac{2^{R_{u\rightarrow v_{i}}(1+R_{u\leftarrow
v_{i}})}}{\log_{2}\left(i+1\right)},$

where $v_{i}$ is the $i$-th element in the ranking induced by the score.

###  4.2. Results & Discussions

We first examined each method’s test data on each fold. Here we fix the
hyperparameter $\eta=0.5$ and evaluate the $DCG@K$ for varying
$K\in\left\\{3,10,20,30\right\\}$.

The results are shown in Table 3. From the table, it is verified that in more
than half of the cases, the proposed method gives the best performance among
the three methods.

Next, we change the hyperparameter $\eta\in\left\\{0.5,0.6,0.8,1.0\right\\}$
and evaluate the test score on the fold-1 data. The results are shown in Table
4. Table 4 shows that the proposed method was relatively robust to the data
generation process.

It is worth noticing that IPW1 gave the worst test scores in many cases. It
appears that in IPW1, the emphasis placed on learning from the proactive
side’s feedback may have inadvertently resulted in a deterioration in the
learning from the reactive side’s feedback.

##  5\. Conclusion

Learning-to-rank (LTR) algorithms have become essential in modern
recommendation systems. However, achieving _unbiased_ LTR is a significant
challenge, particularly when using implicit feedback like clicks, which can be
influenced by factors other than user preferences. Existing unbiased LTR
techniques mainly focus on single-sided markets, which overlook two-sided
markets where mutual preference matching determines final conversion. In such
two-sided market platforms, position biases can emerge from both user sides.
To address such position biases, we proposed a specialized unbiased LTR
framework for two-sided matching platforms. We introduced the two-sided
inverse probability weighting estimator and showed its unbiasedness in two-
sided matching platforms. Our experiments demonstrated that our method
outperformed existing unbiased LTR methods in terms of prediction accuracy and
robustness.

## References

  * (1)
  * Ai et al. (2018) Qingyao Ai, Keping Bi, Cheng Luo, Jiafeng Guo, and W. Bruce Croft. 2018.  Unbiased learning to rank with unbiased propensity estimation. In _Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval_. ACM.  https://doi.org/10.1145/3209978.3209986
  * Bruch (2021) Sebastian Bruch. 2021\.  An alternative cross entropy loss for learning-to-rank. In _Proceedings of the Web Conference 2021_ (Ljubljana, Slovenia). Association for Computing Machinery, New York, NY, USA, 118–126.  https://doi.org/10.1145/3442381.3449794
  * Burges (2010) Christopher JC Burges. 2010\.  From ranknet to lambdarank to lambdamart: An overview.  _Learning_ 11, 23-581 (2010), 81.
  * Cao et al. (2007) Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, and Hang Li. 2007.  Learning to rank: From pairwise approach to listwise approach. In _Proceedings of the 24th International Conference on Machine Learning_ (Corvalis, Oregon, USA). Association for Computing Machinery, New York, NY, USA, 129–136.  https://doi.org/10.1145/1273496.1273513
  * Chen et al. (2023) Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. 2023\.  Bias and debias in recommender system: A survey and future directions.  _ACM Transactions on Information Systems_ 41, 3 (Feb. 2023), 1–39.  https://doi.org/10.1145/3564284
  * Chen et al. (2019) Ruey-Cheng Chen, Qingyao Ai, Gaya Jayasinghe, and W. Bruce Croft. 2019.  Correcting for recency bias in job recommendation. In _Proceedings of the 28th ACM International Conference on Information and Knowledge Management_. ACM.  https://doi.org/10.1145/3357384.3358131
  * Chu and Keerthi (2007) Wei Chu and S. Sathiya Keerthi. 2007.  Support vector ordinal regression.  _Neural Computation_ 19, 3 (March 2007), 792–815.  https://doi.org/10.1162/neco.2007.19.3.792
  * Cooper et al. (1992) William S. Cooper, Fredric C. Gey, and Daniel P. Dabney. 1992\.  Probabilistic retrieval based on staged logistic regression. In _Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '92_. ACM Press.  https://doi.org/10.1145/133160.133199
  * Cossock and Zhang (2006) David Cossock and Tong Zhang. 2006.  Subset ranking using regression.  In _Learning Theory_. Springer Berlin Heidelberg, 605–619.  https://doi.org/10.1007/11776420_44
  * Crammer and Singer (2001) Koby Crammer and Yoram Singer. 2001.  Pranking with ranking. In _Advances in Neural Information Processing Systems_ , T. Dietterich, S. Becker, and Z. Ghahramani (Eds.), Vol. 14. MIT Press.  https://proceedings.neurips.cc/paper/2001/file/5531a5834816222280f20d1ef9e95f69-Paper.pdf
  * Freund et al. (2003) Yoav Freund, Raj Iyer, Robert E. Schapire, and Yoram Singer. 2003\.  An efficient boosting algorithm for combining preferences.  _J. Mach. Learn. Res._ 4, null (dec 2003), 933–969.
  * Fuhr (1989) Norbert Fuhr. 1989\.  Optimum polynomial retrieval functions based on the probability ranking principle.  _ACM Transactions on Information Systems_ 7, 3 (July 1989), 183–204.  https://doi.org/10.1145/65943.65944
  * Guiver and Snelson (2008) John Guiver and Edward Snelson. 2008.  Learning to rank with SoftRank and gaussian processes. In _Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval_. ACM.  https://doi.org/10.1145/1390334.1390380
  * Hu et al. (2019) Ziniu Hu, Yang Wang, Qu Peng, and Hang Li. 2019\.  Unbiased lambdamart: An unbiased pairwise learning-to-rank algorithm. In _The World Wide Web Conference_ (San Francisco, CA, USA). Association for Computing Machinery, New York, NY, USA, 2830–2836.  https://doi.org/10.1145/3308558.3313447
  * Joachims (2006) Thorsten Joachims. 2006\.  Training linear SVMs in linear time. In _Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining_. ACM.  https://doi.org/10.1145/1150402.1150429
  * Joachims et al. (2017) Thorsten Joachims, Adith Swaminathan, and Tobias Schnabel. 2017\.  Unbiased learning-to-rank with biased feedback. In _Proceedings of the Tenth ACM International Conference on Web Search and Data Mining_. ACM.  https://doi.org/10.1145/3018661.3018699
  * Li et al. (2007) Ping Li, Christopher J. C. Burges, and Qiang Wu. 2007.  Mcrank: Learning to rank using multiple classification and gradient boosting. In _Proceedings of the 20th International Conference on Neural Information Processing Systems_ (Vancouver, British Columbia, Canada). Curran Associates Inc., Red Hook, NY, USA, 897–904.
  * Nallapati (2004) Ramesh Nallapati. 2004\.  Discriminative models for information retrieval. In _Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval_. ACM.  https://doi.org/10.1145/1008992.1009006
  * Qin et al. (2009) Tao Qin, Tie-Yan Liu, and Hang Li. 2009.  A general approximation framework for direct optimization of information retrieval measures.  _Information Retrieval_ 13, 4 (Dec. 2009), 375–397.  https://doi.org/10.1007/s10791-009-9124-x
  * Qin et al. (2008) Tao Qin, Xu-Dong Zhang, Ming-Feng Tsai, De-Sheng Wang, Tie-Yan Liu, and Hang Li. 2008\.  Query-level loss functions for information retrieval.  _Information Processing and Management_ 44, 2 (March 2008), 838–855.  https://doi.org/10.1016/j.ipm.2007.07.016
  * Saito and Joachims (2022) Yuta Saito and Thorsten Joachims. 2022.  Fair ranking as fair division: Impact-based individual fairness in ranking. In _Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining_. 1514–1524.
  * Shen and Joshi (2005) Libin Shen and Aravind K. Joshi. 2005.  Ranking and reranking with perceptron.  _Machine Learning_ 60, 1-3 (June 2005), 73–96.  https://doi.org/10.1007/s10994-005-0918-9
  * Su et al. (2022) Yi Su, Magd Bayoumi, and Thorsten Joachims. 2022.  Optimizing rankings for recommendation in matching markets. In _Proceedings of the ACM Web Conference 2022_ (Virtual Event, Lyon, France). Association for Computing Machinery, New York, NY, USA, 328–338.  https://doi.org/10.1145/3485447.3511961
  * Taylor et al. (2008) Michael Taylor, John Guiver, Stephen Robertson, and Tom Minka. 2008.  SoftRank. In _Proceedings of the international conference on Web search and web data mining - WSDM '08_. ACM Press.  https://doi.org/10.1145/1341531.1341544
  * Tu et al. (2014) Kun Tu, Bruno Ribeiro, David Jensen, Don Towsley, Benyuan Liu, Hua Jiang, and Xiaodong Wang. 2014.  Online dating recommendations. In _Proceedings of the 23rd International Conference on World Wide Web_. ACM.  https://doi.org/10.1145/2567948.2579240
  * Volkovs and Zemel (2009) Maksims N. Volkovs and Richard S. Zemel. 2009.  BoltzRank. In _Proceedings of the 26th Annual International Conference on Machine Learning_. ACM.  https://doi.org/10.1145/1553374.1553513
  * Wang et al. (2016) Xuanhui Wang, Michael Bendersky, Donald Metzler, and Marc Najork. 2016.  Learning to rank with selection bias in personal search. In _Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval_. ACM.  https://doi.org/10.1145/2911451.2911537
  * Wang et al. (2018) Xuanhui Wang, Nadav Golbandi, Michael Bendersky, Donald Metzler, and Marc Najork. 2018\.  Position bias estimation for unbiased learning to rank in personal search. In _Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining_ (Marina Del Rey, CA, USA). Association for Computing Machinery, New York, NY, USA, 610–618.  https://doi.org/10.1145/3159652.3159732
  * Wen et al. (2019) Hongyi Wen, Longqi Yang, and Deborah Estrin. 2019.  Leveraging post-click feedback for content recommendations. In _Proceedings of the 13th ACM Conference on Recommender Systems_. ACM.  https://doi.org/10.1145/3298689.3347037
  * Wu et al. (2009) Qiang Wu, Christopher J. C. Burges, Krysta M. Svore, and Jianfeng Gao. 2009.  Adapting boosting for information retrieval measures.  _Information Retrieval_ 13, 3 (Sept. 2009), 254–270.  https://doi.org/10.1007/s10791-009-9112-1
  * Xia et al. (2008) Fen Xia, Tie-Yan Liu, Jue Wang, Wensheng Zhang, and Hang Li. 2008.  Listwise approach to learning to rank. In _Proceedings of the 25th international conference on Machine learning_. ACM Press.  https://doi.org/10.1145/1390156.1390306
  * Xu and Li (2007) Jun Xu and Hang Li. 2007\.  AdaRank. In _Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval_. ACM.  https://doi.org/10.1145/1277741.1277809
  * Yang et al. (2022) Chen Yang, Yupeng Hou, Yang Song, Tao Zhang, Ji-Rong Wen, and Wayne Xin Zhao. 2022\.  Modeling two-way selection preference for person-job fit. In _Proceedings of the 16th ACM Conference on Recommender Systems_ (Seattle, WA, USA). Association for Computing Machinery, New York, NY, USA, 102–112.  https://doi.org/10.1145/3523227.3546752
  * Yue et al. (2007) Yisong Yue, Thomas Finley, Filip Radlinski, and Thorsten Joachims. 2007.  A support vector method for optimizing average precision. In _Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval_. ACM.  https://doi.org/10.1145/1277741.1277790


# An Exact Formula for Quantum Entropy Production along Quantum Trajectories

John E. Gough<EMAIL_ADDRESS>Aberystwyth University, SY23 3BZ, Wales, United
Kingdom  Nina H. Amini<EMAIL_ADDRESS>Laboratoire des
Signaux et Systémes (L2S), CNRS-CentraleSupélec-Université Paris-Sud,
Université Paris-Saclay, 3, Rue Joliot Curie, 91190, Gif-sur-Yvette, France.

###### Abstract

We give an exact formula for the rate of change of the von Neumann entropy for
the conditional state of a quantum system undergoing continuous measurement.
Here we employ Paycha’s Formula Paycha  which gives the noncommutative Taylor
series development.

##  I Introduction

The acquisition of information about a system from observations leads to a
conditioning of prior probability distributions. In continuous time, the
update of the state in this way is known as filtering and it is possible to
study the corresponding entropy and make connections with thermodynamic
concepts, see for instance Mitter and Newton Mitter_Newton . here we examine
the quantum analogue. The dynamical equation for the conditional density
matrix of a quantum system undergoing continuous measurement takes the form
Belavkin ; BvHJ

$\displaystyle
d\widehat{\rho_{t}}=\mathcal{L}^{\star}(\widehat{\rho}_{t})\,dt+\sqrt{\eta}\,\big{(}L\widehat{\rho}_{t}+\widehat{\rho}_{t}L^{\ast}-\lambda_{t}\widehat{\rho}_{t}\big{)}\,dI(t),$
(1)

where $\mathcal{L}(\cdot)$ is a generator of Lindblad type,
$\mathcal{L}(\cdot)=-i[\cdot,H]+\sum_{k}\frac{1}{2}[L_{k}^{\ast},\cdot]L_{k}+\frac{1}{2}L_{k}^{\ast}[\cdot,L_{k}]$,
and $\mathcal{L}(\cdot)^{\star}$ is its adjoint (i.e.,
$\mathrm{tr}\\{\rho\,\mathcal{L}(X)\\}\equiv\mathrm{tr}\\{\mathcal{L}^{\star}(\rho)\,X\\}$
for all trace-class $\rho$ and bounded $X$), $H$ is the Hamiltonian of the
system, $L$ is the coupling operator of the system to the monitored field (the
collapse operator). The term $I(t)$ is the innovations (statistically a Wiener
process) and $\lambda_{t}=\mathrm{tr}\\{\widehat{\rho}_{t}\,(L+L^{\ast})\\}$.

The factor $\eta\in[0,1]$ is the _efficiency_ of the homodyne detector:
$\eta=1$ is perfect detection, while $\eta=0$ effectively means that we are
left with a dissipative dynamics described by a standard master equation.

The conditional state $\widehat{\rho}_{t}$ is a stochastic process adapted to
the filtration generated by the measurements (or equivalently, the innovations
process). Our interest shall be in the associated von Neumann entropies
$\mathsf{S}\big{(}\widehat{\rho}_{t}\big{)}=-\mathrm{tr}_{S}\big{(}\widehat{\rho}_{t}\ln\widehat{\rho}_{t}\big{)}$
and this will likewise be an adapted stochastic process.

We recall that a state state is pure if and only if it is a rank-one
projection; otherwise it is mixed. A mixed state is deemed to be faithful if
its density matrix $\rho$ is strictly positive. In such cases we may define
its inverse $\frac{1}{\rho}$.

Our main result is an exact equation for the rate of change of the von Neumann
entropy for the conditioned state.

###### Theorem 1

On the assumption that the conditional state remains faithful, the von Neumann
entropy for the conditioned state satisfies satisfies the equation

$\displaystyle
d\mathsf{S}\big{(}\widehat{\rho}_{t}\big{)}=-\mathrm{tr}_{s}\bigg{(}\widehat{\rho}_{t}\,\mathcal{L}\ln\widehat{\rho}_{t}\bigg{)}\,dt+\Sigma(\widehat{\rho}_{t})\,dt-\mathrm{tr}_{s}\bigg{(}\big{(}L\widehat{\rho}_{t}+\widehat{\rho}_{t}L^{\ast}-\lambda(t)\widehat{\rho}_{t}\big{)}\,\ln\widehat{\rho}_{t}\bigg{)}\,dI.$
(2)

where

$\displaystyle\Sigma(\widehat{\rho})=\eta\sum_{k_{1},k_{2}=0}^{\infty}\dfrac{1}{(k_{1}+1)(k_{1}+k_{2}+2)}\binom{k_{1}+k_{2}}{k_{1}}\,\,\mathrm{tr}_{s}\bigg{\\{}\big{(}-\frac{1}{\widehat{\rho}}\big{)}^{k_{1}+k_{2}+1}\mathrm{ad}^{k_{1}}_{\widehat{\rho}}(L\widehat{\rho}+\widehat{\rho}L^{\ast})\,\mathrm{ad}^{k_{2}}_{\widehat{\rho}}(L\widehat{\rho}+\widehat{\rho}L^{\ast})\bigg{\\}}.$
(3)

We arrive at the form (3) using a noncommutative Taylor series expansion
formula due to S. Paycha Paycha .

###  I.1 Motivation

Let us first remark that the stochastic master equation 1 admits a pure state
solution provided the following conditions are satisfied: the system is
initially separated from its environment and in a pure state; the environment
consists of just the detector (in which case the Lindblad generator is
precisely
$\mathcal{L}(\cdot)=-i[\cdot,H]+\frac{1}{2}[L^{\ast},\cdot]L+\frac{1}{2}L^{\ast}[\cdot,L]$
where $L$ is the collapse operator); and we have perfect efficiency
$(\eta=1)$. Note that this situation generalizes to more than one monitored
output field in the obvious way.

In this special situations, we have that
$\widehat{\rho}_{t}\equiv|\psi_{t}\rangle\langle\psi_{t}|$ where the pure
stochastic state satisfies (see the contribution of J. Kupsch to the standard
reference book Decoherence )

$\displaystyle
d|\psi_{t}\rangle=-\bigg{(}iH+\frac{1}{2}(L^{\ast}L-\lambda_{t}L+\frac{1}{4}\lambda_{t}^{2})\bigg{)}|\psi_{t}\rangle\,dt+(L-\frac{1}{2}\lambda_{t})\,|\psi_{t}\rangle\,dI(t).$
(4)

The (conditional) von Neumann entropy, of course, vanishes in these cases.

An interesting special situation is where the collapse operator $L$ is self-
adjoint. In particular, the question of whether the stochastic states converge
to a pure eigenstates of the collapse operator become natural ABBH . For
instance, in SvHM  an ensemble of $N$ spin-1/2 particles was effectively
studied through their total spin vector $\mathbf{J}$ and one performed a
continuous homodyne monitoring with collapse operator $M\,J_{z}$ for some
coupling constant $M$. Taking $H\equiv 0$ for convenience and assuming perfect
detection efficiency, the conditional state will almost surely converge into
one of the eigenstates of $J_{z}$, say $|m\rangle$, with probability
$p_{m}=\langle m|\rho_{0}|m\rangle$ in accordance with the Born rule. The
conditional entropy should then converge to zero while the unconditioned
should of course converge to $-\sum_{m}p_{m}\ln p_{m}$. Rather trivially,
Theorem 1 would tell us that the rate of change of the conditional entropy
would vanish for $\widehat{\rho}$ diagonal in the basis of the collapse
operator. (In SvHM , the problem is made more interesting by applying a
feedback control through a Hamiltonian $\gamma\,b(t)J_{y}$ where $b(t)$
depends on the measurement record.)

A related problem arises in the theory of quantum feedback networks GouJam09a
GouJam09b . One may model a quantum system with proportional feedback of
homodyne measurement in a fairly straightforward way using the conditional
dynamics Wiseman . However, since one only needs to know that the measurement
is being performed and the output fedback without actually knowing the
measurement result, one can alternatively describe the average result
coherently - see GouJam09b  section IV B. Here one is choosing to ignore the
measurement record but the fact that a measurement is being made leads to back
action - resulting in a unitary model for the system and the detector. This of
course changes if one tried to look inside to see what the detector actually
detects. An example where this may be of importance is the proposal for
autonomous error-correction of a qubit using quantum switches KNPH . Here
several switches are coupled to the qubit in a coherent quantum feedback
network: again, technically there is no measurement but one could potentially
consider inserting detectors so the the qubit and the switches are driven by
each others homodyne outputs. The set up in KNPH  is specifically designed so
that the switches act as dissipators whenever a particular syndrome error
occurs to the qubit. While the set up is completely coherent (therefore
unitary), one may naturally ask if the energy dissipated by an error
correction could leak out as heat thereby telling us that a error-correction
had been made. In this example, the switches are acting as regulators giving a
quantum analogue of the flyball governor. Related questions about quantum
Maxwell’s demon in the setting of superconducting qubits can be found in the
work Cottet, et al. QMD

##  II Discrete Time Models

Our proof of Theorem 1 relies on formulating the problem in discrete time and
taking the continuum limit. We begin with an exposition of the discrete time
situation.

###  II.1 State Evolution

####  II.1.1 Unconditional State Evolution

We consider a discrete time open quantum dynamics with fixed time step
$\tau>0$. Our system has Hilbert space $\mathfrak{h}_{s}$ and at each time
$t=n\tau$ it comes in contact with a fresh ancillary system. The ancillas have
a Hilbert space which is a copy of a fixed Hilbert space $\mathfrak{h}_{a}$
with a state which is a copy of a fixed state $\sigma$. The system state
updates as

$\mathcal{V}_{s}\,\rho=\mathrm{tr}_{a}\bigg{(}V(\rho\otimes\sigma)V^{\ast}\bigg{)}$
(5)

where $V$ is a (copy of a) unitary operator on
$\mathfrak{h}\otimes\mathfrak{h}_{a}$ and we have the partial trace over the
ancilla. We note that $\mathcal{V}_{s}$ is a TPCP map.

If we start with system state $\rho_{0}$ the we obtain the sequence of states
$(\rho_{n})_{n}$ where $\rho_{n}=\mathcal{V}_{s}\rho_{n-1}$. That is
$\rho_{n}=\mathcal{V}_{s}^{n}\rho_{0}$.

####  II.1.2 Conditional State Evolution

At each time $t=n\tau$ we can measure an observable $Y_{n}$ of the $n$th
ancilla. We assume that $Y_{n}$ is a copy of a fixed observable $Y$ described
by measurement operators $\\{M_{y}\\}$ with $M_{y}^{\ast}M_{y}$ corresponding
to the effect that $Y$ is measured to have value $y$: Note that
$\sum_{y}M_{y}^{\ast}M_{y}=I_{a}$.

If the state of the system at a particular time is $\rho$ then we set

$\mathcal{M}_{y}\rho=\mathrm{tr}_{a}\bigg{(}M_{y}V(\rho\otimes\sigma)V^{\ast}M_{y}^{\ast}\bigg{)}.$
(6)

It follows that $\sum_{y}\mathcal{M}_{y}\equiv\mathcal{V}_{s}$. In particular,
it is not a trace preserving map and we instead have

$\displaystyle
p(y)\triangleq\mathrm{tr}_{s}(\mathcal{M}_{y}\rho)=\mathrm{tr}_{s+a}\bigg{(}M_{y}V(\rho\otimes\sigma)V^{\ast}M_{y}^{\ast}\bigg{)}$
(7)

and we interpret $p(Y=y)$ as the probability that we measure $Y$ to take the
value $y$. The conditioned state of the system immediately after $Y$ is
measured to be $y$ is

$\displaystyle\widehat{\rho}(y)=\dfrac{1}{p(y)}\mathcal{M}_{y}\,\rho.$ (8)

One notes that $\sum_{y}\widehat{\rho}(y)\,p(y)=\mathcal{V}_{s}\rho$.

####  II.1.3 Repeated Measurements

The superoperators $\mathcal{V}_{s}$ and $\mathcal{M}_{y}$ describe the one-
step processes. We now consider the situation where we initialize our system
in state $\rho_{0}$ and at times $m\tau\,(m=1,\cdots n)$ measure an observable
$Y_{m}$ which is a copy of the fixed $Y$ above, but with a fresh ancilla
space. The measurement record will be a sequence of values
$\mathbf{y}=(y_{1},y_{2},\cdots)$. We shall use the notation $\mathbf{y}[m]$
for the subsequence $(y_{1},\cdots,y_{m})$ for finite $m$.

For $n\geq m$, the state of the system at time $t=n\tau$ given the measurement
record $\mathbf{y}[m]$ is given by

$\displaystyle\widehat{\rho}_{n|m}(y_{1},\cdots,y_{m})\triangleq\dfrac{1}{p(y_{1},\cdots,y_{m})}\mathcal{V}_{s}^{n-m}\circ\mathcal{M}_{y_{m}}\circ\cdots\circ\mathcal{M}_{y_{1}}\rho_{0}.$
(9)

where the probability for the record is

$\displaystyle
p(y_{1},\cdots,y_{m})=\mathrm{tr}_{s}\bigg{(}\mathcal{M}_{y_{m}}\circ\cdots\circ\mathcal{M}_{y_{1}}\rho_{0}\bigg{)}.$
(10)

For convenience, we will write $\widehat{\rho}_{n|m}(y_{1},\cdots,y_{m})$ as
$\widehat{\rho}_{n|m}(\mathbf{y})$ and $p(y_{1},\cdots,y_{m})$ as
$p_{m}(\mathbf{y})$ even though the dependence is on $\mathbf{y}[m]$. We also
set

$\displaystyle\widehat{\rho}_{n}(\mathbf{y})\triangleq\widehat{\rho}_{n|n}(\mathbf{y})$
(11)

which is the estimate of the state at time $t=n\tau$ given all measurements up
to that time. This is also called the filtered quantum state at time $t=\tau
n$.

We remark that we may also write

$\displaystyle\widehat{\rho}_{n|m}(\mathbf{y})=\dfrac{1}{p_{m}(\mathbf{y})}\mathrm{tr}_{a^{\otimes
n}}\bigg{(}V_{n}\cdots V_{m+1}M_{y_{m}}V_{m}\cdots
M_{y_{1}}V_{1}(\rho_{0}\otimes\sigma^{\otimes
n})\,V_{1}^{\ast}M_{y_{1}}^{\ast}\cdots
V_{m}^{\ast}M_{y_{m}}^{\ast}V_{m+1}^{\ast}\cdots V_{n}^{\ast}\bigg{)}.$ (12)

Here the $j$th measurement operator $M_{y_{j}}$ acts nontrivially on the $j$th
ancilla space, while $V_{k}$ will couple the $k$th ancilla to the system and
act trivially on all other factors. This implies that $[V_{k},M_{y_{j}}]=0$
when $k>j$. This allows us to rewrite this as

$\displaystyle\widehat{\rho}_{n|m}(\mathbf{y})=\dfrac{1}{p_{m}(\mathbf{y})}\mathrm{tr}_{a^{\otimes
n}}\bigg{(}M_{y_{m}}\cdots M_{y_{1}}U_{n}(\rho_{0}\otimes\sigma^{\otimes
n})U_{n}^{\ast}M_{y_{1}}^{\ast}\cdots M_{y_{m}}^{\ast}\bigg{)},$ (13)

where $U_{n}=V_{n}\cdots V_{1}$ (order is important!). We note that
$[M_{y_{j}},M_{y_{k}}]=0$ for $j\neq k$.

###  II.2 Generic Model

For definiteness, we take $\mathfrak{h}_{a}=\mathbb{C}^{2}$ with basis
$|0\rangle=\left[\begin{array}[]{c}0\\\
1\end{array}\right],\,|1\rangle=\left[\begin{array}[]{c}1\\\
0\end{array}\right]$. This is a qubit probe and we take the probe state before
measurement to be $\sigma=|0\rangle\langle
0|\equiv\left[\begin{array}[]{cc}0&0\\\ 0&1\end{array}\right]$.

It is convenient to represent operators on
$\mathfrak{h}_{s}\otimes\mathfrak{h}_{a}$ as $2\times 2$ matrices whose
entries are operators on $\mathfrak{h}_{s}$. In other words,

$\displaystyle A\otimes|1\rangle\langle 1|+B\otimes|1\rangle\langle
0|+C\otimes|0\rangle\langle 1|+D\otimes|0\rangle\langle
0|\equiv\left[\begin{array}[]{cc}A&B\\\ C&D\end{array}\right].$ (16)

We shall consider a one-step unitary of the form

$\displaystyle V=\exp\big{\\{}-i\tau\,H\otimes
I_{a}\,+\sqrt{\tau}L\otimes|1\rangle\langle
0|-\sqrt{\tau}L^{\ast}|0\rangle\langle 1|\big{\\}}$

where $H=H^{\ast}$ and $L$ are bounded system operators. For our purposes, we
will be interested in the small time step limit, $\tau<<1$, and to this end it
suffices to work with the expansion

$\displaystyle V\approx\left[\begin{array}[]{cc}I_{s}+\tau K&\sqrt{\tau}L\\\
-\sqrt{\tau}L^{\ast}&I_{s}+\tau K\end{array}\right],$ (20)

where $K=-\frac{1}{2}L^{\ast}L-iH$. We find that, for arbitrary system state
$\rho$,

$\displaystyle V(\rho\otimes\sigma)V^{\ast}\equiv
V\left[\begin{array}[]{cc}0&0\\\
0&\rho\end{array}\right]V^{\ast}\approx\left[\begin{array}[]{cc}\tau\,L\rho
L^{\ast}&\sqrt{\tau}\,L\rho\\\ \sqrt{\tau}\rho L^{\ast}&\rho+\tau\,(K\rho+\rho
K^{\ast})\end{array}\right].$ (25)

The unconditional system evolution is then

$\displaystyle\mathcal{V}_{s}\rho$ $\displaystyle=$
$\displaystyle\mathrm{tr}_{a}\bigg{(}V(\rho\otimes\sigma)V^{\ast}\bigg{)}\approx\rho+\tau(L\rho
L^{\ast}+K\rho+\rho K^{\ast}).$ (26)

Here we introduce the Lindblad generator

$\displaystyle\mathcal{L}X=L^{\ast}XL-\frac{1}{2}L^{\ast}LX-\frac{1}{2}XL^{\ast}L-i[X,H]$
(27)

and its adjoint $\mathcal{L}^{\star}\rho=L\rho L^{\ast}+K\rho+\rho K^{\ast}$
defined through the duality
$\mathrm{tr}_{s}(\rho\,\mathcal{L}X)=\mathrm{tr}_{s}(\mathcal{L}^{\star}\rho\,X)$.
With this, we may write

$\displaystyle\mathcal{V}_{s}\rho\approx\rho+\tau\,\mathcal{L}^{\star}\rho.$
(28)

The sequence $(\rho_{n})$ generated by choosing $\rho_{0}$ and iterating
according to $\rho_{n}=\mathcal{V}_{s}\rho_{n-1}$ can be seen as the
approximation with time step $\tau$ to the continuous time $\rho(t)$
satisfying the master equation $\dfrac{d\rho}{dt}=\mathcal{L}^{\star}\rho$.

We now measure the observable $Z=\left[\begin{array}[]{cc}0&1\\\
1&0\end{array}\right]$ which has eigenstates
$|\pm\rangle=\dfrac{1}{\sqrt{2}}\left[\begin{array}[]{c}1\\\ \pm
1\end{array}\right]$. Equivalently, we have the pair of measurement operators

$\displaystyle
M_{\pm}=|\pm\rangle\langle\pm|\equiv\dfrac{1}{2}\left[\begin{array}[]{cc}1&\pm
1\\\ \pm 1&1\end{array}\right]$ (31)

and the associated maps are

$\displaystyle\mathcal{M}_{\pm}\rho\approx\frac{1}{2}\rho\pm\frac{1}{2}\sqrt{\tau}(L\rho+\rho
L^{\ast})$ (32)

with probabilities $p(\pm
1)\approx\frac{1}{2}\pm\frac{1}{2}\sqrt{\tau}\mathrm{tr}_{s}(L\rho+\rho
L^{\ast})$. In the Heisenberg picture, it is better to think of the measured
observable as $Y=\sqrt{\tau}\,V(I_{s}\otimes Z)V^{\ast}$ with
$y=\pm\sqrt{\tau}$ as the two outcomes.

The measurements may be repeated leading to a measurement sequence
$\mathbf{y}=(y_{1},y_{2},\cdots)$, say. The filtered quantum state
$\widehat{\rho}_{n}(\mathbf{y})=\dfrac{1}{p_{n}(\mathbf{y})}\mathcal{M}_{y_{n}}\circ\cdots\circ\mathcal{M}_{y_{1}}\rho_{0}$,
satisfies the following difference equation (expanded to leading order in
$\tau$)

$\displaystyle\widehat{\rho}_{n}=\widehat{\rho}_{n-1}+\mathcal{L}^{\star}\widehat{\rho}_{n-1}\tau+\bigg{(}L\widehat{\rho}_{n-1}+\widehat{\rho}_{n-1}L^{\ast}-\lambda_{n-1}\widehat{\rho}_{n-1}\bigg{)}\Delta
I_{n}$ (33)

where $\lambda_{n-1}=\mathrm{tr}_{s}\big{(}\rho_{n-1}\,(L+L^{\ast})\big{)}$
and

$\displaystyle\Delta I_{n}=y_{n}-y_{n-1}-\lambda_{n-1}\,\tau.$ (34)

The terms $\Delta I_{n}$ are called the innovations and have mean zero - in
the continuous limit there will be a central limit effect which results in
these being increments of a Wiener process: for our purposes, it is enough to
note that to the appropriate order

$\displaystyle(\Delta I_{n})^{2}\approx\tau.$ (35)

For a precise statement of this approximation, see Section 5.2 of BvHJ .

The continuous limit for (33) takes the form

$\displaystyle
d\widehat{\rho}(t)=\mathcal{L}^{\star}\widehat{\rho}(t)\,dt+\big{(}L\widehat{\rho}(t)+\widehat{\rho}(t)L-\lambda(t)\widehat{\rho}(t)\big{)}\,dI(t).$
(36)

where now
$\lambda(t)=\mathrm{tr}_{s}\big{(}\widehat{\rho}(t)\,(L+L^{\ast})\big{)}$ and
$I(t)$ is the limit Wiener process (the innovations process). This is the well
known form of the stochastic master equation for a homodyne measurement.

##  III Entropy Production

Let $\rho$ be a quantum state (density matrix) which we write as
$\rho=e^{-\Phi}$. With the standard notation $\langle
A\rangle_{\rho}=\mathrm{tr}\big{(}\rho A\big{)}$, the von Neumann entropy of a
system in state $\rho$ is given by

$\displaystyle\mathsf{S}(\rho)\triangleq\langle\Phi\rangle_{\rho}=-\mathrm{tr}(\rho\ln\rho).$
(37)

The Kullback-Liebler divergence, or relative entropy, is
$\mathsf{D}(\rho||\sigma)\triangleq\mathrm{tr}\big{(}\rho\ln\rho-\rho\ln\sigma\big{)}$.
The main properties that we shall use are: the Klein’s inequality which states
$\mathsf{D}(\rho||\sigma)\geq 0$ with equality if and only if $\rho=\sigma$;
and the quantum data processing inequality which states that
$\mathsf{D}(\mathcal{N}\rho||\mathcal{N}\sigma)\leq\mathsf{D}(\rho||\sigma)$
for any trace-preserving CP map $\mathcal{N}$.

The unconditional entropy of the system at time $t=n\tau$ is

$\displaystyle S_{n}=\mathsf{S}(\rho_{n}).$ (38)

The conditional entropy at time $t=n\tau$ given the measurement record
$\mathbf{y}[m]$ for $m\leq n$ is
$\mathsf{S}\big{(}\widehat{\rho}_{n|m}(\mathbf{y})\big{)}$, and we denote its
average conditional entropy as

$\displaystyle\overline{S}_{n|m}\triangleq\sum_{y_{1},\cdots,y_{m}}\mathsf{S}\big{(}\widehat{\rho}_{n|m}(\mathbf{y})\big{)}\,p_{m}(\mathbf{y}).$
(39)

###### Definition 2

The Holevo information at time $t=n\tau$ based on the first $m$ measurements
is Holevo

$\displaystyle H_{n|m}\triangleq S_{n}-\overline{S}_{n|m}.$ (40)

The special case $H_{n}=H_{n|n}$ is just referred to as the Holevo information
at time $t=n\tau$.

###### Proposition 3

The Holevo information may be written as

$\displaystyle
H_{n|m}=\sum_{y_{1},\cdots,y_{m}}\mathsf{D}(\widehat{\rho}_{n|m}(\mathbf{y})\big{)}||\rho_{n})\,p_{m}(\mathbf{y}),$
(41)

and, in particular, satisfies $H_{n|m}\geq 0$.

Proof. We first observe that we may write

$\displaystyle\sum_{y_{1},\cdots,y_{m}}\mathsf{D}\big{(}\widehat{\rho}_{n|m}(\mathbf{y})||\rho_{n}\big{)}\,p_{m}(\mathbf{y})$
$\displaystyle=$
$\displaystyle\sum_{y_{1},\cdots,y_{m}}\bigg{\\{}\mathrm{tr}\bigg{(}\widehat{\rho}_{n|m}(\mathbf{y})\,\ln\widehat{\rho}_{n|m}(\mathbf{y})\bigg{)}-\mathrm{tr}\bigg{(}\widehat{\rho}_{n|m}(\mathbf{y})\,\ln\rho_{n}\bigg{)}\bigg{\\}}\,p_{m}(\mathbf{y})$
$\displaystyle=$
$\displaystyle-\overline{S}_{n|m}-\sum_{y_{1},\cdots,y_{m}}\mathrm{tr}\bigg{(}\mathcal{V}_{s}^{n-m}\circ\mathcal{M}_{y_{m}}\circ\cdots\circ\mathcal{M}_{y_{1}}\rho_{0}\,\ln\rho_{n}\bigg{)}$
$\displaystyle=$
$\displaystyle-\overline{S}_{n|m}-\mathrm{tr}\bigg{(}\mathcal{V}_{s}^{n}\rho_{0}\,\ln\rho_{n}\bigg{)}$
$\displaystyle=$
$\displaystyle-\overline{S}_{n|m}-\mathrm{tr}\bigg{(}\rho_{n}\,\ln\rho_{n}\bigg{)}\equiv
H_{n|m}.$

This establishes (41). Non-negativity then follows from Klein’s inequality.

###### Definition 4

The incremental Holevo information is defined to be $\Delta
H_{n}=H_{n}-H_{n-1}$.

The next result is established by Landi, Paternostro, Belenchia in LPB_22 .

###### Proposition 5

The incremental Holevo information can be decomposed as

$\displaystyle\Delta H_{n}=G_{n}-L_{n}$ (42)

where

$\displaystyle G_{n}$ $\displaystyle\triangleq$ $\displaystyle
H_{n|n}-H_{n|n-1},$ (43) $\displaystyle L_{n}$ $\displaystyle\triangleq$
$\displaystyle H_{n-1|n-1}-H_{n|n-1}.$ (44)

In particular, $G_{n}\equiv\overline{S}_{n|n-1}-\overline{S}_{n|n}$ while
$L_{n}$ is strictly positive.

Proof. We sketch the argument given in LPB_22 . The decomposition (42) follows
from subtracting (44) from (43). Likewise, the form
$G_{n}\equiv\overline{S}_{n|n-1}-\overline{S}_{n|n}$ follows immediately from
the definition $H_{n|m}\triangleq S_{n}-\overline{S}_{n|m}$.

This leaves the positivity of $L_{n}$ to prove. We note that

$\displaystyle L_{n}$ $\displaystyle=$
$\displaystyle\sum_{y_{1},\cdots,y_{n-1}}\bigg{\\{}\mathsf{D}\big{(}\widehat{\rho}_{n-1|n-1}(\mathbf{y})||\rho_{n-1}\big{)}-\mathsf{D}\big{(}\widehat{\rho}_{n|n-1}(\mathbf{y})||\rho_{n}\big{)}\bigg{\\}}\,p_{n-1}(\mathbf{y}),$

however, using the quantum data processing inequality, we find

$\displaystyle\mathsf{D}\big{(}\widehat{\rho}_{n|n-1}(\mathbf{y})||\rho_{n}\big{)}$
$\displaystyle=$
$\displaystyle\mathsf{D}\big{(}\mathcal{V}_{s}\widehat{\rho}_{n-1|n-1}(\mathbf{y})||\mathcal{V}_{s}\rho_{n-1}\big{)}$
$\displaystyle\leq$
$\displaystyle\mathsf{D}\big{(}\widehat{\rho}_{n-1|n-1}(\mathbf{y})||\rho_{n-1}\big{)}$

which leads to the desired result.

###  III.1 Entropy Inequalities

Let $\rho_{ab}$ be a state with underlying Hilbert space
$\mathfrak{h}_{a}\otimes\mathfrak{h}_{b}$ and let $\rho_{a}$ and $\rho_{b}$ be
its marginals with respect to this tensor decomposition, i.e.,
$\rho_{a}=\mathrm{tr}_{b}(\rho)$ and $\rho_{b}=\mathrm{tr}_{a}(\rho)$. The
mutual information is defined to be

$\displaystyle\mathsf{I}_{a:b}(\rho)$ $\displaystyle\triangleq$
$\displaystyle\mathsf{D}(\rho||\rho_{a}\otimes\rho_{b})$ (45)
$\displaystyle\equiv$
$\displaystyle\mathsf{S}(\rho_{a})+\mathsf{S}(\rho_{b})-\mathsf{S}(\rho).$

Clearly the mutual information is never negative.

We now return to our iterative interaction model. Before the system and $n$th
ancilla interact there joint state is $\rho_{n-1}\otimes\sigma$; afterwards,
this is $V(\rho_{n-1}\otimes\sigma)V^{\ast}$. The marginals of the latter,
with respect to the system and ancilla tensor decomposition, are
$\mathcal{V}_{s}(\rho_{n-1})=\rho_{n}$ and

$\displaystyle\mathcal{F}_{n}(\sigma)\triangleq\mathrm{tr}_{s}\big{(}V(\rho_{n-1}\otimes\sigma)V^{\ast}\big{)},$
(46)

respectively.

###### Proposition 6

The entropy increment $\Delta S_{n}=S_{n}-S_{n-1}$ satisfies the inequalties

$\displaystyle\Delta
S_{n}+\mathsf{S}(\mathcal{F}_{n}\sigma)-\mathsf{S}(\sigma)\geq 0,$ (47)

and

$\displaystyle\Delta
S_{n}+\mathrm{tr}\big{(}(\sigma-\mathcal{F}_{n}\sigma)\,\ln\sigma\big{)}\geq
0.$ (48)

Proof. The mutual information is therefore

$\displaystyle\mathsf{I}_{s:a}\big{(}V(\rho_{n-1}\otimes\sigma)V^{\ast}\big{)}$
$\displaystyle=$
$\displaystyle\mathsf{S}(\rho_{n})+\mathsf{S}(\mathcal{F}_{n}\sigma)-\mathsf{S}\big{(}V(\rho_{n-1}\otimes\sigma)V^{\ast}\big{)}$
(49) $\displaystyle=$
$\displaystyle\mathsf{S}(\rho_{n})+\mathsf{S}(\mathcal{F}_{n}\sigma)-\mathsf{S}(\rho_{n-1})-\mathsf{S}(\sigma).$

Non-negativity of the mutual information then implies the first inequality.

Noting that
$0\leq\mathsf{D}\big{(}\mathcal{F}_{n}\sigma||\sigma\big{)}=-\mathsf{S}(\mathcal{F}_{n}\sigma)-\mathrm{tr}\big{(}\mathcal{F}_{n}\sigma\,\ln\sigma\big{)}$
and combining this with the first inequality, we arrive at the second
inequality.

##  IV Explicit Formula

We will make extensive use of the following formal noncommutative Taylor
series due to S. Paycha Paycha .

######  Proposition 7 (Paycha’s Formula)

Let $f$ be analytic in a domain of the complex plane with $k$-th derivative
$f^{(k)}$. Then for $\rho,\varepsilon$ noncommutative operators we have

$\displaystyle f(\rho+\varepsilon)=\sum_{n=0}^{\infty}f(n,\rho,\varepsilon),$
(50)

where the $n$th term in the Paycha expansion is

$\displaystyle f(n,\rho,\varepsilon)=\sum_{k_{1},\cdots
k_{m}=0}^{\infty}\dfrac{f^{(k_{1}+\cdots+k_{n}+n)}(\rho)}{(k_{1}+1)\cdots(k_{1}+\cdots+k_{n}+n)}\frac{{\mathrm{ad}^{k_{1}}_{\rho}(\varepsilon)}}{k_{1}!}\cdots\frac{{\mathrm{ad}^{k_{n}}_{\rho}(\varepsilon)}}{k_{n}!}$
(51)

where $\mathrm{ad}_{\rho}=[\rho,\cdot]$.

We note that $f(0,\rho,\varepsilon)\equiv f(\rho)$ while

$\displaystyle
f(1,\rho,\varepsilon)=\sum_{k_{1}}^{\infty}\dfrac{f^{(k_{1}+1)}(\rho)}{(k_{1}+1)!}\mathrm{ad}^{k_{1}}_{\rho}(\varepsilon)$
(52)

and

$\displaystyle
f(2,\rho,\varepsilon)=\sum_{k_{1},k_{2}=0}^{\infty}\dfrac{f^{(k_{1}+k_{2}+2)}(\rho)}{(k_{1}+1)(k_{1}+k_{2}+2)}\frac{{\mathrm{ad}^{k_{1}}_{\rho}(\varepsilon)}}{k_{1}!}\frac{{\mathrm{ad}^{k_{2}}_{\rho}(\varepsilon)}}{k_{2}!}.$
(53)

Our interest will be restricted to quantum diffusions appearing in homodyne
measurement so we will not require the $n>2$ terms. However, this would not be
the case for Poissonian processes encountered in photon number counting
problems.

###  IV.1 The Unconditional Entropy

As a first example of the noncommutative Taylor’s expansion, we consider the
unconditional entropy $S_{n}$. Let us set $f(z)=-z\ln z$, then the derivative
terms are

$\displaystyle f^{(0)}(z)=-z\ln z,\,f^{(1)}(z)=-(1+\ln z),$ $\displaystyle
f^{(m)}(z)=(m-2)!(-\frac{1}{z})^{m-1},\,(m\geq 2).$ (54)

The state updates as $\rho_{n}=\rho_{n-1}+\mathcal{L}^{\star}\rho_{n-1}\tau$
and we use Paycha’s formula with $\rho=\rho_{n-1}$ and
$\varepsilon=\mathcal{L}^{\star}\rho_{n-1}\tau$. The entropy update is

$\displaystyle S_{n}$ $\displaystyle\approx$
$\displaystyle-\mathrm{tr}_{s}f(\rho+\varepsilon)$ (55) $\displaystyle\approx$
$\displaystyle-\mathrm{tr}_{s}f^{(0)}(\rho)+\sum_{k_{1}}\frac{1}{(k_{1}+1)!}\mathrm{tr}_{s}\bigg{(}f^{(k_{1}+1)}(\rho)\,\mathrm{ad}^{k_{1}}_{\rho}(\varepsilon)\bigg{)},$

where we need only expand to the first order term. Moreover, we note that only
the $k_{1}=0$ term gives a non-zero contribution here leading to

$\displaystyle S_{n}$ $\displaystyle\approx$ $\displaystyle
S_{n-1}-\mathrm{tr}_{s}\bigg{(}\rho_{n-1}\,\mathcal{L}(\ln\rho_{n-1})\bigg{)}\tau.$
(56)

Here we have switched to the adjoint generator and used the fact that
$\mathcal{L}(I_{s})=0$.

The continuous limit form of this would be

$\displaystyle\frac{d}{dt}S(t)=-\mathrm{tr}_{s}\bigg{(}\rho(t)\,\mathcal{L}\big{(}\ln\rho(t)\big{)}\bigg{)},$
(57)

which agrees with the calculation in, for instance, Spohn .

###  IV.2 Entropy along a Quantum Trajectory

We now give a proof of Theorem 1.

We will apply the same method to compute the update rule for
$\mathsf{S}\big{(}\widehat{\rho_{n}}(\mathbf{y})\big{)}$ for a fixed quantum
trajectory generated by $\mathbf{y}$. We shall use the update rule (33) with
$\rho=\widehat{\rho}_{n-1}$ and
$\varepsilon=\mathcal{L}^{\star}\widehat{\rho}_{n-1}\tau+\sqrt{\eta}\bigg{(}L\widehat{\rho}_{n-1}+\widehat{\rho}_{n-1}L^{\ast}-\lambda_{n-1}\widehat{\rho}_{n-1}\bigg{)}\Delta
I_{n}$. This time, the bilinear terms in $\varepsilon$ are not negligible so
we have to develop Paycha’s formula to second order:

$\displaystyle\mathsf{S}\big{(}\widehat{\rho_{n}}(\mathbf{y})\big{)}$
$\displaystyle\approx$
$\displaystyle-\mathrm{tr}_{s}f^{(0)}(\rho)+\sum_{k_{1}}\frac{1}{(k_{1}+1)!}\mathrm{tr}_{s}\bigg{\\{}f^{(k_{1}+1)}(\rho)\,\mathrm{ad}^{k_{1}}_{\rho}(\varepsilon)\bigg{\\}}$
(58)
$\displaystyle+\sum_{k_{1},k_{2}=0}^{\infty}\mathrm{tr}_{s}\bigg{\\{}\dfrac{f^{(k_{1}+k_{2}+2)}(\rho)}{(k_{1}+1)(k_{1}+k_{2}+2)}\frac{{\mathrm{ad}^{k_{1}}_{\rho}(\varepsilon)}}{k_{1}!}\frac{{\mathrm{ad}^{k_{2}}_{\rho}(\varepsilon)}}{k_{2}!}\bigg{\\}}.$

The zeroth term in the Paycha expansion is
$\mathsf{S}\big{(}\widehat{\rho}_{n-1}(\mathbf{y})\big{)}$. Here we note that
$\mathrm{ad}^{k}_{\rho}(\varepsilon)\approx\sqrt{\eta}\mathrm{ad}^{k}_{\rho}(L\rho+\rho
L^{\ast})\,\Delta I_{n}$. The first term in the Paycha expansion simplifies
because only the $k_{1}=0$ term survives: all the higher terms involve a trace
of a commutator so they vanish identically. This leads to

$\displaystyle\mathsf{S}\big{(}\widehat{\rho_{n}}(\mathbf{y})\big{)}$
$\displaystyle\approx$
$\displaystyle\mathsf{S}\big{(}\widehat{\rho}_{n-1}(\mathbf{y})\big{)}-\mathrm{tr}_{s}\bigg{(}\big{(}1+\ln\rho\big{)}\varepsilon\bigg{)}$
(59) $\displaystyle+$
$\displaystyle\sum_{k_{1},k_{2}=0}^{\infty}\dfrac{\binom{k_{1}+k_{2}}{k_{1}}}{(k_{1}+1)(k_{1}+k_{2}+2)}\mathrm{tr}_{s}\bigg{\\{}\big{(}-\frac{1}{\rho}\big{)}^{k_{1}+k_{2}+1}\mathrm{ad}^{k_{1}}_{\rho}(\varepsilon)\mathrm{ad}^{k_{2}}_{\rho}(\varepsilon)\bigg{\\}}.$

The result is that we obtain

$\displaystyle\mathsf{S}\big{(}\widehat{\rho_{n}}(\mathbf{y})\big{)}$
$\displaystyle\approx$
$\displaystyle\mathsf{S}\big{(}\widehat{\rho}_{n-1}(\mathbf{y})\big{)}-\mathrm{tr}_{s}\bigg{\\{}\widehat{\rho}_{n-1}(\mathbf{y})\mathcal{L}\ln\widehat{\rho}_{n-1}(\mathbf{y})\big{)}\bigg{\\}}\tau$
(60)
$\displaystyle-\mathrm{tr}_{s}\bigg{\\{}\big{(}L\widehat{\rho}_{n-1}(\mathbf{y})+\widehat{\rho}_{n-1}(\mathbf{y})L^{\ast}-\lambda_{n-1}\widehat{\rho}_{n-1}(\mathbf{y})\big{)}\ln\widehat{\rho}_{n-1}(\mathbf{y})\bigg{\\}}\Delta
I_{n}$ $\displaystyle+$
$\displaystyle\sum_{k_{1},k_{2}=0}^{\infty}\dfrac{\binom{k_{1}+k_{2}}{k_{1}}}{(k_{1}+1)(k_{1}+k_{2}+2)}\mathrm{tr}_{s}\bigg{\\{}\big{(}-\frac{1}{\widehat{\rho}_{n-1}(\mathbf{y})}\big{)}^{k_{1}+k_{2}+1}$
$\displaystyle\times\eta\mathrm{ad}^{k_{1}}_{\widehat{\rho}_{n-1}(\mathbf{y})}(L\widehat{\rho}_{n-1}(\mathbf{y})+\widehat{\rho}_{n-1}(\mathbf{y})L^{\ast})\mathrm{ad}^{k_{2}}_{\widehat{\rho}_{n-1}(\mathbf{y})}(L\widehat{\rho}_{n-1}(\mathbf{y})+\widehat{\rho}_{n-1}(\mathbf{y})L^{\ast})\bigg{\\}}\tau.$

At this stage, we see that the formula for the rate of change of entropy along
a quantum trajectory in the continuous time limit will be (2) with $\Sigma$
given by (3).

##  V Conclusion

In the obtained expression (2) for the rate of change of the entropy along a
quantum trajectory, the expression $\Sigma$ is clearly singular in the
conditioned state $\widehat{\rho}$. This appears to be an intrinsic feature
and faithfulness of the state is needed.

We note that $\mathrm{ad}_{\rho}^{k}(L\rho+\rho
L^{\ast})=\mathrm{ad}_{\rho}^{k}(L)\rho+\rho\,\mathrm{ad}_{\rho}^{k}(L^{\ast})$
and that in general we may write

$\displaystyle\mathrm{tr}_{s}\bigg{\\{}\big{(}\frac{1}{\widehat{\rho}}\big{)}^{\,k_{1}+k_{2}+1}\mathrm{ad}^{k_{1}}_{\widehat{\rho}}(L\widehat{\rho}+\widehat{\rho}L^{\ast})\,\mathrm{ad}^{k_{2}}_{\widehat{\rho}}(L\widehat{\rho}+\widehat{\rho}L^{\ast})\bigg{\\}}=\mathrm{tr}_{s}\bigg{\\{}\widehat{\rho}^{\,-k_{1}-k_{2}-1}\,\mathrm{ad}^{k_{1}}_{\widehat{\rho}}(L)\widehat{\,\rho}^{2}\mathrm{ad}^{k_{2}}_{\widehat{\rho}}(L^{\ast})\bigg{\\}}$
$\displaystyle+\mathrm{tr}_{s}\bigg{\\{}\widehat{\rho}^{\,-k_{1}-k_{2}}\bigg{(}\mathrm{ad}^{k_{1}}_{\widehat{\rho}}(L)\widehat{\rho}\,\mathrm{ad}^{k_{2}}_{\widehat{\rho}}(L)+\mathrm{ad}^{k_{1}}_{\widehat{\rho}}(L^{\ast})\widehat{\rho}\,\mathrm{ad}^{k_{2}}_{\widehat{\rho}}(L^{\ast})\bigg{)}\bigg{\\}}$
$\displaystyle+\mathrm{tr}_{s}\bigg{\\{}\widehat{\rho}^{\,-k_{1}-k_{2}+1}\mathrm{ad}^{k_{1}}_{\widehat{\rho}}(L^{\ast})\mathrm{ad}^{k_{2}}_{\widehat{\rho}}(L)\bigg{\\}}.$
(61)

By inspection, we see that $\Sigma$ is a sum of terms of the form
$\mathrm{tr}_{s}\bigg{\\{}\widehat{\rho}^{-n+1}L_{1}\widehat{\rho}^{n}L_{2}\bigg{\\}}$
for some $n=0,1,2,\cdots$, with various combinatorial coefficients and where
either $L_{1}$ and $L_{2}$ are equal to $L$ or $L^{\ast}$.

The main result, Theorem 1, gives the expression for the rate of change of
entropy of the conditional state along a particular quantum trajectory under
the assumption that the state is faithful. We give a series expansion for an
entropy production rate $\Sigma$ which involves multiple nest commutators of
the collapse operators with the conditional state. Our proof utilized a
specific repeated qubit-probe model to approximate the measurement however the
continuous limit should be universal.

###### Acknowledgements.

This work is supported by the ANR project “Estimation et controle des systémes
quantiques ouverts” QCOAST Projet ANR-19-CE48-0003, the ANR project QUACO
ANR-17-CE40-0007, and the ANR project IGNITION ANR-21-CE47-0015.

## References

  * (1) S.K. Mitter and N.J. Newton, Information and Entropy Flow in the Kalman-Bucy Filter, Journal of Statistical Physics, vol. 118, pp. 145-176, (2005)
  * (2) V.P. Belavkin, Nondemolition measurements, nonlinear filtering and dynamic programming of quantum stochastic processes, In A. Blaquiere (ed.). Proc of Bellmann Continuum Workshop Modelling and Control of Systems. Lecture notes in Control and Inform Sciences. Vol. 121. Sophia-Antipolis: Springer-Verlag. pp. 245-265 (1988); V.P. Belavkin, A new wave equation for a continuous nondemolition measurement, Physics Letters A. 140 (7-8): 355-358 (1989).
  * (3) L. Bouten, R. van Handel, M.R. James, A Discrete Invitation to Quantum Filtering and Feedback Control, Vol. 51, No. 2, pp. 239-316 (2009).
  * (4) E. Joos, H.D. Zeh, C. Kiefer, D.J. Giulini, J. Kupsch, & I.O. Stamatescu, (2013) Decoherence and the Appearance of a Classical World in Quantum Theory. Springer Science & Business Media, Heidelberg.
  * (5) S. L. Adler, D. C. Brody, T. A. Brun, and L. P. Hughston, Martingale Models for Quantum State Reduction, J. Phys. A 34, 8795 (2001).
  * (6) J.K. Stockton, R. van Handel, H. Mabuchi, Deterministic Dicke state preparation with continuous measurement and control, Phys. Rev. A 70, 022106 (2004)
  * (7) J. Gough, M. R. James, Commun. Math. Phys. 287, 1109 (2009).
  * (8) J. Gough, M. R. James, IEEE Trans. on Automatic Control 54, 2530 (2009).
  * (9) H. Wiseman, Quantum theory of continuous feedback, Phys. Rev. A, vol. 49, no. 3, pp. 2133–2150, (1994).
  * (10) J. Kerckhoff, H.I. Nurdin D.S. Pavlichin and Hideo Mabuchi, Designing quantum memories with embedded control: photonic circuits for autonomous quantum error correction, Physical review letters, 105, 4, 040502, (2009)
  * (11) N. Cottet, S. Jezouin, L. Bretheau, P. Campagne-Ibarcq, Q. Ficheux, J. Anders, A. Auffèves, R. Azouit, P. Rouchon, B. Huard, Observing a quantum Maxwell demon at work, PNAS 114, 7561 (2017)
  * (12) H. Spohn, Entropy production for quantum dynamical semigroups, Journ. Math. Phys. 19, 1227 (1978).
  * (13) G.T. Landi, M. Paternostro, A. Belenchia, Informational steady-states and conditional entropy production in continuously monitored systems, PRX Quantum 3, 010303 (2022).
  * (14) A. S. Holevo, Bounds for the Quantity of Information Transmitted by a Quantum Communication Channel, Problems of Information Transmission 9, 177–183 (1973).
  * (15) S. Paycha, Noncommutative formal Taylor expansions and second quantised regularised traces, (Ed. K. Ebrahimi-Fard, M. Marcolli, W.D. van Suijlekom) Contemporary Mathematics 539, p. 349-376, Amer. Math. Soc. (2011).