msbm10 scaled1095

# On quasi-similarity of multiplication operator on the weighted Bergman space
in the unit ball

Cui Chen∗, Ya Wang and Yu-Xia Liang
Cui Chen, Department of Mathematics, Tianjin University of Finance and
Economics, Tianjin 300222, P.R. China<EMAIL_ADDRESS>
Ya Wang, Department of Mathematics, Tianjin University of Finance and
Economics, Tianjin 300222, P.R. China<EMAIL_ADDRESS>
Yu-Xia Liang, School of Mathematical Sciences, Tianjin Normal University,
Tianjin 300378, P.R. China<EMAIL_ADDRESS>
###### Abstract.

For $\alpha>-1$, let $A_{\alpha}^{2}(\mathbb{B}_{N})$ be the weighted Bergman
space on the unit ball $\mathbb{B}_{N}$ in $\mathbb{C}^{N}$. In this paper, we
prove that the multiplication operator $M_{z^{n}}$ is quasi-similar to
$\oplus_{1}^{\prod_{i=1}^{N}n_{i}}M_{z}$ on $A_{\alpha}^{2}(\mathbb{B}_{N})$
for the multi-index $n=(n_{1},n_{2},\cdots,n_{N})$.

###### Key words and phrases:

multiplication operator, weighted Bergman space, quasi-similarity, unit ball

###### 2010 Mathematics Subject Classification:

Primary: 47B38; Secondary: 32H02, 30H05, 30H20, 47B33.

∗Corresponding author.

##  1\. Introduction

Similarity of operators is a weaker concept than unitary equivalence. It is
well known that invariant subspace of an operator can be identified in terms
of invariant subspaces of a similar operator. Over the years, there are some
results characterizing the lattice of closed invariant subspaces by
constructing similar operators, we refer the interested readers to the recent
papers such as [2, 3, 5].

In previous years, the similarity between $M_{z^{n}}$ and
$\oplus_{1}^{n}M_{z}$ acting on a Hilbert space is an active topic which has
be concerned in lots of papers. In 2007, Jiang and Li (see [8]) first obtained
that analytic Toeplitz operator $M_{B(z)}$ is similar to $\oplus_{1}^{n}M_{z}$
on the Bergman space if and only if $B(z)$ is an $n$-Blaschke product. Next,
Li (see [10]) in 2009 proved that multiplication operator $M_{z^{n}}$ is
similar to $\oplus_{1}^{n}M_{z}$ on the weighted Bergman space. And then,
Jiang and Zheng in [9] extended the main result in [8] to the weighted Bergman
space. In 2011, Douglas and Kim in [6] investigated the reducing subspaces for
an analytic multiplication operator $M_{z^{n}}$ on the Bergman space
$A_{\alpha}^{2}(A_{r})$ of the annulus $A_{r}$. The similarity of $M_{z^{n}}$
and $\oplus_{1}^{n}M_{z}$ also holds on the weighted Hardy space and Sobolev
disk algebra, they are shown in [1] and [12]. For further results related, see
[7, 13]. Moreover, in 2017, the unitary equivalence of analytic multipliers on
Sobolev disk algebra was discussed in [4]. But such a characterization is not
always hold for all Hilbert spaces. In Recent paper, Li, Lan and Liu (see
[11]) proved that multiplication operator $M_{z^{n}}$ is not similar to
$\oplus_{1}^{n}M_{z}$ on the Fock space, they are quasi-similar, actually.

However, all these results are considered in the one-dimensional space, such a
characterization is much more complicated for the high-dimension case. Based
on the works above, in this paper we are interested in the corresponding
result on the weighted Bergman space in the unit ball.

Let $\mathbb{B}_{N}$ be the unit ball in $\mathbb{C}^{N}$, where
$\mathbb{C}^{N}$ is the $N$-dimensional complex vector space. Denoted by
$\mathbb{N}_{0}$ and $\mathbb{N}$ the set of all positive and nonnegative
integers, respectively. $H(\mathbb{B}_{N})$ is the class of all holomorphic
functions on $\mathbb{B}_{N}$.

Given three multi-indexes in $\mathbb{N}^{N}$ with
$n=(n_{1},n_{2},\cdots,n_{N}),k=(k_{1},k_{2},\cdots,k_{N})$ and
$j=(j_{1},j_{2},\cdots,j_{N})$, we use the notations

$|n|=n_{1}+n_{2}+\cdots+n_{N},\;\;\;\;\;\;n!=n_{1}!n_{2}!\cdots n_{N}!.$

and

$nk+j=(n_{1}k_{1}+j_{1},n_{2}k_{2}+j_{2},\cdots,n_{N}k_{N}+j_{N}).$

Moreover, we say $j<(\mbox{or}\leq)n$ if $j_{i}<(\mbox{or}\leq)n_{i}$ for each
$i=1,2,\cdots,N$.

For $z\in\mathbb{C}^{N}$,

$z^{n}=z_{1}^{n_{1}}z_{2}^{n_{2}}\cdots z_{N}^{n_{N}}.$

Let $\alpha>-1$, the weighted Bergman space $A_{\alpha}^{2}(\mathbb{B}_{N})$
consists of holomorphic functions $f$ in $L^{2}(\mathbb{B}_{N},dv_{\alpha})$,
that is, $A_{\alpha}^{2}(\mathbb{B}_{N})=L^{2}(\mathbb{B}_{N},dv_{\alpha})\cap
H(\mathbb{B}_{N})$. It is well known that $A_{\alpha}^{2}(\mathbb{B}_{N})$ is
a Hilbert space with the inner product defined by

$\langle
f,g\rangle=\int_{\mathbb{B}_{N}}f\overline{g}dv_{\alpha},\;\;\;\;\;\;\mbox{for}\;f,g\in
A_{\alpha}^{2}(\mathbb{B}_{N}).$

The corresponding norm of $f$ is given by
$\|f\|^{2}=\int_{\mathbb{B}_{N}}|f|^{2}dv_{\alpha}$. As we know, the set of
polynomials is dense in the Bergman space $A_{\alpha}^{2}(\mathbb{B}_{N})$,
and moreover,

$\langle
z^{k},z^{m}\rangle=\int_{\mathbb{B}_{N}}z^{k}\overline{z^{m}}dv_{\alpha}=\left\\{\begin{array}[]{ll}\frac{k!\Gamma(N+\alpha+1)}{\Gamma(N+|k|+\alpha+1)}\;,\;\;\;m=k,\\\
\;\;\;\;\;\;\;\;0\;,\;\;\;\;\;\;\;\;\;\;\;\;\;\;otherwise.\end{array}\right.$

Thus
$\\{e_{k}(z)\triangleq\sqrt{\frac{\Gamma(N+|k|+\alpha+1)}{k!\Gamma(N+\alpha+1)}}z^{k}:k\in\mathbb{N}^{N}\\}$
ia an orthonormal basis of $A_{\alpha}^{2}(\mathbb{B}_{N})$.

Throughout the rest of this paper, we will always assume that $N\geq 2$ since
it has been considered by Li [10] for the case $N=1$. Moreover, we would like
to point out that $n$ stands for the multi-index
$(n_{1},n_{2},\cdots,n_{N})\in\mathbb{N}_{0}^{N}$ with $n_{i}\geq 2$ for at
least one $i\in\\{1,2,\cdots,N\\}$, as it is trivial for otherwise.

##  2\. Quasi-similarity

Let us recall that for two Hilbert spaces $\mathcal{H}$ and $\mathcal{K}$, an
operator $X\in\mathcal{L}(\mathcal{H},\mathcal{K})$ is said to be quasi-
invertible if it has zero kernel and dense range. Let
$S\in\mathcal{L}(\mathcal{H})$ and $T\in\mathcal{L}(\mathcal{K})$, $S$ is
quasi-similar to $T$ if there exist two quasi-invertible operators
$X\in\mathcal{L}(\mathcal{H},\mathcal{K})$ and
$Y\in\mathcal{L}(\mathcal{K},\mathcal{H})$ respectively such that $XS=TX$ and
$SY=YT$. The following theorem is our main result.

######  Theorem 1.

$M_{z^{n}}$ is quasi-similar to $\oplus_{1}^{\prod_{i=1}^{N}n_{i}}M_{z}$
acting on the weighted Bergman space $A_{\alpha}^{2}(\mathbb{B}_{N})$ with
$\alpha>-1$.

Before going to the main theorem, we need the following lemma.

######  Lemma 1.

For a multi-index
$j=(j_{1},j_{2},\cdots,j_{N}),j_{i}=0,1,\cdots,n_{i}-1\;\mbox{for
each}\;i=1,2,\cdots,N$, let
$\mathcal{A}_{j}=span\\{e_{nk+j}:k=(k_{1},k_{2},\cdots,k_{N})\in\mathbb{N}^{N}\\}$.
Then

(i) $\\{e_{nk+j}:k=(k_{1},k_{2},\cdots,k_{N})\in\mathbb{N}^{N}\\}$ form an
orthonormal basis of $\mathcal{A}_{j}$.

(ii) $A_{\alpha}^{2}(\mathbb{B}_{N})=\oplus_{j}\mathcal{A}_{j}$, where the
direct sum is over all multi-indexes $j=(j_{1},j_{2},\cdots,j_{N})$ and
$j_{i}=0,1,\cdots,n_{i}-1$ for each $i=1,2,\cdots,N$.

(iii) $\mathcal{A}_{j}$ is a reducing subspace for $M_{z^{n}}$.

Proof. (i) Note that for all $k,m\in\mathbb{N}^{N}$,

$\displaystyle\langle e_{nk+j},e_{nm+j}\rangle$ $\displaystyle=$
$\displaystyle\int_{\mathbb{B}}\sqrt{\frac{\Gamma(N+|nk+j|+\alpha+1)}{(nk+j)!\Gamma(N+\alpha+1)}}z^{nk+j}\sqrt{\frac{\Gamma(N+|nm+j|+\alpha+1)}{(nm+j)!\Gamma(N+\alpha+1)}}\overline{z}^{nm+j}dv_{\alpha}(z).$

If $m\neq k$, then $\langle e_{nk+j},e_{nm+j}\rangle=0$. If $m=k$, we have

$\displaystyle\langle e_{nk+j},e_{nm+j}\rangle$ $\displaystyle=$
$\displaystyle\frac{\Gamma(N+|nk+j|+\alpha+1)}{(nk+j)!\Gamma(N+\alpha+1)}\int_{\mathbb{B}_{N}}|z^{nk+j}|^{2}dv_{\alpha}(z)$
$\displaystyle=$
$\displaystyle\frac{\Gamma(N+|nk+j|+\alpha+1)}{(nk+j)!\Gamma(N+\alpha+1)}\frac{(nk+j)!\Gamma(N+\alpha+1)}{\Gamma(N+|nk+j|+\alpha+1)}=1.$

Thus (i) holds.

(ii) It is clear that $\mathcal{A}_{j}\perp\mathcal{A}_{t}$ for all $0\leq
j\neq t\leq n-1$. Next, for $f\in A_{\alpha}^{2}(\mathbb{B}_{N})$, it is easy
to see that $f$ has the form

$f=\sum_{k=0}^{\infty}\sum_{j=0}^{n-1}a_{jk}e_{nk+j}.$

Suppose $f=0$, we conclude that $a_{jk}=0$ for all $k\in\mathbb{N}^{N}$ and
$j=(j_{1},j_{2},\cdots,j_{N}),j_{i}=0,1,\cdots,n_{i}-1$, since

$\displaystyle\langle\sum_{k=0}^{\infty}\sum_{j=0}^{n-1}a_{jk}e_{nk+j},e_{l}\rangle=0\;\;\;\;\;\;\mbox{for
each}\;l\in\mathbb{N}^{N}.$

That is $0=\overbrace{0\oplus 0\oplus\cdots\oplus 0}^{\Pi_{i=1}^{N}n_{i}}$,
which yields that $A_{\alpha}^{2}(\mathbb{B}_{N})=\oplus_{j}\mathcal{A}_{j}$.

(iii) It is easy to see that both $\mathcal{A}_{j}$ and
$\mathcal{A}_{j}^{\perp}$ are invariant subspaces for $M_{z^{n}}$. $\Box$

By the previous lemma, it is clear that
$M_{z^{n}}=\oplus_{j}M_{z^{n}}|_{\mathcal{A}_{j}}$. Then we can get the proof
of Theorem 1.

Proof of Theorem 1. Note that

$M_{z}e_{k}=z\sqrt{\frac{\Gamma(N+|k|+\alpha+1)}{k!\Gamma(N+\alpha+1)}}z^{k}=\sqrt{\frac{(k+1)\Gamma(N+|k|+\alpha+1)}{\Gamma(N+|k+1|+\alpha+1)}}e_{k+1}.$

Set $M_{j}=M_{z^{n}}|_{\mathcal{A}_{j}}$, then

$\displaystyle M_{j}e_{nk+j}$ $\displaystyle=$ $\displaystyle
z^{n}\sqrt{\frac{\Gamma(N+|nk+j|+\alpha+1)}{(nk+j)!\Gamma(N+\alpha+1)}}z^{nk+j}$
$\displaystyle=$
$\displaystyle\sqrt{\frac{(n(k+1)+j)!\Gamma(N+|nk+j|+\alpha+1)}{(nk+j)!\Gamma(N+|n(k+1)+j|+\alpha+1)}}e_{n(k+1)+j}$

Define $X_{j}:A_{\alpha}^{2}(\mathbb{B}_{N})\rightarrow\mathcal{A}_{j}$ such
that $X_{j}e_{k}=c_{kj}e_{nk+j}$, where $c_{kj}$ are given by

$\displaystyle
c_{kj}=\sqrt{\frac{(nk+j)!\Gamma(N+|k|+\alpha+1)\Gamma(N+|j|+\alpha+1)}{k!j!\Gamma(N+|nk+j|+\alpha+1)\Gamma(N+\alpha+1)}}.$

Thus we conclude that $X_{j}M_{z}e_{k}=M_{j}X_{j}e_{k}$. In fact,

$\displaystyle X_{j}M_{z}e_{k}$ $\displaystyle=$ $\displaystyle
X_{j}\sqrt{\frac{(k+1)\Gamma(N+|k|+\alpha+1)}{\Gamma(N+|k+1|+\alpha+1)}}e_{k+1}$
$\displaystyle=$
$\displaystyle\sqrt{\frac{(nk+n+j)!\Gamma(N+|k|+\alpha+1)\Gamma(N+|j|+\alpha+1)}{k!j!\Gamma(N+|nk+n+j|+\alpha+1)\Gamma(N+\alpha+1)}}e_{nk+n+j},$

and

$\displaystyle M_{j}X_{j}e_{k}$ $\displaystyle=$ $\displaystyle
M_{j}\sqrt{\frac{(nk+j)!\Gamma(N+|k|+\alpha+1)\Gamma(N+|j|+\alpha+1)}{k!j!\Gamma(N+|nk+j|+\alpha+1)\Gamma(N+\alpha+1)}}e_{nk+j}$
$\displaystyle=$
$\displaystyle\sqrt{\frac{(nk+n+j)!\Gamma(N+|k|+\alpha+1)\Gamma(N+|j|+\alpha+1)}{k!j!\Gamma(N+|nk+n+j|+\alpha+1)\Gamma(N+\alpha+1)}}e_{nk+n+j}.$

Now we will show that $X_{j}$ is not invertible in the following by giving the
fact that $\liminf_{|k|\rightarrow\infty}c_{kj}=0$. Indeed,

$\displaystyle c_{kj}^{2}$ $\displaystyle=$
$\displaystyle\frac{\Gamma(N+|j|+\alpha+1)}{j!\Gamma(N+\alpha+1)}\frac{|nk+j|!\Gamma(N+|k|+\alpha+1)}{|k|!\Gamma(N+|nk+j|+\alpha+1)}\frac{(nk+j)!|k|!}{|nk+j|!k!}$
(1) $\displaystyle\triangleq$
$\displaystyle\frac{\Gamma(N+|j|+\alpha+1)}{j!\Gamma(N+\alpha+1)}I_{1}I_{2},$

where

$I_{1}=\frac{|nk+j|!\Gamma(N+|k|+\alpha+1)}{|k|!\Gamma(N+|nk+j|+\alpha+1)}\;\;\;\;\;\mbox{and}\;\;\;\;\;I_{2}=\frac{(nk+j)!|k|!}{|nk+j|!k!}.$

For $I_{1}$,

$\displaystyle\frac{|nk+j|!\Gamma(N+|k|+\alpha+1)}{|k|!\Gamma(N+|nk+j|+\alpha+1)}$
(2) $\displaystyle=$
$\displaystyle\frac{|nk+j|(|nk+j|-1)\cdots(|k|+1)}{(N+|nk+j|+\alpha)(N+|nk+j|+\alpha-1)\cdots(N+|k|+\alpha+1)}$
$\displaystyle=$
$\displaystyle\frac{|k|+1}{N+|nk+j|+\alpha}\frac{(|nk+j|)(|nk+j|-1)\cdots(|k|+2)}{(N+|nk+j|+\alpha-1))\cdots(N+|k|+\alpha+1)}$
$\displaystyle=$
$\displaystyle\frac{|k|+1}{N+|nk+j|+\alpha}\frac{1}{(1+\frac{N+\alpha-1}{|nk+j|})(1+\frac{N+\alpha-1}{|nk+j|-1})\cdots(1+\frac{N+\alpha-1}{|k|+2})}$

Define

$\displaystyle
a_{kj}(\alpha)=(1+\frac{N+\alpha-1}{|nk+j|})(1+\frac{N+\alpha-1}{|nk+j|-1})\cdots(1+\frac{N+\alpha-1}{|k|+2}).$

Since $\alpha>-1$ and $N\geq 2$, it is clear $N+\alpha-1>0$. Then it easily
follows that

$\displaystyle a_{kj}(\alpha)$ $\displaystyle\leq$
$\displaystyle(1+\frac{N+\alpha-1}{|k|+2})^{|nk+j|-|k|-1}$ (3)
$\displaystyle\leq$
$\displaystyle(1+\frac{N+\alpha-1}{|k|+2})^{|n||k|+|j|-|k|-1}$
$\displaystyle=$
$\displaystyle(1+\frac{N+\alpha-1}{|k|+2})^{\frac{|k|+2}{N+\alpha-1}\cdot\frac{N+\alpha-1}{|k|+2}\cdot[|n||k|+|j|-|k|-1]}$
$\displaystyle\rightarrow$
$\displaystyle\exp\\{(|n|-1)(N+\alpha-1)\\},\;\;\;\mbox{as}\;|k|\rightarrow\infty.$

On the other hand,

$\displaystyle a_{kj}(\alpha)$ $\displaystyle\geq$
$\displaystyle(1+\frac{N+\alpha-1}{|nk+j|})^{|nk+j|-|k|-1}$ (4)
$\displaystyle\geq$
$\displaystyle(1+\frac{N+\alpha-1}{|n||k|+|j|})^{|k|+|j|-|k|-1}$
$\displaystyle=$ $\displaystyle(1+\frac{N+\alpha-1}{|n||k|+|j|})^{|j|-1}$
$\displaystyle\rightarrow$ $\displaystyle
1,\;\;\;\mbox{as}\;|k|\rightarrow\infty.$

From (2)-(4), associated with

$\displaystyle\frac{1}{|n|}$ $\displaystyle=$
$\displaystyle\lim_{|k|\rightarrow\infty}\frac{|k|+1}{N+|n||k|+|j|+\alpha}$
$\displaystyle\leq$
$\displaystyle\lim_{|k|\rightarrow\infty}\frac{|k|+1}{N+|nk+j|+\alpha}$
$\displaystyle\leq$
$\displaystyle\lim_{|k|\rightarrow\infty}\frac{|k|+1}{N+n_{m}|k|+\alpha}=\frac{1}{n_{m}},$

where $m\in\\{1,2,\cdots,N\\}$ is the index such that
$n_{m}=\min\\{n_{1},n_{2},\cdots,n_{N}\\}$, then we conclude that there exist
two positive constants $C_{1}$ and $C_{2}$ such that

$\displaystyle
C_{1}\leq\liminf_{|k|\rightarrow\infty}I_{1}\leq\limsup_{|k|\rightarrow\infty}I_{1}\leq
C_{2}.$ (5)

For $I_{2}$, first note that

$\displaystyle\frac{(n_{1}k_{1}+j_{1})!\cdots(n_{N}k_{N}+j_{N})!}{(n_{1}k_{1}+j_{1}+\cdots+n_{N}k_{N}+j_{N})!}\leq\frac{(n_{m}k_{1}+j_{m})!\cdots(n_{m}k_{N}+j_{m})!}{(n_{m}k_{1}+j_{m}+\cdots+n_{m}k_{N}+j_{m})!}.$
(6)

This is due to that

$\displaystyle\frac{(n_{1}k_{1}+j_{1})!\cdots(n_{N}k_{N}+j_{N})!}{(n_{m}k_{1}+j_{m})!\cdots(n_{m}k_{N}+j_{m})!}\cdot\frac{(n_{m}k_{1}+j_{m}+\cdots+n_{m}k_{N}+j_{m})!}{(n_{1}k_{1}+j_{1}+\cdots+n_{N}k_{N}+j_{N})!}$
$\displaystyle=$
$\displaystyle\frac{\Pi_{i=1}^{N}[(n_{i}k_{i}+j_{i})(n_{i}k_{i}+j_{i}-1)\cdots(n_{m}k_{i}+j_{m}+1)]}{\sum_{i=1}^{N}(n_{i}k_{i}+j_{i})\\{\sum_{i=1}^{N}(n_{i}k_{i}+j_{i})-1\\}\cdots\\{\sum_{i=1}^{N}(n_{m}k_{i}+j_{m})+1\\}}\leq
1,$

which comes from the numbers of factors are both
$\sum_{i=1}^{N}(n_{i}k_{i}+j_{i}-n_{m}k_{i}-j_{m})$ in the numerator and
denominator, and the molecular factors are not greater than the denominator
factors one by one.

Using Stirling’s approximation, we have

$\displaystyle\liminf_{|k|\rightarrow\infty}I_{2}\leq\lim_{k_{1}=k_{2}=\cdots=k_{N}\rightarrow\infty}I_{2}$
$\displaystyle=$
$\displaystyle\lim_{k_{1}\rightarrow\infty}\frac{[(n_{m}k_{1}+j_{m})!]^{N}}{[N(n_{m}k_{1}+j_{m})]!}\frac{(Nk_{1})!}{(k_{1}!)^{N}}$
$\displaystyle=$
$\displaystyle\lim_{k_{1}\rightarrow\infty}\frac{[(n_{m}k_{1}+j_{m})^{n_{m}k_{1}+j_{m}}\sqrt{n_{m}k_{1}+j_{m}}e^{-(n_{m}k_{1}+j_{m})}]^{N}}{[N(n_{m}k_{1}+j_{m})]^{N(n_{m}k_{1}+j_{m})}\sqrt{N(n_{m}k_{1}+j_{m})}e^{-N(n_{m}k_{1}+j_{m})}}$
$\displaystyle\cdot\frac{(Nk_{1})^{Nk_{1}}\sqrt{Nk_{1}}e^{-Nk_{1}}}{[k_{1}^{k_{1}}\sqrt{k_{1}}e^{-k_{1}}]^{N}}$
$\displaystyle=$
$\displaystyle\lim_{k_{1}\rightarrow\infty}n_{m}^{(N-1)/2}(N^{N})^{(1-n_{m})k_{1}-j_{m}}$

Thus if $n_{m}>1$, $\liminf_{|k|\rightarrow\infty}I_{2}=0$.

For the remaining case $n_{m}=1$. Without loss of generality, we may assume
that $\sharp\\{i\in\\{1,2,\cdots,N\\}:n_{i}=1\\}=1$, for otherwise set
$k_{m^{\prime}}=0$ when $n_{m^{\prime}}=1$. Choose $s$ such that
$n_{s}=\min\\{n_{i}:i\in\\{1,2,\cdots,N\\}\backslash\\{m\\}\\}$. From the same
reason of (6) we can get

$\displaystyle\frac{(n_{1}k_{1}+j_{1})!\cdots(n_{N}k_{N}+j_{N})!}{(n_{1}k_{1}+j_{1}+\cdots+n_{N}k_{N}+j_{N})!}$
$\displaystyle\leq$
$\displaystyle\frac{(n_{s}k_{1}+j_{s})!\cdots(n_{s}k_{m-1}+j_{s})!(k_{m}+j_{m})!(n_{s}k_{m+1}+j_{s})!\cdots(n_{s}k_{N}+j_{s})!}{(n_{s}k_{1}+j_{s}+\cdots+n_{s}k_{m-1}+j_{s}+k_{m}+j_{m}+n_{s}k_{m+1}+j_{s}+\cdots+n_{s}k_{N}+j_{s})!}.$

By Stirling’s approximation, we conclude that

$\displaystyle\liminf_{|k|\rightarrow\infty}I_{2}\leq\lim_{k_{1}=k_{2}=\cdots=k_{N}\rightarrow\infty}I_{2}$
$\displaystyle=$
$\displaystyle\lim_{k_{1}\rightarrow\infty}\frac{[(n_{s}k_{1}+j_{s})!]^{N-1}(k_{1}+j_{m})!}{[(N-1)(n_{s}k_{1}+j_{s})+k_{1}+j_{m}]!}\frac{(Nk_{1})!}{(k_{1}!)^{N}}$
$\displaystyle=$
$\displaystyle\lim_{k_{1}\rightarrow\infty}\frac{(n_{s}k_{1}+j_{s})^{(N-1)(n_{s}k_{1}+j_{s})}\sqrt{(n_{s}k_{1}+j_{s})^{N-1}}e^{-(N-1)(n_{s}k_{1}+j_{s})}}{[(N-1)(n_{s}k_{1}+j_{s})+k_{1}+j_{m}]^{(N-1)(n_{s}k_{1}+j_{s})+k_{1}+j_{m}}}$
$\displaystyle\cdot\frac{(k_{1}+j_{m})^{k_{1}+j_{m}}\sqrt{k_{1}+j_{m}}e^{-(k_{1}+j_{m})}}{\sqrt{(N-1)(n_{s}k_{1}+j_{s})+k_{1}+j_{m}}e^{-[(N-1)(n_{s}k_{1}+j_{s})+k_{1}+j_{m}]}}$
$\displaystyle\cdot\frac{(Nk_{1})^{Nk_{1}}\sqrt{Nk_{1}}e^{-Nk_{1}}}{(k_{1}^{k_{1}}\sqrt{k_{1}}e^{-k_{1}})^{N}}$
$\displaystyle=$
$\displaystyle\lim_{k_{1}\rightarrow\infty}\frac{N^{Nk_{1}}}{(N-1+\frac{k_{1}+j_{m}}{n_{s}k_{1}+j_{s}})^{(N-1)(n_{s}k_{1}+j_{s})}(1+\frac{(N-1)(n_{s}k_{1}+j_{s})}{k_{1}+j_{m}})^{k_{1}+j_{m}}}$
$\displaystyle\cdot\frac{\sqrt{(n_{s}k_{1}+j_{s})^{N-1}}\sqrt{k_{1}+j_{m}}\sqrt{Nk_{1}}}{\sqrt{(N-1)(n_{s}k_{1}+j_{s})+k_{1}+j_{m}}\sqrt{k_{1}^{N}}}$
$\displaystyle=$
$\displaystyle\lim_{k_{1}\rightarrow\infty}\Big{(}\frac{N}{(N-1+\frac{1}{n_{s}})^{n_{s}}}\Big{)}^{(N-1)k_{1}}\Big{(}\frac{N}{1+(N-1)n_{s}}\Big{)}^{k_{1}}\sqrt{\frac{Nn_{s}^{N-1}}{(N-1)n_{s}+1}}$
$\displaystyle\cdot\frac{1}{(N-1+\frac{1}{n_{s}})^{j_{s}}(1+(N-1)n_{s})^{j_{m}}}$
$\displaystyle=$ $\displaystyle 0,$

which we have used $\Big{(}N-1+\frac{1}{n_{s}}\Big{)}^{n_{s}}>N$ and
$1+(N-1)n_{s}>N$ in the last equation, since $N,n_{s}\geq 2$. Along with (1)
and (5), we have $\liminf_{|k|\rightarrow\infty}c_{k,j}=0$. This implies that
$X_{j}$ is not invertible.

Next, define $Y_{j}:\mathcal{A}_{j}\rightarrow A_{\alpha}^{2}(\mathbb{B}_{N})$
by $Y_{j}e_{nk+j}=b_{kj}e_{k}$, where $b_{kj}=\frac{1}{c_{kj}}$. It is easy to
prove that $Y_{j}M_{j}=M_{z}Y_{j}$ on $\mathcal{A}_{j}$. From the above
discussion, we have $\limsup_{|k|\rightarrow\infty}b_{kj}=\infty$. Hence,
$Y_{j}$ is also not invertible.

For the rest of the proof we will show that both $X_{j},Y_{j}$ are quasi-
invertible operators. To do this, take $f\in\mbox{ker}X_{j}$ with
$f=\sum_{k=0}^{\infty}d_{k}e_{k},d_{k}\in\mathbb{C}^{N}$. From $0=\langle
X_{j}f,e_{nk+j}\rangle=\langle\sum_{k=0}^{\infty}d_{k}c_{kj}e_{nk+j},e_{nl+j}\rangle$,
we have $d_{l}=0$ for all $l\in\mathbb{N}^{N}$, Hence $X_{j}$ is injective.
Besides, pick $g\in\mbox{ker}X_{j}^{*}$ with
$g=\sum_{k=0}^{\infty}m_{k}e_{nk+j},m_{k}\in\mathbb{C}^{N}$. From $0=\langle
e_{l},X_{j}^{*}g\rangle=\langle
c_{lj}e_{nl+j},\sum_{k=0}^{\infty}m_{k}e_{nk+j}\rangle$, we have $m_{l}=0$ for
all $l\in\mathbb{N}^{N}$. Then
$(\mbox{Ran}X_{j})^{\perp}=\mbox{ker}X_{j}^{*}=\\{0\\}$, so
$\overline{\mbox{Ran}X_{j}}=\mathcal{A}_{j}$, that is $X_{j}$ has dense range.
Similarly, we can check that $Y_{j}$ is injective and has dense range. It
turns out that both $X_{j}$ and $Y_{j}$ are quasi-invertible. To sum up,
$M_{j}$ is quasi-similar to $M_{z}$. Since $M_{z^{n}}=\oplus_{j}M_{j}$, we
conclude that $M_{z^{n}}$ is quasi-similar to
$\oplus_{1}^{\prod_{i=1}^{N}n_{i}}M_{z}$, and we are done. $\;\;\;\;\;\Box$

## References

  * [1] M.F. Ahmadi and K. Hedayatian, On similarity of powers of shift operators, Turk. J. Math., 36 (2012) 596-600.
  * [2] M.R. Alfonso, P.E. Manuel, Invariant subspaces of parabolic self-maps in the Dirichlet space, J. Funct. Anal., 266 (2014) 4115-4120.
  * [3] M.R. Alfonso, P.E. Manuel and A.S. Stanislav, Invariant subspaces of parabolic self-maps in the Hardy space, Math. Res. Lett., 17(1) (2010) 99-107.
  * [4] Y. Chen, C. Qin and Q. Wu, Reducibility and unitary equivalence of analytic multipliers on Sobolev disk algebra, J. Math. Anal. Appl., 455(2) (2017) 1249-1256.
  * [5] $\check{Z}$. $\check{C}$u$\check{c}$kovi$\acute{c}$ and B. Paudyal, Invariant subspaces of the shift plus complex Volterra operator, J. Math. Anal. Appl., 426 (2015) 1174-1181.
  * [6] R.G. Douglas and Y.S. Kim, Reducing subspaces on the annulus, Integr. Equat. Oper. Th., 70 (2011) 1-15.
  * [7] K. Ji and R. Shi, Similarity of multiplication operators on the Sobolev disk algebra, Acta. Math. Sin. Eng. Sers., 29(4) (2013) 789-800.
  * [8] C.L. Jiang and Y.C. Li, The commutant and similarity invariant of analytic Toeplitz operators on Bergman space, Sci. China Ser. A, 50(5) (2007) 651-664.
  * [9] C.L. Jiang and D.C. Zheng, Similarity of analytic Toeplitz operators on the Bergman spaces, J. Funct. Anal., 258 (2010) 2961-2982.
  * [10] Y.C. Li, On similarity of multiplication operator on weighted Bergman space, Integr. Equat. Oper. Th., 63 (2009) 95-102.
  * [11] Y.C. Li, W.H. Lan and J.L. Liu, On quasi-similarity and renducing subspaces of multiplication operator on the Fock space, J. Math. Anal. Appl., 409 (2014) 899-905.
  * [12] Y.C. Li, Q.J. Liu, and W.H. Lan, On similarity and renducing subspaces of multiplication operator on Sobolev disk algebra, J. Math. Anal. Appl., 419 (2014) 1161-1167.
  * [13] R.F. Zhao, A similarity invariant and the commutant of some multiplication operators on the Sobolev disk algebra, Int. J. Math. Math. Sci., Artical ID 378217, (2012) 17p.


LGRptmcmr

# Learning Informative Representations of Biomedical Relations with Latent
Variable Models

Harshil Shah
University College London
BenevolentAI
<EMAIL_ADDRESS>
&Julien Fauqueur
BenevolentAI
<EMAIL_ADDRESS>Work completed during internship at BenevolentAI.

###### Abstract

Extracting biomedical relations from large corpora of scientific documents is
a challenging natural language processing task. Existing approaches usually
focus on identifying a relation either in a single sentence (mention-level) or
across an entire corpus (pair-level). In both cases, recent methods have
achieved strong results by learning a point estimate to represent the
relation; this is then used as the input to a relation classifier. However,
the relation expressed in text between a pair of biomedical entities is often
more complex than can be captured by a point estimate. To address this issue,
we propose a latent variable model with an arbitrarily flexible distribution
to represent the relation between an entity pair. Additionally, our model
provides a unified architecture for both mention-level and pair-level relation
extraction. We demonstrate that our model achieves results competitive with
strong baselines for both tasks while having fewer parameters and being
significantly faster to train. We make our code publicly available.

##  1 Introduction

The vast amounts of scientific literature can provide a significant source of
information for biomedical research. Using this literature to identify
relations between entities is an important task in various applications (van
Mulligen et al., 2012; Segura-Bedmar et al., 2013; Bravo et al., 2015;
Krallinger et al., 2017).

Existing approaches to biomedical relation extraction usually fall into one of
two categories. Mention-level extraction aims to classify the relation between
a pair of entities within a short span of text (usually a sentence). In
contrast, pair-level extraction aims to classify the relation between a pair
of entities across an entire paragraph, document or corpus.

For both mention-level and pair-level relation extraction, recent work has
been focused on representation learning. This is considered to be one of the
major steps towards making progress in artificial intelligence (Bengio et al.,
2013). Representations of relations which understand their context are
particularly important in biomedical research, where identifying fruitful
targets is crucial due to the high costs of experimentation. Learning such
representations is likely to require large amounts of unsupervised data due to
the scarcity of labelled data in this domain.

Recent mention-level methods have been based on using large unsupervised
models with Transformer networks (Vaswani et al., 2017) to learn
representations of sentences containing pairs of entities. These
representations are then used as the inputs to much smaller models, which
perform supervised relation classification (Lee et al., 2019; Beltagy et al.,
2019).

Recent pair-level methods have been based on encoding each mention of a pair
of entities, and designing a mechanism to pool these encodings (across a
paragraph, document, or corpus) into a single representation. This
representation is then used to classify the relation between the entity pair
(Verga et al., 2018; Jia et al., 2019).

However, representation learning methods for both mention-level and pair-level
extraction typically use a point estimate for each representation. As a
result, they may struggle to capture the nature of the true, potentially
complex relations between each pair of entities. For example, Figure 1 shows
sentences for two entity pairs which demonstrate that relation statements can
be very different, typically depending on biological circumstances (e.g.
anatomical location, experimental details, presence of a disease, etc). Such
nuanced relations can be difficult to capture with a single point estimate.

We hypothesise that there is a true underlying relation for each entity pair,
and that this relation can be multimodal (because of the aforementioned
complexities). The sentences containing each pair are textual observations of
these underlying relations.

We therefore propose a probabilistic model which uses a continuous latent
variable to represent the true relation between each entity pair. The
distribution of a sentence containing that pair is then conditioned on this
latent variable. In order to be able to model the complex relations between
each entity pair, we use an infinite mixture distribution for the latent
representation.

Our model provides a unified architecture for learning representations of
relations between entity pairs both at mention and pair level. We show that
(an approximation to) the posterior distribution of the latent variable can be
used for mention-level relation classification. We also demonstrate that the
prior distribution from the same model can be used for pair-level
classification. On both tasks, we achieve results competitive with strong
baselines with a model which has fewer parameters and is significantly faster
to train.

The code is released at https://github.com/BenevolentAI/RELVM

Protein Akt and protein GSK3:
---
“… Akt negatively regulates GSK3 activity…”
“… Akt phosphorylates GSK3…”

Protein EAAT2 and disease ALS:
---
“EAAT2/C1-4 were found to be equally expressed in ALS patients and controls.”
“EAAT2 protein is significantly reduced in ALS in the motor cortex and spinal
cord.”

Figure 1: Two sets of sentences demonstrating the potentially complex nature
of the relation between a pair of entities.

##  2 Model

In this section, we introduce our unified architecture for both mention-level
and pair-level relation extraction. Throughout, we use the following notation:

  * •

$c$ represents a ‘context’, i.e. a sentence (or sequence of tokens) containing
a pair of entities. $c$ has tokens $c_{1},\ldots,c_{T}$.

    * –

$c_{t_{x}}$ and $c_{t_{y}}$ are the tokens representing the two entities. We
replace the actual tokens denoting the two entities with generic <ENT> tokens.
Therefore, a context is given by:

$\displaystyle c=\ $ $\displaystyle c_{1},\ldots,c_{t_{x}-1},\texttt{\textless
ENT\textgreater},c_{t_{x}+1},\ldots,$ $\displaystyle
c_{t_{y}-1},\texttt{\textless ENT\textgreater},c_{t_{y}+1},\ldots,c_{T}$

  * •

$x$ and $y$ are the input representations of the two entities.

    * –

For pair-level classification, $x$ and $y$ will be unique identifiers for the
two entities.

    * –

For mention-level classification, $x$ and $y$ will be the types of the two
entities, e.g. GENE and DISEASE. This is done in order to allow for fair
comparisons with previous methods, which use the entity types for mention-
level classification (see Section 4.2 for further details).

    * –

$x$ and $y$ always refer to the first and second entities in $c$ respectively.

  * •

$\mathbf{e}(c_{t})$ is the embedding of token $c_{t}$. $\mathbf{e}(x)$ and
$\mathbf{e}(y)$ are the embeddings of the entities $x$ and $y$.

  * •

$r$ represents the relation label.

##### Approach

Large corpora of labelled relation statements are often scarce, whereas
unlabelled sentences are usually plentiful. In order to leverage these
unlabelled sentences, we first train an unsupervised model to learn
representations of entity pairs and the contexts in which they occur. We then
train much smaller models to classify relations using the representations from
the unsupervised model.

###  2.1 Representation learning model

When training the unsupervised representation learning model, we assume access
to a corpus of sentences in which entities have been tagged but there are no
relation labels. We train the representation model to maximise the conditional
log-likelihood $\log p(c|x,y)$. $\theta$ will refer to the set of parameters
of the representation model which we wish to optimise. A graph of the
representation model is shown in Figure 2 and a more detailed explanation is
given below.

$x$$y$$\mathbf{z}$$\mathbf{u}$$c$$\theta$ Figure 2: A graph depicting our
unsupervised representation learning model. Clear nodes denote latent
variables and shaded nodes denote observed variables. Representations from
this model are used for both mention-level and pair-level relation
classification.

There are many ways to express the same relation between a given pair of
entities. For example, the sentences “John is Mary’s brother” and “Mary is
John’s sister” express the same relation in different ways. In order to
capture this phenomenon, we introduce a latent variable, $\mathbf{z}$, to
represent the true underlying relation. This will be the representation used
for mention-level and pair-level relation classification. The conditional
distribution is parametrised as:

$\displaystyle
p(c|x,y)=\int_{\mathbf{z}}p_{\theta}(\mathbf{z}|x,y)p_{\theta}(c|\mathbf{z})$
(1)

Intuitively, $p_{\theta}(\mathbf{z}|x,y)$ captures the true underlying
relation between the two entities $x$ and $y$, and $p_{\theta}(c|\mathbf{z})$
captures the variation in the multiple possible ways of expressing that
relation.

For computational simplicity, we could choose $p_{\theta}(\mathbf{z}|x,y)$ to
be Gaussian. However in reality, the true relation between a pair of entities
is probably more complex than can be modelled well with a unimodal
distribution. We therefore introduce another latent variable $\mathbf{u}$ such
that:

$\displaystyle
p_{\theta}(\mathbf{z}|x,y)=\int_{\mathbf{u}}p(\mathbf{u})p_{\theta}(\mathbf{z}|x,y,\mathbf{u})$
(2)

For $p(\mathbf{u})$, we use a standard Gaussian distribution,
$\mathcal{N}(\mathbf{0},\mathbf{I})$. For
$p_{\theta}(\mathbf{z}|x,y,\mathbf{u})$, we again use a Gaussian distribution
whose mean and variance are a function of $x$, $y$ and $\mathbf{u}$. We
concatenate together $\mathbf{e}(x)$, $\mathbf{e}(y)$,
$\mathbf{e}(x)\odot\mathbf{e}(y)$ and $\mathbf{u}$, and pass the resulting
vector into a feedforward network to output the mean and variance of
$p_{\theta}(\mathbf{z}|x,y,\mathbf{u})$ ($\odot$ denotes element-wise
multiplication). Using a nonlinear network allows the marginal distribution
$p_{\theta}(\mathbf{z}|x,y)$ to be an infinite mixture distribution (Mattei
and Frellsen, 2018). The objective becomes:

$\displaystyle\log
p(c|x,y)=\log\int_{\mathbf{u},\mathbf{z}}p(\mathbf{u})p_{\theta}(\mathbf{z}|x,y,\mathbf{u})p_{\theta}(c|\mathbf{z})$
(3)

We parametrise $p_{\theta}(c|\mathbf{z})$ with an LSTM, due to its strong
performance in language modelling (Graves, 2013; Bowman et al., 2016; Melis et
al., 2018). The conditional probabilities for $t=1,\ldots,T$ are:

$\displaystyle
p_{\theta}(c_{t}=v|c_{1:t-1},\mathbf{z})\propto\exp((\mathbf{W}\mathbf{h}^{p}_{t})\cdot\mathbf{e}(v))$
(4)

where $\mathbf{W}$ is a learnable parameter of the model, and
$\mathbf{h}_{t}^{p}$ is computed as:

$\displaystyle\mathbf{h}^{p}_{t}=\textrm{LSTM}(\mathbf{z},\mathbf{h}^{p}_{t-1},\mathbf{e}(c_{t-1}))$
(5)

Complete hyperparameter details are provided in Section 4.1.

####  2.1.1 Training

Because of the nonlinear functions involved in
$p_{\theta}(\mathbf{z}|x,y,\mathbf{u})$ and $p_{\theta}(c|\mathbf{z})$, the
integral in Equation (3) is intractable. We therefore perform approximate
maximum likelihood estimation using stochastic gradient variational Bayes
(SGVB) (Kingma and Welling, 2014; Rezende et al., 2014).

To do this, we parametrise a Gaussian inference distribution
$q_{\phi}(\mathbf{u}|x,y,c)$ (referred to as $q_{\phi}(\mathbf{u})$
henceforth, for brevity) with trainable parameters $\phi$. This allows us to
maximise the following lower bound on the log-likelihood:

$\displaystyle\log p(c|x,y)\\!$
$\displaystyle\geq\mathbb{E}_{q_{\phi}(\mathbf{u})p_{\theta}(\mathbf{z}|x,y,\mathbf{u})}\\!\\!\left[\log\frac{p(\mathbf{u})p_{\theta}(c|\mathbf{z})}{q_{\phi}(\mathbf{u})}\right]$
$\displaystyle\equiv\mathcal{L}_{\theta,\phi}(c,x,y)$ (6)

This bound can be approximated using Monte Carlo integration. It is optimised
with respect to $\theta$ and $\phi$ jointly.

To parametrise $q_{\phi}(\mathbf{u})$, we use a bidirectional LSTM to encode
the context. This is due to its ability to capture useful sentence level
information into a low-dimensional vector (Zhou et al., 2016; Peters et al.,
2018). It is computed as:

$\displaystyle\overrightarrow{\mathbf{h}^{q}_{t}}=\mathrm{LSTM}(\mathbf{e}(c_{t}),\overrightarrow{h^{q}_{t-1}})$
(7)
$\displaystyle\overleftarrow{\mathbf{h}^{q}_{t}}=\mathrm{LSTM}(\mathbf{e}(c_{t}),\overleftarrow{h^{q}_{t+1}})$
(8)
$\displaystyle\mathbf{h}^{q}=[\overrightarrow{\mathbf{h}^{q}_{T}};\overleftarrow{\mathbf{h}^{q}_{1}}]$
(9)

We concatenate $\mathbf{h}^{q}$ to $\mathbf{e}(x)$, $\mathbf{e}(y)$ and
$\mathbf{e}(x)\odot\mathbf{e}(y)$ and pass the resulting vector into a
feedforward network to output the mean and variance of $q_{\phi}(\mathbf{u})$.

###  2.2 Mention-level classification

In this section, we assume that the unsupervised representation model from
Section 2.1 has been trained with $x$ and $y$ being the types of the two
entities. The representations $\mathbf{z}$ can now be used as the inputs to a
supervised mention-level relation classification model.

For mention-level classification, we assume access to a corpus of sentences in
which entities have been tagged and there are labels classifying the type of
relation between the entity pair in each sentence. We train the mention-level
classification model to maximise $p(r|x,y,c)$. $\lambda$ will refer to the set
of parameters of the mention-level classification model which we wish to
optimise.

The representation $\mathbf{z}$ of the entity pair and context would ideally
be distributed according to the posterior $p(\mathbf{z}|x,y,c)$ from the
representation model. We would then optimise the parameters $\lambda$ using
the following objective:

$\displaystyle
p(r|x,y,c)=\int_{\mathbf{z}}p(\mathbf{z}|x,y,c)p_{\lambda}(r|\mathbf{z})$ (10)

However:

$\displaystyle p(\mathbf{z}|x,y,c)$
$\displaystyle=\frac{p_{\theta}(\mathbf{z},c|x,y)}{p(c|x,y)}$ (11)
$\displaystyle=\frac{p_{\theta}(\mathbf{z},c|x,y)}{\int_{\mathbf{u},\mathbf{z}}p(\mathbf{u})p_{\theta}(\mathbf{z}|x,y,\mathbf{u})p_{\theta}(c|\mathbf{z})}$
(12)

As mentioned in Section 2.1.1, the integral in the denominator is intractable.
Instead, the following approximation to the posterior can be used:

$\displaystyle
p(\mathbf{z}|x,y,c)\simeq\int_{\mathbf{u}}q_{\phi}(\mathbf{u})p_{\theta}(\mathbf{z}|x,y,\mathbf{u})$
(13)

This is an approximation to the posterior because maximising the objective in
Equation (6) is equivalent to minimising the KL divergence from
$q_{\phi}(\mathbf{u})p_{\theta}(\mathbf{z}|x,y,\mathbf{u})$ to
$p(\mathbf{z},\mathbf{u}|x,y,c)$ (Kingma and Welling, 2014):

$\displaystyle\mathcal{L}_{\theta,\phi}(c,x,y)=\log p(c|x,y)-$
$\displaystyle\quad
D_{\mathrm{KL}}[q_{\phi}(\mathbf{u})p_{\theta}(\mathbf{z}|x,y,\mathbf{u})||p(\mathbf{z},\mathbf{u}|x,y,c)]$
(14)

Using this approximation, the mention-level classification objective becomes:

$\displaystyle p(r|x,y,c)\\!$
$\displaystyle\simeq\\!\\!\int_{\mathbf{u},\mathbf{z}}q_{\phi}(\mathbf{u})p_{\theta}(\mathbf{z}|x,y,\mathbf{u})p_{\lambda}(r|\mathbf{z})$
(15)
$\displaystyle=\mathbb{E}_{q_{\phi}(\mathbf{u})p_{\theta}(\mathbf{z}|x,y,\mathbf{u})}[p_{\lambda}(r|\mathbf{z})]$
(16)

Empirically, however, we find that the model trains much more easily using the
following objective:

$\displaystyle\mathbb{E}_{q_{\phi}(\mathbf{u})p_{\theta}(\mathbf{z}|x,y,\mathbf{u})}[\log
p_{\lambda}(r|\mathbf{z})]\\!\equiv\\!\mathcal{L}_{\lambda}(r,c,x,y)$ (17)

This is due, particularly at the start of training, to the values of
$p_{\lambda}(r|\mathbf{z})$ being very small. Note that, due to Jensen’s
inequality, the objective in Equation (17) is in fact a lower bound on the
$\log$ of the objective in Equation (16):

$\displaystyle\mathcal{L}_{\lambda}(r,c,x,y)\\!\leq\\!\log\mathbb{E}_{q_{\phi}(\mathbf{u})p_{\theta}(\mathbf{z}|x,y,\mathbf{u})}[p_{\lambda}(r|\mathbf{z})]$
(18)

To parametrise $p_{\lambda}(r|\mathbf{z})$, we use a shallow feedforward
network with a softmax function at the output. Complete hyperparameter details
are provided in Section 4.2.

###  2.3 Pair-level classification

In this section, we assume that the unsupervised representation model from
Section 2.1 has been trained with $x$ and $y$ being unique identifiers for the
two entities. The representations $\mathbf{z}$ can now be used as the inputs
to a supervised pair-level relation classification model.

For pair-level classification, we assume access to a dataset with pairs of
entity identifiers, and labels classifying the type of relation between each
pair. Instead of learning $p(r|x,y,c)$ as in mention-level classification, we
now learn $p(r|x,y)$.

Intuitively, for pair-level classification, we wish to classify the relation
between a pair of entities based on everything that the unsupervised model has
learned about those entities (through the sentences containing them). This is
unlike mention-level classification, where we classify the relation described
in a specific sentence.

For pair-level classification, we follow a very similar approach to that
described in Section 2.2 for mention-level classification. However we no
longer base the input representation on the posterior distribution from the
unsupervised model, $p(\mathbf{z}|x,y,c)$. Instead, the representation used
will be distributed according to:

$\displaystyle
p_{\theta}(\mathbf{z}|x,y)=\int_{\mathbf{u}}p(\mathbf{u})p_{\theta}(\mathbf{z}|x,y,\mathbf{u})$
(19)

Intuitively, this is the natural distribution to use, because we are
interested in the relation between the entities $x$ and $y$, without a
specific context to condition on.

We denote $\psi$ as the parameters of the pair-level supervised model. Then,
following the same reasoning as Section 2.2, the objective for the pair-level
supervised model is:

$\displaystyle\mathbb{E}_{p(\mathbf{u})p_{\theta}(\mathbf{z}|x,y,\mathbf{u})}[\log
p_{\psi}(r|\mathbf{z})]\equiv\mathcal{L}_{\psi}(r,x,y)$ (20)

To parametrise $p_{\psi}(r|\mathbf{z})$, we use a shallow feedforward network
with a softmax function at the output. Complete hyperparameter details are
provided in Section 4.3.

##  3 Related work

Mention-level relation extraction is typically performed using supervised
learning. In the general domain, Zhang et al. (2017) combine an LSTM with a
position-aware attention mechanism to perform multiclass relation extraction.
Soares et al. (2019) fine-tune the BERT (Devlin et al., 2019) architecture to
relation extraction tasks by enforcing similarity between representations of
sentences containing the same pair of entities across a corpus. (Zhang et al.,
2020) construct a teacher model to generate soft labels which guide the
optimisation of a student network via knowledge distillation. In the
biomedical and scientific domains, BioBERT (Lee et al., 2019) and SciBERT
(Beltagy et al., 2019) train the BERT architecture on domain-specific corpora,
achieving state of the art results on mention-level relation extraction tasks.
Zhang et al. (2018) combine an RNN over the sentence’s words and a CNN over
its dependency graph to classify drug-drug and protein-protein interactions.

Pair-level relation extraction usually relies on distant supervision (Mintz et
al., 2009). In the general domain, Hoffmann et al. (2011) develop a latent
variable model to perform multi-instance learning while handling overlapping
relations. Lin et al. (2016) use an attention mechanism to pool the
representations of sentences containing a given pair into a single
representation, which is then used as the input to a classifier. Quirk and
Poon (2017) capture relations across sentences by linking dependency graphs
between sentences. Other pair-level methods build representations using
unsupervised models. Camacho-Collados et al. (2019) use a latent variable
model to learn a point-estimate representation from the unigram distribution
of tokens co-occurring in sentences with the given pair. Joshi et al. (2019)
learn representations of pairs of entities by maximising their pointwise
mutual information (PMI) with the contexts that the entities appear in. In the
biomedical domain, Verga et al. (2018) build a paragraph-level representation
using a modified Transformer network, and aggregate over mentions using a
softmax function. Liang et al. (2019) combine knowledge embeddings and graph
embeddings using a cascade learning framework to predict links in biochemical
networks. Percha and Altman (2015) use a distributional semantics approach to
cluster together drug-gene pairs which are related in similar ways.

Contrary to our work, there does not appear to be prior research performing
both mention-level and pair-level relation extraction with a unified model.

##  4 Experiments

###  4.1 Representation learning model

We train the unsupervised representation model described in Section 2.1 using
sentences from PubMed abstracts, PubMed Central (PMC) open-access full-text
articles, and licensed full-text articles from Wiley and Springer. We take
sentences with a maximum length of 140 tokens and tag the entities with their
type using a dictionary-based method. Entities are linked to unique
identifiers by first disambiguating entity types using a bidirectional LSTM
sentence classifier, followed by type-specific term lookups. Note that if a
sentence contains three or more entities, it is repeated in order to account
for each possible pair of entities.

####  4.1.1 Architectures and training

To parametrise $p_{\theta}(\mathbf{z}|x,y,\mathbf{u})$, we use a 2-layer
feedforward network with the ReLU nonlinearity. To parametrise
$p_{\theta}(c|\mathbf{z})$, we use a 1-layer LSTM. To parametrise
$q_{\phi}(\mathbf{u})$, we use a 1-layer bidirectional LSTM, the output of
which is passed to a 2-layer feedforward network with the ReLU nonlinearity.

In order to evaluate the effect of the number of parameters on performance, we
train four different versions of our representation learning model: {x-small,
small, medium, large}. These correspond to respective hidden state sizes of
{128, 256, 512, 1024} in the networks. For all of the models, both
$\mathbf{u}$ and $\mathbf{z}$, as well as all embeddings, are 300-dimensional.

We train the unsupervised representation models using a single sample
approximation of the objective in Equation (6). We train for 400,000
iterations, using a minibatch size of 192 and optimising the parameters using
Adam (Kingma and Ba, 2015) with a learning rate of 0.0001.

####  4.1.2 Optimisation challenges

The unsupervised objective in Equation (6) can be expressed as:

$\displaystyle\mathcal{L}_{\theta,\phi}(c,x,y)$
$\displaystyle=\mathbb{E}_{q_{\phi}(\mathbf{u})p_{\theta}(\mathbf{z}|x,y,\mathbf{u})}[\log
p_{\theta}(c|\mathbf{z})]$ $\displaystyle\quad-
D_{\mathrm{KL}}[q_{\phi}(\mathbf{u})||p(\mathbf{u})]$ (21)

When training latent variable models with autoregressive observation
distributions (such as that in Equation (4)), this objective can induce local
optima where $q_{\phi}(\mathbf{u})$ = $p(\mathbf{u})$. This results in the KL
divergence term in Equation (21) collapsing to 0, meaning the model ignores
the latent variable altogether. To avoid such local optima, we use the
following two methods (Bowman et al., 2016):

##### KL annealing

We multiply the KL divergence term by a constant weight which is linearly
annealed from 0 to 1 over the first 10,000 iterations of training. This helps
the model to escape local optima where
$D_{\mathrm{KL}}[q_{\phi}(\mathbf{u})||p(\mathbf{u})]=0$ early in training.

##### Token dropout

In Equation (5), we randomly drop the token embedding being passed to the next
LSTM hidden state. We use a dropout rate of 50%. This encourages the LSTM to
rely more on the representation $\mathbf{z}$ than the previous tokens when
modelling the context.

####  4.1.3 Computational costs

We show the computational costs of our unsupervised representation models in
Table 1. We compare against BioBERT (Lee et al., 2019), a language model with
state-of-the-art performance on relation extraction.

All versions of our model have significantly fewer parameters than BioBERT. In
terms of ‘GPU days’111GPU days = No. of GPUs $\times$ training time (in
days)., training BioBERT is approximately 25 to 40 times slower than training
our model. In addition, inference is an order of magnitude faster with our
model compared to BioBERT.

|  | Training | Inference
---|---|---|---
Model |  Params | Hardware |  Time | Hardware | Time
BioBERT | 110M | 8 x V100 GPUs | 10 days | 1 x V100 GPU | 0.0087s/sent.
Ours (x-small) | 2M | 1 x V100 GPU | 2 days | 1 x V100 GPU | 0.0004s/sent.
Ours (small) | 4M | 1 x V100 GPU | 2 days | 1 x V100 GPU | 0.0004s/sent.
Ours (medium) | 10M | 1 x V100 GPU | 3 days | 1 x V100 GPU | 0.0005s/sent.
Ours (large) | 30M | 1 x V100 GPU | 3 days | 1 x V100 GPU | 0.0007s/sent.
Table 1: The computational costs of each of the unsupervised representation
models we train. The inference time for each model is computed on a V100 GPU.

###  4.2 Mention-level classification

After training the unsupervised representation model (using the entity types
for $x$ and $y$), we use it to perform supervised mention-level relation
classification, as described in Section 2.2. We use the EU-ADR (van Mulligen
et al., 2012) and GAD (Bravo et al., 2015) datasets. In both datasets, each
sentence contains a gene and disease. The task is to classify whether the
given sentence either does or does not exhibit a relation between the gene and
the disease. Examples from both datasets are shown in Table 2 and dataset
statistics are shown in Table 3. As per previous work, we report the
performance using 10-fold cross validation on each dataset (Lee et al., 2019).

Dataset | $x$ | $y$ |  $c$ | $r$
---|---|---|---|---
EU-ADR | GENE | DISEASE |  Based on <ENT> analyses, 41 <ENT> patients and 12 healthy controls were studied. | 0
DISEASE | GENE |  <ENT> is associated with decreased expression of mucosal <ENT> . | 1
GAD | GENE | DISEASE |  A broad protective effect of <ENT> S180L against <ENT> per se is not discernible. | 0
GENE | DISEASE |  The <ENT> polymorphism Tyr402His appears indicative of <ENT> pathogenesis. | 1
Table 2: Positive ($r=1$) and negative ($r=0$) examples from the EU-ADR and GAD datasets. Dataset | EU-ADR | GAD
---|---|---
# relations | 355 | 5330
Table 3: Number of relations for the EU-ADR and GAD datasets.

We compare our results with those of BioBERT as reported by Lee et al. (2019).
For a fair comparison, we use the same classifier architecture. This is a
single layer network with a softmax nonlinearity. As well as training the
parameters $\lambda$ of the classifier, we also fine tune the parameters
$\theta$ and $\phi$ of the representation model. Again, this is done to allow
for a fair comparison with BioBERT (which follows the same procedure).

We approximate the objective in Equation (17) using 4 samples during training.
We use a minibatch size of 8 and update the parameters using Adam with a
learning rate of 0.00001. We train on EU-ADR for 200 iterations and on GAD for
3,000 iterations.

Note that the representations for BioBERT are 768-dimensional. This is in
contrast to ours which are 300-dimensional.

Model | EU-ADR | GAD
---|---|---
P | R | F | P | R | F
BioBERT | 80.92 | 90.81 | 84.83 | 75.95 | 88.08 | 81.52
Ours (x-small) | 79.62 | 98.08 | 87.71 | 67.83 | 90.76 | 77.45
Ours (small) | 80.35 | 98.09 | 88.14 | 68.31 | 91.75 | 78.16
Ours (medium) | 80.72 | 98.46 | 88.59 | 69.68 | 91.82 | 78.72
Ours (large) | 82.34 | 98.85 | 89.67 | 72.26 | 92.00 | 80.79
Table 4: Results using 10-fold cross validation on the EU-ADR and GAD
classification tasks. We report the mean precision (P), recall (R) and
F1-score (F) over the 10 folds. For all metrics, higher is better.

####  4.2.1 Results

We perform 10-fold cross validation, and report the mean precision, recall and
F1-score in Table 4. On EU-ADR, all versions of our model outperform BioBERT,
with our Large model achieving a significantly higher F1-score. On this task,
all versions of our model have significantly higher recall than BioBERT, with
the precision being similar. On GAD, BioBERT slightly outperforms our Large
model, thanks to its higher precision. In addition, we find that, on both
tasks, the performance monotonically increases with the size of the
unsupervised representation model.

These results show that it is possible to achieve results competitive with the
state-of-the-art while making significant efficiency gains, both in terms of
memory and time.

###  4.3 Pair-level classification

In this section, we use the large representation model from Section 4.1,
trained using the unique entity identifiers for $x$ and $y$. We fix the
parameters of the unsupervised representation model and use it to perform
supervised pair-level classification, as described in Section 2.3.

We construct a multiclass classification dataset by combining multiple third-
party biomedical datasets. These datasets only provide pairs of entities which
are related. Therefore, if an entity pair does not appear in any of the
datasets, they are assumed to be unrelated and given the label NO-RELATION. If
two entities are related, the label is given by the concatenation of the two
entity types. This is therefore a multiclass classification problem, with the
set of possible classes being {NO-RELATION, DISEASE-GENE, GENE-GENE, CHEMICAL-
GENE, CHEMICAL-DISEASE}. Note that we only include entity pairs that occur in
at least one sentence in the dataset used to train the representation learning
model.

We randomly split the related entity pairs into training, validation and test
sets. The set of entity pairs with label NO-RELATION is extremely large. We
randomly assign a proportion of these to the validation and test sets. During
training, we randomly sample a proportion of each minibatch from the remaining
unrelated entity pairs. The dataset statistics are shown in Table 5.

Dataset | Pair-level
---|---
Train (excl. NO-RELATION) | 263,112
Validation | 691,627
Test | 692,534
Table 5: Total counts across all relation types for the pair-level
classification dataset. The training set excludes NO-RELATION types, as these
are sampled during training.

For the pair-level classifier, we train a 2-layer model which has a
300-dimensional hidden layer with a skip connection. We approximate the
objective in Equation (20) using 4 samples during training. We train for
100,000 iterations, using a minibatch size of 512 (of which 448 are sampled
from the NO-RELATION set). We optimise the parameters using Adam with a
learning rate of 0.0001. When making predictions on unseen data points, we
only predict a label other than NO-RELATION if the predicted probability is
higher than a threshold. This threshold is tuned to maximise the F1-score on
the validation set.

####  4.3.1 Baselines

We compare our method with the following two baselines:

##### Co-occurrences

For every entity pair that occurs in at least one sentence in the dataset used
to train the representation learning model, we predict the relation to be
positive (i.e. the concatenation of the types of the two entities). By design,
this method will have perfect recall.

##### Attention

This method is similar to those presented by Lin et al. (2016) and Verga et
al. (2018). For a given pair of entities, we collect every sentence containing
the pair from the dataset used to train the representation learning model.
Each sentence is passed to an LSTM whose final state is taken as the sentence
representation. The representations for all sentences for the given entity
pair are pooled together into a single representation using an attention
mechanism. This representation is then used as the input to a feedforward
network with a softmax function at the output. This method is therefore
trained on exactly the same dataset as our pair-level classifier.

The attention model is trained for 1,000,000 iterations using a minibatch size
of 100 (of which 50 are sampled from the NO-RELATION set). The parameters are
optimised using Adam with a learning rate of 0.000005. As with our model, when
making predictions on unseen data points, we only predict a label other than
NO-RELATION if the predicted probability is higher than a threshold. This
threshold is tuned to maximise the F1-score on the validation set.

####  4.3.2 Results

Model | P | R | F
---|---|---|---
Co-occurrences | 3.10 | 100.00 | 6.02
Attention | 11.06 | 26.97 | 15.69
Ours | 12.54 | 25.91 | 16.90
Table 6: Results on the test set of the pair-level classification task. We
report the precision (P), recall (R) and the F1-score (F). For all metrics,
higher is better.

The precision, recall, and F1-score on the test set are reported in Table 6.
Our model achieves a higher F1-score than the attention model. Unsurprisingly,
both the attention model and our model achieve significantly higher precision
than the co-occurrence baseline at the expense of lower recall.

In contrast to the attention model, when classifying a new pair, our model
does not need to encode all of the sentences containing that pair. This
provides significant computational advantages, both in terms of memory and
time.

##  5 Conclusion

We have presented a model for learning representations of pairs of biomedical
entities from unlabelled text corpora. We use a latent variable with an
arbitrarily flexible distribution in order to be able to capture the complex
relations between each pair of entities. The unified architecture can be used
for both mention-level and pair-level relation extraction. On both tasks, we
achieve results competitive with strong baselines. We also show significant
computational gains in terms of the number of parameters and training times.

Our model presents many avenues for future work. The results in Table 4 show
that the model’s performance improves with the size of the hidden states in
the networks; this suggests that there are further gains achievable simply by
providing the model with more parameters. The model could be further scaled up
by using a hierarchy of latent variables to increase the expressive power of
the representations.

Other directions include evaluating the benefits of having a representation
which explicitly captures uncertainty about the relations. For example, this
can be done by assessing if the model is less confident when making
predictions about entity pairs which do not occur frequently in the unlabelled
corpus. Additionally, since our model can produce a representation for any
pair of entities (even those which do not occur together in the unlabelled
corpus), it could be used in a link prediction setting to score unseen entity
pairs.

## Acknowledgements

We would like to thank our colleagues Sia Togia and Angus Brayne for their
thorough feedback on this paper and Rogier Hintzen for his precious help in
preparing the datasets.

## References

  * Beltagy et al. (2019) I. Beltagy, K. Lo, and A. Cohan. 2019.  SciBERT: A Pretrained Language Model for Scientific Text.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing_.
  * Bengio et al. (2013) Y. Bengio, A. Courville, and P. Vincent. 2013.  Representation learning: A review and new perspectives.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 35.
  * Bowman et al. (2016) S. Bowman, L. Vilnis, O. Vinyals, A. Dai, R. Jozefowicz, and S. Bengio. 2016.  Generating Sentences from a Continuous Space.  In _Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning_.
  * Bravo et al. (2015) À. Bravo, J. Piñero, N. Queralt-Rosinach, M. Rautschka, and L. Furlong. 2015.  Extraction of Relations Between Genes and Diseases from Text and Large-Scale Data Analysis: Implications for Translational Research.  _BMC Bioinformatics_ , 16.
  * Camacho-Collados et al. (2019) J. Camacho-Collados, L. Espinosa-Anke, S. Jameel, and S. Schockaert. 2019.  A Latent Variable Model for Learning Distributional Relation Vectors.  In _Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence_.
  * Devlin et al. (2019) J. Devlin, M. Chang, K. Lee, and K. Toutanova. 2019.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics_.
  * Graves (2013) A. Graves. 2013.  Generating Sequences With Recurrent Neural Networks.  _CoRR_ , abs/1308.0850.
  * Hoffmann et al. (2011) R. Hoffmann, C. Zhang, X. Ling, L. Zettlemoyer, and D. Weld. 2011.  Knowledge-Based Weak Supervision for Information Extraction of Overlapping Relations.  In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics_.
  * Jia et al. (2019) R. Jia, C. Wong, and H. Poon. 2019.  Document-Level N-ary Relation Extraction with Multiscale Representation Learning.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics_.
  * Joshi et al. (2019) M. Joshi, E. Choi, O. Levy, D. Weld, and L. Zettlemoyer. 2019.  pair2vec: Compositional Word-Pair Embeddings for Cross-Sentence Inference.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics_.
  * Kingma and Ba (2015) D. Kingma and J. Ba. 2015.  Adam: A Method for Stochastic Optimization.  In _International Conference on Learning Representations_.
  * Kingma and Welling (2014) D. Kingma and M. Welling. 2014.  Auto-Encoding Variational Bayes.  In _International Conference on Learning Representations_.
  * Krallinger et al. (2017) M. Krallinger, O. Rabal, S. Akhondi, M. Pérez, J. Santamaría, G. Rodríguez, G. Tsatsaronis, A. Intxaurrondo, J. López, U. Nandal, E. van Buel, A. Chandrasekhar, M. Rodenburg, A. Lægreid, M. Doornenbal, J. Oyarzábal, A. Lourenço, and A. Valencia. 2017.  Overview of the BioCreative VI chemical-protein interaction Track.  In _Proceedings of the BioCreative VI Workshop_.
  * Lee et al. (2019) J. Lee, W. Yoon, S. Kim, D. Kim, S. Kim, C. So, and J. Kang. 2019.  BioBERT: A Pre-Trained Biomedical Language Representation Model for Biomedical Text Mining.  _Bioinformatics_.
  * Liang et al. (2019) X. Liang, D. Li, M. Song, A. Madden, Y. Ding, and Y. Bu. 2019.  Predicting Biomedical Relationships Using the Knowledge and Graph Embedding Cascade Model.  _PLOS ONE_ , 14.
  * Lin et al. (2016) Y. Lin, S. Shen, Z. Liu, H. Luan, and M. Sun. 2016.  Neural Relation Extraction with Selective Attention over Instances.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics_.
  * Mattei and Frellsen (2018) P. Mattei and J. Frellsen. 2018.  Leveraging the Exact Likelihood of Deep Latent Variable Models.  In _Advances in Neural Information Processing Systems 31_.
  * Melis et al. (2018) G. Melis, C. Dyer, and P. Blunsom. 2018.  On the State of the Art of Evaluation in Neural Language Models.  In _International Conference on Learning Representations_.
  * Mintz et al. (2009) M. Mintz, S. Bills, R. Snow, and D. Jurafsky. 2009.  Distant Supervision for Relation Extraction Without Labeled Data.  In _Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP_.
  * van Mulligen et al. (2012) E. van Mulligen, A. Fourrier-Reglat, D. Gurwitz, M. Molokhia, A. Nieto, G. Trifiro, J. Kors, and L. Furlong. 2012.  The EU-ADR Corpus.  _Journal of Biomedical Informatics_ , 45.
  * Percha and Altman (2015) B. Percha and R. Altman. 2015.  Learning the Structure of Biomedical Relationships from Unstructured Text.  _PLoS Computational Biology_ , 11.
  * Peters et al. (2018) M. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer. 2018.  Deep Contextualized Word Representations.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics_.
  * Quirk and Poon (2017) C. Quirk and H. Poon. 2017.  Distant Supervision for Relation Extraction beyond the Sentence Boundary.  In _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics_.
  * Rezende et al. (2014) D. Rezende, S. Mohamed, and D. Wierstra. 2014.  Stochastic Backpropagation and Approximate Inference in Deep Generative Models.  In _Proceedings of the 31st International Conference on Machine Learning_.
  * Segura-Bedmar et al. (2013) I. Segura-Bedmar, P. Martínez, and M. Herrero-Zazo. 2013.  SemEval-2013 Task 9: Extraction of Drug-Drug Interactions from Biomedical Texts (DDIExtraction 2013).  In _Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation_.
  * Soares et al. (2019) L. Soares, N. FitzGerald, J. Ling, and T. Kwiatkowski. 2019.  Matching the Blanks: Distributional Similarity for Relation Learning.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_.
  * Vaswani et al. (2017) A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, Ł Kaiser, and I. Polosukhin. 2017.  Attention is All you Need.  In _Advances in Neural Information Processing Systems_.
  * Verga et al. (2018) P. Verga, E. Strubell, and A. McCallum. 2018.  Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction.  In _North American Chapter of the Association for Computational Linguistics_.
  * Zhang et al. (2018) Y. Zhang, H. Lin, Z. Yang, J. Wang, S. Zhang, Y. Sun, and L. Yang. 2018.  A Hybrid Model Based on Neural Networks for Biomedical Relation Extraction.  _Journal of Biomedical Informatics_ , 81.
  * Zhang et al. (2017) Y. Zhang, V. Zhong, D. Chen, G. Angeli, and C. Manning. 2017.  Position-aware Attention and Supervised Data Improve Slot Filling.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_.
  * Zhang et al. (2020) Zhenyu Zhang, Xiaobo Shu, Bowen Yu, Tingwen Liu, Jiapeng Zhao, Quangang Li, and Li Guo. 2020.  Distilling Knowledge from Well-Informed Soft Labels for Neural Relation Extraction.  In _The Thirty-Fourth AAAI Conference on Artificial Intelligence_.
  * Zhou et al. (2016) P. Zhou, Z. Qi, S. Zheng, J. Xu, H. Bao, and B. Xu. 2016.  Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling.  In _Proceedings of the 26th International Conference on Computational Linguistics_.


11institutetext: University of Toronto 22institutetext: Carleton College

# Using Adaptive Experiments to Rapidly Help Students

Angela Zavaleta-Bernuy 11 Qi Yin Zheng 11 Hammad Shaikh 11 Jacob Nogas 11 Anna
Rafferty 22 Andrew Petersen 11 Joseph Jay Williams 11

###### Abstract

Adaptive experiments can increase the chance that current students obtain
better outcomes from a field experiment of an instructional intervention. In
such experiments, the probability of assigning students to conditions changes
while more data is being collected, so students can be assigned to
interventions that are likely to perform better. Digital educational
environments lower the barrier to conducting such adaptive experiments, but
they are rarely applied in education. One reason might be that researchers
have access to few real-world case studies that illustrate the advantages and
disadvantages of these experiments in a specific context. We evaluate the
effect of homework email reminders in students by conducting an adaptive
experiment using the Thompson Sampling algorithm and compare it to a
traditional uniform random experiment. We present this as a case study on how
to conduct such experiments, and we raise a range of open questions about the
conditions under which adaptive randomized experiments may be more or less
useful.

###### Keywords:

Reinforcement learning Randomized experiments Multi-armed bandits A/B testing
Field deployment

##  1 Introduction

Instructors frequently look for ways to support their students and improve
their performance. With access to online learning environments, instructors
can gather feedback in a larger scale setting. Since optimal instructional
designs and scaffolds may not be known ahead of time, multiple possibilities
can be tested using Uniform Random (UR) A/B experiments, also known as
randomized control trials. In a UR A/B experiment, students are uniformly
assigned to the different conditions that an instructor or researcher would
like to test to learn about their relative effectiveness. One challenge of
this approach is how to use data more rapidly to help current students. To
mitigate this, we can aim to maximize total learning by having most students
being subject to the more effective conditions as they become known.

Adaptive randomization is an effective strategy for assigning more students to
the current most optimal condition, while retaining the ability to test other
conditions. We use a Multi-Armed Bandit (MAB) algorithm that uses machine
learning to increase the number of students assigned to the current most
effective condition (or arm) [1], [7]. MAB are commonly used for rapid use of
data in different areas such as marketing to optimize the benefits of the
users and balance exploration vs. exploitation [1], [3]. For this study, we
used Thompson Sampling (TS), a probability matching algorithm, where the
probability of assignment is proportional to the probability that the arm is
optimal [1].

In this paper, we present a real-world experiment to illustrate the benefits
and limitations of using UR A/B experiments and TS in educational settings.
First, we use UR A/B experiments to evaluate different versions of emails in a
homework reminder intervention to determine if a more effective version can be
identified. We then compare the results of UR A/B experiments with the TS
results to study its performance and benefits. Our experimental design allows
us to compare classical balanced A/B comparisons side-by-side with a TS
adaptive experiment to evaluate the trade-offs of using each of these methods.

##  2 Multi-Armed Bandit (MAB) Algorithms

To optimize the experience of students, we use the TS algorithm designed to
solve MAB problems, useful for adaptively assigning participants to
conditions. [7].

The stochastic MAB problem is the problem of sequentially choosing from a
discrete set of actions to maximize cumulative reward, where a reward is some
measure of the effectiveness of the chosen action (arm). In this paper, we
focus on the MAB problem with binary rewards. More precisely, we choose
between $K$ versions (arms), and we denote the choice of action at step $t$ of
the experiment by $x_{t}$. Assuming we choose the $k-{th}$ arm, where
$k\in\\{1,2,\ldots,K\\}$, then $x_{t}=k$, and we receive a reward $r_{t}$ with
probability $p_{k}$.

TS shows strong empirical performance in maximizing the cumulative reward [5].
TS is a Bayesian algorithm that maintains a posterior distribution over each
reward $p_{k}$. In our case, we use a Beta prior with parameters $\alpha_{k}$
and $\beta_{k}$. Arms are chosen by sampling values from the posteriors over
each arm, and choosing the arm corresponding to the highest sample drawn. The
posterior distribution is then updated based on the chosen action $x_{t}=k$
and observed reward $r_{t}$. We use a uniform prior for each arm,
$\alpha_{k}=\beta_{k}=1$, for all $k\in\\{1,2,...,K\\}$.

$(\alpha_{k},\beta_{k})\leftarrow\begin{cases}(\alpha_{k},\beta_{k})&\text{if
}x_{t}\neq k\\\ (\alpha_{k},\beta_{k})+(r_{t},1-r_{t})&\text{if }x_{t}=k\\\
\end{cases}$

##  3 Methods: Traditional and Adaptive Experimentation

For three consecutive weeks, we tested four different versions of the emails
to investigate which might be more effective in leading students to click on
the homework link appended in the email. To evaluate how TS adapts the
distribution of students to each email version, we sent the messages in four
different batches on consecutive days of the week (Tuesday to Friday).

Using UR, each of the four email reminders has the same probability of being
assigned to a student. For the TS algorithm, the probability of assignment of
the email reminders version is proportional to the reward (in our case, click
rate) in all the previous batches, which is updated after each batch.

For our interventions, we used two different variations of the TS algorithm.
For Weeks 1 and 2, we used a UR-TS Hybrid where the TS updating of the
probability that an arm has the highest click rate used data from the UR
participants too. This hybrid is called $\epsilon$[0.5]-TS, because with
epsilon (epsilon = 0.5) probability, arms are assigned using UR. This takes
inspiration from past algorithms like epsilon-greedy [6] and top-two TS [4].
This is interesting to investigate because scientists may want to get the
benefits of obtaining data under UR (in case TS introduces biases [2]) for
analysis, while also then using that data to improve the performance of the TS
algorithm. For Week 3, we applied traditional TS that did not use the data
from the UR assignment.

##  4 Analysis and Results

Using a panel regression with week fixed effects (i.e., include indicators for
Week2-$\epsilon$[0.5]-TS and Week3-TS), we can aggregate the uniform portion
of the experiment for the three weeks to evaluate the effects of the four arms
on student click rate. We find that the average click rate in our sample
across the three weeks is around $19\%.$ All four arms had click rates are
within 2% of each other and their difference is not statistically significant
regardless of time and participant effects. The lack of an optimal arm
suggests that all students were assigned to fairly similar treatments. These
results are robust to also including student fixed effects within the panel
regression model.

For both of the later weeks, $\epsilon$[0.5]-TS behaves similarly, favouring
one particular arm. Some arms are assigned a substantially higher number of
students and the others far fewer, but the seemingly favoured arm varies
across the two weeks and is not consistent in Week2-$\epsilon$[0.5]-TS. In
Table 1, we show the cumulative reward per arm and per batch, which influenced
the probability of assignment. Even though the probability of assignment
aligns with the click rates from the previous batch, the algorithm drastically
shifts to the arm with the highest reward and leaves minimal room for
exploration in batch 3. As the click rates were not consistent across the
different batches, we are unable to identify the presence of the most
efficient arm, which is reflected on the cumulative rewards in Table 1, and
could be a result of there being minimal differences between arms.

|  | Arm 1 | Arm 2 | Arm 3 | Arm 4
---|---|---|---|---|---
|  | CCR | PA | CCR | PA | CCR | PA | CCR | PA
Week1-$\epsilon$[0.5]-TS | Batch 1 | 0.200 | 0.117 | 0.277 | 0.659 | 0.212 | 0.177 | 0.167 | 0.047
Batch 2 | 0.219 | 0.466 | 0.22 | 0.443 | 0.149 | 0.040 | 0.152 | 0.052
Batch 3 | 0.206 | 0.434 | 0.209 | 0.452 | 0.137 | 0.017 | 0.163 | 0.097
Batch 4 | 0.163 | 0.077 | 0.205 | 0.697 | 0.161 | 0.104 | 0.162 | 0.122
Week2-$\epsilon$[0.5]-TS | Batch 1 | 0.213 | 0.116 | 0.086 | 0.000 | 0.246 | 0.225 | 0.300 | 0.659
Batch 2 | 0.198 | 0.082 | 0.14 | 0.004 | 0.244 | 0.311 | 0.266 | 0.603
Batch 3 | 0.197 | 0.065 | 0.186 | 0.041 | 0.261 | 0.666 | 0.235 | 0.229
Batch 4 | 0.194 | 0.052 | 0.209 | 0.109 | 0.249 | 0.505 | 0.240 | 0.334
Week3-TS | Batch 1 | 0.231 | 0.477 | 0.153 | 0.056 | 0.22 | 0.389 | 0.157 | 0.078
Batch 2 | 0.233 | 0.926 | 0.137 | 0.030 | 0.135 | 0.033 | 0.120 | 0.011
Batch 3 | 0.183 | 0.510 | 0.129 | 0.039 | 0.133 | 0.070 | 0.174 | 0.381
Batch 4 | 0.181 | 0.405 | 0.152 | 0.086 | 0.144 | 0.076 | 0.181 | 0.433
Table 1: The Table presents the cumulative click rate (CCR) and the
probability of assignment (PA) for each arm in the next batch for the three
weeks of interventions. The arms represent a version of the message. Each
batch represents a separate day of message deployments: Monday (Batch 1),
Tuesday (Batch 2), Wednesday (Batch 3), and Thursday (Batch 4). The cumulative
click rate of each arm represents the percentage of people who received the
message that clicked accumulated up to and including that batch. The
probability of assignment is the probability that a particular arm will be
chosen in the next batch calculated using 1000000 Monte Carlo simulations.
Bold values highlight the arm with the highest probability of assignment for
each batch on every week.

##  5 Discussion & Limitations

We present a case study of adaptive experimentation, using the Thompson
Sampling MAB algorithm, when a difference between arms is not observed (i.e.,
the arms/conditions are equally effective), according to a traditional Uniform
Random experiment. We illustrate how TS may randomly favour an arm, even when
giving this arm more frequently has no consequences for participants. The TS
algorithm minimizes regret — it aims to keep participants from being assigned
to sub-optimal arms — so in the case where arms are equivalent, one could
argue that any of them could be presented. However, this can be problematic
for scientific inference and statistical analysis [2].

One limitation is that there might have been unobserved confounding variables
that caused the click rates to change in one particular batch (e.g., students
had an assignment due or a test), which will also affect the algorithm – this
is one concern to keep in mind in applying MAB algorithms for adaptive
experimentation.

##  6 Conclusion & Future Work

This paper provides an example of a real-world intervention using adaptive
experiments, which can help instructors and researchers to use the results
from experiments to more rapidly benefit students. We illustrate an instance
of conducting such experiments using the TS algorithm, where the results
suggest there is no difference between arms/conditions. We hope that this
paper provides a first step for instructors and researchers to investigate
adaptive experimentation in education. Future work can explore how the
algorithm behaves in a wider variety of scenarios, such as different batch
sizes and structure, and alternative differences between the arm/conditions.

##  7 Acknowledgements

This work was partially supported by the Natural Sciences and Engineering
Research Council of Canada (NSERC) (#RGPIN-2019-06968), as well as by the
Office of Naval Research (ONR) (#N00014-21-1-2576).

## References

  * [1] Lomas, J.D., Forlizzi, J., Poonwala, N., Patel, N., Shodhan, S., Patel, K., Koedinger, K., Brunskill, E.: Interface design optimization as a multi-armed bandit problem. In: Proceedings of the 2016 CHI conference on human factors in computing systems. pp. 4142–4153 (2016)
  * [2] Rafferty, A., Ying, H., Williams, J.: Statistical consequences of using multi-armed bandits to conduct adaptive educational experiments. JEDM— Journal of Educational Data Mining 11(1), 47–79 (2019)
  * [3] Rafferty, A.N., Ying, H., Williams, J.J.: Bandit assignment for educational experiments: Benefits to students versus statistical power. In: International Conference on Artificial Intelligence in Education. pp. 286–290. Springer (2018)
  * [4] Russo, D.: Simple bayesian algorithms for best arm identification. In: Conference on Learning Theory. pp. 1417–1418 (2016)
  * [5] Sutton, R.S., Barto, A.G.: Reinforcement learning: An introduction. MIT press (2018)
  * [6] Watkins, C.J.C.H.: Learning from delayed rewards (1989)
  * [7] Williams, J.J., Rafferty, A.N., Tingley, D., Ang, A., Lasecki, W.S., Kim, J.: Enhancing online problems through instructor-centered tools for randomized experiments. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. pp. 1–12 (2018)