$\tau(\widehat{\theta}_{n,M})-\tau(\bar{\theta})$ with an error
$O\left(\frac{1}{\sqrt{n}}\right)$ so $\widehat{C}^{*}_{n,\alpha}$ has a
probability $1-\alpha+O\left(\frac{1}{\sqrt{n}}\right)$ of containing
$\tau(\bar{\theta})$. Let $E_{1}$ denotes the event such that the CI
$\widehat{C}^{*}_{n,\alpha}$ covers $\tau(\bar{\theta})$. Then the above
analysis shows that

$P(E_{1})=P(\tau(\bar{\theta})\in\widehat{C}^{*}_{n,\alpha})\geq
1-\alpha+O\left(\frac{1}{\sqrt{n}}\right).$ (31)

Let $E_{2}$ denotes the event that
$\bar{\theta}\in{{\mathcal{C}}}^{\pi}_{M,\delta}$. Then when both
$E_{1},E_{2}$ are true,
$\widehat{C}^{*}_{n,\alpha}\cap\tau({{\mathcal{C}}}^{\pi}_{M,\delta})\neq\emptyset$.
So the coverage of $\widehat{C}^{*}_{n,\alpha}$ can be derived as

$\displaystyle
P(\widehat{C}^{*}_{n,\alpha}\cap\tau({{\mathcal{C}}}^{\pi}_{M,\delta})\neq\emptyset)$
$\displaystyle\geq P(E_{1},E_{2})$ $\displaystyle\geq
1-P(E_{1}^{c})-P(E_{2}^{c})$ $\displaystyle=P(E_{1})-P(E_{2}^{c}).$

In the proof of Theorem 5, we have shown

$P(E_{2})=P(\bar{\theta}\in{{\mathcal{C}}}^{\pi}_{M,\delta})\geq
1-\delta+O\left(\sqrt{\frac{\log n}{n}}\right).$

This, together with equation (31), implies

$\displaystyle
P(\widehat{C}^{*}_{n,\alpha}\cap\tau({{\mathcal{C}}}^{\pi}_{M,\delta})\neq\emptyset)$
$\displaystyle\geq P(E_{1})-P(E_{2}^{c})$ $\displaystyle\geq
1-\alpha-\delta+O\left(\sqrt{\frac{\log n}{n}}\right),$

which completes the proof.

$\square$

Before proving Theorem 7, we first recall a useful theorem from Balakrishnan
et al. (2017).

######  Theorem 13 (Theorem 1 in Balakrishnan et al. (2017)).

Let $M(\theta)={\sf argmax}_{\theta^{\prime}}Q(\theta^{\prime}|\theta)$ be the
updated point from the population EM algorithm with a starting point $\theta$.
Assume that for every $\theta_{1},\theta_{2}\in B(\theta_{MLE},r_{1})$, the
following holds with some $\lambda>\gamma\geq 0$:

  * •

($\lambda$-strongly concave)

$q(\theta_{1})-q(\theta_{2})-(\theta_{1}-\theta_{2})^{T}\nabla
q(\theta_{2})\leq-\frac{\lambda}{2}\|\theta_{1}-\theta_{2}\|^{2}_{2}.$

  * •

(First-order stability)

$\|\nabla Q(M(\theta)|\theta)-\nabla
Q(M(\theta)|\theta_{MLE})\|_{2}\leq\gamma\|\theta-\theta_{MLE}\|.$

Then the EM algorithm is contractive within $B(\theta_{MLE},r_{1})$ toward
$\theta_{MLE}$. Namely,

$B(\theta_{MLE},r_{1})\subset\mathcal{A}^{EM}(\theta_{MLE}).$

ProofProof of Theorem 7.

To prove Theorem 7, we first show that the sample EM algorithm is contractive
to $\widehat{\theta}_{MLE}$ within $B(\theta_{MLE},r_{0}/3)$ and then we
calculate the probability of selecting an initial point within this ball.

Under assumption (EM2-1), the function $q(\theta)$ is $\lambda_{0}$-strongly
concave within $B(\theta_{MLE},r_{0})$, where $\lambda_{0}$ is the constant in
assumption (EM2-1). To see this, the Taylor’s theorem implies that for any
$\theta_{1},\theta_{2}\in B(\theta_{MLE},r_{0})$,

$\displaystyle q(\theta_{1})-q(\theta_{2})$
$\displaystyle=(\theta_{1}-\theta_{2})^{T}\nabla
q(\theta_{2})+\frac{1}{2}(\theta_{1}-\theta_{2})^{T}\int_{s=0}^{s=\theta_{2}-\theta_{1}}\nabla\nabla
q(\theta_{1}+s)ds$ (32) $\displaystyle=(\theta_{1}-\theta_{2})^{T}\nabla
q(\theta_{2})+\frac{1}{2}(\theta_{1}-\theta_{2})^{T}\left(\int_{t=0}^{t=1}\nabla\nabla
q(\theta_{1}+t(\theta_{2}-\theta_{1}))dt\right)(\theta_{1}-\theta_{2})$
$\displaystyle\leq(\theta_{1}-\theta_{2})^{T}\nabla
q(\theta_{2})+\frac{1}{2}\|\theta_{1}-\theta_{2}\|^{2}\sup_{\theta\in
B(\theta_{MLE},r_{0})}\lambda_{\max}(\nabla\nabla q(\theta))$

because for any $t\in[0,1]$, $\theta_{1}+t(\theta_{2}-\theta_{1})\in
B(\theta_{MLE},r_{0})$. Recall that assumption (EM2-1) requires

$\sup_{\theta\in B(\theta_{MLE},r_{0})}\lambda_{\max}(\nabla\nabla
q(\theta))\leq-\lambda_{0}<0.$

After rearrangement, the equation (32) becomes

$q(\theta_{1})-q(\theta_{2})-(\theta_{1}-\theta_{2})^{T}\nabla
q(\theta_{2})\leq-\frac{\lambda_{0}}{2}\|\theta_{1}-\theta_{2}\|^{2},$

which implies that $q(\theta)$ is $\lambda_{0}$-strongly concave.

Moreover, assumption (EM2-2) implies a first-order stability with
$\gamma=\kappa_{0}$, where $\kappa_{0}$ is defined in assumption (EM2-2). To
see this, again we use the Taylor’s theorem

$\displaystyle\nabla Q(M(\theta)|\theta)$ $\displaystyle-\nabla
Q(M(\theta)|\theta_{MLE})$ (33)
$\displaystyle=\int_{s=0}^{s=\theta-\theta_{MLE}}\nabla\nabla
Q(M(\theta)|\theta_{MLE}+s)ds$
$\displaystyle=(\theta-\theta_{MLE})^{T}\int_{t=0}^{t=1}\nabla\nabla
Q(M(\theta)|\theta_{MLE}+t(\theta-\theta_{MLE}))dt.$

Because for every $t\in[0,1]$, $\theta_{MLE}+t(\theta-\theta_{MLE})\in
B(\theta_{MLE},r_{0})$, assumption (EM2-2) implies

$\left\|\int_{t=0}^{t=1}\nabla\nabla
Q(M(\theta)|\theta_{MLE}+t(\theta-\theta_{MLE}))dt\right\|_{2}\leq\sup_{\theta,\theta^{\prime}\in
B(\theta_{MLE},r_{0})}\|\nabla\nabla
Q(\theta|\theta^{\prime})\|_{2}\leq\kappa_{2}.$

Thus, taking the 2-norm in both sides of equation (33) leads to

$\|\nabla Q(M(\theta)|\theta)-\nabla
Q(M(\theta)|\theta_{MLE})\|\leq\kappa_{0}\|\theta-\theta_{MLE}\|,$

which proves the first-order stability.

With the two properties (strong concavity and first-order stability), we will
prove the theorem by using the following events:

$\displaystyle E_{1}$
$\displaystyle=\left\\{\|\widehat{\theta}_{MLE}-\theta_{MLE}\|\leq
r_{0}/3\right\\}$ $\displaystyle E_{2}$
$\displaystyle=\\{\|\nabla\nabla\widehat{q}(\theta)-\nabla\nabla
q(\theta)\|_{2}\leq\lambda_{0}/2\\}$ $\displaystyle E_{3}$
$\displaystyle=\\{\|\nabla_{\theta}\nabla_{\theta^{\prime}}\widehat{Q}(\theta|\theta^{\prime})-\nabla_{\theta}\nabla_{\theta^{\prime}}Q(\theta|\theta^{\prime})\|\leq\kappa_{0}/2\\}$
$\displaystyle E_{4}$
$\displaystyle=\left\\{\left|\widehat{\Pi}_{n}\left(B\left(\theta_{MLE},\frac{r_{0}}{3}\right)\right)-\Pi\left(B\left(\theta_{MLE},\frac{r_{0}}{3}\right)\right)\right|\leq
q_{EM}\right\\}.$

When $E_{1}$, $E_{2}$, and $E_{3}$ occur, the sample EM algorithm is
contractive within $B(\widehat{\theta}_{MLE},2r_{0}/3)\subset
B(\theta_{MLE},r_{0})$ by Theorem 13 because $\widehat{q}(\theta)$ is
$\lambda_{0}/2$-strongly concave and the first-order stability holds with
$\gamma=\kappa_{0}$ in $B(\widehat{\theta}_{MLE},2r_{0}/3)$. Thus, Theorem 13
implies that

$B(\widehat{\theta}_{MLE},2r_{0}/3)\subset\widehat{\mathcal{A}}^{EM}(\widehat{\theta}_{MLE}),$

where $\widehat{\mathcal{A}}^{EM}(\widehat{\theta}_{MLE})$ is the basin of
attraction of $\widehat{\theta}_{MLE}$ using the gradient of
$\widehat{L}_{n}$.

Moreover, under $E_{1}$, $\|\widehat{\theta}_{MLE}-\theta_{MLE}\|\leq r_{0}/3$
so the ball

$B(\theta_{MLE},r_{0}/3)\subset
B(\widehat{\theta}_{MLE},2r_{0}/3)\subset\widehat{\mathcal{A}}^{EM}(\widehat{\theta}_{MLE}).$

Thus, when we initializes the EM algorithm many times, as long as one initial
point falls within $B(\theta_{MLE},r_{0}/3)$,
$\widehat{\theta}^{EM}_{n,M}=\widehat{\theta}_{MLE}$.

Because we choose the initial point from $\widehat{\Pi}_{n}$, the chance of
having an initial point within $B(\theta_{MLE},r_{0}/3)$ is
$\widehat{\Pi}_{n}(B(\theta_{MLE},r_{0}/3)).$ Thus, after $M$ initialization,
the chance of having at least one initial point within
$B(\theta_{MLE},r_{0}/3)$ is

$1-\left(1-\widehat{\Pi}_{n}\left(B\left(\theta_{MLE},\frac{r_{0}}{3}\right)\right)\right)^{M}.$

When $E_{4}$ occurs,

$\displaystyle\widehat{\Pi}_{n}\left(B\left(\theta_{MLE},\frac{r_{0}}{3}\right)\right))$
$\displaystyle=\underbrace{\Pi\left(B\left(\theta_{MLE},\frac{r_{0}}{3}\right)\right)}_{=2q_{EM}}-\underbrace{\Pi\left(B\left(\theta_{MLE},\frac{r_{0}}{3}\right)\right)+\widehat{\Pi}_{n}\left(B\left(\theta_{MLE},\frac{r_{0}}{3}\right)\right)}_{\leq
q_{EM}}$ $\displaystyle\geq 2q_{EM}-q_{EM}=q_{EM}.$

Therefore,

$P\left(\widehat{\theta}^{EM}_{n,M}=\widehat{\theta}_{MLE}|E_{1},E_{2},E_{3},E_{4}\right)\geq
1-\left(1-q_{EM}\right)^{M}.$

Using the above inequality, we can bound the probability

$\displaystyle
P\left(\widehat{\theta}^{EM}_{n,M}=\widehat{\theta}_{MLE}\right)$
$\displaystyle\geq
P\left(E_{1},E_{2},E_{3},E_{4},\widehat{\theta}^{EM}_{n,M}=\widehat{\theta}_{MLE}\right)$
(34) $\displaystyle\geq
P\left(\widehat{\theta}^{EM}_{n,M}=\widehat{\theta}_{MLE}|E_{1},E_{2},E_{3},E_{4}\right)P(E_{1},E_{2},E_{3},E_{4})$
$\displaystyle\geq\left(1-\left(1-q_{EM}\right)^{M}\right)\cdot
P(E_{1},E_{2},E_{3},E_{4}).$

Because

$\displaystyle P(E_{1},E_{2},E_{3},E_{4})$ $\displaystyle=1-P(E_{1}^{c}\cup
E_{2}^{c}\cup E^{c}_{3}\cup E^{c}_{4})$ $\displaystyle\geq
1-P(E_{1}^{c})-P(E_{2}^{c})-P(E_{3}^{c})-P(E_{4}^{c}),$

we only need to bound each $P(E_{j}^{c})$. For the event $E_{1}$, a slightly
modification of the derivation of equation (19) shows that
$\widehat{\theta}_{MLE}-\theta_{MLE}=H^{-1}(\theta_{MLE})(\widehat{S}(\theta_{MLE})-S(\theta_{MLE})))(1+o_{P}(1))$,
which leads to

$\widehat{\theta}_{MLE}-\theta_{MLE}=O(\epsilon_{1,n}).$

Thus, a necessary condition to $E_{1}^{c}$ is $\epsilon_{1,n}>C_{0}\cdot
r_{0}/3$ for some positive number $C_{0}$. Again, by Lemma 12 and assumption
(A3L),

$P(E_{1}^{c})=P(\|\widehat{\theta}_{MLE}-\theta_{MLE}\|>r_{0}/3)\leq
A_{1,1}e^{-A_{1,2}n},$

for some positive number $A_{1,1}$ and $A_{1,2}$. Similarly, assumption (A3L)
implies that the gradient and the Hessian of the log-likelihood function are
Lipschitz so Lemma 12 implies

$\displaystyle P(E_{2}^{c})$ $\displaystyle\leq A_{2,1}e^{-A_{2,2}n},$
$\displaystyle P(E_{3}^{c})$ $\displaystyle\leq A_{3,1}e^{-A_{3,2}n},$

for some positive numbers $A_{2,1},A_{2,2},A_{3,1},A_{3,2}$. Finally, by
assumption (EM4),

$P(E_{4}^{c})\leq\eta_{n}(q_{EM}).$

Therefore, putting it altogether,

$P(E_{1},E_{2},E_{3},E_{4})\geq 1-c_{1}e^{-c_{2}n}-\eta_{n}(q_{EM})$

for some positive numbers $c_{1}$ and $c_{2}$. Putting this into equation
(34), we obtain

$P\left(\widehat{\theta}^{EM}_{n,M}=\widehat{\theta}_{MLE}\right)\geq
1-\left(1-q_{EM}\right)^{M}-\eta_{n}(q_{EM})-c_{1}e^{-c_{2}n},$

which is the desired probability bound.

To see that $\widehat{\theta}_{MLE}\in{\mathcal{A}}^{EM}(\theta_{MLE})$,
Theorem 13 implies that

$B(\theta_{MLE},r_{0})\subset{\mathcal{A}}^{EM}(\theta_{MLE}).$

In the event $E_{1}$, $\|\widehat{\theta}_{MLE}-\theta_{MLE}\|\leq r_{0}/3$ so
this implies $\widehat{\theta}_{MLE}\in
B(\theta_{MLE},r_{0})\subset{\mathcal{A}}^{EM}(\theta_{MLE})$, which completes
the proof.

$\square$

ProofProof of Theorem 9.

The proof of Theorem 9 is similar to the proof of Theorem 5 and 6, so we only
focus on the key steps. The proof consists of four parts. In the first part,
we will be verifying the assumptions of Theorem 3. In the second part, we will
derive a Gaussian approximation between the estimated mode $\widehat{\sf
Mode}(p)$ and $\overline{\sf Mode}(p)$, an element of the local mode of
$p_{h}={\mathbb{E}}(\widehat{p}_{h})$. $\overline{\sf Mode}(p)$ behaves like
the quantity $\bar{\theta}$ in the proof of Theorem 5 and 6. The third part is
a bootstrap approximation and wrapping up each component to obtain the final
bound. The first three parts prove the coverage of containing an element of
${{\mathcal{C}}}^{\pi}_{n,\delta}$, the first assertion of the theorem. The
last part of the proof is to derive the coverage of containing the mode in the
CI.

Part I: Verifying the working assumptions. We will first identify
$\eta_{j,n}(t)$ for $j=1,\cdots,4$ in Theorem 3. Under the assumption (K1–2)
and (A1–2), Lemma 10 implies that when $h\rightarrow 0$ and
$\frac{nh^{d+4}}{\log n}\rightarrow\infty$,

$\displaystyle\eta_{1,n}(t)$ $\displaystyle=A_{1}e^{-A_{2}nh^{d+2}t^{2}}$
$\displaystyle\eta_{2,n}(t)$ $\displaystyle=A_{1}e^{-A_{2}nh^{d+4}t^{2}}$

for some positive number $A_{1},A_{2}$ due to Lemma 10. Moreover,
$\eta_{3,n}(t)$ can be obtained by the Hoeffding’s inequality:

$\eta_{3,n}(t)=2Ke^{-2nt^{2}},$

where $K$ is the number of local modes. For $\epsilon_{4,n}(t)$, because the
collection of regions $\\{{\mathcal{B}}\oplus r:r>0\\}$ has a VC dimension
$1$, by the VC theory (see, e.g., Theorem 2.43 in Wasserman 2006), we have

$\eta_{4,n}(t)=8(n+1)e^{-nt^{2}/32}.$

Moreover, Lemma 2 holds whenever $\epsilon_{1,n},\epsilon_{2,n}<C_{0}$ for
some constant $C_{0}$ so when $h\rightarrow 0$,
$\xi_{n}=P(\mbox{Lemma~{}\ref{lem::sep} is not true})\leq c_{0}e^{c_{1}\cdot
nh^{d+4}}$ for some positive constants $c_{0},c_{1}$. The above derivation
shows that assumption (A3) and (A4) both hold and we have explicit forms of
$\xi_{n}$ and each $\eta_{n,j}(t)$.

With the assumptions from (A1–4) and $\xi_{n}\leq c_{0}e^{c_{1}\cdot
nh^{d+4}}$, Theorem 3 implies that there exists a local mode $\overline{\sf
Mode}(p)$ of the population density $p$ such that

$\|\widehat{\sf Mode}^{\dagger}(p)-\overline{\sf
Mode}(p)\|=O(h^{2})+O_{P}\left(\sqrt{\frac{\log n}{nh^{d+2}}}\right),$

with a probability of at least $1-O\left(\sqrt{\frac{\log
n}{nh^{d+2}}}\right)$. Note that this comes from choosing $t=\sqrt{\frac{k\log
n}{nh^{d+2}}}$ in Theorem 3 with a very large $k$ and use the second assertion
of Theorem 3. With this choice of $t$, each $\eta_{n,j}(t)$ will be at rate
$O(n^{-k^{\prime}})$ for some large number $k^{\prime}$ so they are dominated
by $O(t)=O\left(\sqrt{\frac{k\log n}{nh^{d+2}}}\right)$.

When $h\rightarrow 0$, the smoothed density
$p_{h}={\mathbb{E}}(\widehat{p}_{h})$ converges to the true density $p$ and
moreover, the gradient $g_{h}=\nabla p_{h}\rightarrow g=\nabla p$ and the
Hessian $H_{h}=\nabla\nabla p_{h}\rightarrow H=\nabla\nabla p$ in the maximum
norm. Thus, Lemma 2 implies that local maxima of $p_{h}$ and local maxima of
$p$ have a 1-1 correspondence. Using this fact, we define $\overline{\sf
Mode}(p_{h})$ to be the local mode of $p_{h}$ that corresponds to
$\overline{\sf Mode}(p)$. With this definition, equation (19) implies

$\displaystyle\overline{\sf Mode}(p_{h})-\overline{\sf
Mode}(p)=O(\sup_{x}\|g_{h}(x)-g(x)\|_{\max})=O(h^{2}).$ (35)

Thus, we have

$\displaystyle\widehat{\sf Mode}^{\dagger}(p)-\overline{\sf Mode}(p)$
$\displaystyle=\widehat{\sf Mode}^{\dagger}(p)-\overline{\sf
Mode}(p_{h})+\overline{\sf Mode}(p_{h})-\overline{\sf Mode}(p)$ (36)
$\displaystyle=\widehat{\sf Mode}^{\dagger}(p)-\overline{\sf
Mode}(p_{h})+O(h^{2})$

with a probability of at least $1-O\left(\sqrt{\frac{\log
n}{nh^{d+2}}}\right).$

Moreover, using the same derivation as in the proof of Theorem 5, we can show
that when we initialize a gradient ascent method $n$ times, the chance that
$\bar{\theta}=\overline{\sf Mode}(p)\in S^{\pi}_{n,\delta}$ is at least

$1-\delta+O\left(\sqrt{\frac{\log n}{nh^{d+2}}}\right).$ (37)

This gives a probability bound that will be used later to calculate the
coverage of a CI.

Part II: Gaussian approximation. In this part, we will show that the
difference $\widehat{\sf Mode}^{\dagger}(p)-\overline{\sf Mode}(p_{h})$ has a
Gaussian limit.

Using the same derivation as the proof of Theorem 4 and equation (19), we have

$\displaystyle\widehat{\sf Mode}^{\dagger}(p)-\overline{\sf Mode}(p_{h})$
$\displaystyle=H_{h}^{-1}(\overline{\sf
Mode}(p_{h}))(\widehat{g}_{h}(\overline{\sf Mode}(p_{h}))-g_{h}(\overline{\sf
Mode}(p_{h})))$ $\displaystyle+O\left(\|\overline{\sf
Mode}(p_{h})-\widehat{\sf
Mode}^{\dagger}(p_{h})\|\cdot\epsilon^{\prime}_{2,n}+\|\overline{\sf
Mode}(p_{h})-\widehat{\sf Mode}^{\dagger}(p_{h})\|^{2}\right),$

where
$\epsilon^{\prime}_{2,n}=\sup_{x}\|\widehat{H}_{h}-H_{h}\|_{\max}=O_{P}\left(\sqrt{\frac{\log
n}{nh^{d+4}}}\right)$.

The requirement $\frac{nh^{d+4}}{\log n}\rightarrow\infty$ implies
$\epsilon^{\prime}_{2,n}=o_{P}(1)$ and

$\displaystyle\sqrt{nh^{d+2}}\bigg{(}$ $\displaystyle\widehat{\sf
Mode}^{\dagger}(p)-\overline{\sf Mode}(p_{h})\bigg{)}$ (38)
$\displaystyle=\sqrt{nh^{d+2}}H^{-1}_{h}(\overline{\sf
Mode}(p_{h}))(\widehat{g}_{h}(\overline{\sf Mode}(p_{h}))-g_{h}(\overline{\sf
Mode}(p_{h})))+o_{P}(1)$
$\displaystyle=\frac{1}{\sqrt{n}}\sum_{i=1}^{n}W_{i}+o_{P}(1)$

where

$W_{i}=\frac{1}{\sqrt{h^{d+2}}}H^{-1}_{h}(\overline{\sf
Mode}(p_{h}))\left(\nabla K\left(\frac{X_{i}-\overline{\sf
Mode}(p_{h})}{h}\right)-{\mathbb{E}}\left(\nabla
K\left(\frac{X_{i}-\overline{\sf Mode}(p_{h})}{h}\right)\right)\right)$

is a random variable with mean $0$ and a covariance matrix $\Sigma_{h}$ with
bounded entries.

By the multivariate Berry-Esseen bound (Gotze, 1991; Sazonov, 1968), there
exists a $d$-dimensional normal random variable $Z_{0}$ with mean $0$ and
identity covariance matrix such that

$\sup_{t}\left|P\left(\|\frac{1}{\sqrt{n}}\sum_{i=1}^{n}W_{i}\|\leq
t\right)-P\left(\|\Sigma_{h}^{1/2}Z\|\leq t\right)\right|\leq
c^{\prime}_{BE}\frac{{\mathbb{E}}\|W_{i}\|^{3}}{\sqrt{n}},$

where $c^{\prime}_{BE}$ is a constant. Note that
${\mathbb{E}}\|W_{i}\|^{3}=O\left(\sqrt{\frac{1}{h^{d+2}}}\right)$.

Therefore, equation (38) and the above Berry-Esseen bound implies that

$\sup_{t}\left|P\left(\sqrt{nh^{d+2}}\left\|\widehat{\sf
Mode}^{\dagger}(p)-\overline{\sf Mode}(p_{h})\right\|\leq
t\right)-P\left(\|\Sigma_{h}^{1/2}Z\|\leq t\right)\right|\leq
c^{\prime}_{BE}\frac{2{\mathbb{E}}\|W_{i}\|^{3}}{\sqrt{n}}.$ (39)

Note that we use the fact that the $o_{P}(1)$ part in equation (38) has an
expectation shrinking toward $0$ so we can bound its contribution using
$2\times{\mathbb{E}}\|W_{i}\|^{3}$.

Part III: Bootstrap approximation. In the bootstrap case, we are sampling from
$\widehat{P}_{n}$ so $\widehat{P}_{n}$ will be treated as the underlying
population distribution function. The smoothed density of $\widehat{P}_{n}$
turns out to be the original KDE $\widehat{p}_{h}$. By applying the same
derivation as in Part II with replacing $(\widehat{p}_{h},p_{h})$ with
$(\widehat{p}^{*}_{n},\widehat{p}_{h})$ and using the fact $\overline{\sf
Mode}(\widehat{p}_{h})=\widehat{\sf Mode}^{\dagger}(p)$, equation (39) implies

$\displaystyle\sup_{t}\bigg{|}P\left(\sqrt{nh^{d+2}}\left\|\widehat{\sf
Mode}^{\dagger*}(p)-\underbrace{\overline{\sf
Mode}(\widehat{p}_{h})}_{=\widehat{\sf Mode}^{\dagger}(p)}\right\|\leq
t|\mathcal{X}_{n}\right)$
$\displaystyle-P\left(\|\widehat{\Sigma}_{h}^{1/2}Z\|\leq
t|\mathcal{X}_{n}\right)\bigg{|}$ (40) $\displaystyle\leq
c^{\prime}_{BE}\frac{2\widehat{{\mathbb{E}}}_{n}\|W_{i}\|^{3}}{\sqrt{n}},$

where $\widehat{\Sigma}_{h}$ is the (conditional) covariance matrix of

$W^{*}_{i}=\frac{1}{\sqrt{h^{d+2}}}\widehat{H}^{-1}_{h}(\widehat{\sf
Mode}^{\dagger}(p))\left(\nabla K\left(\frac{X^{*}_{i}-\widehat{\sf
Mode}^{\dagger}(p)}{h}\right)-{\mathbb{E}}\left(\nabla
K\left(\frac{X^{*}_{i}-\widehat{\sf
Mode}^{\dagger}(p)}{h}\right)|\mathcal{X}_{n}\right)\right)$

given the original sample $\mathcal{X}_{n}=\\{X_{1},\cdots,X_{n}\\}$.

Essentially, $\widehat{\Sigma}_{h}$ is just a sample analogue of $\Sigma_{h}$
so it is easy to see that

$\left\|\widehat{\Sigma}_{h}-\Sigma_{h}\right\|_{\max}=O_{P}\left(\sqrt{\frac{1}{n}}\right),\quad{\mathbb{E}}\left\|\widehat{\Sigma}_{h}-\Sigma_{h}\right\|_{\max}=O\left(\sqrt{\frac{1}{n}}\right).$

Using the delta method (Wasserman, 2006),

$\sup_{t}\left|P\left(\|\Sigma_{h}^{1/2}Z\|\leq
t\right)-P\left(\|\widehat{\Sigma}_{h}^{1/2}Z\|\leq
t|\mathcal{X}_{n}\right)\right|=O\left(\left\|\widehat{\Sigma}_{h}-\Sigma_{h}\right\|_{\max}\right).$

This, together with equations (39) and (40), implies

$\displaystyle\sup_{t}\Bigg{|}P\Bigg{(}\sqrt{nh^{d+2}}$
$\displaystyle\bigg{\|}\widehat{\sf Mode}^{\dagger*}(p)-\widehat{\sf
Mode}^{\dagger}(p)\bigg{\|}\leq t|\mathcal{X}_{n}\Bigg{)}-$ $\displaystyle
P\Bigg{(}\sqrt{nh^{d+2}}\left\|\widehat{\sf Mode}^{\dagger}(p)-\overline{\sf
Mode}(p_{h})\right\|\leq t\Bigg{)}\Bigg{|}$ $\displaystyle\leq
O\left(\sqrt{\frac{1}{n}}\right)+c^{\prime}_{BE}\frac{2\widehat{{\mathbb{E}}}_{n}\|W^{*}_{i}\|^{3}}{\sqrt{n}}+O\left(\left\|\widehat{\Sigma}_{h}-\Sigma_{h}\right\|_{\max}\right).$

By taking expectation in both sides and using the Jensen’s inequality, we can
convert the conditional probability into a marginal probability:

$\displaystyle\sup_{t}\Bigg{|}P\Bigg{(}\sqrt{nh^{d+2}}$
$\displaystyle\bigg{\|}\widehat{\sf Mode}^{\dagger*}(p)-\widehat{\sf
Mode}^{\dagger}(p)\bigg{\|}\leq t\Bigg{)}-$ $\displaystyle
P\Bigg{(}\sqrt{nh^{d+2}}\left\|\widehat{\sf Mode}^{\dagger}(p)-\overline{\sf
Mode}(p_{h})\right\|\leq t\Bigg{)}\Bigg{|}$ $\displaystyle\leq
O\left(\sqrt{\frac{1}{n}}\right)+c^{\prime}_{BE}\frac{2{\mathbb{E}}(\widehat{{\mathbb{E}}}_{n}\|W^{*}_{i}\|^{3})}{\sqrt{n}}+O\left({\mathbb{E}}\left\|\widehat{\Sigma}_{h}-\Sigma_{h}\right\|_{\max}\right)$
$\displaystyle=O\left(\sqrt{\frac{1}{nh^{d+2}}}\right).$

Note that the rate $O\left(\sqrt{\frac{1}{nh^{d+2}}}\right)$ comes from

$c^{\prime}_{BE}\frac{2{\mathbb{E}}(\widehat{{\mathbb{E}}}_{n}\|W^{*}_{i}\|^{3})}{\sqrt{n}}=c^{\prime}_{BE}\frac{2{\mathbb{E}}\|W_{i}\|^{3}}{\sqrt{n}}=O\left(\sqrt{\frac{1}{nh^{d+2}}}\right)$

because ${\mathbb{E}}\|W_{i}\|^{3}=O\left(\sqrt{\frac{1}{h^{d+2}}}\right)$.

To infer the population quantity $\overline{\sf Mode}(p)$, we first recall
from equation (35) that

$\overline{\sf Mode}(p_{h})-\overline{\sf Mode}(p)=O(h^{2}).$

Therefore, by replacing $\overline{\sf Mode}(p_{h})$ with $\overline{\sf
Mode}(p)$, we obtain

$\displaystyle\sup_{t}\Bigg{|}P\Bigg{(}\sqrt{nh^{d+2}}$
$\displaystyle\bigg{\|}\widehat{\sf Mode}^{\dagger*}(p)-\widehat{\sf
Mode}^{\dagger}(p)\bigg{\|}\leq t\Bigg{)}-$ (41) $\displaystyle
P\Bigg{(}\sqrt{nh^{d+2}}\left\|\widehat{\sf Mode}^{\dagger}(p)-\overline{\sf
Mode}(p)\right\|\leq t\Bigg{)}\Bigg{|}$
$\displaystyle=O\left(\sqrt{\frac{1}{nh^{d+2}}}\right)+O\left(\sqrt{nh^{d+6}}\right).$

Recall that our CI is

$\mathbb{M}_{n,\alpha}=\left\\{x:\left\|x-\widehat{\sf
Mode}^{\dagger}(p)\right\|\leq\widehat{d}^{\dagger}_{1-\alpha}\right\\}.$

Thus,

$P\left(\overline{\sf Mode}(p)\in\mathbb{M}_{n,\alpha}\right)\geq
1-\alpha+O\left(\sqrt{\frac{1}{nh^{d+2}}}\right)+O\left(\sqrt{nh^{d+6}}\right).$
(42)

To calculate the coverage of containing an element within
${{\mathcal{C}}}^{\pi}_{n,\delta}$, the contribution in the analysis of Part I
must be considered. By equation (37) and (42),

$\displaystyle
P\left(\mathbb{M}_{n,\alpha}\cap{{\mathcal{C}}}^{\pi}_{n,\delta}\neq\emptyset\right)$
$\displaystyle\geq P\left(\overline{\sf
Mode}(p)\in\mathbb{M}_{n,\alpha},\overline{\sf Mode}(p)\in
S^{\pi}_{n,\delta}\right)$ $\displaystyle\geq 1-P\left(\overline{\sf
Mode}(p)\notin\mathbb{M}_{n,\alpha}\right)-P\left(\overline{\sf
Mode}(p)\notin{{\mathcal{C}}}^{\pi}_{n,\delta}\right)$ $\displaystyle\geq
1-\alpha+O\left(\sqrt{\frac{1}{nh^{d+2}}}\right)+O\left(\sqrt{nh^{d+6}}\right)-\delta+O\left(\sqrt{\frac{\log
n}{nh^{d+2}}}\right)$ $\displaystyle=1-\alpha-\delta+O\left(\sqrt{\frac{\log
n}{nh^{d+2}}}\right)+O\left(\sqrt{nh^{d+6}}\right).$

Part IV: Coverage of containing the mode. The proof of this statement is very
similar to the proof of Theorem 7 so we only highlight the key quantities that
we need to consider. Let

$E_{1}=\\{\mbox{Lemma~{}\ref{lem::sep} is true}\\}.$

Under $E_{1}$, there is a local mode $\widetilde{\sf Mode}(\widehat{p}_{h})$
of $\widehat{p}_{h}$ that corresponds to ${\sf Mode}(p)$. Then we define the
event

$E_{2}=\\{\mbox{$\widetilde{\sf Mode}(\widehat{p}_{h})={\sf
Mode}(\widehat{p}_{h})$.}\\}$

Namely, $E_{2}$ is the event that the local mode $\widetilde{\sf
Mode}(\widehat{p}_{h})$ is also the global mode of $\widehat{p}_{h}.$ Under
events $E_{1}$, define another event

$E_{3}=\left\\{\widehat{\sf Mode}^{\dagger}(p)=\widetilde{\sf
Mode}(\widehat{p}_{h})\right\\}.$

In other words, $E_{3}$ is the event that the estimator we compute is
$\widetilde{\sf Mode}(\widehat{p}_{h})$. When events $E_{1},E_{2},E_{3}$ all
happen, our estimator $\widehat{\sf Mode}^{\dagger}(p)$ is the global mode of
$\widehat{p}_{h}$ and is close to ${\sf Mode}(p)$ so $\overline{\sf
Mode}(p)={\sf Mode}(p)$. Thus,

$\displaystyle P({\sf Mode}(p)\in\mathbb{M}_{n,\alpha})$ $\displaystyle\geq
P(\overline{\sf Mode}(p)\in\mathbb{M}_{n,\alpha},E_{1},E_{2},E_{3})$ (43)
$\displaystyle\geq 1-P(\overline{\sf
Mode}(p)\notin\mathbb{M}_{n,\alpha})-P(E_{1}^{c}\cup E_{2}^{c}\cup E_{3}^{c})$
$\displaystyle\geq
1-\alpha+O\left(\sqrt{\frac{1}{n}}\right)+O\left(\sqrt{nh^{d+6}}\right)-P(E_{1}^{c}\cup
E_{2}^{c}\cup E_{3}^{c}).$

We can bound the probability

$\displaystyle P(E_{1}^{c}\cup E_{2}^{c}\cup E_{3}^{c})$ $\displaystyle\geq
P(E_{1}^{c}\cup E_{2}^{c})+P(E_{3}^{c})$ (44)
$\displaystyle=1-P(E_{1},E_{2})+P(E_{3}^{c})$
$\displaystyle=1-P(E_{1})P(E_{2}|E_{1})+P(E_{3}^{c})$
$\displaystyle=1-(1-P(E_{1}^{c}))\times(1-P(E_{2}^{c}|E_{1}))+P(E_{3}^{c}).$

Because $E_{1}$ is true whenever the gradient and Hessian difference are
sufficiently small,

$P(E_{1}^{c})\leq A_{1}e^{-A_{2}nh^{d+4}}$

by Lemma 10. Given $E_{1}$, $E_{2}$ occurs when the
$\|\widehat{p}_{h}-p\|_{\infty}$ is sufficiently small because assumption (A1)
requires that the mode is unique. Thus,

$P(E^{c}_{2}|E_{1})\leq A_{3}e^{-A_{4}nh^{d}}$

for some constants $A_{3}$ and $A_{4}$.

To bound $P(E^{c}_{3}),$ note that
$P(E^{c}_{3})={\mathbb{E}}(P(E^{c}_{3}|E_{1},E_{2},\mathcal{X}_{n}))$. Let
$\widehat{{\mathcal{A}}}_{\sf mode}$ be the basin of attraction of
$\widetilde{\sf Mode}(\widehat{p}_{h})$ based on the gradient of
$\widehat{p}_{h}$. Given the original sample $\mathcal{X}_{n}$, if we run the
initialization once, the chance of obtaining $\widetilde{\sf
Mode}(\widehat{p}_{h})$ as our estimator is
$\widehat{P}_{n}(\widehat{{\mathcal{A}}}_{\sf mode})$. Thus, the chance of
running $n$ initializations but have no starting points within
$\widehat{{\mathcal{A}}}_{\sf mode}$ is
$(1-\widehat{P}_{n}(\widehat{{\mathcal{A}}}_{\sf mode}))^{n}$. Namely,
$P(E^{c}_{3}|E_{1},E_{2},\mathcal{X}_{n})=(1-\widehat{P}_{n}(\widehat{{\mathcal{A}}}_{\sf
mode}))^{n}$.

By triangle inequality, a lower bound on
$\widehat{P}_{n}(\widehat{{\mathcal{A}}}_{\sf mode})$ can be derived as

$\displaystyle\widehat{P}_{n}(\widehat{{\mathcal{A}}}_{\sf mode})$
$\displaystyle=\widehat{P}_{n}(\widehat{{\mathcal{A}}}_{\sf
mode})-\widehat{P}_{n}({\mathcal{A}}_{\sf
mode})+\widehat{P}_{n}({\mathcal{A}}_{\sf mode})-P({\mathcal{A}}_{\sf
mode})+P({\mathcal{A}}_{\sf mode})$ $\displaystyle\geq P({\mathcal{A}}_{\sf
mode})-|\widehat{P}_{n}(\widehat{{\mathcal{A}}}_{\sf
mode})-\widehat{P}_{n}({\mathcal{A}}_{\sf
mode})|-|\widehat{P}_{n}({\mathcal{A}}_{\sf mode})-P({\mathcal{A}}_{\sf
mode})|.$

Using assumption (A2) and (A4), one can show that there exists some constant
$C_{0}$ such that $\sup_{x}\|\nabla\widehat{p}_{h}(x)-\nabla
p(x)\|_{\max}<C_{0}$ implies $|\widehat{P}_{n}(\widehat{{\mathcal{A}}}_{\sf
mode})-\widehat{P}_{n}({\mathcal{A}}_{\sf
mode})|<\frac{1}{3}P({\mathcal{A}}_{\sf mode})$. Moreover, the Hoeffding’s
inequality implies

$P\left(|\widehat{P}_{n}({\mathcal{A}}_{\sf mode})-P({\mathcal{A}}_{\sf
mode})|>t\right)\leq 2e^{-2nt^{2}}$

so with a probability of at least $2e^{-2nC_{1}}$ for some positive $C_{1}$,
$|\widehat{P}_{n}({\mathcal{A}}_{\sf mode})-P({\mathcal{A}}_{\sf
mode})|<\frac{1}{3}P({\mathcal{A}}_{\sf mode})$. Therefore,
$\widehat{P}_{n}(\widehat{{\mathcal{A}}}_{\sf
mode})>\frac{1}{3}P({\mathcal{A}}_{\sf mode})$ with a probability of at least
$1-A_{5}e^{-nh^{d+2}A_{6}}$ for some positive number $A_{5}$ and $A_{6}$.
Define another event

$E_{4}=\left\\{\widehat{P}_{n}(\widehat{{\mathcal{A}}}_{\sf
mode})\geq\frac{1}{3}P({\mathcal{A}}_{\sf mode})\right\\}.$

The above analysis shows that $P(E_{4})\geq 1-A_{5}e^{-nh^{d+2}A_{6}}$.

Using event $E_{4}$, we obtain

$\displaystyle P(E^{c}_{3})$
$\displaystyle={\mathbb{E}}(P(E^{c}_{3}|E_{1},E_{2},\mathcal{X}_{n}))$
$\displaystyle={\mathbb{E}}(P(E^{c}_{3}|E_{1},E_{2},\mathcal{X}_{n})I(E_{4}))+{\mathbb{E}}(P(E^{c}_{3}|E_{1},E_{2},\mathcal{X}_{n})I(E^{c}_{4}))$
$\displaystyle\leq{\mathbb{E}}\left((1-\widehat{P}_{n}(\widehat{{\mathcal{A}}}_{\sf
mode}))^{n}I\left(\widehat{P}_{n}(\widehat{{\mathcal{A}}}_{\sf
mode})\geq\frac{1}{3}P({\mathcal{A}}_{\sf mode})\right)\right)+P(E_{4}^{c})$
$\displaystyle\leq\left(1-\frac{1}{3}P({\mathcal{A}}_{\sf
mode})\right)^{n}-A_{5}e^{-nh^{d+2}A_{6}}.$

So putting it altogether into equations (43) and (44) and absorb the
exponential terms into the big $O$’s, we obtain

$\displaystyle P({\sf Mode}(p)\in\mathbb{M}_{n,\alpha})\geq
1-\alpha-\left(1-\frac{1}{3}P({\mathcal{A}}_{\sf
mode})\right)^{n}+O\left(\sqrt{\frac{1}{nh^{d+2}}}\right)+O\left(\sqrt{nh^{d+6}}\right).$

which is the desired result.

$\square$

## References

  * Arias-Castro et al. (2016) E. Arias-Castro, D. Mason, and B. Pelletier.  On the estimation of the gradient lines of a density and the consistency of the mean-shift algorithm.  _The Journal of Machine Learning Research_ , 17(1):1487–1514, 2016.
  * Azizyan et al. (2015) M. Azizyan, Y.-C. Chen, A. Singh, and L. Wasserman.  Risk bounds for mode clustering.  _arXiv preprint arXiv:1505.00482_ , 2015.
  * Balakrishnan et al. (2017) S. Balakrishnan, M. J. Wainwright, and B. Yu.  Statistical guarantees for the em algorithm: From population to sample-based analysis.  _The Annals of Statistics_ , 45(1):77–120, 2017\.
  * Banyaga and Hurtubise (2013) A. Banyaga and D. Hurtubise.  _Lectures on Morse homology_ , volume 29.  Springer Science & Business Media, 2013.
  * Berry (1941) A. C. Berry.  The accuracy of the gaussian approximation to the sum of independent variates.  _Transactions of the American Mathematical Cociety_ , 49(1):122–136, 1941.
  * Beyn (1987) W.-J. Beyn.  On the numerical approximation of phase portraits near stationary points.  _SIAM journal on numerical analysis_ , 24(5):1095–1113, 1987.
  * Blei et al. (2017) D. M. Blei, A. Kucukelbir, and J. D. McAuliffe.  Variational inference: A review for statisticians.  _Journal of the American Statistical Association_ , (just-accepted), 2017.
  * Bottou (2010) L. Bottou.  Large-scale machine learning with stochastic gradient descent.  In _Proceedings of COMPSTAT’2010_ , pages 177–186. Springer, 2010\.
  * Burman and Polonik (2009) P. Burman and W. Polonik.  Multivariate mode hunting: Data analytic tools with measures of significance.  _Journal of Multivariate Analysis_ , 100(6):1198–1218, 2009.
  * Chacón et al. (2015) J. E. Chacón et al.  A population background for nonparametric density-based clustering.  _Statistical Science_ , 30(4):518–532, 2015.
  * Chazal et al. (2014) F. Chazal, B. T. Fasy, F. Lecci, B. Michel, A. Rinaldo, and L. Wasserman.  Robust topological inference: Distance to a measure and kernel distance.  _arXiv preprint arXiv:1412.7197_ , 2014.
  * Chen et al. (2015) Y.-C. Chen, C. R. Genovese, and L. Wasserman.  Asymptotic theory for density ridges.  _The Annals of Statistics_ , 43(5):1896–1928, 2015.
  * Chen et al. (2016) Y.-C. Chen, C. R. Genovese, L. Wasserman, et al.  A comprehensive approach to mode clustering.  _Electronic Journal of Statistics_ , 10(1):210–241, 2016.
  * Chen et al. (2017) Y.-C. Chen, C. R. Genovese, L. Wasserman, et al.  Statistical inference using the morse-smale complex.  _Electronic Journal of Statistics_ , 11(1):1390–1433, 2017.
  * Cheng (1995) Y. Cheng.  Mean shift, mode seeking, and clustering.  _Pattern Analysis and Machine Intelligence, IEEE Transactions on_ , 17(8):790–799, 1995.
  * Comaniciu and Meer (2002) D. Comaniciu and P. Meer.  Mean shift: A robust approach toward feature space analysis.  _Pattern Analysis and Machine Intelligence, IEEE Transactions on_ , 24(5):603–619, 2002.
  * Dempster et al. (1977) A. Dempster, N. Laird, and D. Rubin.  Maximum likelihood from incomplete data via the $em$ algorithm.  _Journal of the Royal Statistical Society. Series B (Methodological)_ , 39(1):1–38, 1977.
  * Efron (1982) B. Efron.  _The jackknife, the bootstrap and other resampling plans_.  SIAM, 1982.
  * Efron (1992) B. Efron.  Bootstrap methods: another look at the jackknife.  In _Breakthroughs in Statistics_ , pages 569–593. Springer, 1992\.
  * Einmahl and Mason (2005) U. Einmahl and D. M. Mason.  Uniform in bandwidth consistency for kernel-type function estimators.  _The Annals of Statistics_ , 2005.
  * Elsener and van de Geer (2018) A. Elsener and S. van de Geer.  Sharp oracle inequalities for stationary points of nonconvex penalized m-estimators.  _arXiv preprint arXiv:1802.09733_ , 2018.
  * Esseen (1942) C.-G. Esseen.  _On the Liapounoff limit of error in the theory of probability_.  Almqvist & Wiksell, 1942.
  * Fukunaga and Hostetler (1975) K. Fukunaga and L. Hostetler.  The estimation of the gradient of a density function, with applications in pattern recognition.  _Information Theory, IEEE Transactions on_ , 21(1):32–40, 1975.
  * Gelfand and Mitter (1991) S. B. Gelfand and S. K. Mitter.  Recursive stochastic algorithms for global optimization in r^d.  _SIAM Journal on Control and Optimization_ , 29(5):999–1018, 1991.
  * Genovese and Wasserman (2000) C. R. Genovese and L. Wasserman.  Rates of convergence for the gaussian mixture sieve.  _The Annals of Statistics_ , 28(4):1105–1127, 2000.
  * Genovese et al. (2014) C. R. Genovese, M. Perone-Pacifico, I. Verdinelli, and L. Wasserman.  Nonparametric ridge estimation.  _The Annals of Statistics_ , 42(4):1511–1545, 2014.
  * Genovese et al. (2016) C. R. Genovese, M. Perone-Pacifico, I. Verdinelli, and L. Wasserman.  Non-parametric inference for density modes.  _Journal of the Royal Statistical Society: Series B (Statistical Methodology)_ , 78(1):99–126, 2016.
  * Gine and Guillou (2002) E. Gine and A. Guillou.  Rates of strong uniform consistency for multivariate kernel density estimators.  _In Annales de l’Institut Henri Poincare (B) Probability and Statistics_ , 2002.
  * Good and Gaskins (1980) I. Good and R. Gaskins.  Density estimation and bump-hunting by the penalized likelihood method exemplified by scattering and meteorite data.  _Journal of the American Statistical Association_ , 75(369):42–56, 1980.
  * Gotze (1991) F. Gotze.  On the rate of convergence in the multivariate clt.  _The Annals of Probability_ , 19(2):724–739, 1991\.
  * Hall (2013) P. Hall.  _The bootstrap and Edgeworth expansion_.  Springer Science & Business Media, 2013.
  * Hall et al. (2004) P. Hall, M. C. Minnotte, and C. Zhang.  Bump hunting with non-gaussian kernels.  _The Annals of Statistics_ , 32(5):2124–2141, 2004.
  * Horn and Johnson (1990) R. A. Horn and C. R. Johnson.  _Matrix analysis_.  Cambridge university press, 1990.
  * Jin et al. (2016) C. Jin, Y. Zhang, S. Balakrishnan, M. J. Wainwright, and M. I. Jordan.  Local maxima in the likelihood of gaussian mixture models: Structural results and algorithmic consequences.  In _Advances in Neural Information Processing Systems_ , pages 4116–4124, 2016.
  * Kiefer et al. (1952) J. Kiefer, J. Wolfowitz, et al.  Stochastic estimation of the maximum of a regression function.  _The Annals of Mathematical Statistics_ , 23(3):462–466, 1952.
  * Kim and Lindsay (2011) D. Kim and B. G. Lindsay.  Comparing wald and likelihood regions applied to locally identifiable mixture models.  _Mixtures: Estimation and Applications_ , pages 77–100, 2011.
  * Kim et al. (2016) J. Kim, Y.-C. Chen, S. Balakrishnan, A. Rinaldo, and L. Wasserman.  Statistical inference for cluster trees.  In _Advances In Neural Information Processing Systems_ , pages 1839–1847, 2016.
  * Kushner and Yin (2003) H. Kushner and G. G. Yin.  _Stochastic approximation and recursive algorithms and applications_ , volume 35.  Springer Science & Business Media, 2003.
  * Lee et al. (2016) J. D. Lee, M. Simchowitz, M. I. Jordan, and B. Recht.  Gradient descent only converges to minimizers.  In _Conference on Learning Theory_ , pages 1246–1257, 2016.
  * Li et al. (2007) J. Li, S. Ray, and B. G. Lindsay.  A nonparametric statistical approach to clustering via mode identification.  _Journal of Machine Learning Research_ , 2007.
  * Li and Barron (1999) J. Q. Li and A. R. Barron.  Mixture density estimation.  In _Proceedings of the 12th International Conference on Neural Information Processing Systems_ , pages 279–285. MIT Press, 1999.
  * Liang and Su (2017) T. Liang and W. Su.  Statistical inference for the population landscape via moment adjusted stochastic gradients.  _arXiv preprint arXiv:1712.07519_ , 2017.
  * Loh and Wainwright (2015) P.-L. Loh and M. J. Wainwright.  Regularized m-estimators with nonconvexity: Statistical and algorithmic theory for local optima.  _Journal of Machine Learning Research_ , 16:559–616, 2015\.
  * Loh et al. (2017) P.-L. Loh et al.  Statistical consistency and asymptotic normality for high-dimensional robust $m$-estimators.  _The Annals of Statistics_ , 45(2):866–896, 2017\.
  * McLachlan and Krishnan (2007) G. McLachlan and T. Krishnan.  _The EM algorithm and extensions_ , volume 382.  John Wiley & Sons, 2007.
  * McLachlan and Peel (2004) G. McLachlan and D. Peel.  _Finite mixture models_.  John Wiley & Sons, 2004.
  * Mei et al. (2016) S. Mei, Y. Bai, and A. Montanari.  The landscape of empirical risk for non-convex losses.  _arXiv preprint arXiv:1607.06534_ , 2016.
  * Merlet and Pierre (2010) B. Merlet and M. Pierre.  Convergence to equilibrium for the backward euler scheme and applications.  _Commun. Pure Appl. Anal_ , 9, 2010.
  * Milnor (1963) J. W. Milnor.  _Morse theory_.  Number 51. Princeton university press, 1963.
  * Morse (1925) M. Morse.  Relations between the critical points of a real function of n independent variables.  _Transactions of the American Mathematical Society_ , 27(3):345–396, 1925.
  * Morse (1930) M. Morse.  The foundations of a theory of the calculus of variations in the large in m-space (second paper).  _Transactions of the American Mathematical Society_ , 32(4):599–631, 1930.
  * Owen (1990) A. Owen.  Empirical likelihood ratio confidence regions.  _The Annals of Statistics_ , pages 90–120, 1990.
  * Panageas and Piliouras (2016) I. Panageas and G. Piliouras.  Gradient descent only converges to minimizers: Non-isolated critical points and invariant regions.  _arXiv preprint arXiv:1605.00405_ , 2016.
  * Parzen (1962) E. Parzen.  On estimation of a probability density function and mode.  _The annals of mathematical statistics_ , 33(3):1065–1076, 1962.
  * Pensia et al. (2018) A. Pensia, V. Jog, and P.-L. Loh.  Generalization error bounds for noisy, iterative algorithms.  _arXiv preprint arXiv:1801.04295_ , 2018.
  * Raginsky et al. (2017) M. Raginsky, A. Rakhlin, and M. Telgarsky.  Non-convex learning via stochastic gradient langevin dynamics: a nonasymptotic analysis.  _arXiv preprint arXiv:1702.03849_ , 2017.
  * Rao (1948) C. R. Rao.  Large sample tests of statistical hypotheses concerning several parameters with applications to problems of estimation.  In _Mathematical Proceedings of the Cambridge Philosophical Society_ , volume 44, pages 50–57. Cambridge University Press, 1948.
  * Redner (1981) R. Redner.  Note on the consistency of the maximum likelihood estimate for nonidentifiable distributions.  _The Annals of Statistics_ , 9(1):225–228, 1981\.
  * Redner and Walker (1984) R. A. Redner and H. F. Walker.  Mixture densities, maximum likelihood and the em algorithm.  _SIAM review_ , 26(2):195–239, 1984.
  * Romano (1988a) J. P. Romano.  Bootstrapping the mode.  _Annals of the Institute of Statistical Mathematics_ , 40(3):565–586, 1988a.
  * Romano (1988b) J. P. Romano.  On weak convergence and optimality of kernel density estimates of the mode.  _The Annals of Statistics_ , pages 629–647, 1988b.
  * Sazonov (1968) V. Sazonov.  On the multi-dimensional central limit theorem.  _Sankhyā: The Indian Journal of Statistics, Series A_ , pages 181–204, 1968.
  * Sundberg (1974) R. Sundberg.  Maximum likelihood theory for incomplete data from an exponential family.  _Scand J Statist_ , 1:49–58, 1974.
  * Talagrand (1994) M. Talagrand.  Sharper bounds for gaussian and empirical processes.  _The Annals of Probability_ , 22(1):28–76, 1994\.
  * Teh et al. (2016) Y. W. Teh, A. H. Thiery, and S. J. Vollmer.  Consistency and fluctuations for stochastic gradient langevin dynamics.  _The Journal of Machine Learning Research_ , 17(1):193–225, 2016.
  * Titterington et al. (1985) D. Titterington, A. Smith, and U. Makov.  Statistical analysis of finite mixture distributions.  1985\.
  * Van der Vaart (1998) A. W. Van der Vaart.  _Asymptotic statistics_ , volume 3.  Cambridge university press, 1998.
  * Wald (1943) A. Wald.  Tests of statistical hypotheses concerning several parameters when the number of observations is large.  _Transactions of the American Mathematical society_ , 54(3):426–482, 1943.
  * Wasserman (2006) L. Wasserman.  _All of nonparametric statistics_.  Springer Science+ Business Media, New York, 2006.
  * Wu (1983) C. J. Wu.  On the convergence properties of the em algorithm.  _The Annals of Statistics_ , 11(1):95–103, 1983\.


$A_{1},\ A_{2},\ \psi$ in such a way that the terms dependent on $\langle
q_{12}(\bm{\sigma}^{(1)},\bm{\sigma}^{(2)})\rangle_{t,a}$ cancel out,

$\begin{array}[]{lll}&\psi=\beta^{\prime}\dfrac{P}{2M}(1+\rho)^{P/2}\bar{n}^{P-1},\\\
\\\
&A_{1}^{\mbox{\tiny$2$}}=\beta^{\prime}\dfrac{P}{2}\dfrac{(1+\sqrt{\rho_{P}})^{2}}{(1+\rho)^{P}}\dfrac{KP!}{2N^{P-1}}\bar{q}_{1}^{{}^{P-1}},\\\
\\\
&A_{2}^{\mbox{\tiny$2$}}=\beta^{\prime}\dfrac{P}{2}\dfrac{(1+\sqrt{\rho_{P}})^{2}}{(1+\rho)^{P}}\dfrac{KP!}{2N^{P-1}}(\bar{q}_{2}^{{}^{P-1}}-\bar{q}_{1}^{{}^{P-1}})\end{array}$
(148)

This makes $d\mathcal{F}^{(P)}/dt$ $t$-independent and leads to the thesis
(40).

###  B.2 Proof of Corollary 2

Let us start from the self consistency equations in the $M\to\infty$ limit
introduced in Corollary 1, we recognize that as $\tilde{\beta}\to\infty$, we
have $\bar{q}_{2}\to 1$, therefore in order to perform the limit we will
introduce the reparametrization

$\begin{array}[]{lll}\bar{q}_{2}=1-\dfrac{\delta\bar{q}_{2}}{\tilde{\beta}}&\mathrm{as}&\tilde{\beta}\to\infty.\end{array}$
(149)

It is now useful to insert an additional term $\tilde{\beta}y$ in the
expression of $g(\beta,\gamma,\bm{Y})$ in (76), which now reads as

$\displaystyle g(\beta,\gamma,\bm{Y})=$
$\displaystyle\tilde{\beta}\dfrac{P}{2}\bar{m}^{{}^{P-1}}+\tilde{\beta}Y^{(2)}\sqrt{\gamma\dfrac{P}{2}(1+\sqrt{\rho_{P}})^{2}(\bar{q}_{2}^{{}^{P-1}}-\bar{q}_{1}^{{}^{P-1}})}$
$\displaystyle+\tilde{\beta}Y^{(1)}\sqrt{\rho\left(\dfrac{P}{2}\bar{m}^{{}^{P-1}}\right)^{2}+\gamma\dfrac{P}{2}(1+\sqrt{\rho_{P}})^{2}\,\bar{q}_{1}^{{}^{P-1}}\;}\;+\tilde{\beta}y.$
(150)

Using this new parameter $y$, we can recast the equation for $\bar{q}_{2}$ as
a derivative of the magnetization

$\begin{array}[]{lll}\dfrac{\partial\bar{m}}{\partial
y}=\delta\bar{q}_{2}-D\Delta\bar{q}\Longrightarrow\delta\bar{q}_{2}=\dfrac{\partial\bar{m}}{\partial
y}+D\Delta\bar{q}\end{array}$ (151)

where we have used $\Delta\bar{q}=\bar{q}_{2}-\bar{q}_{1}$ and, as
$\tilde{\beta}\to\infty$, $\tilde{\beta}\theta\to D\in\mathbb{R}$. Thus, in
the zero temperature limit the last three equations in Eq. (70) become

$\begin{array}[]{lll}\bar{m}&\to&\mathbb{E}_{Y^{(1)}}\left\\{\dfrac{\mathbb{E}_{Y^{(2)}}\left[\mathrm{sign}{[g(\bm{Y},\bar{m},\Delta\bar{q})]}\>e^{D|g(\bm{Y},\bar{m},\Delta\bar{q})|}\,\right]}{\mathbb{E}_{Y^{(2)}}\left[e^{D|g(\bm{Y},\bar{m},\Delta\bar{q})|}\right]}\right\\}\\\
\Delta\bar{q}&\to&1-\mathbb{E}_{Y^{(1)}}\left\\{\dfrac{\mathbb{E}_{Y^{(2)}}\left[\mathrm{sign}{[g(\bm{Y},\bar{m},\Delta\bar{q})]}\>e^{D|g(\bm{Y},\bar{m},\Delta\bar{q})|}\,\right]}{\mathbb{E}_{Y^{(2)}}\left[e^{D|g(\bm{Y},\bar{m},\Delta\bar{q})|}\right]}\right\\}^{2}\\\
\bar{q}_{2}&\to&1.\end{array}$ (152)

Now, if we suppose $\Delta\bar{q}\ll 1$ the (150) reduces to

$g(\bm{Y},\bar{m},\Delta\bar{q})=C_{0}+C_{1}Y^{(1)}+C_{2}Y^{(2)}+\mathcal{O}(\Delta\bar{q})$
(153)

where

$\begin{array}[]{lll}C_{0}=\dfrac{P}{2}\bar{m}^{P-1}\,,&C_{1}=\sqrt{\rho\left(\dfrac{P}{2}\bar{m}^{P-1}\right)^{2}+\gamma(1+\sqrt{\rho_{P}})^{2}\dfrac{P}{2}}\,,&C_{2}=\sqrt{\gamma(1+\sqrt{\rho_{P}})^{2}\dfrac{P(P-1)}{2}\Delta\bar{q}}\,.\end{array}$
(154)

Performing the integral over $Y^{(2)}$ we reach Eqs. (77)-(78).

## References

  * (1) J. J. Hopfield.  Neural networks and physical systems with emergent collective computational abilities.  Proceedings of the National Academy of Sciences of the United States of America, 79:2554–2558, 1982.
  * (2) M. Mézard, G. Parisi, and M. A. Virasoro.  Spin glass theory and beyond: An Introduction to the Replica Method and Its Applications, volume 9.  World Scientific Publishing Company, 1987.
  * (3) M. Talagrand.  Spin glasses: a challenge for mathematicians: cavity and mean field models, volume 46.  Springer Science & Business Media, 2003.
  * (4) D. J. J Amit.  Modeling brain function: The world of attractor neural networks.  Cambridge university press, 1989.
  * (5) V. Dotsenko.  An introduction to the theory of spin glasses and neural networks.  World Scientific, 1994.
  * (6) A. Engel.  Statistical mechanics of learning.  Cabridge Press, 2001.
  * (7) H. Nishimori.  Statistical physics of spin glasses and information processing: an introduction.  Clarendon Press, 2001.
  * (8) A. C. C. Coolen, R. Kuhn, and P. Sollich.  Theory of Neural Information Processing Systems.  Oxford University Press, Inc., USA, 2005.
  * (9) H. Huang.  Statistical mechanics of neural networks.  Springer Press, 2021.
  * (10) E. Agliari, A. Barra, P. Sollich, and L. Zdeborová.  Machine learning and statistical physics.  J. Phys. A, 53:500401, 2020.
  * (11) G. Carleo, I. Cirac, K. Cranmer, L. Daudet, M. Schuld, N. Tishby, L. Vogt-Maranto, and L. Zdeborová.  Machine learning and the physical sciences.  Reviews of Modern Physics, 91(4):045002, 2019.
  * (12) D. J. Amit, H. Gutfreund, and H. Sompolinsky.  Storing infinite numbers of patterns in a spin-glass model of neural networks.  Physical Review Letters, 55(14):1530, 1985.
  * (13) I. Kanter.  Asymmetric neural networks with multispin interactions.  Physical Review A, 38(11):5972, 1988.
  * (14) E. Gardner.  Multiconnected neural network models.  Journal of Physics A: General Physics, 20, 1987.
  * (15) P. Baldi and S. S. Venkatesh.  Number of stable points for spin-glasses and neural networks of higher orders.  Physical Review Letters, 58:913, 1987.
  * (16) D. Krotov and J. J. Hopfield.  Dense associative memory for pattern recognition.  Advances in Neural Information Processing Systems, pages 1180–1188, 2016.
  * (17) H. Ramsauer, B. Schäfl, J. Lehner, P. Seidl, M. Widrich, T. Adler, L. Gruber, M. Holzleitner, M. Pavlović, G. K. Sandve, et al.  Hopfield networks is all you need.  arXiv preprint arXiv:2008.02217, 2020.
  * (18) E. Agliari and et al.  Neural networks with a redundant representation: Detecting the undetectable.  Physical Review Letters, 124:28301, 2020.
  * (19) D. Krotov and J. J. Hopfield.  Dense associative memory is robust to adversarial inputs.  Neural Computation, 30:3151–3167, 2018.
  * (20) C. Lucibello and M. Mezard.  The exponential capacity of dense associative memories.  arXiv, page 2304.14964, 2023.
  * (21) A. Barra, A. Bernacchia, E. Santucci, and P. Contucci.  On the equivalence of hopfield networks and boltzmann machines.  Neural Networks, 34:1–9, 2012.
  * (22) G. E Hinton, T. J. Sejnowski, and D. H. Ackley.  Boltzmann machines: Constraint satisfaction networks that learn.  Carnegie-Mellon University, Department of Computer Science Pittsburgh, PA, 1984.
  * (23) E. Agliari, A. Barra, A. Galluzzi, F. Guerra, and F. Moauro.  Multitasking associative networks.  Physical review letters, 109(26):268101, 2012.
  * (24) P. Sollich, D. Tantari, A. Annibale, and A. Barra.  Extensive parallel processing on scale-free networks.  Physical review letters, 113(23):238106, 2014.
  * (25) J. Tubiana and R. Monasson.  Emergence of compositional representations in restricted boltzmann machines.  Phys. Rev. Lett., 118.13:138301, 2017.
  * (26) A. Barra, G. Genovese, P. Sollich, and D. Tantari.  Phase transitions in restricted boltzmann machines with generic priors.  Physical Review E, 2017(04):042156, 2017.
  * (27) A. Barra, G. Genovese, P. Sollich, and D. Tantari.  Phase diagram of restricted boltzmann machines and generalized hopfield networks with arbitrary priors.  Physical Review E, 2018(02):022310, 2018.
  * (28) E. Agliari and et al.  A transport equation approach for deep neural networks with quenched random weights.  Journal of Physics A: Mathematical and Theoretical, 54, 2021.
  * (29) E. Agliari and G. Sebastiani.  Learning and retrieval operational modes for three-layer restricted boltzmann machines.  J. Stat. Phys., 185.2:10, 2021.
  * (30) D. Alberici, A. Barra, P. Contucci, and E. Mingione.  Annealing and replica-symmetry in deep boltzmann machines.  Journal of Statistical Physics, 180(1):665–677, 2020.
  * (31) D. Alberici, P. Contucci, and E. Mingione.  Deep boltzmann machines: rigorous results at arbitrary depth.  In Annales Henri Poincaré, volume 22, pages 2619–2642. Springer, 2021.
  * (32) P. Mehta and D. Schwab.  An exact mapping between the variational renormalization group and deep learning.  arXiv, page 1410.3831, 2014.
  * (33) E. Agliari and et al.  The emergence of a concept in shallow neural networks.  Neural Networks, 148:232–253, 2022.
  * (34) F. Alemanno, M. Aquaro, I. Kanter, A. Barra, and E. Agliari.  Supervised hebbian learning.  Europhysics Letters, 141(1):11001, 2023.
  * (35) H. B. Barlow.  Unsupervised learning.  Neural computation, 1(3):295–311, 1989.
  * (36) P. Cunningham, M. Cord, and S. J. Delany.  Supervised learning.  In Machine learning techniques for multimedia, pages 21–49. Springer, 2008.
  * (37) E. Agliari, L. Albanese, F. Alemanno, A. Alessandrelli, A. Barra, F. Giannotti, D. Lotito, and D. Pedreschi.  Dense hebbian neural networks: A replica symmetric picture of unsupervised learning.  Physica A: Statistical Mechanics and its Applications, 627:129143, 2023.
  * (38) E. Agliari, L. Albanese, F. Alemanno, A. Alessandrelli, A. Barra, F. Giannotti, D. Lotito, and D. Pedreschi.  Dense hebbian neural networks: a replica symmetric picture of supervised learning.  Physica A: Statistical Mechanics and its Applications, 626:129076, 2023.
  * (39) A. Albanese, L. Alessandrelli, A. Barra, and A. Annibale.  About the de almeida-thouless line in neural networks.  Physica A (in press), preprint arXiv:2303.06375, 2023.
  * (40) A. Bovier and P. Picco.  Mathematical aspects of spin glasses and neural networks, volume 41.  Springer Science & Business Media, 2012.
  * (41) G. Gradenigo, M. C. Angelini, L. Leuzzi, and F. Ricci-Tersenghi.  Solving the spherical p-spin model with the cavity method: equivalence with the replica results.  Journal of Statistical Mechanics: Theory and Experiment, 2020(11):113302, 2020.
  * (42) A. Barra, A. Di Biasio, and F. Guerra.  Replica symmetry breaking in mean-field spin glasses through the hamilton–jacobi technique.  Journal of Statistical Mechanics: Theory and Experiment, 2010(09):P09006, 2010.
  * (43) F. Guerra.  Broken replica symmetry bounds in the mean field spin glass model.  Communications in Mathematical Physics, 233:1–12, 2003.
  * (44) E. Agliari and et al.  Replica symmetry breaking in neural networks: A few steps toward rigorous results.  Journal of Physics A: Mathematical and Theoretical, 53, 2020.
  * (45) R. Rammal, G. Toulouse, and M.A. Virasoro.  Ultrametricity for physicists.  Rev. Mod. Phys., 58.3:765, 1986.
  * (46) L. F. Cugliandolo.  The effective temperature.  Journal of Physics A: Mathematical and Theoretical, 44(48):483001, 2011.
  * (47) J. Van Mourik and A.C.C. Coolen.  Cluster derivation of parisi’s rsb solution for disordered systems.  Journal of Physics A: Mathematical and General, 34, 2001.
  * (48) A. Barra, F. Guerra, and E. Mingione.  Interpolating the sherrington–kirkpatrick replica trick.  Philosophical Magazine, 92, 2012.
  * (49) D. Sherrington and S. Kirkpatrick.  Solvable model of a spin-glass.  Phys. Rev. Lett., 35:1792–1796, Dec 1975.
  * (50) G. Parisi.  Infinite number of order parameters for spin-glasses.  Physical Review Letters, 43:23:1754–1758, 1979.
  * (51) L. Albanese, F. Alemanno, A. Alessandrelli, and A. Barra.  Replica symmetry breaking in dense hebbian neural networks.  Journal of Statistical Physics, 189(2):1–41, 2022.
  * (52) D. Amit and H. Gutfreund.  Statistical mechanics of neural networks near saturation.  Ann. of Phys., 173:30–67, 1987.
  * (53) A. Crisanti, D. J. Amit, and H. Gutfreund.  Saturation level of the hopfield model for neural network.  Europhysics Letters (EPL), 2:337–341, 8 1986.
  * (54) B. Derrida.  Random-energy model: An exactly solvable model of disordered systems.  Physical Review B, 24(5):2613, 1981.
  * (55) H. Steffan and R. Kühn.  Replica symmetry breaking in attractor neural network models.  Zeitschrift für Physik B Condensed Matter, 95:249–260, 1994\.
  * (56) G. Biroli and M. Mezard.  Generative diffusion in very large dimensions.  arXiv preprint, page 2306.03518, 2023.
  * (57) E. Agliari, A. Alessandrelli, A. Barra, and F. Ricci-Tersenghi.  Parallel learning by multitasking neural networks.  Journal of Statistical Mechanics: Theory and Experiment, 2023(11):113401, 2023.
  * (58) J. R. L. de Almeida and D. J Thouless.  Stability of the sherrington-kirkpatrick solution of a spin glass model.  Journal of Physics A: Mathematical and General, 11(5):983, 1978\.


# Multi-Task End-to-End Training Improves Conversational Recommendation

Naveen Ram, Dima Kuzmin, Ellie Ka In Chio
Moustafa Farid Alzantot, Santiago Ontanon, Ambarish Jash, and Judith Yue Li
Google Research
<EMAIL_ADDRESS>
###### Abstract

In this paper, we analyze the performance of a multitask end-to-end
transformer model on the task of conversational recommendations, which aim to
provide recommendations based on a user’s explicit preferences expressed in
dialogue. While previous works in this area adopt complex multi-component
approaches where the dialogue management and entity recommendation tasks are
handled by separate components, we show that a unified transformer model,
based on the T5 text-to-text transformer model, can perform competitively in
both recommending relevant items and generating conversation dialogue. We
fine-tune our model on the ReDIAL conversational movie recommendation dataset,
and create additional training tasks derived from MovieLens (such as the
prediction of movie attributes and related movies based on an input movie), in
a multitask learning setting. Using a series of probe studies, we demonstrate
that the learned knowledge in the additional tasks is transferred to the
conversational setting, where each task leads to a $9\%-52\%$ increase in its
related probe score.

##  1 Introduction

The modern recommendation systems found in commercial applications are largely
based on implicit preferences, such as a user’s history of web page clicks,
item purchases, or media streams, with the record of these actions used to
retrieve relevant recommendations Rendle et al. (2012). This approach often
works, but in the case where a user might not have an extensive history, or
might desire a recommendation which doesn’t match their usual niche, we might
want a system which can take advantage of explicit preferences. With the
growing success of deep learning language models, it has become possible to
design conversational recommendation models which can communicate with a user
directly while retrieving custom recommendations based on the user’s explicit
preferences.

Most previous work on conversational recommender systems adopts a multi-
component approach (Gao et al., 2021). These models often are implemented
using a recommendation component, which analyzes the mentioned entities in
order to predict a related item, and a dialogue component, which analyzes the
input phrases and generates a conversational response Jannach et al. (2020).
Multi-component approaches are appealing because they can be built directly
from standard models in the dialogue and recommendation fields. However, the
knowledge learned by each component is not immediately available to the other
components (i.e., the item recommendation model does not benefit directly from
conversation state, and vice versa), preventing these approaches from taking
advantage of the data to its fullest extent. Ideally, a conversational
recommendation model should be able to both use descriptive language in the
dialogue to retrieved relevant items and generate engaging dialogue about the
items simultaneously. To address this problem, in this paper we investigate
whether an end-to-end approach to conversational recommendations using a
single component model can improve dialogue and recommendation generation by
allowing the model to fully utilize the conversation features for both tasks.

This paper strives to show the feasibility of a unified model for
conversational recommendations by leveraging a single large transformer model
to generate both relevant recommendations and natural dialogue and evaluating
the benefits of a fully unified dialogue and recommendation module. To
determine whether single end-to-end model match or outperform multi-component
approaches, we train our model on several standard datasets in the domain of
movie recommendations and compare our results to previous work. To measure the
benefit of generating dialogue and recommendations in the same model, we
follow a common procedure in related work Penha and Hauff (2020) and design a
series of probes to assess how the model leverages different types of
information to generate dialogue and recommendations. One potential problem of
a single component system is the reliance on a large dataset of sample
dialogues containing both recommendation and language information. To bypass
the need for a single large dialogue dataset, we finetune the pretrained T5
model on a relatively small dataset of dialogues, and incorporate movie
relationship, attribute, and description information from additional datasets
using a multitask setup.

The main contributions of this paper are:

  * •

A fully end-to-end approach to conversational recommendation that uses a
unified model for both dialogue and item recommendation.

  * •

Conducting a series of probe studies that shows how conversational
recommendation tasks benefits from knowledge learned by the model via multi-
task training on a number of separate, small datasets.

The remainder of this paper is structured as follows. First, we briefly
present some related work on conversational recommender systems, transformer
models and probes studies. After that, we describe our T5-based approach, and
the datasets, tasks and training procedure used to train our model. We then
describe our experimental methodology, and a series of probe studies showing
how dialogue and recommendation mutually improved by sharing a common model.

###  1.1 Related Work

The section presents a brief background on conversational recommendations,
multitask transformer models, and the evaluation of conversational
recommendation models through probe studies.

A recent survey published by Gao et al. (2021) highlights the range of
strategies used to address the different challenges faced by a conversational
recommendation system (CRS): Question-based User Preference Elicitation,
performed by models which use natural dialogue questions to build an accurate
user representation Zou et al. (2020), Multi-turn Conversational
Recommendation Strategies, which use various dialogue representations to keep
track of user preferences over a long form dialogue Li et al. (2020), and
Natural Language Understanding and Generation, which often relies on large
pretrained language models to translate recommendations into natural text Wang
et al. (2021). While these challenges have been approached using a variety of
multi-component models, our model aims to demonstrate that a single-component
transformer model can perform the task of conversational recommendations, and
even benefit from cross-task transfer due to its unified design. A different
approach to unified conversational recommendations, by Deng et al. (2021),
succeeds in building a single-component system based on a graph-based Markov
Decision Process which switches between predefined question-asking,
recommendation, and conversation patterns in order to lead a user to a
recommendation in a multi-turn dialogue. This approach, however, is fixed to a
rigid flow of conversation patterns and does not contain natural language
understanding or generation components necessary to create or understand free-
form dialogues or unstructured conversations.

Dialogue generation has historically been approached in many ways, with recent
efforts focusing on RNNs models (like LSTMs Hochreiter and Schmidhuber
(1997)), and transformers (Vaswani et al., 2017). Recommendation systems
typically perform collaborative filtering on a set of user-item associations
using a range of models such as matrix factorization systems or autoencoders
(Ricci et al., 2011). Li et al. (2018) proposed an approach to combining these
two areas into a functional conversational recommendation model, using an
autoencoder recommender in conjunction with a GRU based hierarchical encoder-
decoder system to generate the dialogue. There is some interplay between
components, with mentioned movies and sentiments being fed into the
autoencoder in order to retrieve a relevant recommendation based on a user’s
liked and disliked movies, but the generation of dialogues and recommendations
are still largely separate. Chen et al. (2019) took this approach one step
further, creating a conversational recommendation system which would use
mentioned entities in the dialogue to conduct a knowledge graph search of
related items and add a vocabulary bias based on the user representation back
into the transformer-based dialogue generation module. Although this model
demonstrates the potential for transfer between dialogue and recommendation
tasks, it requires a complex structure where incomplete representations of
both the dialogue and recommendation features are passed to separate
components and then joined with a switching network. In this paper we attempt
to fully leverage this cross-task transfer without the need for separate
components.

In recent years, many studies have demonstrated the effectiveness of large,
pre-trained transformer-based language models on a range of natural language
generation tasks. The architecture, which makes use of self attention blocks
in order to model language, was proposed by Vaswani et al. (2017) and achieved
state-of-the-art performance on a variety of benchmarks. When pretrained on a
large corpus of text, transformer models such as BERT Devlin et al. (2018),
GPT-3 Brown et al. (2020), and UniT Hu and Singh (2021) have shown the ability
to handle multiple language-based tasks with minimal finetuning. The T5 model,
introduced by Raffel et al. (2019), has demonstrated a distinct ability to
incorporate different types of knowledge from multiple sources and handle
several disparate tasks in the text-to-text format.

In regards to evaluating these large transformer models on the task of
conversational recommendations, one effective approach proposed by Penha and
Hauff (2020) is to use probe studies to measure the model’s ability to score
the likelihood of certain entities when conditioned on a set of generated
inputs. Penha and Hauff evaluate BERT’s performance on conversational
recommendation tasks by using BERT’s prediction, similarity, and next sentence
prediction function to score the model’s ability to associate a book, movie,
or song with a related item or attribute. Although Penha and Hauff evaluate
BERT’s knowledge on conversational recommendations, it is important to note
that in their study BERT is not acting as a full conversational recommendation
system on its own and cannot be considered an example of an end-to-end CRS. It
is only being used to rank probes against each other, and not to generate
dialogues and recommendation or manage a conversation with a user.

##  2 Our Approach

The main idea of our approach is to formulate the conversational
recommendation task as an instance of the text-to-text problem. We finetune a
pre-trained transformer model on the movie recommendation dialogues contained
in the ReDial dataset, and improve the model’s ability to utilize movie
attributes and descriptive details within the dialogues through the
introduction of additional training tasks in a multi-task learning setting. In
this section we present a background on the T5 transformer model, a summary of
each of the training datasets, and an overview of the finetuning parameters we
used.

###  2.1 T5 Model

T5 is a large, publicly available, encoder-decoder transformer based model
created by Raffel et al. (2019). The model was trained and structured with the
intent to support as many different use cases as possible using a text-to-text
format. In the context of recommendation systems, T5 and related models are
attractive because they perform well on natural language understanding and
generation tasks and has demonstrated the ability to train on multiple
disparate types of text data within one model.

###  2.2 ReDial Dialogue Task

|  ReDial Dialogues |  MovieLens Sequences |  MovieLens Tags |  MovieLens Reviews
---|---|---|---|---
Training Inputs |  [User] I’m in the mood to watch a romantic comedy. What do you suggest? [Assistant] @ 50 First Dates (2004) @ |  @ The Incredibles (2004) @ Harry Potter and the Chamber of Secrets (2002) The Hunger Games Mockingjay - Part 1 (2014) @ |  drama, based on a book, adapted from:book |  Review for @ Alice in Wonderland (1951) @:
Training Targets |  Yes she is good. Do you like @ The Wedding Singer (1998) @ |  Underworld: Awakening (2012) |  The Book Thief (2013) |  Perhaps because its surrealism matched the hippy culture of psychedelia, Alice in Wonderland (1951) 1 …
Knowledge |  Dialogue |  Recommendation |  Attributes |  Description

Table 1: Comparison of the primary training task (ReDial Dialogues) and the
three auxilary training tasks designed to increase recommendation, attribute,
and description knowledge.

The ReDial (Recommendation Dialogues) dataset is an annotated set of 11248
dialogues collected through Amazon Mechanical Turk Li et al. (2018). Each
dialogue contains the movies and messages sent between two parties acting as
either a "recommender" or a "recommendation seeker". Although this dataset is
relatively small, and doesn’t necessarily capture as much movie relationship
and attribute data as other recommendation-focused datasets, we have found
that it provides enough examples for the T5 to learn the style and structure
of conversational recommendations.

For each conversation in the dataset we create a training example
corresponding to each response from the human recommender. The model inputs
contain the conversation up to a certain recommender utterance, with the
outputs containing the next utterance from the recommender party. Using this
format the T5 model can learn to parse relevant movie, attribute, and dialogue
details from the previous messages in the conversation and formulate an
appropriate response. We use the T5’s standard vocabulary, so movie titles are
processed by the word, the same as any other piece of the input. To help the
model learn these titles, @ signs are used to separate movie titles from the
rest of the dialogues.

Each message in the ReDial conversation is preceded by either a [USER] or a
[ASSISTANT] tag to indicate its source. The redial conversation shown in
appendix A has been processed into multiple training examples corresponding to
each response by the recommender. Table 1 shows a sample training example from
this process.

###  2.3 MovieLens Sequences Task

The MovieLens 25m dataset is a collection of 25 million ratings and one
million tag associations often used to quantify movie relationships and
attributes (Harper and Konstan, 2015). We utilize this data for multiple
tasks, as it can be used to quantify different types of movie information. The
first additional training task is to recommend a movie given a sequence of 1-9
related movies. This task incorporates movie relationship information in order
to increase the quality of recommendations in the ReDial Dialogues task, and
is referred to as the ML Sequences task.

In order to use the user ratings contained in the MovieLens 25m dataset to
generate movie associations, we create sequences of movies wherever there are
10 movies rated higher than 4.0 / 5.0 by the same user. From these sequences
we create examples for each $n$ where $1<n<10$ by mapping the first $n$ movies
as the inputs and the movie in position $(n+1)$ as the target. An example of
this format is shown in Table 1.

###  2.4 MovieLens Tags Task

The MovieLens 25m dataset contains a tag genome which scores each movie’s
relevance across a set of 1,129 tags Vig et al. (2012). These tags are movie
attributes or descriptive words which often correspond to genres ("horror",
"action", "mystery"), plot elements ("alien invasion", "character study",
"father daughter relationship"), opinion ("excellent script", "boring", "over
the top"), or general information ("oscar (best actor)", "based on a book",
"stanley kubrick"). For each movie, we add each tag with a relevance score
over 0.8 to the movies tag list. From these tag lists we randomly sample
examples containing 1-5 tags as the input and the related movie as a the
target. This mapping allows the model to associate movie attribute information
and better recommend movies based on descriptive details in-dialogue. Table 1
displays an example of a tag-to-movie mapping.

###  2.5 MovieLens Reviews Task

The final training task, referred to as the MovieLens Review task, uses a
joint dataset created by Penha and Hauff (2020) to incorporate additional
movie description and opinion data. The training examples for this task are
generated from the reviews portion of Penha and Hauff (2020)’s search dataset,
which contains the IMDB user reviews associated with each movie in the
MovieLens database. These reviews contain movie attribute data written in the
kind of casual, natural dialogue style found in the ReDial dataset, so they
aid the model’s natural text generation and descriptive capabilities. As shown
in Table 1, these reviews are processed into examples where the model is asked
to predict the next sentence of a review given a movie title and the truncated
review 111Because movie titles and title fragments in the MovieLens Reviews
dataset are not delimited, the MovieLens Reviews training task does not use
’@’ signs to separate movie titles..

###  2.6 Multitask Training

The T5 module supports multitask training, where examples from training
dataset are loaded through their own preprocessing steps (in our case only
lowercasing). We opt to finetune the base size (220 million parameters) with a
learning rate of 0.003 for 40,000 steps and batch size 128 222The T5 was
finetuned with using the public T5 codebase: https://github.com/google-
research/text-to-text-transfer-transformer. Texts longer than the maximum
sequence length, i.e., 512 for inputs and 128 for targets are truncated. We
train variants with different combinations of the four training tasks in order
to isolate their effects. Examples from each task were loaded equally often.
As suggested by Raffel et al. (2019), we prepend the inputs to each task with
a task label: “redial conversation:", “movielens sequence:", “movielens
tags:", or “movielens review:". From this point, the name T5 will be used to
refer to the out-of-the-box pretrained T5 model and the name T5-CR will be
used to refer to our custom T5 model with all four finetuning tasks.

##  3 Baseline Evaluations

In order to determine whether our end-to-end approach can perform
competitively on dialogue and recommendation, we compare our performance using
BLEU score and Recall. These metrics are both run on the evaluation set
provided with the ReDial dataset. The BLEU score acts as a measure of dialogue
quality, by measuring the similarity of the model and human responses. The
Recall metric is calculated by comparing the movies mentioned by the model in
the evaluation dialogues to the set of movies mentioned by the human
recommender in the ReDial dialogues evaluation set. End-to-End Recall refers
to the Recall@1 caluclated in the dialogue task, while Rec Module Recall is
calculated on the same movies run through only the isolated recommendation
module if one exists. These two metrics were selected as they are standards
which have been run on many of the previous works in the area. We calculated
baselines for the ReDial and KBRD models, which did not have reported BLEU or
Recall scores, and sources our other baselines from Wang et al.. In Table 2 we
compare the most relevant models: (1) ReDial Li et al. (2018) an HRED CR
system, (2) KBRD Chen et al. (2019) which uses a transformer for dialogue and
a knowledge graph for recommendation, (3) KGSF Zhou et al. (2020) which uses
fuses word and entity knowlege using a knowlege graph and transformer, (4)
GPT-2 a pretrained transformer, (5) RID Wang et al. (2021) which uses a
pretrained language model with a relational graph convolutional network
(RGCN).

Model Name | Model Type | BLEU | Rec Module Recall | End-to-End Recall
---|---|---|---|---
ReDial | Autoencoder + LSTM | 8.38 | 2.30 | 0.70
KBRD | Knowledge Graph + Transformer | 11.00 | 3.00 | 0.80
KGSF | Knowledge Graph Semantic Fusion | —– | 3.90 | 0.90
GPT-2 | Pretrained Transformer | —– | —– | 1.40
RID | RGCN + Pretrained Transformer | 20.70 | —– | 3.10
T5-CR | Finetuned T5 (4 Tasks) | 15.39 | —– | 6.93
Table 2: BLEU and Recall@1 metric comparisons between T5-CR, our T5 variant
finetuned on 4 tasks, and the previous approaches to conversational
recommendations. All evaluation scores are calculated based on the model’s
performance on the ReDial validation dialogues. The baseline scores for the
KGSF, GPT-2, and RID models along with the End-to-End Recall scores for the
KBRD and ReDial models were taken from Wang et al. (2021).

###  3.1 BLEU

BLEU score is a standard metric used in machine translation and text
generation tasks which quantifies how similar a generated phrase is to the
expected target phrase Papineni et al. (2002). We postprocess our ReDial
predictions to replace movie titles with a "__unk__" token before calculating
the metric. This ensures that our BLEU score only captures information on the
closeness of the dialogue to our target, and isn’t influenced by
correct/incorrect movie titles and recommendations. Our T5-CR model, trained
on all four training tasks was able to outperform the KBRD and ReDial models
approaches, achieving a BLEU score of 15.39. The increase in BLEU score is
likely a result of the introduction of movie description and attribute data
through the multitask training setup as well as general increased fluency of
large pre-trained language models such as the T5. The RID model trained by
Wang et al. was also built using a pretrained tranformer and performed the
best, with a score of 20.70.

###  3.2 Recall

In order to evaluate the quality of the recommendations we calculate End-to-
End Recall as the percent of movies generated by the model in-dialogue which
correspond to one of the known recommendations given by the human recommender
in the corresponding redial dialogue. Rec Module Recall refers to the Recall@1
score when only the isolated recommendation modules are used. In previous
efforts such as the ReDial, KBRD, and KGSF models, the End-to-End Recall
scores were significantly lower than the Rec Module Recall scores, suggesting
that the models were less likely to apply their recommendation knowledge
accurately in-dialogue compared to as a separated recommendation task. This
highlights the advantage of the end-to-end approach, as the unified structures
ensures the model can generate high quality dialogue and recommendations
simultaneously. The multitask T5-CR model achieved a Recall score of 6.93,
which outperforms all baseline models. The increase in Recall is likely due to
the end-to-end structure of the model allowing it to use dialogue features to
retrieve better recommendations, as well as the movie relationship and
attribute training tasks allowing for more accurate analysis of user
preferences.

##  4 Probe Studies

Although the BLEU and Recall scores on the ReDial Evaluation Dataset prove
that an end-to-end model can outperform multi-component approaches, the scores
do not give us insight on the extent to which our multitask training setup
benefited the model’s ability to generate dialogue and recommendations. Also,
the ReDial evaluation slice covers a small selection of movies and dialogue
interactions. In order to determine the contribution of each of the training
tasks, as well as any measurable advantages of cross-task transfer within the
same T5 architecture, we present four probe studies in the style of Penha and
Hauff (2020). Each probe tests a specific dialogue interaction by measuring
the T5-CR’s ability to distinguish between relevant and unrelated information
conditioned on different types of sample dialogues. In order to filter out
misspellings, alternate titles, and rare movies which the model has little
information on, the probes are generated using the set of movies which occur
over 30 times in the ML Sequences dataset. With probe examples generated from
this set of around 5,000 movies, we are able to run evaluations on a much
larger range of data than the limited ReDial evaluation set. These probes are
designed to measure the model’s ability to apply the information gained
through its multitask training in a dialogue setting, therefore all probe data
is evaluated through the ReDial Dialogue tasks.

T5 Finetuning Tasks | Rec Probe | Attr Probe | Combo Probe | Desc Probe
---|---|---|---|---
None (T5) | 0.5493 | 0.4908 | 0.5597 | 0.5936
ReDial | 0.4716 | 0.5046 | 0.5731 | 0.7097
ReDial + |  |  |  |
ML Sequences | 0.6359 | 0.5869 | 0.7367 | 0.7307
ReDial + |  |  |  |
ML Tags | 0.5670 | 0.7826 | 0.8016 | 0.7133
ReDial + |  |  |  |
ML Reviews | 0.4771 | 0.5091 | 0.5833 | 0.7763
All (T5-CR) | 0.6599 | 0.7678 | 0.8418 | 0.7928
Table 3: Comparison of probe scores across T5 models with different finetuning
tasks.

###  4.1 Recommendation Probe

The recommendation probe measures the model’s ability to distinguish a related
movie from a popular movie chosen at random. In order to quantify related
movies based on cooccurrence in the ML Sequences dataset, we rank movies based
on $PMI^{2}$ Role and Nadif (2011), a variation on pointwise mutual
information ($PMI$). $PMI^{2}$ is a commonly used variation on $PMI$ which
reduces $PMI$’s known bias toward rare and infrequent items Role and Nadif
(2011). For each of the top ten related movies we sample a random popular
movie from the top $10\%$ of movies (ranked by frequency in the ML Sequences
dataset). For each of the ten $(related_{i},\ popular_{i})$ pairs generated
for each movie, we create a probe by swapping in the movies to a generic piece
of dialogue, as seen in Table 4. The probe score is calculated as the percent
of probes where the models log-likelihood score, $L(\theta)$, of the target
containing the related movie was higher than that of the random popular movie.
Note that different phrasings and dialogue formats were tested with little
effect on the probe results.

As shown in Figure 3, the introduction of the ML Sequences task improved the
model’s ability to differentiate between related and random movies, reflected
by a $30\%$ increase in the recommendation probe scores between the ReDial-
only model and the ReDial + ML Sequences model. This increase demonstrates
that the patterns in the movie sequences fed in the ML Sequences tasks can be
generalized and applied within the dialogue tasks. Interestingly, the ReDial +
ML Tags model also outperformed the ReDial-only model, with an increase of
$23\%$ in recommendation probe scores over the ReDial-only model.

This increase demonstrates an advantage of the end-to-end format: data
incorporated to help the understanding of descriptive words in the dialogues
also boosted performance on movie-to-movie recommendation, despite the
additional data not directly specifying any movie relationships. Because
recommendation and dialogue are handled in the same model, it can leverage
patterns in seemingly unrelated data. Here, the model is likely using the
overlap of tags associated with different movies to help determine whether
they are related. In the combined model, where all four training tasks were
included, the model performed the best ($+37\%$ over ReDial-only), a score
which demonstrates the viability of multitask transformers to incorporate many
different data sources and tasks without losing performance.

Overall the performance of the recommendation probe represents a transfer of
movie-relationship knowledge between training tasks, but this transfer is not
perfect. While the probes (fed into the ReDial Dialogues task) achieved a
score of $.6599$ in the Combined model, the same pairs fed into the ML
Sequences task without any dialogue achieved a score of $.7711$. This increase
indicates either an incomplete transfer of knowledge from the MovieLens
Sequences task to the ReDial Dialogues task, or a bias from the the movie
recommendation data already present in the ReDial Conversations. Similarly,
the T5’s performance on the movie recommendation probes is lower than that of
a purely Matrix Factorization model, which achieved a score of $.8096$ on the
movie pairs.

###  4.2 Attributes Probe

The attributes probe measures the model’s ability to use details and
descriptive words appearing in-dialogue to retrieve relevant movies. As shown
in Table 4, a probe is generated for each movie-tag association in the
MovieLens Tags dataset, with a random popular movie used as the negative.
Because many of the most popular tags (such as "action" or "excellent") might
apply to a large portion of the popular movies, we filter the negative to
ensure it isn’t associated with the given tag.

The attribute probe scores also demonstrated the effectiveness of multitask
learning, with the introduction of the ML Tags task leading to a $52\%$
increase in performance over the ReDial-only model. This probe directly shows
one of the advantages of end-to-end learning. Because dialogue analysis and
recommendation generation occurs in the same model, the descriptive attributes
mentioned in the input dialogue (or movie "tags") can help the model retrieve
a movie relevant to that attribute, even when no movie titles are mentioned in
the input. While the Combined model didn’t out-perform the RD Tags model, it
did perform consistently, with an accuracy of $.7689$ over the probe set.

|  Recommendation Probe |  Attribute Probe |  Combination Probe |  Description Probe
---|---|---|---|---
Input 1 (Related) |  [User] Can you recommend me a movie like @ Zootopia (2016) @ |  [User] Can you recommend me a vampire movie? |  [User] Can you recommend me a science fiction movie like @ Looper (2012) @? |  [User] What is your opinion on @ Ringing Bell (1978) @?
Input 2 (Rand. Popular) |  |  |  |  [User] What is your opinion on @ Robin Hood: Men in Tights (1993) @?
Target 1 (Related) |  Sure, have you seen @ Inside Out (2015) @? |  Sure, have you seen @ Interview with the Vampire: the Vampire Chronicles (1994) @? |  Sure, have you seen @ Edge of Tomorrow(2014) @? |  Watching this several times as a child was quite the…
Target 2 (Rand. Popular) |  Sure, have you seen @ I Am Sam (2001) @? |  Sure, have you seen @ Sicko (2007) @? |  Sure, have you seen @ Zoolander (2001) @? |
Data |  ML Sequences |  ML Tags |  ML Sequences + Tags |  ML Reviews
Metric |  $L(T_{1}\ |\ I_{1})>$ $L(T_{2}\ |\ I_{1})$ |  $L(T_{1}\ |\ I_{1})>$ $L(T_{2}\ |\ I_{1})$ |  $L(T_{1}\ |\ I_{1})>$ $L(T_{2}\ |\ I_{1})$ |  $L(T_{1}\ |\ I_{1})>$ $L(T_{1}\ |\ I_{2})$

Table 4: Comparison of the four probe sets, which determine whether the model
can correctly rank related entities as more likely than random negatives.

###  4.3 Combination Probe

The combination probe measures the multitask capabilities of the model,
determining whether attribute and movie entity data can be used simultaneously
to generate a relevant response. As shown in Table 4, a probe is generated for
each shared tag among each of a movies top 10 most related movies. As in the
attribute probe, we filter out the popular negative to ensure it does not
match the given tag.

The combination probe extends the findings of the previous two probes: not
only can the model use mentioned movies or movie attributes to influence its
recommendations, it can do both at the same time. Whereas a multi-component
approach to the problem would base its recommendation solely on the previously
mentioned movies or the attributes mentioned in-dialogue, an end-to-end
approach uses these pieces of information together. The Combined model was
able to differentiate $84.18\%$ of the probe pairs when given a movie and a
tag in the input dialogue, an improvement over its performance on either the
recommendation or attribute probes. This improvement demonstrates that when
using both types of information together, the model can more accurately
recommend a related movie.

###  4.4 Movie Description Probe

The previous three probes test whether the model can retrieve a relevant movie
title when conditioned on a dialogue. The movie description probe tests the
reverse direction: can the model retrieve a piece of relevant or descriptive
dialogue when conditioned on a certain movie title. To do this, we measure the
likelihood of a given review snippet taken from the first four sentences of a
review in the ML Reviews dataset. In previous probes, we have ranked two
different targets based on likelihood, but because review snippets differ
greatly in length, phrasing, style of language, and other factors which can
influence likelihood, we opt to keep the target the same and compare the
likelihood of a given review snippet when conditioned on a related/unrelated
movie. As shown in Table 4, for a related input $I_{1}$, a random popular
input $I_{2}$, and a review snippet $T$ we compare the log likelihood scores
and measure how often $L(T\ |\ I_{1})>L(T\ |\ I_{2})$.

The description probe demonstrates that in an end-to-end model, mentioning a
movie can prompt the model to retrieve relevant dialogue. This functionality
wouldn’t be in traditional multi-component approaches where mentioned movies
are processed separately from dialogue. The ML Reviews training task led to a
$9.38\%$ increase over the ReDial-only model, while the combined model was
able to achieve a score of $0.7929$, an $11.72\%$ increase over the ReDial-
only model.

##  5 Conclusion

In this paper, we presented a multitask approach to end-to-end conversational
recommendations. In direct comparison to two previously published models in
the domain, our T5-based architecture outperformed the baselines in both its
quality of dialogue and recommendation. When probed on recommendation,
attribute knowledge, and description, our model demonstrates that dialogues
and recommendations can be mutually improved by sharing a model architecture.
Specifically, the probes prove that the model is able to use dialogue features
to inform its recommendations and movie mentions to influence its dialogue
generation. These findings support a general trend in current natural language
processing landscape, where large pretrained transformer models are rapidly
becoming the state-of-the-art in many domains. In fact, our research has
implication on the broader area of multitask models, highlighting how a
limited dataset (such as ReDial) can be injected with information from several
auxiliary datasets, regardless of format. In the future, this effect might
shift the focus from training and combining optimized components for each
functionality of a system, to simply incorporating all desired information as
different tasks in a pretrained multitask model.

## References

  * Brown et al. (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language models are few-shot learners.  _CoRR_ , abs/2005.14165.
  * Chen et al. (2019) Qibin Chen, Junyang Lin, Yichang Zhang, Ming Ding, Yukuo Cen, Hongxia Yang, and Jie Tang. 2019.  Towards knowledge-based recommender dialog system.  _CoRR_ , abs/1908.05391.
  * Deng et al. (2021) Yang Deng, Yaliang Li, Fei Sun, Bolin Ding, and Wai Lam. 2021.  Unified conversational recommendation policy learning via graph-based reinforcement learning.  _CoRR_ , abs/2105.09710.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  BERT: pre-training of deep bidirectional transformers for language understanding.  _CoRR_ , abs/1810.04805.
  * Gao et al. (2021) Chongming Gao, Wenqiang Lei, Xiangnan He, Maarten de Rijke, and Tat-Seng Chua. 2021.  Advances and challenges in conversational recommender systems: A survey.  _CoRR_ , abs/2101.09459.
  * Harper and Konstan (2015) F. Maxwell Harper and Joseph A. Konstan. 2015.  The movielens datasets: History and context.  _ACM Trans. Interact. Intell. Syst._ , 5(4).
  * Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber. 1997.  Long short-term memory.  _Neural computation_ , 9(8):1735–1780.
  * Hu and Singh (2021) Ronghang Hu and Amanpreet Singh. 2021.  Transformer is all you need: Multimodal multitask learning with a unified transformer.  _CoRR_ , abs/2102.10772.
  * Jannach et al. (2020) Dietmar Jannach, Ahtsham Manzoor, Wanling Cai, and Li Chen. 2020.  A survey on conversational recommender systems.  _CoRR_ , abs/2004.00646.
  * Li et al. (2018) Raymond Li, Samira Kahou, Hannes Schulz, Vincent Michalski, Laurent Charlin, and Chris Pal. 2018.  Towards deep conversational recommendations.  _arXiv preprint arXiv:1812.07617_.
  * Li et al. (2020) Shijun Li, Wenqiang Lei, Qingyun Wu, Xiangnan He, Peng Jiang, and Tat-Seng Chua. 2020.  Seamlessly unifying attributes and items: Conversational recommendation for cold-start users.  _CoRR_ , abs/2005.12979.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002.  Bleu: A method for automatic evaluation of machine translation.  In _Proceedings of the 40th Annual Meeting on Association for Computational Linguistics_ , ACL ’02, page 311–318, USA. Association for Computational Linguistics.
  * Penha and Hauff (2020) Gustavo Penha and Claudia Hauff. 2020.  What does bert know about books, movies and music? probing bert for conversational recommendation.  In _Fourteenth ACM Conference on Recommender Systems_ , RecSys ’20, page 388–397, New York, NY, USA. Association for Computing Machinery.
  * Raffel et al. (2019) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2019.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _arXiv preprint arXiv:1910.10683_.
  * Rendle et al. (2012) Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2012\.  Bpr: Bayesian personalized ranking from implicit feedback.  _arXiv preprint arXiv:1205.2618_.
  * Ricci et al. (2011) Francesco Ricci, Lior Rokach, Bracha Shapira, and Paul B. Kantor. 2011.  _Recommender Systems Handbook_.  Springer.
  * Role and Nadif (2011) François Role and Mohamed Nadif. 2011.  Handling the impact of low frequency events on co-occurrence based measures of word similarity - a case study of pointwise mutual information.  In _Proceedings of the International Conference on Knowledge Discovery and Information Retrieval, KDIR 2011_.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in neural information processing systems_ , pages 5998–6008.
  * Vig et al. (2012) Jesse Vig, Shilad Sen, and John Riedl. 2012.  The tag genome: Encoding community knowledge to support novel interaction.  _ACM Trans. Interact. Intell. Syst._ , 2(3).
  * Wang et al. (2021) Lingzhi Wang, Huang Hu, Lei Sha, Can Xu, Kam-Fai Wong, and Daxin Jiang. 2021.  Finetuning large-scale pre-trained language models for conversational recommendation with knowledge graph.  _CoRR_ , abs/2110.07477.
  * Zhou et al. (2020) Kun Zhou, Wayne Xin Zhao, Shuqing Bian, Yuanhang Zhou, Ji-Rong Wen, and Jingsong Yu. 2020.  Improving conversational recommender systems via knowledge graph based semantic fusion.  _CoRR_ , abs/2007.04032.
  * Zou et al. (2020) Jie Zou, Yifan Chen, and Evangelos Kanoulas. 2020.  Towards question-based recommender systems.  _CoRR_ , abs/2005.14255.

##  Appendix A Appendix - Redial Dataset Sample

An example of the beginning of a ReDial con-versation, randomly selected from
the ReDial dataset is shown at 1.

REDIAL CONVERSATION:

Sender: I’m in the mood to watch a romantic comedy. What do you suggest?

Responder: @ 50 First Dates (2004) @ Have you seen that one?

Sender: Oh, I’ve seen that one. I really like... Figure 1: Beginning of a
randomly selected example from the ReDial dataset.