# Neutrino mixing and CP violation phases in Zee-Babu model

Vo Van Vien<EMAIL_ADDRESS>Department of Physics, Tay Nguyen University,
567 Le Duan, Buon Ma Thuot, DakLak, Vietnam  Hoang Ngoc Long
<EMAIL_ADDRESS>Institute of Physics, VAST, 10 Dao Tan, Ba Dinh, Hanoi,
Vietnam  Pham Ngoc Thu<EMAIL_ADDRESS>Faculty of Mathematics-
Physics-Informatics, Tay Bac University, Son La city, Vietnam

###### Abstract

We show that the neutrino mass matrix of the Zee-Babu model is able to fit the
most recent data on neutrino masses and mixing with large $\theta_{13}$ and
provides the Dirac and Majorana CP violation phases. For the normal hierarchy,
the Majorana phases ($\alpha_{21},\alpha_{31}$) are equal to zero, while for
the inverted pattern, one phase ($\alpha_{31}$) takes the value $2\pi$. The
Dirac phase ($\delta$) is predicted to either $0$ or $\pi$. The effective mass
governing neutrinoless double beta decay and the sum of neutrino masses are
consistent with the recent analysis. The model gives some regions of the
parameters of neutrino mixing angles in both normal and inverted neutrino mass
hierarchy.

###### pacs:

14.60.Pq, 14.60.St, 12.60.Fr, 11.30.Er

##  I Introduction

At present, neutrino and Higgs physics are hot topics in current Particle
Physics. The neutrino mass and mixing are the first evidence of beyond
Standard Model (SM) physics. Despite the Higgs boson bas been discovered by
the ATLAS and the CMS but in which model it belongs is still open question.
For the aforementioned reasons, the search for an extended model coinciding
with the current data on neutrino physics is one of our top priorities. In our
opinion, the model with the simplest particle content is preferred. By this
criterion, the Zee-Babu model zee ; zeebabu ; babu  is very attractive. In our
previous work lvZB , we have derived the exact solution for the neutrino mass
matrix in the model under consideration and derived some regions of the
parameters. In this paper, we focus on the Majorana phases, the effective mass
governing neutrinoless double beta decay and the sum of neutrino masses. As we
discuss below, the Zee-Babu can fit very well data on recent neutrino
experiments.

In this paper, starting from the neutrino mass matrix, we get the exact
solution, i.e., the eigenstates and the eigenvalues. As a consequence, the
neutrino mixing with large $\theta_{13}$ is generated. On the other hand, the
Dirac CP violation phase is predicted to either $0$ or $\pi$. In this work,
the Majorana phases is included and for the inverted pattern, one phase
($\alpha_{31}$) takes the value $2\pi$. The effective mass governing
neutrinoless double beta decay and the sum of neutrino mass are consistent
with the recent analysis. The model gives some regions of the parameters where
neutrino mixing angles and the normal neutrino mass hierarchy obtained
consistent with the recent experimental data. With this exact solution, we can
fit the most recent data on neutrino mass and mixing.

This paper is organized as follows. In Sec. II, we briefly present the Zee-
Babu model and its neutrino mass matrix. Sec. III is devoted for the solution
and phenomenology with focus on the Dirac and Majorana violation phases. We
summarize our result in the last section - Sec. IV.

##  II Zee-Babu model and neutrino mass matrix

With just two $\mbox{SU}(2)_{L}$ singlet Higgs fields, a singly charged field
$h^{-}$ and a doubly charged field $k^{--}$ and without right-handed
neutrinos, the new Yukawa interactions in the Zee-Babu model zeebabu  are

${\cal
L}_{Y}=f_{ab}\overline{(\psi_{aL})^{C}}\psi_{bL}h^{+}+h^{\prime}_{ab}\overline{(l_{aR})^{C}}l_{bR}k^{++}+H.c.,$
(1)

where $\psi_{L}$ stands for the left-handed lepton doublet, $l_{R}$ for the
right-handed charged lepton singlet and ($a,b=e,\mu,\tau$) being the
generation indices, a superscript C indicating charge conjugation. Note that
$f_{ab}$ is antisymmetric ($f_{ab}=-f_{ba}$) and $h^{\prime}_{ab}$ is
symmetric ($h^{\prime}_{ab}=h^{\prime}_{ba}$). In terms of the component
fields, the interaction Lagrangian is given by

$\displaystyle{\cal L}_{Y}$ $\displaystyle=$ $\displaystyle
2\left[f_{e\mu}(\bar{\nu_{e}^{c}}\mu_{L}-\bar{\nu_{\mu}^{c}}e_{L})+f_{e\tau}(\bar{\nu_{e}^{c}}\tau_{L}-\bar{\nu_{\tau}^{c}}e_{L})+f_{\mu\tau}(\bar{\nu_{\mu}^{c}}\tau_{L}-\bar{\nu_{\tau}^{c}}\mu_{L})\right]h^{+}$
$\displaystyle+$
$\displaystyle\left[h_{ee}\bar{e^{c}}e_{R}+h_{\mu\mu}\bar{\mu^{c}}\mu_{R}+h_{\tau\tau}\bar{\tau^{c}}\tau_{R}+h_{e\mu}\bar{e^{c}}\mu_{R}+h_{e\tau}\bar{e^{c}}\tau_{R}+h_{\mu\tau}\bar{\mu^{c}}\tau_{R}\right]k^{++}$
$\displaystyle+$ $\displaystyle H.c.$

where we have used $h_{aa}=h^{\prime}_{aa},h_{ab}=2h^{\prime}_{ab}$ for $a\neq
b$. In Eq. (1), the lepton number is conserved, and neutrino mass will be
generated due to the Higgs potential given by:

$V(\phi,h^{+},k^{++})=\mu(h^{-}h^{-}k^{++}+h^{+}h^{+}k^{--})+\cdot\cdot\cdot.$
(3)

Here, the lepton number is violated by two units, hence one expects the
Majorana neutrino masses. From Eq.(1), it follows both $h^{-}$ and $k^{--}$
carry lepton number two, so the coefficient $\mu$ in (3) also carries lepton
number two. Therefore it is expected that the Majorana neutrino masses are
generated by loop quantum effects. At the two-loop level, the mass matrix for
Majorana neutrinos is given by

$M_{ab}=8\mu f_{ac}h^{*}_{cd}m_{c}m_{d}I_{cd}(f^{+})_{db},$ (4)

where $I_{cd}$ has the form tinhI

$\displaystyle I_{cd}$ $\displaystyle=$
$\displaystyle\int\frac{d^{4}k}{(2\pi)^{4}}\int\frac{d^{4}q}{(2\pi)^{4}}\frac{1}{k^{2}-m^{2}_{c}}\frac{1}{k^{2}-M^{2}_{h}}\frac{1}{q^{2}-m^{2}_{d}}$
(5) $\displaystyle\times$
$\displaystyle\frac{1}{q^{2}-M^{2}_{h}}\frac{1}{(k-q)^{2}-M^{2}_{k}}.$

Assuming masses of new Higgses are much larger than lepton ones, we can
evaluate $I_{cd}$ as follows

$I_{cd}\simeq
I=\frac{1}{(16\pi^{2})^{2}}\frac{1}{M^{2}}\frac{\pi^{2}}{3}\tilde{I}(r),\,\,M\equiv\textrm{max}(M_{k},M_{h}).$
(6)

Here $\tilde{I}(r)$ is a function of the ratio of the masses of the charged
Higgses $r\equiv M^{2}_{k}/M^{2}_{h}$,

$\tilde{I}(r)=\left\\{\begin{array}[]{ll}\hbox{$1+\frac{3}{\pi^{2}}(\log^{2}r-1)$}&\mbox{for}\,\,r\gg
1\\\ \hbox{$1$}&\mbox{for}\,\,r\rightarrow 0,\\\ \end{array}\right.$ (7)

which is close to $1$ for a wide range of scalar masses.

The neutrino mass matrix in (4) is symmetric and given by babum

$\displaystyle{\cal M}_{\nu}=-I\mu f_{\mu\tau}^{2}\times$
$\displaystyle\left(\begin{array}[]{ccc}\epsilon^{2}\omega_{\tau\tau}+2\epsilon\epsilon^{\prime}\omega_{\mu\tau}+\epsilon^{\prime
2}\omega_{\mu\mu}&\epsilon\omega_{\tau\tau}+\epsilon^{\prime}(\omega_{\mu\tau}-\epsilon\omega_{e\tau}-\epsilon^{\prime}\omega_{e\mu})&\,\,\,-\epsilon^{\prime}\omega_{\mu\mu}-\epsilon(\omega_{\mu\tau}+\epsilon\omega_{e\tau}+\epsilon^{\prime}\omega_{e\mu})\\\
\star&\omega_{\tau\tau}+\epsilon^{\prime
2}\omega_{ee}-2\epsilon^{\prime}\omega_{e\tau}&\epsilon\epsilon^{\prime}\omega_{ee}-\omega_{\mu\tau}-\epsilon\omega_{e\tau}+\epsilon^{\prime}\omega_{e\mu}\\\
\star&\star&\omega_{\mu\mu}+2\epsilon\omega_{e\mu}+\epsilon^{2}\omega_{ee}\\\
\end{array}\right)$ (11) (12)

where we have redefined parameters:

$\epsilon\equiv\frac{f_{e\tau}}{f_{\mu\tau}},\hskip
8.5359pt\epsilon^{\prime}\equiv\frac{f_{e\mu}}{f_{\mu\tau}}\hskip
8.5359pt\omega_{ab}\equiv m_{a}h^{*}_{ab}m_{b}.$ (13)

Let us denote ageng ; lvZB

$\displaystyle\omega_{\tau\tau}^{{}^{\prime}}$ $\displaystyle\equiv$
$\displaystyle\omega_{\tau\tau}+\epsilon^{\prime
2}\omega_{ee}-2\epsilon^{\prime}\omega_{e\tau},$
$\displaystyle\omega_{\mu\tau}^{{}^{\prime}}$ $\displaystyle\equiv$
$\displaystyle\omega_{\mu\tau}+\epsilon\omega_{e\tau}-\epsilon^{\prime}\omega_{e\mu}-\epsilon\epsilon^{\prime}\omega_{ee},$
$\displaystyle\omega_{\mu\mu}^{{}^{\prime}}$ $\displaystyle\equiv$
$\displaystyle\omega_{\mu\mu}+2\epsilon\omega_{e\mu}+\epsilon^{2}\omega_{ee},$

then the neutrino mass matrix can be rewritten in the compact form

$\displaystyle{\cal M}_{\nu}=-I\mu
f_{\mu\tau}^{2}\left(\begin{array}[]{ccc}\epsilon^{2}\omega_{\tau\tau}^{\prime}+2\epsilon\epsilon^{\prime}\omega_{\mu\tau}^{\prime}+\epsilon^{\prime
2}\omega_{\mu\mu}^{\prime}&\epsilon\omega^{\prime}_{\tau\tau}+\epsilon^{\prime}\omega^{\prime}_{\mu\tau}&-\epsilon\omega_{\mu\tau}^{\prime}-\epsilon^{\prime}\omega_{\mu\mu}^{\prime}\\\
\star&\omega_{\tau\tau}^{\prime}&-\omega_{\mu\tau}^{\prime}\\\
\star&\star&\omega^{\prime}_{\mu\mu}\\\ \end{array}\right).$ (17)

Next we turn to solution and implication to current neutrino data with a
rather large $\theta_{13}$.

##  III Solution and phenomenology

To begin this section, let us present the newest data on neutrino mass and
mixing. The best fit values of neutrino mass squared differences and the
leptonic mixing angles in Nuexp1 ; Nuexp2  have been given to be slightly
deviation from Tri-bimaximal mixing form, as shown in Tables 1 and 2 with a
rather large $\theta_{13}$.

Table 1:  The experimental values of neutrino mass squared splittings and leptonic mixing parameters, taken from Nuexp1 ; Nuexp2  for normal hierarchy. Parameter | Best fit |  $1\sigma$ range |  $2\sigma$ range
---|---|---|---
$\Delta m_{21}^{2}$($10^{-5}$eV2) | $7.62$ | $7.43-7.81$ | $7.27-8.01$
$\Delta m_{31}^{2}$($10^{-3}$eV2) | $2.55$ | $2.64-2.61$ | $2.38-2.68$
$\sin^{2}\theta_{12}$ | $0.320$ | $0.303-0.336$ | $0.29-0.35$
$\sin^{2}\theta_{23}$ | $0.613$ | $0.573-0.635$ | $0.38-0.66$
$\sin^{2}\theta_{13}$ | $0.0246$ | $0.0218-0.0275$ | $0.019-0.03$
Table 2: The experimental values of neutrino mass squared splittings and leptonic mixing parameters, taken from Nuexp1 ; Nuexp2  for inverted hierarchy. Parameter | Best fit |  $1\sigma$ range |  $2\sigma$ range
---|---|---|---
$\Delta m_{21}^{2}$($10^{-5}$eV2) | $7.62$ | $7.43-7.81$ | $7.27-8.01$
$\Delta m_{13}^{2}$($10^{-3}$eV2) | $2.43$ | $2.37-2.50$ | $2.29-2.58$
$\sin^{2}\theta_{12}$ | $0.32$ | $0.303-0.336$ | $0.29-0.35$
$\sin^{2}\theta_{23}$ | $0.60$ | $0.569-0.626$ | $0.39-0.65$
$\sin^{2}\theta_{13}$ | $0.025$ | $0.0223-0.0276$ | $0.02-0.03$

These large neutrino mixing angles are completely different from the quark
mixing ones defined by the Cabibbo- Kobayashi-Maskawa (CKM) matrix pmns1 ;
pmns2 ; pmns3 . This has stimulated works on flavor symmetries and non-Abelian
discrete symmetries, which are considered in the models based on the non-
Abelian discrete symmetries dlshA4 ; dlsvS4 ; dlnvS3 ; vlD4 ; vlS4 ; vlS3 ;
vlT7 ; vD4 ; vT7  (and the references therein).

The matrix ${\cal M}_{\nu}$ in (17) has three exact eigenvalues given by

$\displaystyle\lambda_{1}$ $\displaystyle=$ $\displaystyle 0,$
$\displaystyle\lambda_{2,3}$ $\displaystyle=$
$\displaystyle\frac{1}{2}\left(-kF\pm\sqrt{k^{2}\left[F^{2}+4(1+\epsilon^{2}+\epsilon^{\prime
2})(\omega^{\prime
2}_{\mu\tau}-\omega^{\prime}_{\mu\mu}\omega^{\prime}_{\tau\tau})\right]}\right),$
(18)

where we have denoted

$k=\mu If^{2}_{\mu\tau},\hskip 8.5359ptF=(1+\epsilon^{\prime
2})\omega^{\prime}_{\mu\mu}+2\epsilon\epsilon^{\prime}\omega^{\prime}_{\mu\tau}+(1+\epsilon^{2})\omega^{\prime}_{\tau\tau}.$
(19)

The massless eigenstate is given by

$\displaystyle\nu_{1}$ $\displaystyle=$
$\displaystyle\frac{1}{\sqrt{f^{2}_{e\mu}+f^{2}_{e\tau}+f^{2}_{\mu\tau}}}(f_{\mu\tau}\nu_{e}-f_{e\tau}\nu_{\mu}+f_{e\mu}\nu_{\tau}).$

Until now values of neutrino masses (or the absolute neutrino masses) as well
as the mass ordering of neutrinos are unknown. An upper bound on the absolute
value of neutrino mass was found from the analysis of the cosmological data
Tegmark

$m_{i}\leq 0.6\,\mathrm{eV},$ (21)

while arguments from the growth of large-scale structure in the early Universe
yield the upper bound on the sum of neutrino mass Weiler

$\sum^{3}_{i=1}m_{i}\leq 0.5\,\mathrm{eV}.$ (22)

The neutrino mass spectrum can be the normal hierarchy
($|m_{1}|\simeq|m_{2}|<|m_{3}|$), the inverted hierarchy
($|m_{3}|<|m_{1}|\simeq|m_{2}|$) or nearly degenerate
($|m_{1}|\simeq|m_{2}|\simeq|m_{3}|$). The mass ordering of neutrino depends
on the sign of $\Delta m^{2}_{23}$ which is currently unknown. In the case of
3-neutrino mixing, in the model under consideration, the two possible signs of
$\Delta m^{2}_{23}$ corresponding to two types of neutrino mass spectrum can
be provided as follows

  1. 1.

Normal hierarchy (NH): $m_{1}=0,\,m_{2}=\sqrt{\Delta
m^{2}_{21}},\,m_{3}=\sqrt{\Delta m^{2}_{31}}$ with $m_{2}\ll m_{3}$.

  2. 2.

Inverted hierarchy (IH): $m_{3}=0,m_{1}=\sqrt{\Delta
m^{2}_{13}},\,m_{2}=\sqrt{m^{2}_{1}-\Delta m^{2}_{21}}$ with $m_{1}\simeq
m_{2}$.

As we discuss below the neutrino mass matrix in (17) is able to provide both
mass schemes

###  III.1 Normal hierarchy ($\Delta m^{2}_{23}>0$)

In this case, three neutrino masses are

$m_{1}=\lambda_{1}=0,\,\,m_{2}=\lambda_{2},\,\,m_{3}=\lambda_{3},$ (23)

with $\lambda_{i}\,\,\,(i=1,2,3)$ is defined in (18), and the corresponding
eigenstates put in the neutrino mixing matrix:

$\displaystyle U_{\nu N}$ $\displaystyle=$
$\displaystyle\left(\begin{array}[]{ccc}\,\,\,\frac{1}{\sqrt{1+\epsilon^{2}+\epsilon^{\prime
2}}}&-\frac{A_{1}}{\sqrt{1+A^{2}_{1}+B^{2}_{1}}}&\frac{A_{2}}{\sqrt{1+A^{2}_{2}+B^{2}_{2}}}\\\
-\frac{\epsilon}{\sqrt{1+\epsilon^{2}+\epsilon^{\prime
2}}}&-\frac{B_{1}}{\sqrt{1+A^{2}_{1}+B^{2}_{1}}}&\frac{B_{2}}{\sqrt{1+A^{2}_{2}+B^{2}_{2}}}\\\
\,\,\,\frac{\epsilon^{\prime}}{\sqrt{1+\epsilon^{2}+\epsilon^{\prime
2}}}&-\frac{1}{\sqrt{1+A^{2}_{1}+B^{2}_{1}}}&\frac{1}{\sqrt{1+A^{2}_{2}+B^{2}_{2}}}\end{array}\right)$
(27)

where lvZB

$\displaystyle A_{1,2}$ $\displaystyle=$
$\displaystyle\frac{-k\left[\epsilon(\epsilon^{\prime
2}-1)\omega^{\prime}_{\mu\mu}+2\epsilon^{\prime}(1+\epsilon^{2})\omega^{\prime}_{\mu\tau}+\epsilon(1+\epsilon^{2})\omega^{\prime}_{\tau\tau}\right]\pm\epsilon\sqrt{k^{2}F^{\prime}}}{2k\left[\epsilon\epsilon^{\prime}\omega^{\prime}_{\mu\mu}+(1+\epsilon^{2})\omega^{\prime}_{\mu\tau}\right]},$
(28) $\displaystyle B_{1,2}$ $\displaystyle\equiv$
$\displaystyle\frac{k(1+\epsilon^{\prime
2})\omega^{\prime}_{\mu\mu}-k(1+\epsilon^{2})\omega^{\prime}_{\tau\tau}\pm\sqrt{k^{2}F^{\prime}}}{2k\left[\epsilon\epsilon^{\prime}\omega^{\prime}_{\mu\mu}+(1+\epsilon^{2})\omega^{\prime}_{\mu\tau}\right]},$
(29)

and

$F^{\prime}=F^{2}+4(1+\epsilon^{2}+\epsilon^{\prime 2})(\omega^{\prime
2}_{\mu\tau}-\omega^{\prime}_{\mu\mu}\omega^{\prime}_{\tau\tau}).$ (30)

The eigenstates $\nu_{i}$ corresponding to the eigenvalues $m_{i}\,(i=1,2,3)$
are found to be

$\displaystyle\nu_{1}$ $\displaystyle=$
$\displaystyle\frac{1}{\sqrt{f^{2}_{e\mu}+f^{2}_{e\tau}+f^{2}_{\mu\tau}}}(f_{\mu\tau}\nu_{e}-f_{e\tau}\nu_{\mu}+f_{e\mu}\nu_{\tau}),$
$\displaystyle\nu_{2}$ $\displaystyle=$
$\displaystyle-\frac{A_{1}}{\sqrt{1+A^{2}_{1}+B^{2}_{1}}}\nu_{e}-\frac{B_{1}}{\sqrt{1+A^{2}_{1}+B^{2}_{1}}}\nu_{\mu}-\frac{1}{\sqrt{1+A^{2}_{1}+B^{2}_{1}}}\nu_{\tau},$
$\displaystyle\nu_{3}$ $\displaystyle=$
$\displaystyle\frac{A_{2}}{\sqrt{1+A^{2}_{2}+B^{2}_{2}}}\nu_{e}+\frac{B_{2}}{\sqrt{1+A^{2}_{2}+B^{2}_{2}}}\nu_{\mu}+\frac{1}{\sqrt{1+A^{2}_{2}+B^{2}_{2}}}\nu_{\tau}.$
(31)

From the explicit expressions of $A_{1,2}$ and $B_{1,2}$ in (28) and (29),
some useful relations are in order lvZB

$\displaystyle A_{1}A_{2}+B_{1}B_{2}+1$ $\displaystyle=$ $\displaystyle 0,$
$\displaystyle A_{1}-\epsilon B_{1}+\epsilon^{\prime}$ $\displaystyle=$
$\displaystyle 0,$ $\displaystyle A_{2}-\epsilon B_{2}+\epsilon^{\prime}$
$\displaystyle=$ $\displaystyle 0,$ $\displaystyle(A_{1}-A_{2})/(B_{1}-B_{2})$
$\displaystyle=$ $\displaystyle\epsilon.$ (32)

One also has

$\displaystyle A_{1}A_{2}$ $\displaystyle=$
$\displaystyle\frac{(\epsilon^{\prime
2}-\epsilon^{2})\omega^{\prime}_{\mu\tau}+\epsilon\epsilon^{\prime}(\omega^{\prime}_{\tau\tau}-\omega^{\prime}_{\mu\mu})}{\epsilon\epsilon^{\prime}\omega^{\prime}_{\mu\mu}+(1+\epsilon^{2})\omega^{\prime}_{\mu\tau}},$
$\displaystyle B_{1}B_{2}$ $\displaystyle=$
$\displaystyle-\frac{(1+\epsilon^{\prime
2})\omega^{\prime}_{\mu\tau}+\epsilon\epsilon^{\prime}\omega^{\prime}_{\tau\tau}}{\epsilon\epsilon^{\prime}\omega^{\prime}_{\mu\mu}+(1+\epsilon^{2})\omega^{\prime}_{\mu\tau}}.$

In the standard Particle Data Group (PDG) parametrization, the neutrino mixing
matrix ($U_{PMNS}$) can be parametrized as PDG2012

$U_{PMNS}=\left(\begin{array}[]{ccc}c_{12}c_{13}&s_{12}c_{13}&s_{13}e^{-i\delta}\\\
-s_{12}c_{23}-c_{12}s_{23}s_{13}e^{i\delta}&c_{12}c_{23}-s_{12}s_{23}s_{13}e^{i\delta}&s_{23}c_{13}\\\
s_{12}s_{23}-c_{12}c_{23}s_{13}e^{i\delta}&-c_{12}s_{23}-s_{12}c_{23}s_{13}e^{i\delta}&c_{23}c_{13}\\\
\end{array}\right)\times P,$ (33)

where
$P=\mathrm{diag}\left(1,e^{i\frac{\alpha_{21}}{2}},e^{i\frac{\alpha_{31}}{2}}\right)$,
and $c_{ij}=\cos\theta_{ij}$, $s_{ij}=\sin\theta_{ij}$ with $\theta_{12}$,
$\theta_{23}$ and $\theta_{13}$ being the solar, atmospheric and the reactor
angles, respectively, the angles $\theta_{ij}=[0,\frac{\pi}{2}]$.
$\delta=[0,2\pi]$ is the Dirac CP violation phase and
$\alpha_{21},\alpha_{31}$ are two Majorana CP violation phases. It is to be
mentioned that in our previous work lvZB , the Majorana has been included in a
simple form.

By comparing Eqs. (27) and (33), all the parameters in the lepton mixing
matrix in (27) can be parameterized in terms of three Euler’s angles
$\theta_{ij}$ as follows:

$\displaystyle A_{2}$ $\displaystyle=$
$\displaystyle\frac{c_{12}s^{2}_{13}\sqrt{1+\epsilon^{2}+\epsilon^{\prime
2}}}{c_{13}\left(s_{12}s_{23}\sqrt{1+\epsilon^{2}+\epsilon^{\prime
2}}-\epsilon^{\prime}\right)},\hskip 8.5359ptB_{2}=\frac{s_{23}}{c_{23}},$
(34) $\displaystyle A_{1}$ $\displaystyle=$
$\displaystyle\frac{A_{2}B_{1}c^{2}_{13}c_{23}s_{12}}{A_{2}c_{12}c_{13}c^{2}_{23}-s_{12}s_{23}s^{2}_{13}},\hskip
8.5359ptB_{1}=\frac{-A_{2}.c_{12}c_{13}c^{2}_{23}+s_{12}s_{23}s^{2}_{13}}{c_{23}(s_{12}s^{2}_{13}+A_{2}.c_{12}c_{13}s_{23})},$
(35) $\displaystyle e^{i\frac{\alpha_{21}}{2}}$ $\displaystyle=$
$\displaystyle-\frac{1}{s_{12}c_{13}}\frac{A_{1}}{\sqrt{1+A^{2}_{1}+B^{2}_{1}}},$
$\displaystyle e^{i\frac{\alpha_{31}}{2}}$ $\displaystyle=$
$\displaystyle\frac{1}{c_{13}c_{23}}\frac{1}{\sqrt{1+A^{2}_{2}+B^{2}_{2}}},$
$\displaystyle e^{i\delta}$ $\displaystyle=$
$\displaystyle\frac{s_{13}}{c_{13}c_{23}A_{2}},$ (36)

and two solutions with $\epsilon,\epsilon^{\prime}$:

$\displaystyle\epsilon$ $\displaystyle=$
$\displaystyle\frac{s_{12}c_{23}-c_{12}s_{13}s_{23}}{c_{12}c_{13}}\equiv\epsilon_{1},\hskip
8.5359pt\epsilon^{\prime}=\frac{s_{12}s_{23}+c_{12}s_{13}c_{23}}{c_{12}c_{13}}\equiv\epsilon^{\prime}_{1},$
(37)

or

$\displaystyle\epsilon$ $\displaystyle=$
$\displaystyle\frac{s_{12}c_{23}+c_{12}s_{13}s_{23}}{c_{12}c_{13}}\equiv\epsilon_{2},\hskip
8.5359pt\epsilon^{\prime}=\frac{s_{12}s_{23}-c_{12}s_{13}c_{23}}{c_{12}c_{13}}\equiv\epsilon^{\prime}_{2}.$
(38)

We are going to consider in detail the first case.

####  III.1.1  The solution with $\epsilon_{1},\epsilon^{\prime}_{1}$

With $\epsilon=\epsilon_{1},\epsilon^{\prime}=\epsilon^{\prime}_{1}$ given in
Eq. (37) the parameters in Eqs. from (34) to (36) become:

$\displaystyle A_{1}$ $\displaystyle=$
$\displaystyle-\frac{s_{12}c_{13}}{c_{12}s_{23}-s_{12}s_{13}c_{23}},\hskip
8.5359ptA_{2}=-\frac{s_{13}}{c_{13}c_{23}},$ $\displaystyle B_{1}$
$\displaystyle=$
$\displaystyle\frac{s_{12}s_{13}s_{23}+c_{12}c_{23}}{s_{12}s_{13}c_{23}-c_{12}s_{23}},\hskip
8.5359ptB_{2}=\frac{s_{23}}{c_{23}},$ $\displaystyle
e^{i\frac{\alpha_{21}}{2}}$ $\displaystyle=$ $\displaystyle 1,\hskip
8.5359pte^{i\frac{\alpha_{31}}{2}}=1,\hskip 8.5359pte^{i\delta}=-1.$ (39)

Taking the data in Refs.Nuexp1 ; Nuexp2  given in Tab. 1, we obtain

$\displaystyle A_{1}$ $\displaystyle=$ $\displaystyle-0.946222,\hskip
8.5359ptA_{2}=-0.255282,\hskip 8.5359ptB_{1}=-0.986486,\hskip
8.5359ptB_{2}=1.25856,$ (40) $\displaystyle\epsilon$ $\displaystyle=$
$\displaystyle 0.307762,\hskip 8.5359pt\epsilon^{\prime}=0.64262,$ (41)

Eq. (39) implies $\alpha_{21}=\alpha_{31}=0$, $\delta=\pi$. The neutrino
mixing matrix then takes the form:

$U_{\nu N}=\left(\begin{array}[]{ccc}0.814415&0.558684&-0.156844\\\
-0.250646&0.582457&0.773253\\\ 0.523359&-0.590437&0.614394\\\
\end{array}\right).$ (42)

Taking the best fit values of the mass squared differences as displayed in
Tab.1, we obtain

$m_{2}=8.72926\times 10^{-3}\,\mathrm{eV},\,m_{3}=5.04975\times
10^{-2}\,\mathrm{eV},$ (43)

and the effective masses $\langle m_{ee}\rangle,m_{\beta}$ governing
neutrinoless double beta decay betdecay1 ; betdecay2 ; betdecay3 ; betdecay4 ;
betdecay5  as well as the sum of the neutrino masses are given by:

$\displaystyle\langle
m_{ee}\rangle=\mid\sum^{3}_{i=1}U_{ei}^{2}m_{i}\mid=0.00396688\,\mathrm{eV},$
(44) $\displaystyle m_{\beta}=\sum_{i=1}^{3}\mid
U_{ei}\mid^{2}m_{i}^{2}=0.00930129\,\mathrm{eV},$ (45) $\displaystyle
m_{1}+m_{2}+m_{3}=0.0592268\,\mathrm{eV}.$ (46)

Now, substituting $\epsilon,\epsilon^{\prime}$ from (41) into (18) and (23)
yields:

$\displaystyle m_{1}$ $\displaystyle=$ $\displaystyle
0,\,\,m_{2,3}=-\frac{k}{2}\left(1.41296\omega^{\prime}_{\mu\mu}+0.395548\omega^{\prime}_{\mu\tau}+1.09472\omega^{\prime}_{\tau\tau}\mp\sqrt{V_{1}}\right),$
(47)

where

$\displaystyle V_{1}$ $\displaystyle=$ $\displaystyle 1.99646\omega^{\prime
2}_{\mu\mu}+1.11779\omega^{\prime}_{\mu\mu}\omega^{\prime}_{\mu\tau}+6.18717\omega^{\prime
2}_{\mu\tau}$ (48) $\displaystyle-$ $\displaystyle
2.93713\omega^{\prime}_{\mu\mu}\omega^{\prime}_{\tau\tau}+0.866027\omega^{\prime}_{\mu\tau}\omega^{\prime}_{\tau\tau}+1.19841\omega^{\prime
2}_{\tau\tau}.$

Without loss of generality, we assume
$\omega^{\prime}_{\mu\mu}=\omega^{\prime}_{\tau\tau}=\omega^{\prime}$ lvZB .
Comparing (43) and (47) we have two solutions:

$\omega^{\prime}_{\mu\tau}=-0.771825\omega^{\prime},\,\,k=-\frac{0.0268921}{\omega^{\prime}},$
(49)

or

$\omega^{\prime}_{\mu\tau}=0.608604\omega^{\prime},\,\,k=-\frac{0.0215495}{\omega^{\prime}}.$
(50)

Now we consider the second case.

####  III.1.2  The solution with $\epsilon_{2},\epsilon^{\prime}_{2}$

With $\epsilon=\epsilon_{2},\epsilon^{\prime}=\epsilon^{\prime}_{2}$ given in
Eq. (38) the parameters in Eqs. from (34) to (36) become:

$\displaystyle A_{1}$ $\displaystyle=$
$\displaystyle-\frac{s_{12}c_{13}}{c_{12}s_{23}+s_{12}s_{13}c_{23}},\hskip
8.5359ptA_{2}=\frac{s_{13}}{c_{13}c_{23}},$ $\displaystyle B_{1}$
$\displaystyle=$
$\displaystyle\frac{s_{12}s_{13}s_{23}-c_{12}c_{23}}{s_{12}s_{13}c_{23}+c_{12}s_{23}},\hskip
8.5359ptB_{2}=\frac{s_{23}}{c_{23}},$ $\displaystyle
e^{i\frac{\alpha_{21}}{2}}$ $\displaystyle=$ $\displaystyle 1,\hskip
8.5359pte^{i\frac{\alpha_{31}}{2}}=1,\hskip 8.5359pte^{i\delta}=1.$ (51)

Taking the data in Refs.Nuexp1 ; Nuexp2  given in Tab. 1, we obtain

$\displaystyle A_{1}$ $\displaystyle=$ $\displaystyle-0.797179,\hskip
8.5359ptA_{2}=0.255282,\hskip 8.5359ptB_{1}=-0.63286,\hskip
8.5359ptB_{2}=1.25856,$ (52) $\displaystyle\epsilon$ $\displaystyle=$
$\displaystyle 0.556439,\hskip 8.5359pt\epsilon^{\prime}=0.445031.$ (53)

Eq. (70) implies $\alpha_{21}=\alpha_{31}=\delta=0$, i.e, the Dirac and
Majorana CP violation phases are equal to zero.

The neutrino mixing matrix then takes the form:

$U_{\nu N}=\left(\begin{array}[]{ccc}0.814415&0.558684&0.156844\\\
-0.453172&0.443525&0.773253\\\ 0.36244&-0.700826&0.614394\\\
\end{array}\right).$ (54)

In this case, the neutrino masses are given in (43), and the effective masses
$\langle m_{ee}\rangle,m_{\beta}$ governing neutrinoless double beta decay as
well as the sum of the neutrino masses are given by (44), (45) and (46).

Substituting $\epsilon,\epsilon^{\prime}$ from (41) into (18), (23) yields:

$\displaystyle m_{1}$ $\displaystyle=$ $\displaystyle
0,\,\,m_{2,3}=-\frac{k}{2}\left(1.19805\omega^{\prime}_{\mu\mu}+0.495265\omega^{\prime}_{\mu\tau}+1.30962\omega^{\prime}_{\tau\tau}\mp\sqrt{V_{2}}\right),$
(55)

where

$\displaystyle V_{2}$ $\displaystyle=$ $\displaystyle 1.43533\omega^{\prime
2}_{\mu\mu}+1.18671\omega^{\prime}_{\mu\mu}\omega^{\prime}_{\mu\tau}+6.276\omega^{\prime
2}_{\mu\tau}$ (56) $\displaystyle-$ $\displaystyle
2.89271\omega^{\prime}_{\mu\mu}\omega^{\prime}_{\tau\tau}+1.29722\omega^{\prime}_{\mu\tau}\omega^{\prime}_{\tau\tau}+1.71512\omega^{\prime
2}_{\tau\tau}.$

In similarity, we assume again
$\omega^{\prime}_{\mu\mu}=\omega^{\prime}_{\tau\tau}=\omega^{\prime}$ lvZB ,
and by comparing (43) and (55) we have two solutions:

$\displaystyle\omega^{\prime}_{\mu\tau}=-0.671538\omega^{\prime},\,\,k=-\frac{0.00518566}{\omega^{\prime}},$
(57)

or

$\displaystyle\omega^{\prime}_{\mu\tau}=0.391254\omega^{\prime},\,\,k=-\frac{0.00417526}{\omega^{\prime}}.$
(58)

To finish this section, we note that in the normal scheme, the Majorana
violation phases are equal to the zero.

###  III.2 Inverted hierarchy ($\Delta m^{2}_{23}<0$)

In this case, three neutrino masses are

$m_{1}=\lambda_{3},\,\,m_{2}=\lambda_{2},\,\,m_{3}=0,$ (59)

with $\lambda_{i}\,\,\,(i=1,2,3)$ is defined in (18), and the corresponding
eigenstates put in the neutrino mixing matrix:

$\displaystyle U_{\nu I}$ $\displaystyle=$
$\displaystyle\left(\begin{array}[]{ccc}\frac{A_{2}}{\sqrt{1+A^{2}_{2}+B^{2}_{2}}}&-\frac{A_{1}}{\sqrt{1+A^{2}_{1}+B^{2}_{1}}}&\frac{1}{\sqrt{1+\epsilon^{2}+\epsilon^{\prime
2}}}\\\
\frac{B_{2}}{\sqrt{1+A^{2}_{2}+B^{2}_{2}}}&-\frac{B_{1}}{\sqrt{1+A^{2}_{1}+B^{2}_{1}}}&-\frac{\epsilon}{\sqrt{1+\epsilon^{2}+\epsilon^{\prime
2}}}\\\
\frac{1}{\sqrt{1+A^{2}_{2}+B^{2}_{2}}}&-\frac{1}{\sqrt{1+A^{2}_{1}+B^{2}_{1}}}&\frac{\epsilon^{\prime}}{\sqrt{1+\epsilon^{2}+\epsilon^{\prime
2}}}\end{array}\right)$ (63)

where $A_{1,2}$ and $B_{1,2}$ are given in (28) and (29), respectively.

The eigenstates $\nu_{i}$ corresponding to the eigenvalues $m_{i}\,(i=1,2,3)$
are found to be

$\displaystyle\nu_{1}$ $\displaystyle=$
$\displaystyle\frac{A_{2}}{\sqrt{1+A^{2}_{2}+B^{2}_{2}}}\nu_{e}+\frac{B_{2}}{\sqrt{1+A^{2}_{2}+B^{2}_{2}}}\nu_{\mu}+\frac{1}{\sqrt{1+A^{2}_{2}+B^{2}_{2}}}\nu_{\tau},$
$\displaystyle\nu_{2}$ $\displaystyle=$
$\displaystyle-\frac{A_{1}}{\sqrt{1+A^{2}_{1}+B^{2}_{1}}}\nu_{e}-\frac{B_{1}}{\sqrt{1+A^{2}_{1}+B^{2}_{1}}}\nu_{\mu}-\frac{1}{\sqrt{1+A^{2}_{1}+B^{2}_{1}}}\nu_{\tau},$
$\displaystyle\nu_{3}$ $\displaystyle=$
$\displaystyle\frac{1}{\sqrt{f^{2}_{e\mu}+f^{2}_{e\tau}+f^{2}_{\mu\tau}}}(f_{\mu\tau}\nu_{e}-f_{e\tau}\nu_{\mu}+f_{e\mu}\nu_{\tau}).$
(64)

Similar to the normal case, comparing Eqs. (63) and (33) yields:

$\displaystyle\epsilon$ $\displaystyle=$
$\displaystyle\frac{c_{13}\left(\frac{B_{2}}{\sqrt{1+A^{2}_{2}+B^{2}_{2}}}+s_{12}c_{23}\right)}{c_{12}s^{2}_{13}},\,\,\epsilon^{\prime}=-\frac{c_{23}}{s_{23}}.\epsilon,$
(65) $\displaystyle e^{i\frac{\alpha_{21}}{2}}$ $\displaystyle=$
$\displaystyle-\frac{1}{s_{12}c_{13}}\frac{A_{1}}{\sqrt{1+A^{2}_{1}+B^{2}_{1}}},$
$\displaystyle e^{i\frac{\alpha_{31}}{2}}$ $\displaystyle=$
$\displaystyle-\frac{1}{c_{13}s_{23}}\frac{\epsilon}{\sqrt{1+\epsilon^{2}+\epsilon^{\prime
2}}},$ $\displaystyle e^{i\delta}$ $\displaystyle=$
$\displaystyle-\frac{s_{13}}{c_{13}s_{23}}\epsilon,$

and two solutions with $A_{2},B_{2}$:

$\displaystyle A_{2}$ $\displaystyle=$
$\displaystyle\frac{c_{12}c_{13}}{s_{12}s_{23}+c_{12}s_{13}c_{23}}\equiv
A^{+}_{2},$ $\displaystyle B_{2}$ $\displaystyle=$
$\displaystyle\frac{c_{12}s_{12}c_{13}+s_{23}c_{23}(c^{2}_{12}c^{2}_{13}-1)}{s^{2}_{12}+c^{2}_{23}(c^{2}_{12}c^{2}_{13}-1)}\equiv
B^{+}_{2},$ (66)

or

$\displaystyle A_{2}$ $\displaystyle=$
$\displaystyle\frac{c_{12}s_{12}c_{13}}{s^{2}_{12}s_{23}-c_{12}s_{12}s_{13}c_{23}}\equiv
A^{-}_{2},$ $\displaystyle B_{2}$ $\displaystyle=$
$\displaystyle\frac{-c_{12}s_{12}c_{13}+s_{23}c_{23}(c^{2}_{12}c^{2}_{13}-1)}{s^{2}_{12}+c^{2}_{23}(c^{2}_{12}c^{2}_{13}-1)}\equiv
B^{-}_{2}.$ (67)

Let us consider both cases.

####  III.2.1  The solution with $A^{+}_{2},B^{+}_{2}$

As is easily shown that, in this case, the model is consistent because the
five experimental constraints on the mixing angles and squared mass
differences of neutrinos can be respectively fitted with all parameters of the
model. Indeed, with $A_{2}=A^{+}_{2},B_{2}=B^{+}_{2}$ given in Eq. (66),
taking the data in Refs.Nuexp1 ; Nuexp2  given in Tab. 2, we obtain

$\displaystyle A_{1}$ $\displaystyle=$ $\displaystyle-0.959444,\hskip
8.5359ptA_{2}=1.56394,\hskip 8.5359ptB_{1}=-1.01484,\hskip
8.5359ptB_{2}=-0.493193,$ (68) $\displaystyle\epsilon$ $\displaystyle=$
$\displaystyle 4.83735,\hskip 8.5359pt\epsilon^{\prime}=-3.94968,\hskip
8.5359pt\frac{\epsilon}{\epsilon^{\prime}}=-1.22474,$ (69) $\displaystyle
e^{i\frac{\alpha_{21}}{2}}$ $\displaystyle=$ $\displaystyle 1,\hskip
8.5359pte^{i\frac{\alpha_{31}}{2}}=-1,\hskip 8.5359pte^{i\delta}=-1.$ (70)

Eq. (70) implies $\alpha_{21}=0$, $\alpha_{31}=2\pi$, $\delta=\pi$. The
neutrino mixing matrix then takes the form:

$U_{\nu I}=\left(\begin{array}[]{ccc}0.814248&0.55857&0.158114\\\
-0.256776&0.590818&-0.764853\\\ 0.52064&-0.58218&-0.6245\\\
\end{array}\right).$ (71)

The physical neutrino masses are obtained as

$\displaystyle m_{1}$ $\displaystyle=$ $\displaystyle\sqrt{\Delta
m^{2}_{13}}=4.9295\times 10^{-2}\,\mathrm{eV},$ $\displaystyle m_{2}$
$\displaystyle=$ $\displaystyle\sqrt{m^{2}_{1}-\Delta m^{2}_{21}}=4.8516\times
10^{-2}\,\mathrm{eV},\,m_{3}=0,$ (72)

and the effective masses $\langle m_{ee}\rangle,m_{\beta}$ governing
neutrinoless double beta decay betdecay1 ; betdecay2 ; betdecay3 ; betdecay4 ;
betdecay5  as well as the sum of the neutrino masses are given by:

$\displaystyle\langle
m_{ee}\rangle=\mid\sum^{3}_{i=1}U_{ei}^{2}m_{i}\mid=0.0478196\,\mathrm{eV},$
(73) $\displaystyle m_{\beta}=\sum_{i=1}^{3}\mid
U_{ei}\mid^{2}m_{i}^{2}=0.0484301\,\mathrm{eV},$ (74) $\displaystyle
m_{1}+m_{2}+m_{3}=0.097811\,\mathrm{eV}.$ (75)

As before, we assume
$\omega^{\prime}_{\mu\mu}=\omega^{\prime}_{\tau\tau}=\omega^{\prime}$ lvZB .
Substituting $\epsilon,\epsilon^{\prime}$ in Eq. (69) into (18) and (59)
yields

$\omega^{\prime}_{\mu\tau}=1.07194\omega^{\prime},\,\,k=-\frac{0.0200278}{\omega^{\prime}},$
(76)

or

$\omega^{\prime}_{\mu\tau}=1.07399\omega^{\prime},\,\,k=-\frac{0.0197385}{\omega^{\prime}}.$
(77)

Eq. (70) shows that in this case one Majorana violation phase is nonzero. Our
next step is the second case.

####  III.2.2  The solution with $A^{-}_{2},B^{-}_{2}$

In this case, taking the data in Nuexp1 ; Nuexp2  given in Tab. 2, we obtain

$\displaystyle A_{1}=-0.80333,\hskip 8.5359ptA_{2}=2.28904,\hskip
8.5359ptB_{1}=-0.650428,\hskip 8.5359ptB_{2}=-1.2897,$ (78)
$\displaystyle\epsilon=-4.83735,\hskip
8.5359pt\epsilon^{\prime}=3.94968,\hskip
8.5359pt\frac{\epsilon}{\epsilon^{\prime}}=-1.22474,$ (79) $\displaystyle
e^{i\frac{\alpha_{21}}{2}}=e^{i\frac{\alpha_{31}}{2}}=e^{i\delta}=1.$ (80)

Eq. (80) implies $\alpha_{21}=\alpha_{31}=\delta=0$. Then, the neutrino mixing
matrix is

$U_{\nu I}=\left(\begin{array}[]{ccc}0.814248&0.55857&0.158114\\\
-0.458766&0.452254&0.764853\\\ 0.355716&-0.695317&0.6245\\\
\end{array}\right).$ (81)

Three neutrino masses are given in (75), and the effective masses $\langle
m_{ee}\rangle,m_{\beta}$ governing neutrinoless double beta decay as well as
the sum of the neutrino masses are given by (73), (74) and (75). The relation
between $\omega^{\prime}_{\mu\tau},k$ and $\omega^{\prime}$ are given in
Eqs.(76) and (77). Note that in this case, all Dirac and Majorana violation
phases vanish.

##  IV Conclusion

In this paper we have derived the exact eigenvalues and eigenstates of the
neutrino mass matrix in the Zee-Babu model in which the most recent data on
neutrino masses and mixing with large $\theta_{13}$ are updated, and with
focus on Majorana violation phases. For the normal hierarchy, the Majorana
phases ($\alpha_{21},\alpha_{31}$) are equal to zero, while for the inverted
pattern, one phase ($\alpha_{31}$) takes the value $2\pi$. The Dirac phase
($\delta$) is predicted to either $0$ or $\pi$. Taking into account of the
effective mass governing neutrinoless double beta decay and the sum of
neutrino, we have showed that the model fits well with the recent most
experimental data. Therefore we conclude that the Zee-Babu model is
fascinating one for neutrino physics.

## Acknowledgments

This research is funded by the Vietnam National Foundation for Science and
Technology Development (NAFOSTED) under grant number 103.01-2014.51.

## References

  * (1) A. Zee, Phys. Lett. B 93, 389 (1980).
  * (2) A. Zee, Nucl. Phys. B 264, 99 (1986).
  * (3) K. S. Babu, Phys. Lett. B 203, 132 (1988).
  * (4) Hoang Ngoc Long and Vo Van Vien, Int. J. Mod. Phys. A 29 (2014) 1450072, arXiv:1405.1622 [hep-ph].
  * (5) K. L. McDonald and B. H. J. McKellar , _Evaluating The Two Loop Diagram Responsible For Neutrino Mass In Babu’s Model_ , arXiv:0309270 [hep- ph] (2003).
  * (6) K. S. Babu, C. Macesanu, Phys. Rev. D 67 (2003) 073010, arXiv: 0212058 [hep-ph].
  * (7) T. Araki and C. Q. Geng, Phys. Lett. B 694, 113 (2010).
  * (8) D. V. Forero, M. Tortola and J. W. F. Valle, Phys. Rev. D 86, 073012 (2012), arXiv:1205.4018 [hep-ph].
  * (9) M. D. Campos et al., Phys. Rev. D 90, 016006, arXiv:1403.2525 [hep-ph].
  * (10) Z. Maki, M. Nakagawa and S. Sakata, Prog. Theor. Phys. 28, 870 (1962).
  * (11) B. Pontecorvo, Zh. Eksp. Teor. Fiz. 53, 1717 (1967).
  * (12) B. Pontecorvo, Sov. Phys. JETP 26, 984(1968).
  * (13) P. V. Dong, L. T. Hue, H. N. Long and D. V. Soa, _Phys. Rev. D_ 81, 053004 (2010), arXiv: 1001.4625 [hep-ph].
  * (14) P. V. Dong, H. N. Long, D. V. Soa, and V. V. Vien, _Eur. Phys. J. C_ 71, 1544 (2011), arXiv: 1009.2328 [hep-ph].
  * (15) P. V. Dong, H. N. Long, C. H. Nam, and V. V. Vien, _Phys. Rev. D_ 85, 053001 (2012), arXiv: 1111.6360 [hep-ph].
  * (16) V. V. Vien and H. N. Long, Int. J. Mod. Phys. A 28 (2013) 1350159, arXiv: 1312.5034 [hep-ph].
  * (17) V. V. Vien and H. N. Long, Adv. High Energy Phys. 2014, 192536 (2014).
  * (18) V. V. Vien and H. N. Long, J.Exp.Theor.Phys. 118 (2014) 869, arXiv:1404.6119 [hep-ph].
  * (19) V. V. Vien and H. N. Long, J.High Energy Phys. 04 (2014) 133, arXiv:1402.1256 [hep-ph].
  * (20) V. V. Vien, Mod. Phys. Lett. A 29, No. 23 (2014) 1450122.
  * (21) V. V. Vien, _$T_{7}$ flavor symmetry scheme for understanding neutrino mass and mixing in 3-3-1 model with neutral leptons_, to be published in Mod. Phys. Lett. A (2014).
  * (22) M. Tegmark et al, Phys. Rev. D 69 (2004) 103501.
  * (23) T. J. Weiler (2013), _Oscillation and Mixing Among the Three Neutrino Flavors_ , arXiv:1308.1715 [hep-ph].
  * (24) J. Beringer, _et al._(2012), _Review of Particle Physics_ (Particle Data Group), Phys. Rev. D. 86, 010001.
  * (25) W. Rodejohann, Int. J. Mod. Phys. E 20, 1833 (2011).
  * (26) M. Mitra, G. Senjanovic, F. Vissani, Nucl. Phys. B 856, 26 (2012);
  * (27) S. M. Bilenky, C. Giunti, Mod. Phys. Lett. A 27, 1230015 (2012),arXiv:1203.5250;
  * (28) W. Rodejohann, J. Phys. G 39 (2012) 124008, arXiv:1206.2560 [hep-ph].
  * (29) A. Merle, Int. J. Mod. Phys. D 22, 1330020 (2013).


# Hierarchical Aspect-guided Explanation Generation for Explainable
Recommendation

Yidan Hu, Yong Liu, Chunyan Miao, Gongqi Lin, Yuan Miao Yidan Hu and Chunyan
Miao are currently with School of Computer Science and Engineering, Nanyang
Technological University, Singapore 639798. Email<EMAIL_ADDRESS><EMAIL_ADDRESS>Yong Liu is with Joint NTU-UBC Research Centre of
Excellence in Active Living for the Elderly (LILY), and AlibabaNTU Singapore
Joint Research Institute, Nanyang Technological University, Singapore 639798.
Email<EMAIL_ADDRESS>Miao and Gongqi Lin are currently with IT
Discipline, Victoria University, Australia. Email<EMAIL_ADDRESS><EMAIL_ADDRESS>received XXX; revised XXX.

###### Abstract

Explainable recommendation systems provide explanations for recommendation
results to improve their transparency and persuasiveness. The existing
explainable recommendation methods generate textual explanations without
explicitly considering the user’s preferences on different aspects of the
item. In this paper, we propose a novel explanation generation framework,
named Hierarchical Aspect-guided explanation Generation (HAG), for explainable
recommendation. Specifically, HAG employs a review-based syntax graph to
provide a unified view of the user/item details. An aspect-guided graph
pooling operator is proposed to extract the aspect-relevant information from
the review-based syntax graphs to model the user’s preferences on an item at
the aspect level. Then, a hierarchical explanation decoder is developed to
generate aspects and aspect-relevant explanations based on the attention
mechanism. The experimental results on three real datasets indicate that HAG
outperforms state-of-the-art explanation generation methods in both single-
aspect and multi-aspect explanation generation tasks, and also achieves
comparable or even better preference prediction accuracy than strong baseline
methods.

###### Index Terms:

Explanation generation, explainable recommendation, hierarchical graph
pooling.

##  1 Introduction

RECOMMENDATION systems have been widely used to help users make decisions by
suggesting them a list of items they may have interests. Various types of
recommendation methods have been developed based on collaborative filtering
[1] and deep learning techniques [2]. Although these methods can usually
achieve satisfactory performances, it is still very hard to explain their
recommendation mechanisms. Thus, many recent research efforts [3, 4, 5, 6, 7]
have been devoted to building explainable recommendation models that explain
why an item is recommended by generating high-quality explanations, which can
help improve the transparency and persuasiveness of recommendation systems. In
practice, different strategies may be adopted to explain the recommendation
results, _e.g._ , images, the behaviors of relevant users, and textual
descriptions of relevant items [8]. In this work, we focus on generating high
quality review text to explain the recommendation results presented to the
user.

The review generation methods for explainable recommendation can be roughly
classified into two groups: template-based approaches and natural language
generation approaches [8]. The template-based methods generate explanations by
filling the generated words in a sentence template. For example, in the
template “You might be interested in [aspect], on which this product performs
well”, we can replace [aspect] by a generated aspect to produce text
explanation for item recommendation [9]. However, template-based explanations
are uninformative and not persuasive. Moreover, designing high-quality
templates usually requires domain knowledge. Natural language generation
approaches can generate more natural and flexible sentences. Such approaches
have recently attracted increasing research attention. However, the pioneer
works [10, 11] focus on generating short reviews or tips only based on given
attributes (e.g., user ID, item ID, and rating value). Thus, it is difficult
for them to generate reliable explanations without considering other
generative signals [9, 11].

Aspects, an important type of generative signal, which usually represent item
features (e.g., ”price” and ”romance”), have recently been exploited to build
aspect-aware explanation generation models [12, 13, 14]. In these methods, the
aspects are extracted from the user-generated reviews and used to train the
explanation generation model. These methods assume the user’s interested
aspects are available for explanation generation. In practice, this assumption
usually does not hold, because we need to predict the user’s preferences for
different aspects of a target item in many application scenarios. The
hierarchical generation framework [15] provides a potential solution to
address this problem. However, this method only considers the user ID, item
ID, and the rating value as inputs. Thus, it cannot be effective in capturing
the aspect-relevant details of the user and item for generating long and
informative explanations.

Figure 1: An example of the review-based syntax graph. (a) shows the
dependency tree structures of two reviews. (b) shows the review-based syntax
graph built based on the nodes and relations extracted from the dependency
trees of reviews.

To address this problem, we build a review-based syntax graph to provide a
unified view of the user/item details based on the review data. Firstly, a
syntax dependency tree is built from each review. The relations in the
dependency tree provide important clues to mine aspects, details, and
opinions. From Figure 1 (a), we observe two sub-trees: story
$\xrightarrow[]{\text{nmod:with}}$ twists $\xrightarrow[]{\text{amod}}$
interesting, story $\xrightarrow[]{\text{nmod:with}}$ characters
$\xrightarrow[]{\text{amod}}$ memorable. They are both built up with the
structure of aspect $\xrightarrow[]{}$ details $\xrightarrow[]{}$ opinion. To
aggregate the details of the same aspect in different reviews, we construct
the review-based syntax graph by connecting details in different dependency
trees. As shown in Figure 1 (b), details about love story, such as characters
and twists, can be directly connected to love story in the review-based syntax
graph.

Moreover, we propose a novel explainable recommendation framework, i.e.,
Hierarchical Aspect-guided explanation Generation (HAG). Specifically, HAG
performs hierarchical aspect-guided graph pooling on the user/item review-
based syntax graph to extract the aspect-relevant information for building the
user/item representation. The user’s interests are matched with the item
properties at the aspect level to predict her preferences for different
aspects of the item. Then, a hierarchical explanation decoder is developed to
generate aspects and aspect-relevant explanations based on attention
mechanisms. To demonstrate the effectiveness of the proposed HAG model, we
perform extensive experiments on three real-world datasets. The experimental
results indicate that HAG outperforms state-of-the-art explanation generation
methods, and achieves comparable or even better preference prediction accuracy
than baseline methods.

Figure 2: Overall structure of the proposed HAG model. In this figure, we use
the aspect and explanation generation processes of a single word ”characters”
as an example.

##  2 Related Work

There are two main groups of explanation generation methods for explainable
recommendation, namely template-based and generation-based methods.

Template-based methods generate recommendation explanations by filling pre-
defined templates with different words for different users. For example, the
explicit factor model [9] generates the explanations by filling in the aspect
in the pre-defined template. Moreover, [16] introduces a template-based
explainable recommendation model with aspects and opinions. However, these
methods cannot provide more details about user preferences, and manually
designing the template is also time-consuming.

Generation-based methods focus on developing natural language generation
methods for explainable recommendation. For example, [10, 11, 17] employ
Recurrent Neural Networks (RNNs) based methods to encode the attribute
information, e.g., user ID, item ID, and rating value, to generate
explanations for recommendation results. These methods cannot generate
reliable and precise explanations due to the lack of guided information. To
address this problem, [12] exploits auxiliary information to generate
explanations from given aspects. Moreover, [14] proposes the neural template-
based explanation generation framework, which integrates the advantages of
both the template-based and language generation methods. This method first
uses the given aspect as a template and then generates template-controlled
explanations.

The aforementioned methods usually focus on short explanation generation with
only one aspect. To support long and informative explanation generation, [15]
proposes a coarse-to-fine generation framework that first generates a sentence
skeleton and then generates the aspect-aware explanation. This method only
considers the user ID, item ID, and rating value as input attributes. As these
input attributes usually contain less information, [13] introduces the
reference-based seq2seq model that treats historical justifications as
references. In addition, [18] presents a knowledge enhanced review generation
model to combine review and knowledge graph information.

##  3 PRELIMINARY

In this section, we introduce some background about the construction of
review-based syntax graph and the research problem studied in this work.

###  3.1 Review-based Syntax Graph Construction

In this work, we denote the historical reviews of a user $u$ by
$\mathcal{D}_{u}=\\{d_{u}^{1},d_{u}^{2},\cdots,d_{u}^{n_{du}}\\}$ and the
historical reviews of an item $i$ by
$\mathcal{D}_{i}=\\{d_{i}^{1},d_{i}^{2},\cdots,d_{i}^{n_{di}}\\}$, where
$n_{du}$ and $n_{di}$ denote the number of reviews associated with $u$ and
$i$, respectively. To better understand the review data, we build a user
review-based syntax graph $\mathcal{G}_{u}$ and an item review-based syntax
graph $\mathcal{G}_{i}$ from the review sets $\mathcal{D}_{u}$ and
$\mathcal{D}_{i}$, respectively. For each user $u$, we first apply text pre-
processing techniques (_e.g._ , tokenization and spelling correct) on
$\mathcal{D}_{u}$. Then, dependency parsing is used to automatically generate
constituent-based representation (_i.e._ , dependency tree) for each review
sentence based on syntax [19]. Moreover, we also perform pruning to remove the
words with little information. For example, as ”det” in Figure 1 (a) in the
main content is the relation between ”story” and its determiner ”a” which has
little semantic information, thus we remove the relation ”det” and only keep
the head and tail nodes. Similarly, we also remove the relations such as
”nmod:poss” and ”punct”. Then, we connect all the dependency trees of the
review sentences in $\mathcal{D}_{u}$ by replacing the same words in different
dependency trees with the same node. After that, we can obtain the user
review-based syntax graph
$\mathcal{G}_{u}=\\{\mathcal{X}_{u},\mathcal{E}_{u}\\}$, where
$\mathcal{X}_{u}$ denotes the set of nodes (_i.e._ , words), and
$\mathcal{E}_{u}$ denotes the set of edges
$\mathcal{E}_{u}=\\{(x_{h},r,x_{t})|x_{h},x_{t}\in\mathcal{X}_{u},r\in\mathcal{R}\\}$.
Here, $r$ denotes the relation connecting the two nodes, and $\mathcal{R}$
denotes the set of all possible relations. Similar operations are performed on
the item reviews $\mathcal{D}_{i}$ to obtain the item review-based syntax
graph $\mathcal{G}_{i}=\\{\mathcal{X}_{i},\mathcal{E}_{i}\\}$.

###  3.2 Problem Formulation

Following [20, 14], we first extract aspects from the observed review data
using the tool developed in [9], and use $\mathcal{L}_{u}$ and
$\mathcal{L}_{i}$ to denote the lists of aspects derived from the user review
set $\mathcal{D}_{u}$ and item review set $\mathcal{D}_{i}$, respectively. In
$\mathcal{L}_{u}$, we sort aspects according to their occurrence frequency in
descending order, and choose the top-$n$ ranked aspects to describe the user
properties, which are denoted by
$\mathcal{Q}_{u}=\\{q_{u}^{1},q_{u}^{2},\cdots,q_{u}^{n}\\}$. Similarly, we
choose the top-$n$ ranked aspects from $\mathcal{L}_{i}$ and denote them by
$\mathcal{Q}_{i}=\\{q_{i}^{1},q_{i}^{2},\cdots,q_{i}^{n}\\}$. In this work, we
study the following explainable recommendation problem: given a user $u$ and
an item $i$, their historical aspect sets $\mathcal{Q}_{u}$ and
$\mathcal{Q}_{i}$, and review-based syntax graphs $\mathcal{G}_{u}$ and
$\mathcal{G}_{i}$, we aim to predict the user’s preference $r_{ui}$ on the
item, mine the user’s interested aspects
$\mathcal{A}_{ui}=\\{a_{1},a_{2},\cdots,a_{m}\\}$ of the item, and generate
aspect-aware explanations $\mathcal{P}_{ui}=\\{p_{1},p_{2},\cdots,p_{m}\\}$,
in the form of a set of sentences.

##  4 The Proposed HAG Framework

Figure 2 shows the overall framework of the proposed explainable
recommendation framework HAG. As shown in Figure 2, HAG contains the following
main components: 1) review-based syntax graph representation learning (RGRL),
2) aspect matching, 3) preference prediction, and 4) hierarchical explanation
decoder. Next, we introduce the details of each component.

###  4.1 Review-based Syntax Graph Representation Learning

The objective of the RGRL module is to extract representations for each
review-based syntax graph. Firstly, we apply a BiLSTM $f$ to encode the word
embedding of an aspect $q$, and obtain output backward hidden state as the
representation $f(q)$. Then, hierarchical aspect-guided graph pooling is used
to extract the review-based syntax graph representation at the aspect level.
We propose an aspect-guided graph pooling (AGP) operator to effectively
extract the aspect-specific knowledge from the review-based syntax graph. AGP
employs the user/item historical aspects to guide review-based syntax graph
representation learning.

####  4.1.1 Aspect-guided Graph Pooling Operator

Figure 3 (a) shows the workflow of the AGP operator. The inputs of an AGP
operator include an input graph
$\mathcal{G}=\\{\mathcal{X},\mathcal{E},\mathbf{X},\mathbf{A}\\}$ and an
aspect $q$ with its representation ${f(q)}$. Here, $\mathcal{X}$ and
$\mathcal{E}$ denote the set of nodes and edges in $\mathcal{G}$,
respectively. $\mathbf{X}$ is the node feature matrix and $\mathbf{A}$ is the
adjacency matrix. Firstly, a graph attention network (GAT) [21] is used to
encode graph $\mathcal{G}$. For each node $x_{h}\in\mathcal{X}$, we denote its
first-hop neighbors in the graph by $\mathcal{N}_{h}$. The feature of $x_{h}$
is updated by aggregating the input features of neighborhood nodes and adding
its input feature $\mathbf{x}_{h}$ by self-loop as,

$\overline{\mathbf{x}}_{h}=\mathbf{x}_{h}\
+\sum_{{x}_{t}\in\mathcal{N}_{h}}\alpha(x_{h},r,x_{t})\mathbf{x}_{t}\mathbf{W}_{1},$
(1)

where $\mathbf{W}_{1}\in\mathbb{R}^{d_{0}\times d_{0}}$ is the weight matrix,
and $\alpha(x_{h},r,x_{t})$ denotes the attention score between two nodes
$x_{t}$ and $x_{h}$. Following [21], we define the attention score
$\alpha(x_{h},r,x_{t})$ as,

$\displaystyle\alpha(x_{h},r,x_{t})=\frac{\exp\big{[}\pi(x_{h},r,x_{t})\big{]}}{{\sum_{{x}_{t^{\prime}}\in\mathcal{N}_{h}}}\exp\big{[}\pi(x_{h},{r},{x}_{t^{\prime}})\big{]}}.$
(2)

Here, $\pi(x_{h},r,x_{t})$ is implemented by the following attentional
mechanism,

$\pi(x_{h},r,x_{t})=\sigma_{1}\big{(}(\mathbf{x}_{h}\mathbf{W}_{2})(\mathbf{x}_{t}\mathbf{W}_{3}+\mathbf{r}\mathbf{W}_{4})^{\top}\big{)},$
(3)

where $\mathbf{r}$ is the embedding of the relation $r$, $\sigma_{1}(\cdot)$
is LeakyReLU activation function, $\mathbf{W}_{2}$, $\mathbf{W}_{3}$,
$\mathbf{W}_{4}\in\mathbb{R}^{d_{0}\times d_{0}}$ are weight matrixes. We use
a matrix $\overline{\mathbf{X}}$ to denote the updated features of all nodes.

Inspired by previous graph pooling methods [22, 23, 24], we define the
following aspect-aware importance score to describe the relevance of each node
in $\mathcal{G}$ to the given aspect $q$,

$\delta(x_{h})=\mbox{abs}\big{(}\overline{\mathbf{x}}_{h}{f(q)}^{\top}\big{)},$
(4)

where $\mbox{abs}(\cdot)$ denotes the absolute value function. The nodes in
$\mathcal{G}$ can be ranked according to $\delta(x_{h})$ in descending order.
Then, we denote the set of top-$K$ ranked nodes by $\widetilde{\mathcal{X}}$
and their indices by $\mbox{idx}(s)$. In this work, we empirically set
$K=\lceil\rho|\mathcal{X}|\rceil$, where $\rho$ is the pooling ratio,
$|\cdot|$ denotes the cardinality of a set, $\lceil\cdot\rceil$ is the ceiling
function. The new features of nodes in $\widetilde{\mathcal{X}}$ and adjacency
matrix $\widetilde{\mathbf{A}}$ of the corresponding graph are defined as,

$\displaystyle\widetilde{\mathbf{X}}=\mbox{ReLU}\big{(}\overline{\mathbf{X}}[\mbox{idx}(s),:]\mathbf{W}_{5}+\mathbf{b}_{1}\big{)},$
(5)
$\displaystyle\widetilde{\mathbf{A}}=\mathbf{A}\big{[}\mbox{idx}(s),\mbox{idx}(s)\big{]},$
(6)

where $\mathbf{W}_{5}\in\mathbb{R}^{K\times d_{0}}$ and
$\mathbf{b}_{1}\in\mathbb{R}^{1\times d_{0}}$ are the weight matrix and bias
vector respectively. $\overline{\mathbf{X}}[\mbox{idx}(s),:]$ is row-wise
indexed feature matrix. $\mathbf{A}[\mbox{idx}(s),\mbox{idx}(s)]$ aims to get
the row-wise and column-wise indexed adjacency matrix from $\mathbf{A}$. Then,
$\widetilde{\mathbf{X}}$ and $\widetilde{\mathbf{A}}$ are the new feature
matrix and the corresponding adjacency matrix after pooling. Moreover, we use
$\widetilde{\mathcal{E}}$ to denote the set of edges that describe the
connecting relationships between the nodes in $\widetilde{\mathcal{X}}$. Then,
the output of the AGP operator is denoted by
$\widetilde{\mathcal{G}}=\\{\widetilde{\mathcal{X}},\widetilde{\mathcal{E}},\widetilde{\mathbf{X}},\widetilde{\mathbf{A}}\\}$.

Figure 3: (a) shows the flow of the AGP operator where $q$ and $\mathcal{G}$
denote the input aspect and graph. (b) shows the flow of review-based syntax
graph representation learning.

####  4.1.2 Hierarchical Graph Pooling

We perform hierarchical graph pooling on the item/user review-based syntax
graph, by staking AGP operators. Here, we only introduce the hierarchical
graph pooling on the user review-based syntax graph $\mathcal{G}_{u}$. The
same process is also followed on the item review-based syntax graph
$\mathcal{G}_{i}$. As shown in Figure 3 (b), we conduct $L$ layers graph
pooling on $\mathcal{G}_{u}$, guided by the user aspect set
$\mathcal{Q}_{u}=\\{q_{u}^{1},q_{u}^{2},\cdots,q_{u}^{n}\\}$.

At the $\ell$-th layer, there are $n$ AGP operators. The input graph of the
$k$-th AGP operator is
$\mathcal{G}_{u,k}^{\ell-1}=\\{\mathcal{X}_{u,k}^{\ell-1},\mathcal{E}_{u,k}^{\ell-1},\mathbf{X}_{u,k}^{\ell-1},\mathbf{A}_{u,k}^{\ell-1}\\}$.
The representation ${f({q}}_{u}^{k})$ of aspect $q_{u}^{k}$ is used to guide
the pooling in the $k$-th AGP operator. The output graph of this AGP operator
is $\mathcal{G}_{u,k}^{\ell}$, which will be used as the input of the $k$-th
AGP operator at the $(\ell+1)$-th layer. Note that, at the first layer, the
inputs $\mathcal{G}_{u,k}^{0}=\mathcal{G}_{u}$, for $k=1,2,\cdots,n$. We
perform max pooling on $\mathbf{X}_{u,k}^{\ell}$ to obtain the aspect-aware
graph representation $\mathbf{g}_{u,k}^{\ell}$ at the $\ell$-th layer. After
performing the above pooling operation $L$ times, we can obtain multiple
representations of $\mathcal{G}_{u}$ that are relevant to the aspect
$q_{u}^{k}$, i.e.,
$\mathbf{g}_{u,k}^{1},\mathbf{g}_{u,k}^{2},\cdots,\mathbf{g}_{u,k}^{L}$. To
fuse the graph representations from fine-grained to coarse, we concatenate
these representations to form the representation $\mathbf{g}_{u}^{k}$ of the
review-based syntax graph $\mathcal{G}_{u}$ as
$\mathbf{g}_{u}^{k}=\mathbf{g}_{u,k}^{1}\oplus\mathbf{g}_{u,k}^{2}\oplus\cdots\mathbf{g}_{u,k}^{L}$.

###  4.2 Aspect Matching

To better describe the user’s preferences for different aspects, we compute
aspect level user representation ${\mathbf{S}}_{u}$ and item representation
${\mathbf{S}}_{i}$ by concatenating the representations of historical aspects
with those of the review-based syntax graphs as follows,

${\mathbf{S}}_{u}=\begin{bmatrix}\mathbf{g}_{u}^{1}\oplus f(q_{u}^{1})\\\
\mathbf{g}_{u}^{2}\oplus f(q_{u}^{2})\\\ \cdots\\\ \mathbf{g}_{u}^{n}\oplus
f(q_{u}^{n})\end{bmatrix},{\mathbf{S}}_{i}=\begin{bmatrix}\mathbf{g}_{i}^{1}\oplus
f(q_{i}^{1})\\\ \mathbf{g}_{i}^{2}\oplus f(q_{i}^{2})\\\ \cdots\\\
\mathbf{g}_{i}^{n}\oplus f(q_{i}^{n})\end{bmatrix}.$ (7)

Following [25, 26], we use ${\mathbf{S}}_{u}$ and ${\mathbf{S}}_{i}$ as
features to define the following aspect level importance weight matrix,

$\mathbf{M}_{s}=\mbox{ReLU}\big{(}{\mathbf{S}}_{u}\mathbf{W}_{s}{\mathbf{S}}_{i}^{\top}\big{)},$
(8)

where $\mathbf{W}_{s}\in\mathbb{R}^{d^{\prime}\times d^{\prime}}$ is a
learnable weight matrix, and $d^{\prime}=d_{0}\times L+d_{1}$. In
$\mathbf{M}_{s}\in\mathbb{R}^{n\times n}$, each element $\mathbf{M}_{s}[x,y]$
describes the importance of the $y$-th item aspect to the $x$-th user aspect.
Then, we fuse the aspect level information of the user and item with
$\mathbf{M}_{s}$ as follows,

$\displaystyle\mathbf{v}_{u}$
$\displaystyle=\phi\bigg{(}{\mathbf{S}}_{u}\mathbf{W}_{u}^{s}\oplus\mathbf{M}_{s}({\mathbf{S}}_{i}\mathbf{W}_{i}^{s})\bigg{)},$
(9) $\displaystyle\mathbf{v}_{i}$
$\displaystyle=\phi\bigg{(}{\mathbf{S}}_{i}\mathbf{W}_{i}^{s}\oplus\mathbf{M}^{\top}_{s}({\mathbf{S}}_{u}\mathbf{W}_{u}^{s})\bigg{)},$
(10)

where $\mathbf{W}_{u}^{s}$, $\mathbf{W}_{i}^{s}\in\mathbb{R}^{n\times
d^{\prime}}$ are trainable parameters, and $\phi(\cdot)$ denotes the mean
pooling operation. Here, $\mathbf{v}_{u}$ aims to incorporate user interested
aspects and the user preferences for the item aspect. Similarly,
$\mathbf{v}_{i}$ aims to combine item typical aspect and the representation of
aspects that are highly relevant to the user.

###  4.3 Preference Prediction

For each user $u$, we feed the user ID embedding $\mathbf{e}_{u}$ into a
Multi-Layer Perceptron (MLP) and concatenate its output with her aspect level
preference vector $\mathbf{v}_{u}$ to form the final representation as
follows,

$\mathbf{x}_{u}=\mbox{MLP}_{u}(\mathbf{e}_{u})\oplus\mathbf{W}_{u}^{v}\mathbf{v}_{u},$
(11)

where $\mathbf{W}_{u}^{v}\in\mathbb{R}^{(2d^{\prime})\times d_{2}}$ is a
learnable parameter. Similarly, we can obtain the final representation
$\mathbf{x}_{i}$ of the item $i$. Then, we concatenate $\mathbf{x}_{u}$ and
$\mathbf{x}_{i}$ to form $\mathbf{x}$. A Factorization Machine (FM) layer [27]
is applied to predict the user $u$’s preference on the item $i$ as follows,

$\hat{r}_{ui}=b_{0}+b_{u}+b_{i}+\mathbf{x}\mathbf{w}^{\top}+\sum_{i=1}^{n}\sum_{j=i+1}^{n}\left\langle\mathbf{v}_{i},\mathbf{v}_{j}\right\rangle
x_{i}x_{j},$ (12)

where $b_{0}$, $b_{u}$ and $b_{i}$ are global bias, user bias, and item bias.
$\mathbf{v}_{i}$ and $\mathbf{v}_{j}$ are the $i$-th and $j$-th variants.
$\mathbf{w}\in\mathbb{R}^{1\times(4d_{2})}$ is coefficient vector, and
$\left\langle\cdot,\cdot\right\rangle$ is the dot product of two vectors.

###  4.4 Hierarchical Explanation Generation

In HAG, we adopt two attention-based Long Short-Term Memory (LSTM) models as
the aspect decoder and explanation decoder, respectively. This section
introduces the details of these two decoders.

####  4.4.1 Aspect Decoder

As shown in Figure 3, for each aspect $q_{u}^{k}$, we can obtain the node
feature matrix $\mathbf{X}_{u,k}^{L}$ of the graph $\mathcal{G}_{u,k}^{L}$
after graph pooling on the user review-based syntax graph. Similarly, we can
also obtain the node feature matrix $\mathbf{X}_{i,k}^{L}$ after graph pooling
on the item review-based syntax graph. Then, we stack all aspect-relevant node
feature matrices to form the following matrices,

$\mathbf{X}_{u}=\begin{bmatrix}\mathbf{X}_{u,1}^{L}\\\ \cdots\\\
\mathbf{X}_{u,n}^{L}\end{bmatrix},\quad\mathbf{X}_{i}=\begin{bmatrix}\mathbf{X}_{i,1}^{L}\\\
\cdots\\\ \mathbf{X}_{i,n}^{L}\end{bmatrix}.$ (13)

To initial the hidden state, we first map the predicted rating $\hat{r}_{ui}$
into a sentiment representation $\mathbf{v}_{r}$ to guide aspect and
explanation generation as,

$\displaystyle\mathbf{v}_{r}=\mbox{ReLU}(\mathbf{W}_{u}^{r}\hat{r}_{ui}+\mathbf{b}_{u}^{r}),$
(14)

where $\mathbf{W}_{u}^{r}\in\mathbb{R}^{1\times d_{1}}$ and
$\mathbf{b}_{u}^{r}\in\mathbb{R}^{1\times d_{1}}$ are a trainable weight
matrix and a bias vector. Then, we feed $\mathbf{v}_{r}$, $\mathbf{v}_{u}$,
and $\mathbf{v}_{i}$ into an MLP layer as,

$\displaystyle\mathbf{h}_{0}^{a}$
$\displaystyle=\mbox{MLP}_{h}(\mathbf{v}_{r}\oplus\mathbf{v}_{u}\oplus\mathbf{v}_{i}).$
(15)

To fully exploit the review-based syntax graph information, at each decoding
time-step $j-1$, we incorporate hidden state $\mathbf{h}_{j-1}^{a}$ and node
feature $\mathbf{x}_{z}\in\mathbf{X}_{u}$ to calculate attention vector
$\mathbf{c}_{u,j-1}^{a}$ as,

$\displaystyle\beta_{u}^{j-1}$
$\displaystyle=\frac{\exp(\mbox{MLP}(\mathbf{x}_{z}\oplus\mathbf{h}_{j-1}^{a}))}{\sum_{\acute{\mathbf{x}}_{z}\in\mathbf{X}_{u}}\exp(\mbox{MLP}(\acute{\mathbf{x}}_{z}\oplus\mathbf{h}_{j-1}^{a})},$
$\displaystyle\mathbf{c}_{u,j-1}^{a}$
$\displaystyle=\sum_{\mathbf{x}_{z}\in\mathbf{X}_{u}}\beta_{u}^{j-1}\mathbf{x}_{z}.$
(16)

Similarly, we can obtain the attention vector $\mathbf{c}_{i,j-1}^{a}$ from
$\mathbf{X}_{i}$. The next time-step hidden state $\mathbf{h}_{j}^{a}$ is,

$\displaystyle\hat{\mathbf{h}}_{j-1}^{a}$
$\displaystyle=\mathbf{h}_{j-1}^{a}\oplus\mathbf{c}_{u,j-1}^{a}\oplus\mathbf{c}_{i,j-1}^{a}\oplus\mathbf{h}_{0}^{a},$
$\displaystyle\mathbf{h}_{j}^{a}$
$\displaystyle=\mbox{LSTM}(\hat{\mathbf{h}}_{j-1}^{a},E(w_{j-1}^{a})),$ (17)

where the $E(w_{j-1}^{a})$ is the embedding of the previous aspect
$w_{j-1}^{a}$. $\mathbf{h}_{j}^{a}$ is fed into an MLP layer to obtain the
probability of target aspect $w_{j}^{a}$ as,

$\mathcal{P}(w_{j}^{a})=\mbox{softmax}(\mathbf{W}_{a}\mathbf{h}_{j}^{a}+\mathbf{b}_{a}),$
(18)

where $\mathbf{W}_{a}\in\mathbb{R}^{d_{1}\times d_{a}}$ is a trainable
parameter, $d_{a}$ is the size of the aspect vocabulary.

####  4.4.2 Explanation Decoder

After the user interested aspect set $w^{a}$ has been generated, we can
further generate aspect-relevant explanations. For the $j$-th explanation
$p_{j}\in P$, we utilize the $j$-th hidden state $\mathbf{h}_{j}^{a}$ of
aspect encoder as the initial hidden state $\mathbf{h}_{j,0}^{p}$. Similar to
Eq. (16), we further use review-based syntax graph to obtain attention vector
$\mathbf{c}_{u,j,t-1}^{p}$ and $\mathbf{c}_{i,j,t-1}^{p}$ at each decoding
time-step $t-1$. We also concatenate $\mathbf{h}_{j,0}^{p}$,
$\mathbf{c}_{u,j,t-1}^{p}$, $\mathbf{c}_{i,j,t-1}^{p}$ and previous hidden
state to obtain $\hat{\mathbf{h}}_{j,t-1}^{p}$. Then
$\hat{\mathbf{h}}_{j,t-1}^{p}$ and the embedding of previous predicted word
$E(w_{j,t-1}^{p})$ are fed into the decoder as,

$\mathbf{h}_{j,t}^{p}=\mbox{LSTM}(\hat{\mathbf{h}}_{j,t-1}^{p},E(w_{j,t-1}^{p})).$
(19)

The probability of target word $w_{j,t}^{p}$ is calculated as,

$\mathcal{P}(w_{j,t}^{p})=\mbox{softmax}(\mathbf{W}_{p}\mathbf{h}_{j,t}^{p}+b_{p}),$
(20)

where $\mathbf{W}_{p}\in\mathbb{R}^{d_{1}\times d_{v}}$ is a weight parameter
and $d_{v}$ is the size of the vocabulary.

###  4.5 Multi-task Learning Objective Function

For the explanation generation task, we define the following cross-entropy
losses for aspect and explanation generation respectively,

$\displaystyle\ell_{ui}^{(a)}$
$\displaystyle=\frac{1}{|\mathcal{A}_{ui}|}\sum_{j=1}^{|\mathcal{A}_{ui}|}-\log(\mathcal{P}(w_{j}^{a})),$
$\displaystyle\ell_{ui}^{(p)}$
$\displaystyle=\frac{1}{|\mathcal{P}_{ui}|}\sum_{j=1}^{|\mathcal{P}_{ui}|}\frac{1}{|p_{j}|}\sum_{t=1}^{|p_{j}|}-\log(\mathcal{P}(w_{j,t}^{p})),$
(21)

where $|\mathcal{A}_{ui}|$ and $|\mathcal{P}_{ui}|$ denote the length of
golden aspect and explanation sets for a given user-item pair $(u,i)$.
$|p_{j}|$ denotes the length of the ground-truth explanation for the $j$-th
aspect. $\mathcal{P}(w_{j}^{a})$ and $\mathcal{P}(w_{j,t}^{p})$ denote the
probability of aspect $w_{j}^{a}$ and word $w_{j,t}^{p}$. In addition, we also
choose preference prediction as an auxiliary task to learn the HAG model and
define the loss function as follows,

$\ell_{ui}^{(r)}=(\hat{r}_{ui}-r_{ui})^{2},$ (22)

where $\hat{r}_{ui}$ and $r_{ui}$ denote the predicted and ground-truth rating
values respectively. The final loss function of the proposed HAG model is
defined as follows,

$\frac{1}{|\mathcal{O}|}\sum_{(u,i)\in\mathcal{O}}\ell_{ui}^{(a)}+\ell_{ui}^{(p)}+\ell_{ui}^{(r)},$
(23)

where $\mathcal{O}$ denotes the set of observed user-item pairs in the
training data, and $|\mathcal{O}|$ is the cardinality of set $\mathcal{O}$.
The entire framework can be effectively trained by minimizing Eq. (23) using
end-to-end back propagation.

##  5 Experiments

In this work, we perform experiments to evaluate both the explanation
generation performance and preference prediction performance of the proposed
HAG model.

###  5.1 Experimental Settings

####  5.1.1 Experimental Datasets

The experiments are conducted on the Amazon Review dataset [28] and Yelp
Challenge 2019 dataset111https://www.yelp.com/dataset/challenge, which have
been widely used for explanation generation. For the Amazon review dataset, we
choose the following 5-core subsets for evaluation: ”Kindle Store” and
”Electronics” (respectively denoted by Kindle and Elec.). For the Yelp dataset
and Amazon review dataset, we keep users and items that have more than 20 and
5 reviews for experiments respectively, due to the limitation of computation
resources. In each dataset, a record consists of user ID, item ID, overall
rating, and textual review. Following previous studies[29, 20], we first
extract aspects from the review data by the tool developed in [9]. Then, we
only keep records that contain more than one aspect and extract aspect-
relevant sentences from reviews as target explanations. Table I summarizes the
statistics of experimental datasets. For each dataset, we randomly split the
data into training, validation, and test data by the ratio 8:1:1.

Dataset | # Users | # Items | # Reviews | # Aspects
---|---|---|---|---
Kindle | 44,589 | 46,691 | 650,575 | 2,683
Elec. | 118,607 | 45,334 | 1,036,965 | 7,345
Yelp | 22,982 | 15,525 | 1,072,495 | 7,070
TABLE I: Statistics of the experimental datasets.

####  5.1.2 Evaluation Metrics

For the explanation generation task, we use BLEU [30], ROUGE [31], and METEOR
[32] as the evaluation metrics, which evaluate the text similarity between the
generated and gold explanations. BLEU evaluates the n-gram overlap between
gold and generated explanations. ROUGE evaluates the recall, precision, and
accuracy of the n-gram overlap. METEOR calculates the harmonic mean of each
word precision and recall based on the whole corpus. However, these
traditional language generation metrics can not measure whether the predicted
explanations could express the gold aspects. To better evaluate the generated
explanation, we also employ the Feature Matching Ratio (FMR) [14] to measure
whether generated explanation can include the target aspects.

To evaluate preference prediction performance of different methods, we use
Mean Absolute Error (MAE) as the evaluation metric. The definition of MAE is
as follows,

$\mbox{MAE}=\frac{1}{|\mathcal{T}|}\sum_{(u,i)\in\mathcal{T}}\big{|}\hat{r}_{ui}-r_{ui}\big{|},$
(24)

where $\mathcal{T}$ denotes the set of test data, $\hat{r}_{ui}$ denotes the
predicted rating value, $r_{ui}$ denotes the rating value in test data,
$|\cdot|$ denotes the size of a set.

Note that larger BLEU, ROUGE, METEOR, and FMR values indicate better results
for explanation generation task, and lower MAE values indicate better
performance for preference prediction task.

Datasets | Methods | Single-aspect Generation | Multi-aspect Generation
---|---|---|---
FMR | B-1 (%) | B-4 (%) | R-1 (%) | R-L (%) | M (%) | FMR | B-1 (%) | B-4 (%) | R-1 (%) | R-L (%) | M (%)
Kindle | Att2Seq | 0.22 | 13.96 | 1.95 | 14.96 | 11.77 | 5.81 | 0.46 | 18.01 | 2.19 | 19.80 | 13.47 | 6.88
ExpNet | 0.27 | 14.80 | 2.86 | 15.65 | 12.80 | 5.95 | 0.39 | 12.87 | 2.13 | 17.31 | 11.97 | 5.09
Ref2Seq | 0.29 | 15.46 | 3.07 | 17.66 | 14.42 | 6.68 | 0.46 | 13.47 | 2.29 | 20.91 | 14.34 | 6.27
NETE-PMI | 0.16 | 12.09 | 1.21 | 13.20 | 10.32 | 4.96 | - | - | - | - | - | -
ACF | 0.13 | 14.55 | 3.08 | 15.58 | 13.14 | 5.96 | 0.18 | 12.52 | 2.20 | 17.43 | 11.80 | 5.20
HAG | 0.43 | 16.42 | 3.34 | 18.08 | 14.65 | 7.15 | 0.65 | 19.10 | 2.66 | 23.13 | 14.88 | 8.37
Elec. | Att2Seq | 0.06 | 11.08 | 0.43 | 11.35 | 8.81 | 4.25 | 0.18 | 14.93 | 0.67 | 16.25 | 11.03 | 5.29
ExpNet | 0.05 | 12.03 | 0.36 | 12.20 | 9.51 | 4.60 | 0.18 | 14.64 | 0.58 | 16.69 | 11.45 | 4.97
Ref2Seq | 0.10 | 12.70 | 0.66 | 14.05 | 10.99 | 4.78 | 0.21 | 11.78 | 0.64 | 17.82 | 11.98 | 4.86
NETE-PMI | 0.11 | 11.28 | 0.62 | 12.08 | 9.45 | 4.90 | - | - | - | - | - | -
ACF | 0.08 | 11.31 | 0.34 | 12.51 | 9.57 | 3.33 | 0.18 | 11.99 | 0.31 | 14.69 | 9.38 | 4.08
HAG | 0.11 | 13.84 | 0.65 | 14.10 | 10.90 | 5.44 | 0.31 | 16.78 | 0.86 | 18.54 | 13.34 | 6.97
Yelp | Att2Seq | 0.06 | 12.06 | 0.92 | 12.45 | 9.84 | 4.33 | 0.19 | 16.70 | 0.91 | 17.19 | 13.00 | 5.61
ExpNet | 0.08 | 8.53 | 0.69 | 14.34 | 11.02 | 4.44 | 0.22 | 16.13 | 1.03 | 18.23 | 14.47 | 5.55
Ref2Seq | 0.09 | 13.32 | 1.05 | 16.16 | 12.87 | 5.15 | 0.23 | 13.45 | 0.96 | 18.93 | 14.67 | 5.42
NETE-PMI | 0.10 | 11.73 | 0.84 | 14.22 | 11.05 | 4.40 | - | - | - | - | - | -
ACF | 0.08 | 12.26 | 0.54 | 14.36 | 11.87 | 3.74 | 0.22 | 14.09 | 0.65 | 15.47 | 11.63 | 4.08
HAG | 0.25 | 17.40 | 1.17 | 19.05 | 14.64 | 7.37 | 0.27 | 17.07 | 1.07 | 18.30 | 14.70 | 6.88
TABLE II: Explanation generation performance achieved by different methods in
terms of FMR, BLEU (%), ROUGE (%), METEOR (%). B, R and M refer to BLEU, ROUGE
and METEOR. Note that NETE-PMI generates explanations with a single aspect,
thus we only report its single-aspect explanation generation performance.

####  5.1.3 Baseline Methods

We compare HAG with the following state-of-the-art explanation generation
methods,

  * •

Att2Seq [10]: It incorporates the Seq2Seq model [33] and attention mechanism
to learn the user’s preference from the user attributes and generate review
explanations.

  * •

ExpNet [12]: It utilizes an encoder-decoder framework to expand a short phrase
to a long review by combining the user and item information with other
auxiliary side information.

  * •

Ref2Seq [13]: This method follows the structure of Seq2Seq and learns the
representation from the user and item reviews to generate explanations.

  * •

NETE-PMI [14]: It adopts MLP to predict the rating and then generates a
template-controlled sentence with the predicted aspect.

  * •

ACF [15]: It uses MLP to encode different attributes and applies a coarse-to-
fine decoding model to generate long reviews.

Moreover, we also compare HAG with the following rating prediction methods to
evaluate its ability in predicting users’ preferences,

  * •

PMF [34]: This is the probabilistic matrix factorization method developed for
rating prediction.

  * •

SVD++ [35]: This method exploits both the user’s preferences on items and the
influences between items for recommendation.

  * •

CARL [36]: This method uses CNNs to learn relevant aspects from the review
data;

  * •

RMG [37]: It uses a multi-view learning framework to incorporate the review
contents and the users’ rating behaviors for the recommendation.

  * •

NETE-PMI [14]: This method feeds user ID and Item ID to an MLP to predict
rating scores.

####  5.1.4 Implementation Details

In this work, all the evaluated methods are implemented by PyTorch [38]. For
explanation generation methods, the learning rate is chosen from
$\\{0.0005,0.0007,0.002\\}$, and the batch size is set to 16. We empirically
set the max vocabulary size $d_{v}$ to 30,000. All the remaining words are
replaced by the special token ⟨UNK⟩. For single-aspect explanation generation,
we set the max sentence length to 15. For multi-aspect explanation generation,
we set the max feature length to 4 and set the max sentence length to 25.
During the inference process, we use the greedy search algorithm to generate
explanations.

In HAG, we set the hidden size $d_{1}$ and word embedding size to 128. The
pre-trained Google News vectors222https://code.google.com/archive/p/word2vec/
are used to initialize the word embeddings. The graph node embedding size
$d_{0}$ is chosen from $\\{8,16,32,64,128\\}$, and the dimension of user and
item id embeddings $d_{2}$ is chosen from $\\{8,16,32,64,128\\}$. Adam is used
to optimize the model with a cosine annealing learning rate decay [39]. The
number of graph pooling layers $L$ is set to 2. Moreover, we set the number of
top-ranked aspects $n$ extracted from the user/item reviews to 4. In Att2Seq,
we set the dimension of attribute embeddings to 64. For the decoding process,
the dimension of word embeddings and hidden vectors are set to 512. The
dropout rate is set to 0.2. For the ExpNet model, we set the dimension of
attribute embeddings and word embedding to 64 and 512 respectively. The
dimension of the aspect embedding is 15. For the decoder part, we set the
hidden size of GRU to 512, and the dropout rate is set to 0.1. For Ref2Seq,
the dimension of word embeddings and hidden vector are 256. For encoder and
decoder, the dropout rates are set to 0.5 and 0.2. For NETE-PMI, we set the
dimensions of word embeddings and attribute vectors to 200. The size of RNN
hidden states is set to 256. The dropout rate is 0.2, and the regularization
parameter is set to 0.0001. In ACF, aspects are extracted by TwitterLDA, and
the number of aspects is set to 10. For each aspect, the number of keywords is
set to 50. The dimension of word embeddings is set to 512. The hidden size of
the 2-layer GRU is set to 512.

The hyper-parameters of preference prediction methods are set as follows. For
PMF and SVD++, the learning rate is chosen from {0.0001, 0.0005, 0.001, 0.005,
0.01, 0.05}, the dimensionality of embeddings is chosen in {8, 16, 32, 48, 64,
128}, and the batch size is set to 512. For RMG, the learning rate is selected
from {0.01, 0.005, 0.001, 0.0005}, the number of CNNs filters is set to 150
and the kernel size is 3. For CARL, the learning rate is chosen from the
{0.05, 0.01, 0.005, 0.001}. The dimension of ID embedding and hidden state are
50 and 200. The number of CNN filters is 40.

Methods | Kindle | Electronics | Yelp
---|---|---|---
PMF | 0.6214 | 0.8292 | 0.7982
SVD++ | 0.5723 | 0.8492 | 0.8022
CARL | 0.5223 | 0.7794 | 0.7766
RMG | 0.5555 | 0.8203 | 0.8035
NETE-PMI | 0.5284 | 0.8537 | 0.7763
HAG | 0.5069 | 0.7836 | 0.7760
TABLE III: The preference prediction performance achieved by different methods
in terms of MAE. The best results are in bold faces and the second-best
results are underlined.

###  5.2 Performance Comparison

(a) FMR

(b) BLEU-4

(c) ROUGE-L

(d) MAE

Figure 4: Performance of HAG with respect to different settings of historical
aspect number $n$ on Kindle dataset.

Table II summarizes the performance of the single-aspect and multi-aspect
explanation generation tasks achieved by different methods. As shown in Table
II, ExpNet usually achieves better performance than Att2Seq by using aspects
to guide the generation process. Ref2Seq obtains better results than ExpNet
and Att2Seq. One potential reason is that Ref2Seq uses the user’s historical
reviews as inputs for explanation generation. In addition, ExpNet outperforms
NETE-PMI on the Kindle dataset, in terms of all metrics. It may because that
ExpNet uses an attention fusion layer to control the generation outputs.
Compared with Ref2Seq, ACF does not extract side information from the review
data as input, and it also generates sentences by filling the generated
templates based on predicted aspects. Its explanation generation performance
is highly dependent on the quality of input aspects. In ACF, the aspects are
extracted by TwitterLDA [40], thus the aspect quality could be influenced by
the review data quality and the given number of latent topics.

The preference prediction performance achieved by different methods is shown
in Table III. We can note that the review-based methods usually achieve better
performance than traditional matrix factorization method (_i.e._ , PMF and
SVD++). Moreover, the proposed HAG method achieves the best preference
prediction performance on Kindle and Yelp datasets, and achieves the second
best prediction performance on Electronics dataset, in terms of MAE. These
observations indicate that the user preference predicted by HAG can support
the high-quality explanation generation.

For both tasks, HAG usually achieves the best performance on all datasets, by
enhancing explanation generation with user and item review-based syntax
graphs. The review-based syntax graph aggregates information of all the
reviews associated with the user/item to provide a unified view about the
aspect-relevant details about the user/item. Thus, the review-based syntax
graph can also help weaken the impacts caused by noise reviews.

Methods | FMR | BLUE-4 (%) | ROUGE-L (%) | MAE
---|---|---|---|---
HAG | 0.65 | 2.66 | 14.88 | 0.5069
HAGw/o AGP | 0.68 | 2.39 | 14.40 | 0.5075
HAGGAT | 0.65 | 2.33 | 14.41 | 0.5099
HAGLSTM | 0.63 | 2.43 | 14.71 | 0.5127
HAGMLP | 0.57 | 2.41 | 12.24 | 0.5095
HAGimplicit | 0.64 | 2.22 | 14.90 | 0.5084
HAGw/o Relation | 0.57 | 2.58 | 15.50 | 0.5141
TABLE IV: The performance achieved by different variants of HAG on Kindle
dataset.

###  5.3 Ablation Study

In this section, we perform ablation study to analyze the effectiveness of
different components of HAG.

####  5.3.1 Impacts of AGP Operator

To study the contributions of the AGP operator, we study the performance of
the following three variants of HAG,

  * •

HAGw/o AGP: In this variant, the AGP operator is removed from the model. For
the aspect and explanation attention, we use the node feature of the original
review-based syntax graph to replace the node feature of the $L$-th graph.

  * •

HAGGAT: In this variant, the aspect-aware graph pooling module of AGP is
removed. We only use GAT to learn the representation of the review-based
syntax graph.

  * •

HAGLSTM: In this variant, the AGP operator is replaced with LSTM to update the
representations of nodes in the graph. For the aspect and explanation
attention calculation, it is same as HAGw/o AGP.

As shown in Table IV, HAG outperforms HAGw/o AGP, HAGGAT, and HAGLSTM, in
terms of BLEU-4, ROUGE-L, and MAE. This demonstrates that the AGP operator can
benefit both the prediction of user preferences and the explanation
generation.

####  5.3.2 Impacts of Historical Aspects

To study the impacts of historical aspects, we consider the following two
variants of HAG for evaluation,

  * •

HAGMLP: In this variant, we replace BiLSTM with MLP to extract features from
the historical aspects.

  * •

HAGImplicit: In this variant, we replace the feature of the historical aspect
with a randomly initialized vector, which can be learned in model training.

As shown in Table IV, HAG achieves better performance than HAGMLP and
HAGImplicit. This observation indicates that the historical aspect set is
helpful to mine the user preferences for aspects, and the BiLSTM structure can
help learn better representations for historical aspects.

####  5.3.3 Impacts of Grammatical Relations

To study the impacts of grammatical relations in the review-based syntax
graph, we remove the relation embedding in Eq. (3) and denote this variant of
HAG by HAGw/o Relation. As shown in Table IV, HAG achieves better FMR, BLUE-4
and MAE values than HAGw/o Relation. This indicates that the grammatical
relations can help improve the explanation generation performance.

$\rho$ | FMR | BLEU-4 (%) | ROUGE-L (%) | MAE
---|---|---|---|---
0.1 | 0.66 | 2.25 | 14.13 | 0.5107
0.3 | 0.53 | 2.57 | 14.67 | 0.5069
0.5 | 0.65 | 2.66 | 14.88 | 0.5069
0.7 | 0.66 | 2.49 | 14.68 | 0.5159
0.9 | 0.52 | 2.54 | 14.99 | 0.5126
TABLE V: Performance of HAG with respect to different graph pooling ratio $\rho$ on Kindle dataset. $L$ | FMR | BLEU-4 (%) | ROUGE-L (%) | MAE
---|---|---|---|---
1 | 0.69 | 2.22 | 13.78 | 0.5144
2 | 0.65 | 2.66 | 14.88 | 0.5069
3 | 0.65 | 2.62 | 15.79 | 0.5081
4 | 0.68 | 2.26 | 13.70 | 0.5113
TABLE VI: Performance of HAG with respect to different settings of the number
of pooling layers $L$ on Kindle dataset.

| Single-aspect Explanation | Multi-aspect Explanation
---|---|---
Reference Text |  | Good price and has protected my phone with no
---
complaints.
| Fairy tales and romance to read it.
---
The characters were enjoyable.
Att2Seq | And charges quickly. | No complaints.
ExpNet | I’ve used this case for my ipad mini and tablet. | I can’t wait for the next one.
ACF | The could is great, the could is great. | I found the characters to be perfectly and the plot driven.
Ref2Seq | Nice and nice protection. | A very enjoyable read. A very good read. A very good read.
NETE-PMI | I have a very good price and the product is very good. | -
HAG | The case is very good and the price is great. |  | I would recommend this book to anyone who likes a good romance.
---
The characters were interesting and the story line was good.

TABLE VII: Explanations generated by different methods. The target aspects are
in bold faces. Note that NETE-PMI cannot generate multi-aspect relevant
explanations. Compared with baseline methods, the proposed HAG model can
generate more informative explanations, which cover more target aspects.

###  5.4 Parameter Sensitivity Study

In this work, we employ the AGP operator to mine the aspect relevant
information from the review-based syntax graph. We perform experiments to
evaluate the impacts of the number of historical aspects $n$ on the
performance of HAG. Figure 4 shows performance trends of HAG with respect to
different settings of $n$. As shown in Figure 4, we can note that HAG achieves
the best FMR value when $n$ is set to 2. This indicates that the generated
sentences have the highest probability to cover the user’s interested aspects.
The best BLEU-4 and MAE values are achieved when $n$ is set to 4. Moreover,
the best ROUGE-L value is achieved when $n$ is set to 8. In the experiments,
we empirically set the number of historical aspects $n$ to 4, due to its best
performance on BLEU-4.

Moreover, we also study the performance of HAG with respect to different
settings of the graph pooling ratio $\rho$, which represents the proportion of
nodes retained. When $\rho$ is set to 0.1, it means that the model only
retains 10% of nodes in each graph pooling operation. As we set the graph
layer to 2, only 1% of nodes in the review-based syntax graph are remained
after two AGP operations. Table V summarizes the performance of HAG with
respect to different settings of $\rho$. We can note that the best FMR values
are achieved by setting $\rho$ to 0.1 and 0.7. For text generation metrics,
the best BLEU-4 and ROUGE-L values are achieved when $\rho$ is set to 0.5 and
0.9, respectively. The best preference prediction performance in terms of MAE
is achieved when $\rho$ is set to 0.3 and 0.5.

Furthermore, we also conduct experiments to study the performance of HAG with
respect to different number of graph layers $L$ on the Kindle dataset. As
shown in Table VI, HAG achieves better FMR values when $L$ is set to 1 and 4.
However, better BLEU-4, ROUGE-L, and MAE values are achieved when $L$ is set
to 2 and 3. When $L=2$, HAG can achieve the best performance in terms of
BLEU-4 and MAE, thus it is appropriate to set $L$ to 2 in the experiments.

###  5.5 Case Study

Table VII shows examples of the single-aspect and multi-aspect explanations
generated by different methods. For the single-aspect generation task, we can
note that the ”price” is the target aspect. As shown in Table VII, the
explanations generated by HAG can better express the target aspect. The
explanations generated by Att2Seq and ExpNet include irrelevant properties of
the product. The sentences generated by ACF and Ref2Seq are not natural enough
as the outputs have repetitive words or sentences. Moreover, we can note that
ACF cannot generate a meaningful explanation. The main reason is that the
quality of generated explanations highly depends on the accuracy of the
predicted aspects, and ACF cannot predict the user’s interested aspects
accurately. Although NETE-PMI can cover the target aspect, the subject of the
sentence is not correct. For the multi-aspect generation task, we can note
that most outputs of baseline models cannot cover the target aspects such as
“romance” and “characters”. Thus, they cannot generate high-quality
explanations based on the user preferences on aspects. The explanation
sentences generated by HAG not only cover specific aspects but also are more
natural. Compared to baseline methods, the proposed HAG model can generate
more expressive and reliable explanations.

Model | FMR | BLEU-4 (%) | ROUGE-L (%) | MAE
---|---|---|---|---
HAGP | 0.65 | 2.66 | 14.88 | 0.5069
HAGw/o P | 0.67 | 2.60 | 15.30 | 0.5102
Ref2SeqP | 0.46 | 2.29 | 14.34 | -
Ref2Seqw/o P | 0.45 | 2.22 | 14.22 | -
TABLE VIII: Performance of HAG and Ref2Seq with/without pre-trained word
embeddings on Kindle dataset, where “P” means with pre-trained embedding and
“w/o P” means without pre-trained embedding. Note that Ref2Seq does not
include the preference prediction module.

###  5.6 Impacts of Pre-trained Embeddings

Pre-trained embeddings are obtained from models trained on large datasets,
thus they usually contain rich semantic information. To evaluate the
performance of HAG without pre-trained embeddings and the effectiveness of
pre-trained embeddings in our tasks, we summarize the performance of HAG
with/without pre-trained embeddings on the Kindle dataset in Table VIII. Here,
we only compare HAG with Ref2Seq, which is the best baseline method in
explanation generation, and it also uses reviews as inputs. As shown in Table
VIII, the pre-trained embeddings do not help improve the performance of the
proposed HAG model. The performance of HAG without pre-trained embeddings are
even better than the performance of HAG with pre-trained embeddings. For the
Ref2Seq model, the performance decreases without using the pre-trained
embeddings. In conclusion, the proposed HAG model achieves excellent
performance even without pre-trained embeddings, and the effectiveness of pre-
trained embeddings in the studied explanation generation task is limited.

##  6 Conclusion

This paper proposes a novel explainable recommendation model, namely
Hierarchical Aspect-guided explanation Generation (HAG). Specifically, HAG
employs a review-based syntax graph that captures the user/item details from
the review data to enhance explanation generation. An aspect-guided graph
pooling (AGP) operator is proposed to distill the aspect-relevant information
from the review-based syntax graph. Moreover, an aspect matching mechanism is
developed to match the user preferences and item properties at the aspect
level. Furthermore, a hierarchical decoder is also developed to first predict
the user interested aspects and then generate the aspect-relevant
explanations. The experimental results on real datasets demonstrate that HGA
outperforms state-of-the-art explanation generation methods.

##  7 Acknowledgments

This research is supported by the National Research Foundation, Prime
Minister’s Office, Singapore under its AI Singapore Programme (AISG Award No:
AISG-GC-2019-003) and under its NRF Investigatorship Programme (NRFI Award No.
NRF-NRFI05-2019-0002). Any opinions, findings and conclusions or
recommendations expressed in this material are those of the authors and do not
reflect the views of National Research Foundation, Singapore.

## References

  * [1] Y. Shi, M. Larson, and A. Hanjalic, “Collaborative filtering beyond the user-item matrix: A survey of the state of the art and future challenges,” _ACM Computing Surveys (CSUR)_ , vol. 47, no. 1, pp. 1–45, 2014.
  * [2] S. Zhang, L. Yao, A. Sun, and Y. Tay, “Deep learning based recommender system: A survey and new perspectives,” _ACM Computing Surveys (CSUR)_ , vol. 52, no. 1, pp. 1–38, 2019.
  * [3] R. Catherine and W. Cohen, “Transnets: Learning to transform for recommendation,” in _Proceedings of the eleventh ACM conference on recommender systems_ , 2017, pp. 288–296. [Online]. Available: https://arxiv.org/pdf/1704.02298.pdf
  * [4] C. Chen, M. Zhang, Y. Liu, and S. Ma, “Neural attentional rating regression with review-level explanations,” in _Proceedings of the 2018 World Wide Web Conference_ , 2018, pp. 1583–1592.
  * [5] H. Chen, X. Chen, S. Shi, and Y. Zhang, “Generate natural language explanations for recommendation,” in _Proceedings of SIGIR’19 Workshop on ExplainAble Recommendation and Search. ACM_ , 2019. [Online]. Available: https://arxiv.org/pdf/2101.03392.pdf
  * [6] X. Chen, H. Chen, H. Xu, Y. Zhang, Y. Cao, Z. Qin, and H. Zha, “Personalized fashion recommendation with visual explanations based on multimodal attention network: Towards visually explainable recommendation,” in _Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval_ , 2019, pp. 765–774.
  * [7] P. Sun, L. Wu, K. Zhang, Y. Fu, R. Hong, and M. Wang, “Dual learning for explainable recommendation: Towards unifying user preference prediction and review generation,” in _Proceedings of The Web Conference 2020_ , 2020, pp. 837–847.
  * [8] Y. Zhang and X. Chen, “Explainable recommendation: A survey and new perspectives,” _arXiv preprint arXiv:1804.11192_ , 2018.
  * [9] Y. Zhang, G. Lai, M. Zhang, Y. Zhang, Y. Liu, and S. Ma, “Explicit factor models for explainable recommendation based on phrase-level sentiment analysis,” in _Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval_, 2014, pp. 83–92.
  * [10] L. Dong, S. Huang, F. Wei, M. Lapata, M. Zhou, and K. Xu, “Learning to generate product reviews from attributes,” in _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers_ , 2017, pp. 623–632. [Online]. Available: https://www.aclweb.org/anthology/E17-1059.pdf
  * [11] P. Li, Z. Wang, Z. Ren, L. Bing, and W. Lam, “Neural rating regression with abstractive tips generation for recommendation,” in _Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval_ , 2017, pp. 345–354. [Online]. Available: https://arxiv.org/pdf/1708.00154.pdf
  * [12] J. Ni and J. McAuley, “Personalized review generation by expanding phrases and attending on aspect-aware representations,” in _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , 2018, pp. 706–711.
  * [13] J. Ni, J. Li, and J. McAuley, “Justifying recommendations using distantly-labeled reviews and fine-grained aspects,” in _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , 2019, pp. 188–197.
  * [14] L. Li, Y. Zhang, and L. Chen, “Generate neural template explanations for recommendation,” in _Proceedings of the 29th ACM International Conference on Information & Knowledge Management_, 2020, pp. 755–764. [Online]. Available: http://yongfeng.me/attach/li-cikm2020.pdf
  * [15] J. Li, W. X. Zhao, J.-R. Wen, and Y. Song, “Generating long and informative reviews with aspect-aware coarse-to-fine decoding,” _arXiv preprint arXiv:1906.05667_ , 2019. [Online]. Available: https://arxiv.org/pdf/1906.05667.pdf
  * [16] N. Wang, H. Wang, Y. Jia, and Y. Yin, “Explainable recommendation via multi-task learning in opinionated text data,” in _The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval_, 2018, pp. 165–174.
  * [17] J. Tang, Y. Yang, S. Carton, M. Zhang, and Q. Mei, “Context-aware natural language generation with recurrent neural networks,” _arXiv preprint arXiv:1611.09900_ , 2016.
  * [18] J. Li, S. Li, W. X. Zhao, G. He, Z. Wei, N. J. Yuan, and J.-R. Wen, “Knowledge-enhanced personalized review generation with capsule graph neural network,” _arXiv preprint arXiv:2010.01480_ , 2020.
  * [19] D. Jurafsky and J. H. Martin, “Speech and language processing (draft),” _Chapter A: Hidden Markov Models (Draft of September 11, 2018). Retrieved March_ , vol. 19, p. 2019, 2018.
  * [20] X. He, T. Chen, M.-Y. Kan, and X. Chen, “Trirank: Review-aware explainable recommendation by modeling aspects,” in _Proceedings of the 24th ACM International on Conference on Information and Knowledge Management_ , 2015, pp. 1661–1670.
  * [21] P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio, “Graph attention networks,” _arXiv preprint arXiv:1710.10903_ , 2017.
  * [22] H. Gao and S. Ji, “Graph u-nets,” in _international conference on machine learning_. PMLR, 2019, pp. 2083–2092. [Online]. Available: http://proceedings.mlr.press/v97/gao19a/gao19a.pdf
  * [23] H. Gao, Y. Chen, and S. Ji, “Learning graph pooling and hybrid convolutional operations for text representations,” in _The World Wide Web Conference_ , 2019, pp. 2743–2749. [Online]. Available: https://arxiv.org/pdf/1901.06965.pdf
  * [24] J. Lee, I. Lee, and J. Kang, “Self-attention graph pooling,” in _International Conference on Machine Learning_. PMLR, 2019, pp. 3734–3743. [Online]. Available: http://proceedings.mlr.press/v97/lee19c/lee19c.pdf
  * [25] J. Y. Chin, K. Zhao, S. Joty, and G. Cong, “Anr: Aspect-based neural recommender,” in _Proceedings of the 27th ACM International Conference on Information and Knowledge Management_ , 2018, pp. 147–156. [Online]. Available: https://kaiqizhao.github.io/cikm18-anr.pdf
  * [26] J. Lu, J. Yang, D. Batra, and D. Parikh, “Hierarchical question-image co-attention for visual question answering,” in _Proceedings of the 30th International Conference on Neural Information Processing Systems_ , 2016, pp. 289–297. [Online]. Available: https://arxiv.org/pdf/1606.00061
  * [27] S. Rendle, “Factorization machines,” in _2010 IEEE International Conference on Data Mining_. IEEE, 2010, pp. 995–1000.
  * [28] R. He and J. McAuley, “Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering,” in _proceedings of the 25th international conference on world wide web_ , 2016, pp. 507–517. [Online]. Available: https://arxiv.org/pdf/1602.01585.pdf
  * [29] Z. Cheng, Y. Ding, L. Zhu, and M. Kankanhalli, “Aspect-aware latent factor model: Rating prediction with ratings and reviews,” in _Proceedings of the 2018 world wide web conference_ , 2018, pp. 639–648.
  * [30] K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in _Proceedings of the 40th annual meeting of the Association for Computational Linguistics_ , 2002, pp. 311–318. [Online]. Available: https://www.aclweb.org/anthology/P02-1040.pdf
  * [31] C.-Y. Lin, “Rouge: A package for automatic evaluation of summaries,” in _Text summarization branches out_ , 2004, pp. 74–81. [Online]. Available: https://www.aclweb.org/anthology/W04-1013.pdf
  * [32] S. Banerjee and A. Lavie, “Meteor: An automatic metric for mt evaluation with improved correlation with human judgments,” in _Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization_ , 2005, pp. 65–72. [Online]. Available: https://www.aclweb.org/anthology/W05-0909.pdf
  * [33] I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” in _Advances in neural information processing systems_ , 2014, pp. 3104–3112.
  * [34] A. Mnih and R. R. Salakhutdinov, “Probabilistic matrix factorization,” in _Advances in neural information processing systems_ , 2008, pp. 1257–1264.
  * [35] Y. Koren, “Factorization meets the neighborhood: a multifaceted collaborative filtering model,” in _Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining_ , 2008, pp. 426–434.
  * [36] L. Wu, C. Quan, C. Li, Q. Wang, B. Zheng, and X. Luo, “A context-aware user-item representation learning for item recommendation,” _ACM Transactions on Information Systems (TOIS)_ , vol. 37, no. 2, pp. 1–29, 2019.
  * [37] C. Wu, F. Wu, T. Qi, S. Ge, Y. Huang, and X. Xie, “Reviews meet graphs: Enhancing user and item representations for recommendation with hierarchical attentive graph neural network,” in _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , 2019, pp. 4886–4895.
  * [38] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, high-performance deep learning library,” in _Advances in Neural Information Processing Systems 32_ , 2019, pp. 8024–8035.
  * [39] G. Huang, Y. Li, G. Pleiss, Z. Liu, J. E. Hopcroft, and K. Q. Weinberger, “Snapshot ensembles: Train 1, get m for free,” _arXiv preprint arXiv:1704.00109_ , 2017. [Online]. Available: https://arxiv.org/pdf/1704.00109.pdf
  * [40] W. X. Zhao, J. Jiang, J. Weng, J. He, E.-P. Lim, H. Yan, and X. Li, “Comparing twitter and traditional media using topic models,” in _European conference on information retrieval_. Springer, 2011, pp. 338–349.