# Molecular nature of $P_{cs}(4459)$ and its heavy quark spin partners

C. W. Xiao<EMAIL_ADDRESS>School of Physics and Electronics, Central
South University, Changsha 410083, China  J. J. Wu<EMAIL_ADDRESS>School
of Physical Sciences, University of Chinese Academy of Sciences (UCAS),
Beijing 100049, China  B. S. Zou<EMAIL_ADDRESS>Key Laboratory of
Theoretical Physics, Institute of Theoretical Physics, Chinese Academy of
Sciences, Beijing 100190, China  School of Physical Sciences, University of
Chinese Academy of Sciences (UCAS), Beijing 100049, China  School of Physics
and Electronics, Central South University, Changsha 410083, China

###### Abstract

Inspired by the observation of the $P_{cs}(4459)$ state by LHCb recently, we
reexamine the results of the interaction of the $J/\psi\Lambda$ channel with
its coupled channels, exploiting the coupled channel unitary approach combined
with heavy quark spin and local hidden gauge symmetries. By tuning the only
free parameter, we find a pole of $(4459.07+i6.89)\textrm{ MeV}$ below the
$\bar{D}^{*}\Xi_{c}$ threshold, which was consistent well with the mass and
width of the $P_{cs}(4459)$ state. Thus, we assume the $P_{cs}(4459)$ state to
be a $\bar{D}^{*}\Xi_{c}$ bound state with the uncertainties on its degeneracy
with $J^{P}=\frac{1}{2}^{-}$ and $J^{P}=\frac{3}{2}^{-}$. For the degeneracy,
it would have two-poles structure, like $P_{c}(4450)$ before. There is another
pole in the $J^{P}=\frac{1}{2}^{-}$ sector, $(4310.53+i8.23)\textrm{ MeV}$,
corresponding to a deep bound state of $\bar{D}\Xi_{c}$. Furthermore, the
previously predicted loose bound states of $\bar{D}\Xi^{\prime}_{c}$,
$\bar{D}^{*}\Xi^{\prime}_{c}$, $\bar{D}^{*}\Xi^{*}_{c}$ with $J=1/2,~{}I=0$
and $\bar{D}^{*}\Xi^{\prime}_{c}$, $\bar{D}\Xi^{*}_{c}$,
$\bar{D}^{*}\Xi_{c}^{*}$ with $J=3/2,~{}I=0$ may exist as either bound states
or unbound virtual states. We hope that future experiments can search for the
$\bar{D}^{(*)}\Xi_{c}$ molecular states in their dominant decay channels of
$\bar{D}^{(*)}_{s}\Lambda_{c}$, also in the $J/\psi\Lambda$ and
$\eta_{c}\Lambda$ channels to reveal their different nature.

##  I Introduction

Early in 2010, with the coupled channel unitary approach (CCUA) Oller:1997ti ;
Oset:1997it ; Oller:2000fj  and the local hidden gauge (LHG) formalism
Bando:1984ej ; Bando:1987br ; Meissner:1987ge ; Nagahiro:2008cv  combined with
SU(4) symmetry, several hidden charm and hidden charm strangeness resonances
were predicted around the energy range of 4200 MeV – 4600 MeV in Ref.
Wu:2010jy , where the possible decay channel of $\eta_{c}N$ or $J/\psi N$ was
suggested for searching for the hidden charm ones in the experiments and its
further investigation was given in detail in Ref. Wu:2010vk . After that, the
pentaquark states caught the theoretical interest once again and had many
predictions in the hidden charm sector in Refs. Wang:2011rga ; Yang:2011wz ;
Yuan:2012wz ; Wu:2012md ; Garcia-Recio:2013gaa ; Xiao:2013yca ; Uchino:2015uha
; Karliner:2015ina . In 2015, the LHCb Collaboration reported two pentaquark-
like resonances found in the $J/\psi p$ invariant mass distributions of the
$\Lambda_{b}^{0}\to J/\psi K^{-}p$ decay Aaij:2015tga ; Aaij:2015fea , denoted
as $P_{c}(4380)^{+}$ with a large width about $205\textrm{ MeV}$ and
$P_{c}(4450)^{+}$ with a small width of $39\textrm{ MeV}$, which were
confirmed by a model-independent re-analysis of the experimental data
Aaij:2016phn  and in the $\Lambda_{b}^{0}\to J/\psi p\pi^{-}$ decay
Aaij:2016ymb . Furthermore, in 2019, with the Run-2 data the LHCb
Collaboration updated the new results for the $P_{c}$ states as three clear
narrow structures Aaij:2019vzc ,

$\displaystyle M_{P_{c1}}=(4311.9\pm 0.7^{+6.8}_{-0.6})\textrm{
MeV},\quad\Gamma_{P_{c1}}=(9.8\pm 2.7^{+3.7}_{-4.5})\textrm{ MeV},$
$\displaystyle M_{P_{c2}}=(4440.3\pm 1.3^{+4.1}_{-4.7})\textrm{
MeV},\quad\Gamma_{P_{c2}}=(20.6\pm 4.9^{+8.7}_{-10.1})\textrm{ MeV},$
$\displaystyle M_{P_{c3}}=(4457.3\pm 0.6^{+4.1}_{-1.7})\textrm{
MeV},\quad\Gamma_{P_{c3}}=(6.4\pm 2.0^{+5.7}_{-1.9})\textrm{ MeV},$

where the $P_{c}(4450)$ was split into two states of $P_{c}(4440)$ and
$P_{c}(4457)$, in addition to a new narrow resonance $P_{c}(4312)$. Whereas,
the broad old one $P_{c}(4380)$ could neither be confirmed nor refuted in the
updated results.

In principle, due to the fact that the $P_{c}$ states were found in the
experiments, the $P_{cs}$ states as their strangeness partners should also
exist as predicted in Refs. Wu:2010jy ; Wu:2010vk  from SU(4) flavour
symmetry. Note that, the first predictions for the masses and the widths of
the molecular resonances in Refs. Wu:2010jy ; Wu:2010vk  were due to the lack
of the experimental informations to determine the only free parameter of
$a_{\mu}$ ($\mu$ is not an independent one Oller:2000fj ; Ozpineci:2013zas ;
Xiao:2020frg ) in the loop functions. Therefore, after the updated results of
LHCb Collaboration Aaij:2019vzc  became available, this free parameter was
fitted as $a_{\mu}(\mu=1\textrm{ GeV})=-2.09$ in Ref. Xiao:2019aya  with the
masses of three $P_{c}$ states based on the former work of Xiao:2013yca ,
where seven hidden charm molecular states were predicted with the CCUA
combined with the LHG symmetry and the heavy quark spin symmetry (HQSS)
Isgur:1989vq ; Neubert:1993mb ; MW00 . Inspired by the experimental findings
of the $P_{c}$ states, with the same fitted parameter of $a_{\mu}$ from the
$P_{c}$ states, and also combined with the LHG symmetry and the HQSS, several
bound states of $\bar{D}^{(*)}\Xi_{c}^{(*)}$ and
$\bar{D}^{(*)}\Xi_{c}^{\prime}$ were predicted in Ref. Xiao:2019gjd , where
some of them were analogous to the ones in Refs. Wu:2010jy ; Wu:2010vk  and
seeking for them in the reaction of $\Xi_{b}^{-}\to J/\psi\Lambda K^{-}$ was
commented at the end as also suggested in the work of Chen:2015sxa ;
Santopinto:2016pkp . The possibility to look for them in the $\Lambda_{b}$
decays was also discussed in the early work of Feijoo:2015kts ; Lu:2016roh .
Furthermore, the decay properties of the hidden charm strangeness resonances
predicted in Refs. Wu:2010jy ; Wu:2010vk  were investigated in detail in Ref.
Shen:2019evi , where the partial decay widths of possible decay channels were
obtained by exploiting the effective Lagrangian framework through the triangle
loops. On the other hand, with the one-boson-exchange model, the work of Ref.
Chen:2016ryt  also predicted that the possible
$\bar{D}_{s}^{*}\Sigma_{c}^{(*)}$ and $\bar{D}^{*}\Xi_{c}^{(\prime,\,*)}$
pentaquark states existed. Taking into account the spin-flavour symmetric
states, the possible $SU_{f}(3)$ multiplets for the charmonium compact
pentaquark states were obtained in Ref. Santopinto:2016pkp , where the
possible decay channels and the partial decay widths were discussed for these
predictions. Using the chiral effective field theory up to the next-to-leading
order, Ref. Wang:2019nvm  found ten bound states in the hidden charm
strangeness systems of $\bar{D}^{(*)}\Xi_{c}^{\prime}$ and
$\bar{D}^{(*)}\Xi_{c}^{(*)}$ with different spin, where a mass difference of
about 6 MeV for two $\bar{D}^{*}\Xi_{c}$ molecular states with spins
$J=\frac{1}{2}$ and $J=\frac{3}{2}$ was obtained and looking for these
$P_{cs}$ states in the decay channel of $J/\psi\Lambda$ was suggested.

Just recently, the LHCb Collaboration had reported the results of
$\Xi_{b}^{-}\to J/\psi\Lambda K^{-}$ decay in Ref. Aaij:2020gdg , where a
resonance structure of $P_{cs}(4459)$ state was found in the invariant mass
distributions of $J/\psi\Lambda$, given as

$M_{P_{cs}}=(4458.8\pm 2.9^{+4.7}_{-1.1})\textrm{
MeV},\quad\Gamma_{P_{cs}}=(17.3\pm 6.5^{+8.0}_{-5.7})\textrm{ MeV},$

which is just about 19 MeV below the $\bar{D}^{*0}\Xi_{c}^{0}$ threshold. Note
that, analogous to the degenerate $P_{c}(4450)$ state, the possible
hypothetical structure as predicted in Refs. Xiao:2019gjd ; Wang:2019nvm  with
different spins was unsure for the current data sample. The two-poles
structure was also suggested in the results of the QCD sum rules in Ref.
Chen:2020uif , where the $P_{cs}(4459)$ state was assumed to be a
$\bar{D}^{*}\Xi_{c}$ molecular state with spin-parity $J^{P}=\frac{1}{2}^{-}$
or $J^{P}=\frac{3}{2}^{-}$ and a $\bar{D}\Xi_{c}$ bound state at
$4.29^{+0.13}_{-0.12}\textrm{ GeV}$ with $J^{P}=\frac{1}{2}^{-}$ was found
too. Also with the QCD sum rules, Ref. Wang:2020eep  assigned the
$P_{cs}(4459)$ state as a hidden-charm compact pentaquark state with
$J^{P}=\frac{1}{2}^{-}$, which was confirmed in the later work of
Azizi:2021utt ; 1844557 . On the other hand, with a combined effective field
theory and phenomenological assumptions, the work of Peng:2020hql  proposed
that the $P_{cs}(4459)$ state could be a $\bar{D}^{*}\Xi_{c}$ molecular
pentaquark state with $J^{P}=\frac{3}{2}^{-}$ more likely, or possibly
$J^{P}=\frac{1}{2}^{-}$ with more uncertainties on its mass. Using a coupled
channel analysis based on the one-boson-exchange model, Ref. Chen:2020kco
concluded that the $P_{cs}(4459)$ state was not a pure $\bar{D}^{*}\Xi_{c}$
molecular state and other possible resonances of
$\bar{D}^{(*)}\Xi_{c}^{(\prime,\,*)}$ could exist, and its decay behavior was
further discussed in Ref. Chen:2021tip . In Ref. Liu:2020hcv , the existence
of the $\bar{D}^{(*)}\Xi_{c}^{(\prime,\,*)}$ molecular states were also found
under the effective field theory by taking into account the HQSS and SU(3)
flavor symmetry, which were the SU(3)-flavour partners of the
$\bar{D}^{(*)}\Sigma_{c}^{(*)}$ molecular states. With the quasipotential
Bethe-Salpeter equation approach, Ref. Zhu:2021lhd  assigned the
$P_{cs}(4459)$ state as the $\bar{D}^{*}\Xi_{c}$ molecular state with
$J^{P}=\frac{3}{2}^{-}$ and predicted other
$\bar{D}^{(*)}\Xi_{c}^{(\prime,\,*)}$ states with different spins.
Furthermore, the decay of $\Lambda_{b}\to J/\psi\Lambda\phi$ was suggested to
search for the $P_{cs}(4459)$ state in Ref. Liu:2020ajv .

Motivated by the new findings of Ref. Aaij:2020gdg , we reexamine the results
of Refs. Wu:2010jy ; Wu:2010vk ; Xiao:2019gjd  to understand the differences
between our predictions and experimental data, then fix the model parameters
to make further predictions.

##  II Formalism

There are various approaches dealing with hadronic molecules as recently
reviewed in Refs. (Guo:2017jvc, ; Chen:2016qju, ). The LHG formalism seems
working well to give a general consistent explanation for various observed
hadronic molecular candidates with hidden charm Dong:2021juy ; Dong:2020hxe .
In Ref. Xiao:2019gjd , using the CCUA with LHG formalism, we considered the
coupled channels of the $J/\psi\Lambda$ channel, where there were nine
channels of $\eta_{c}\Lambda$, $J/\psi\Lambda$, $\bar{D}\Xi_{c}$,
$\bar{D}_{s}\Lambda_{c}$, $\bar{D}\Xi_{c}^{\prime}$, $\bar{D}^{*}\Xi_{c}$,
$\bar{D}^{*}_{s}\Lambda_{c}$, $\bar{D}^{*}\Xi_{c}^{\prime}$,
$\bar{D}^{*}\Xi_{c}^{*}$ in the $J^{P}=\frac{1}{2}^{-},\ I=0$ sector, and six
channels of $J/\psi\Lambda$, $\bar{D}^{*}\Xi_{c}$,
$\bar{D}_{s}^{*}\Lambda_{c}$, $\bar{D}^{*}\Xi_{c}^{\prime}$,
$\bar{D}\Xi_{c}^{*}$, $\bar{D}^{*}\Xi_{c}^{*}$ in the $J^{P}=\frac{3}{2}^{-},\
I=0$ sector. In addition, a single channel of $\bar{D}^{*}\Xi_{c}^{*}$ with
$J^{P}=\frac{5}{2}^{-}$ was also found in $s$ wave within the HQSS, which was
not taken into account in our work since it could not couple to the
$J/\psi\Lambda$ channel in $s$ wave. Note that the interaction of the
$\bar{D}^{*}\Xi_{c}^{*}$, which can be specified with spin
$J=\frac{1}{2},\,\frac{3}{2},\,\frac{5}{2}$, was included in the coupled
channels in Ref. Xiao:2019gjd  under the constraint of the HQSS, and not
considered in Refs. Wu:2010jy ; Wu:2010vk .

Within the CCUA, the scattering amplitudes ($T$) are evaluated by the coupled
channel Bethe-Salpeter equation with the on-shell prescription,

$T=[1-V\,G]^{-1}\,V,$ (1)

where $G$ is constructed by the loop functions with meson-baryon intermediate
states and $V$ are the potentials of the coupled channel interactions. Note
that, $G$ is a diagonal matrix with elements of meson-baryon loop functions,
where we take the ones with the dimensional regularization, see more details
in Refs. Wu:2010jy ; Wu:2010vk . Thus, the free parameter is the only one of
$a_{\mu}$, see the discussions later. Respecting the HQSS, the elements of the
potential $V$ matrix are given in Tables 1 and 2 for the $J=1/2,\,I=0$ and
$J=3/2,\,I=0$ sectors, respectively, where we only show $V_{ij}$ for $j\geq i$
for simplicity due to the fact that $V_{ji}=V_{ij}$ in the CCUA. In Tables 1
and 2, the coefficients $\mu_{i}$, $\mu_{ij}$ ($i,j=1,2,3,4$) and $\lambda$
are the unknown low energy constants with the HQSS constraint, see more
details in Ref. Xiao:2019gjd .

Table 1: Potential matrix elements $V_{ij}$ of Eq. (1) for the $J=1/2,~{}I=0$ sector. $\eta_{c}\Lambda$ | $J/\psi\Lambda$ | $\bar{D}\Xi_{c}$ | $\bar{D}_{s}\Lambda_{c}$ | $\bar{D}\Xi^{\prime}_{c}$ | $\bar{D}^{*}\Xi_{c}$ | $\bar{D}^{*}_{s}\Lambda_{c}$ | $\bar{D}^{*}\Xi^{\prime}_{c}$ | $\bar{D}^{*}\Xi^{*}_{c}$
---|---|---|---|---|---|---|---|---
$\mu_{1}$ | 0 | $-\frac{\mu_{12}}{2}$ | $-\frac{\mu_{13}}{2}$ | $\frac{\mu_{14}}{2}$ | $\frac{\sqrt{3}\mu_{12}}{2}$ | $\frac{\sqrt{3}\mu_{13}}{2}$ | $\frac{\mu_{14}}{2\sqrt{3}}$ | $\sqrt{\frac{2}{3}}\mu_{14}$
| $\mu_{1}$ | $\frac{\sqrt{3}\mu_{12}}{2}$ | $\frac{\sqrt{3}\mu_{13}}{2}$ | $\frac{\mu_{14}}{2\sqrt{3}}$ | $\frac{\mu_{12}}{2}$ | $\frac{\mu_{13}}{2}$ | $\frac{5\mu_{14}}{6}$ | $-\frac{\sqrt{2}\mu_{14}}{3}$
|  | $\mu_{2}$ | $\mu_{23}$ | 0 | 0 | 0 | $\frac{\mu_{24}}{\sqrt{3}}$ | $-\sqrt{\frac{2}{3}}\mu_{24}$
|  |  | $\mu_{3}$ | 0 | 0 | 0 | $\frac{\mu_{34}}{\sqrt{3}}$ | $-\sqrt{\frac{2}{3}}\mu_{34}$
|  |  |  | $\frac{1}{3}(2\lambda+\mu_{4})$ | $\frac{\mu_{24}}{\sqrt{3}}$ | $\frac{\mu_{34}}{\sqrt{3}}$ | $-\frac{2(\lambda-\mu_{4})}{3\sqrt{3}}$ | $\frac{1}{3}\sqrt{\frac{2}{3}}(\mu_{4}-\lambda)$
|  |  |  |  | $\mu_{2}$ | $\mu_{23}$ | $\frac{2\mu_{24}}{3}$ | $\frac{\sqrt{2}\mu_{24}}{3}$
|  |  |  |  |  | $\mu_{3}$ | $\frac{2\mu_{34}}{3}$ | $\frac{\sqrt{2}\mu_{34}}{3}$
|  |  |  |  |  |  | $\frac{1}{9}(2\lambda+7\mu_{4})$ | $\frac{1}{9}\sqrt{2}(\lambda-\mu_{4})$
|  |  |  |  |  |  |  | $\frac{1}{9}(\lambda+8\mu_{4})$
Table 2: Potential matrix elements $V_{ij}$ of Eq. (1) for the $J=3/2,~{}I=0$ sector. $J/\psi\Lambda$ | $\bar{D}^{*}\Xi_{c}$ | $\bar{D}_{s}^{*}\Lambda_{c}$ | $\bar{D}^{*}\Xi^{\prime}_{c}$ | $\bar{D}\Xi^{*}_{c}$ | $\bar{D}^{*}\Xi_{c}^{*}$
---|---|---|---|---|---
$\mu_{1}$ | $\mu_{12}$ | $\mu_{13}$ | $-\frac{\mu_{14}}{3}$ | $\frac{\mu_{14}}{\sqrt{3}}$ | $\frac{\sqrt{5}\mu_{14}}{3}$
| $\mu_{2}$ | $\mu_{23}$ | $-\frac{\mu_{24}}{3}$ | $\frac{\mu_{24}}{\sqrt{3}}$ | $\frac{\sqrt{5}\mu_{24}}{3}$
|  | $\mu_{3}$ | $-\frac{\mu_{34}}{3}$ | $\frac{\mu_{34}}{\sqrt{3}}$ | $\frac{\sqrt{5}\mu_{34}}{3}$
|  |  | $\frac{1}{9}(8\lambda+\mu_{4})$ | $\frac{\lambda-\mu_{4}}{3\sqrt{3}}$ | $\frac{\sqrt{5}}{9}(\lambda-\mu_{4})$
|  |  |  | $\frac{1}{3}(2\lambda+\mu_{4})$ | $\frac{1}{3}\sqrt{\frac{5}{3}}(\mu_{4}-\lambda)$
|  |  |  |  | $\frac{1}{9}(4\lambda+5\mu_{4})$

Using the LHG formalism, we obtain the values of these low energy constants
Xiao:2019gjd ,

$\displaystyle\mu_{1}$ $\displaystyle=$
$\displaystyle\mu_{3}=\mu_{24}=\mu_{34}=0$ (2) $\displaystyle\mu_{2}$
$\displaystyle=$ $\displaystyle\mu_{23}/\sqrt{2}=\mu_{4}=\lambda=-F,\quad
F=\frac{1}{4f^{2}}(p^{0}+p^{\prime\,0})$ (3) $\displaystyle\mu_{12}$
$\displaystyle=$
$\displaystyle-\mu_{13}/\sqrt{2}=\mu_{14}/\sqrt{3}=-\sqrt{\frac{2}{3}}\
\frac{m_{V}^{2}}{m_{D^{*}}^{2}}\ F,$ (4)

with $f_{\pi}=93\textrm{ MeV}$ and $m_{V}=800\textrm{ MeV}$, where $p^{0}$ and
$p^{\prime}\,{}^{0}$ are the energies of the incoming and outgoing mesons in a
certain channel. Note that, we have explicitly taken the reduction factor
$m_{V}^{2}/m_{D^{*}}^{2}$ in the matrix elements that involve the transition
processes with the exchange of $D^{*}$ meson. The null $\mu_{24}$ and
$\mu_{34}$ values are due to the pion exchange neglect in our formalism, as
done in Ref. Xiao:2013yca , see more discussions in Ref. Xiao:2020frg .

##  III Results and discussions

As discussed above, the subtraction constant $a_{\mu}$ in the meson-baryon
loop functions is a free parameter in our formalism, and thus, we cannot get a
precise value for it theoretically. It is even worse that the prediction will
significantly rely on its value for a loose bound system, since its value will
influence the lowest strength of attractive potential to form a bound state
(we will discuss this issue in detail later). In practice, the only way to get
its accurate value is using some experimental data to fix it. Therefore, using
the newest experimental results of Aaij:2020gdg , the value of $a_{\mu}$ can
be determined as $a_{\mu}(\mu=1\textrm{ GeV})=-1.94$ for the $P_{cs}$ case. It
is similar to what we have done in Ref. Xiao:2019aya  for the $P_{c}$ case,
where a value of $a_{\mu}(\mu=1\textrm{ GeV})=-2.09$ was obtained. In the
early prediction of Refs. Wu:2010jy ; Wu:2010vk , the central value of
$a_{\mu}(\mu=1\textrm{ GeV})=-2.3$ was used to match the cutoff in the loop
function as $0.8\textrm{ GeV}$, which is close to the masses of exchange
vectors, $\rho$ and $\omega$. Indeed, now the values of $a_{\mu}(\mu=1\textrm{
GeV})=-1.94$ and the one of $a_{\mu}(\mu=1\textrm{ GeV})=-2.09$ Xiao:2019aya
are really around the “natural values” of $a_{\mu}=-2$ Oller:2000fj .

Then, using the fitted $a_{\mu}(\mu=1\textrm{ GeV})=-1.94$, we obtain the
results of the modulus squared of the amplitudes in Figs. 1 and 2 for
$J=1/2,~{}I=0$ and $J=3/2,~{}I=0$, respectively, where the peak structures are
analogous to the ones obtained in Ref. Xiao:2019gjd  but more narrow and with
higher energies. The corresponding poles and its coupling constants to all the
channels are given in Tables 3 111There is an extra pole around
$(4291.05+i12.80)\textrm{ MeV}$ in the $\bar{D}^{*}\Xi_{c}$ channel, see the
left panel of Fig. 1, which couples strongly to $\bar{D}^{*}_{s}\Lambda_{c}$
channel (threshold $4398.66\textrm{ MeV}$) and looks like unnormal. It is due
to the fact that the $G$ functions for the channels of $\bar{D}^{*}\Xi_{c}$
and $\bar{D}^{*}_{s}\Lambda_{c}$ (denoted as channel 6 and 7, respectively)
become positive far below their thresholds, which also lead to the “effective”
potential of $V_{66}+V_{67}^{2}G_{77}$ changing to a positive one (see the
discussion later). Thus, a repulsive potential lead to a bound state
unusually, see more discussions in Refs. Xiao:2013yca ; Wu:2010rv . and 4 for
$J=1/2,~{}I=0$ and $J=3/2,~{}I=0$, respectively, where the thresholds of each
channel are shown accordingly. In Tables 3 and 4, with the couplings obtained,
the partial decay widths and the branching ratios for each channel are
evaluated. One thing should be mentioned, that the poles given in Tables 3 and
4 are the ones located in the general “second Riemann sheet”, which means that
all the channels below the certain bound channel are extrapolated to the
second Riemann sheet (these channels are always called the open channels),
whereas, the other coupled channels including the bound channel are in the
first Riemann sheet. As found from our results of Tables 3 and 4, the bound
state of $\bar{D}^{*}\Xi_{c}$ is tuned as $4459\textrm{ MeV}$ to make it
consistent with the mass of observed $P_{cs}$ state, which has increased about
30 MeV with respect to the one obtained in Ref. Xiao:2019gjd . Then the pole
of the $\bar{D}^{*}\Xi_{c}$ channel is $(4459.07+i6.89)\textrm{ MeV}$, where
the width is quite consistent with the experimental results Aaij:2020gdg , and
it just has 3 MeV difference. Note that, owing to the pion exchange neglect in
our formalism as discussed above, this pole of the $\bar{D}^{*}\Xi_{c}$
channel is degenerate with spins $J=1/2$ and $J=3/2$, as shown in Tables 3 and
4. Therefore, one can conclude from our formalism that the $P_{cs}(4459)$
state can be a bound state of $\bar{D}^{*}\Xi_{c}$ with spin uncertainty of
$J=1/2$ or $J=3/2$. Besides, in Table 3 for the $J=1/2,~{}I=0$ sector, we have
another very stable pole, $(4310.53+i8.23)\textrm{ MeV}$, which is bound by
the $\bar{D}\Xi_{c}$ channel and has 56 MeV binding energy. On the other hand,
other three peak structures are all very close to the corresponding
thresholds. The peak around 4445 MeV is contributed from the pole at
$(4445.12+i0.19)\textrm{ MeV}$ bound by the $\bar{D}\Xi^{\prime}_{c}$ channel,
of which the binding energy is just 0.23 MeV. Thus, in our model this bound
state becomes unstable if the parameter $a_{\mu}$ changes a little to move it
to the threshold, which will cause the threshold effect and then lead to no
pole in the general second Riemann sheet, as shown in Tables 3 and 4 for some
channels. Furthermore, the other two peak structures are actually due to the
poles from the other Riemann sheets. Thus, these poles can not be recognized
as normal bound states of the certain channels. It is proper to say that these
two peak structures are the threshold effects because of very weak attractive
interaction potentials. Indeed, the three poles for the channels of
$\bar{D}\Xi^{\prime}_{c}$, $\bar{D}^{*}\Xi^{\prime}_{c}$ and
$\bar{D}^{*}\Xi^{*}_{c}$ were just loosely bound as found in the results of
Ref. Xiao:2019gjd , where these poles were only a few MeV below the
corresponding thresholds and had narrow widths, a few MeV. Similarly, in Table
4 for $J=3/2,~{}I=0$ sector, except for the one of $\bar{D}^{*}\Xi_{c}$
stable, the other three poles of the channels $\bar{D}^{*}\Xi^{\prime}_{c}$,
$\bar{D}\Xi^{*}_{c}$ and $\bar{D}^{*}\Xi_{c}^{*}$ are not stable for the same
reason. Three of them were also loosely bound as given in the results of Ref.
Xiao:2019gjd  with a few MeV for the binding energies and the widths.

As shown in Tables 3 and 4, the degenerate molecular states of
$\bar{D}^{*}\Xi_{c}$ with $J=1/2$ and $J=3/2$ correspond to nearly the same
pole $(4459.07+i6.89)\textrm{ MeV}$. Their main decay channel is
$\bar{D}^{*}_{s}\Lambda_{c}$ with the branch ratio larger than $80\%$, which
can be understood as this channel coming from the strong interaction by
exchanging light vector meson in our model, and its large decay width was also
found in Ref. Chen:2021tip . But, the branch ratios of the $J/\psi\Lambda$
channel in the two cases are a little different because of the Clebsch-Gordan
coefficients in the HQSS model. Besides, the $\eta_{c}\Lambda$ channel is the
only decay channel of the bound state with $J^{P}=1/2^{-}$, while it requires
the $d$ wave interaction for the $J^{P}=3/2^{-}$ state, which is neglected in
our model. Therefore, if we assume the $\bar{D}^{*}\Xi_{c}$ bound state to be
the $P_{cs}(4459)$ state, it may have two-poles structure with different
spins, like the one of $P_{c}(4450)$ before, and it can be helpful to reveal
their nature with more experimental information in the future for the partial
decay widths and the branching ratios of the channels $J/\psi\Lambda$ and
$\eta_{c}\Lambda$. In Fig. 3, we show our results of Fig. 1 for $J=1/2,~{}I=0$
compared with the experimental data, where the full data are compared on the
left and the parts with the cut of $1.9\textrm{ GeV}<m_{\Lambda
K^{-}}<2.2\textrm{ GeV}$ on the right. As one can see from Fig. 3, at the
present Run II data, there is not enough statistics to draw a strong
conclusion for the existence of more bound states in the $J/\psi\Lambda$
invariant mass distributions.

Figure 1: Results of the modulus squared of the amplitudes for $J=1/2,~{}I=0$
sector.

Figure 2: Results of the modulus squared of the amplitudes for $J=3/2,~{}I=0$ sector. Table 3: Coupling constants to all channels for certain poles in $J=1/2,~{}I=0$ sector. Chan. | $\eta_{c}\Lambda$ | $J/\psi\Lambda$ | $\bar{D}\Xi_{c}$ | $\bar{D}_{s}\Lambda_{c}$ | $\bar{D}\Xi^{\prime}_{c}$ | $\bar{D}^{*}\Xi_{c}$ | $\bar{D}^{*}_{s}\Lambda_{c}$ | $\bar{D}^{*}\Xi^{\prime}_{c}$ | $\bar{D}^{*}\Xi^{*}_{c}$
---|---|---|---|---|---|---|---|---|---
Thres. | 4099.58 | 4212.58 | 4366.61 | 4254.80 | 4445.34 | 4477.92 | 4398.66 | 4586.66 | 4654.48
$4310.53+i8.23$ |  |  |  |  |  |  |  |
$|g_{i}|$ | $0.15$ | $0.27$ | $\mathbf{2.33}$ | $0.69$ | $0.00$ | $0.04$ | $0.09$ | $0.01$ | $0.02$
$\Gamma_{i}$ | $0.57$ | $1.18$ | $-$ | $13.86$ | $-$ | $-$ | $-$ | $-$ | $-$
Br. | 3.47% | 7.16% | $-$ | 84.21% | $-$ | $-$ | $-$ | $-$ | $-$
$4445.12+i0.19$ |  |  |  |  |  |  |  |
$|g_{i}|$ | $0.10$ | $0.06$ | $0.00$ | $0.00$ | $\mathbf{0.72}$ | $0.08$ | $0.04$ | $0.01$ | $0.01$
$\Gamma_{i}$ | $0.29$ | $0.08$ | $0.00$ | $0.00$ | $-$ | $-$ | $0.04$ | $-$ | $-$
Br. | 74.74% | 21.22% | 0.01% | 0.01% | $-$ | $-$ | 10.62% | $-$ | $-$
$4459.07+i6.89$ |  |  |  |  |  |  |  |
$|g_{i}|$ | $0.22$ | $0.13$ | $0.00$ | $0.00$ | $0.07$ | $\mathbf{2.16}$ | $0.61$ | $0.03$ | $0.02$
$\Gamma_{i}$ | $1.59$ | $0.46$ | $0.00$ | $0.00$ | $0.01$ | $-$ | $11.14$ | $-$ | $-$
Br. | 11.57% | 3.31% | 0.00% | 0.00% | 0.70% | $-$ | 80.86% | $-$ | $-$
$4586.66?$ |  |  |  |  |  |  |  |
$|g_{i}|$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$
$4654.48?$ |  |  |  |  |  |  |  |
$|g_{i}|$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$
Table 4: Coupling constants to all channels for certain poles in $J=3/2,~{}I=0$ sector.. Chan. | $J/\psi\Lambda$ | $\bar{D}^{*}\Xi_{c}$ | $\bar{D}_{s}^{*}\Lambda_{c}$ | $\bar{D}^{*}\Xi^{\prime}_{c}$ | $\bar{D}\Xi^{*}_{c}$ | $\bar{D}^{*}\Xi_{c}^{*}$
---|---|---|---|---|---|---
Thres. | 4212.58 | 4477.92 | 4398.66 | 4586.66 | 4513.17 | 4654.48
$4459.02+i6.83$ |  |  |  |  |
$|g_{i}|$ | $0.28$ | $\mathbf{2.16}$ | $0.61$ | $0.02$ | $0.04$ | $0.03$
$\Gamma_{i}$ | $2.00$ | $-$ | $11.15$ | $-$ | $-$ | $-$
Br. | 14.68% | $-$ | 81.64% | $-$ | $-$ | $-$
$4586.66?$ |  |  |  |  |
$|g_{i}|$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$
$4513.17?$ |  |  |  |  |
$|g_{i}|$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$
$4654.48?$ |  |  |  |  |
$|g_{i}|$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$

Figure 3: Results of the modulus squared of the amplitudes for $J=1/2,~{}I=0$
sector compared with the experimental data, Left: with full data; Right: with
$1.9\textrm{ GeV}<m_{\Lambda K^{-}}<2.2\textrm{ GeV}$. The vertical lines
correspond to the thresholds of the certain channels.

For the results of the partial decay widths and the branching ratios shown in
Table 3, it seems strange that the bound state of $\bar{D}\Xi_{c}$ with its
pole at $(4310.53+i8.23)\textrm{ MeV}$ decays more strongly to the
$J/\psi\Lambda$ channel than to the $\eta_{c}\Lambda$, whereas, the molecular
state of $\bar{D}^{*}\Xi_{c}$ with the pole $(4459.07+i6.89)\textrm{ MeV}$ is
just the opposite, which is different from the cases of the
$\bar{D}^{(*)}\Sigma_{c}$ bound states studied in Ref. Xiao:2020frg . This is
because of the constraint of the HQSS, see the transition elements of
$\frac{\mu_{12}}{2}$ and $\frac{\sqrt{3}\mu_{12}}{2}$ for the related channels
in Table 1. But, in Ref. Shen:2019evi  the bound state of $\bar{D}^{*}\Xi_{c}$
predicted in Refs. Wu:2010jy ; Wu:2010vk  was found to decay stronger to the
$J/\psi\Lambda$ channel than the $\eta_{c}\Lambda$. Indeed, the earlier work
of Wu:2010jy ; Wu:2010vk ; Shen:2019evi  did not consider the HQSS as well as
the transition between Pseudoscalar-Baryon (PB) and Vector-Baryon(VB)
channels. However, after taking into account the HQSS, the interactions
between PB and VB are almost the same except for the Clebsch-Gordan
coefficients. On the other hand, the bound states of $\bar{D}\Xi_{c}$ and
$\bar{D}^{*}\Xi_{c}$ decay mostly to the channels of $\bar{D}_{s}\Lambda_{c}$
and $\bar{D}^{*}_{s}\Lambda_{c}$, respectively, which have quite large partial
decay widths and the branching ratios (see the results in Tables 3 and 4),
whereas, the cases of the $\bar{D}^{(*)}\Sigma_{c}$ bound states Xiao:2020frg
cannot decay to the channels $\bar{D}^{(*)}\Lambda_{c}$ with the vector meson
exchange. The reason is that the $\Lambda_{c}$ particle is in the same spin
1/2 antitriplet (the $\bar{3}$ multi-states) with the $\Xi_{c}$ particle in
the SU(3) flavour symmetry, but not with the $\Sigma_{c}$ particle (belonging
to the $6$ multi-states).

As discussed above, when we take the new value of $a_{\mu}(\mu=1\textrm{
GeV})=-1.94$ for the only free parameter in the loop functions, the loose
bound systems of $\bar{D}\Xi^{\prime}_{c}$, $\bar{D}^{*}\Xi^{\prime}_{c}$,
$\bar{D}^{*}\Xi^{*}_{c}$ in $J=1/2,~{}I=0$ sector, and
$\bar{D}^{*}\Xi^{\prime}_{c}$, $\bar{D}\Xi^{*}_{c}$, $\bar{D}^{*}\Xi_{c}^{*}$
in $J=3/2,~{}I=0$ sector have become unstable with the poles moving to the
thresholds, except for three strongly bound ones of $\bar{D}\Xi_{c}$ and
$\bar{D}^{*}\Xi_{c}$. Note that, all of them were stable bound states in the
results of Ref. Xiao:2019gjd  with $a_{\mu}(\mu=1\textrm{ GeV})=-2.09$, and
even more bound with $a_{\mu}(\mu=1\textrm{ GeV})=-2.3$ used in Refs.
Wu:2010jy ; Wu:2010vk , taking the one of $\bar{D}^{*}\Xi_{c}$ for example,
the pole at 4370 MeV, which had been below the threshold of the
$\bar{D}^{*}_{s}\Lambda_{c}$ channel. Indeed, the reduced $a_{\mu}$ leads to
all of the poles less bound, and thus, some of the loosely bound states moved
to the thresholds, see the left part of Fig. 4, where we plot the real parts
of the loop functions $G$ with different $a_{\mu}$ and the inverse potential
$\frac{1}{V}$, taking the $\bar{D}^{*}\Xi_{c}$ channel for example (denoted as
the channel 6 in the figure), see more discussions about the pole moving by
the free parameter of the loop functions in Refs. Sekihara:2012xp ;
Ahmed:2020kmp . Furthermore, there is a strong coupling between the
$\bar{D}^{*}_{s}\Xi_{c}$ (the channel 6 in the figure) and
$\bar{D}^{*}_{s}\Lambda_{c}$ (the channel 7 in the figure) channels, and then
such off-diagonal transition potential will strengthen the attractive
interaction from pure diagonal potential $V_{66}$ to the “effective” potential
of $V_{66}+V^{2}_{67}G_{77}$ owing to $V_{77}=\mu_{3}=0$ (see Table 1) . In
the right panel of Fig. 4, we compare the real parts of the loop functions $G$
with the one of $1/(V_{66}+V^{2}_{67}G_{77})$ (taking $a_{\mu}=-1.94$), where
the cross point is just the pole position. Thus, one can expect that the
$\bar{D}^{*}\Xi_{c}$ system will be more bound when $a_{\mu}(\mu=1\textrm{
GeV})=-2.3$ are taken as in Refs. Wu:2010jy ; Wu:2010vk , which is analogous
to the case of the $\bar{D}\Xi_{c}$ system with the contribution from the
$\bar{D}_{s}\Lambda_{c}$ channel .

Figure 4: Results of the potential $\frac{1}{V}$ versus the loop functions for
the $\bar{D}^{*}\Xi_{c}$ channel, where the vertical line locates at the
threshold.

##  IV Conclusion

In summary, we revisit the interactions of the $J/\psi\Lambda$ channel and its
coupled channels in $s$ wave, using the CCUA in combination with HQSS and LHG
symmetry. With the new observation of the $P_{cs}(4459)$ state by LHCb to fix
the only free parameter, we obtain a pole of $(4459.07+i6.89)\textrm{ MeV}$
located below the threshold of the $\bar{D}^{*}\Xi_{c}$ channel, which
represents effectively the two nearly degenerate states with spin parity
$J^{P}=\frac{1}{2}^{-}$ and $J^{P}=\frac{3}{2}^{-}$ in our approach, which
keeps in the potential only the constant leading term by dropping the momentum
dependent terms. To remove the degeneracy, we will make further investigations
in the future by taking into account the momentum dependent terms, including
the pion exchange mechanism. Thus, by assuming the LHCb observed
$P_{cs}(4459)$ peak to be the degenerated $\bar{D}^{*}\Xi_{c}$ molecular
states from our formalism, it would split to two peaks with higher statistics
in the future, like the previous $P_{c}(4450)$ peak observed by LHCb. From our
results, there is also a $\bar{D}\Xi_{c}$ bound state with
$J^{P}=\frac{1}{2}^{-}$ and a pole at $(4310.53+i8.23)\textrm{ MeV}$, which is
consistent with the predictions of Ref. Chen:2020uif  within the
uncertainties. Owing to the uncertainties of the experimental data and hence
their corresponding constraint to the free parameter, the possible loosely
bound states of $\bar{D}\Xi^{\prime}_{c}$, $\bar{D}^{*}\Xi^{\prime}_{c}$,
$\bar{D}^{*}\Xi^{*}_{c}$ with $J=1/2,~{}I=0$ and
$\bar{D}^{*}\Xi^{\prime}_{c}$, $\bar{D}\Xi^{*}_{c}$, $\bar{D}^{*}\Xi_{c}^{*}$
with $J=3/2,~{}I=0$, which were predicted previously, are in fact suffering a
large model dependence, since there are no coupled channels to strengthen the
attractive interaction. Thus, unfortunately even their existence is now put
into question in our present work. Although their LHG potentials are
attractive, whether they are bound states or virtual states would sensitively
depend on the model parameter and neglected momentum dependent terms. These
results are consistent with recent general analysis in Ref. (Dong:2021juy, ).
To look for these molecular states, especially for the $\bar{D}\Xi_{c}$ bound
state and two $\bar{D}^{*}\Xi_{c}$ states (corresponding to the observed
$P_{cs}(4459)$ peak), the decay channels of $\bar{D}_{s}\Lambda_{c}$ and
$\bar{D}^{*}_{s}\Lambda_{c}$, respectively, are strongly suggested due to
their large decay branching ratios. Also searching for them both in the
$J/\psi\Lambda$ and $\eta_{c}\Lambda$ channels can be helpful to distinguish
their different nature. We hope that future experiments, the Run-3 in LHCb for
example, can make further test on our predictions and suggestions to reveal
the properties of the $P_{cs}$ states.

## Acknowledgments

We thank useful discussions and valuable comments from Bo Fang about the
experimental information, and acknowledge Eulogio Oset for useful comments and
careful reading the paper. This work is partly supported by the Fundamental
Research Funds for the Central Universities (J.J.Wu), and NSFC under Grant No.
12070131001 (CRC110 cofunded by DFG and NSFC), Grant No. 11835015, No.
12047503, and by the Chinese Academy of Sciences (CAS) under Grants No.
XDB34030000 (B. S. Zou).

## References

  * (1) J. A. Oller and E. Oset, Nucl. Phys. A 620, 438-456 (1997) [erratum: Nucl. Phys. A 652, 407-409 (1999)] [arXiv:hep-ph/9702314 [hep-ph]].
  * (2) E. Oset and A. Ramos, Nucl. Phys. A 635, 99-120 (1998) [arXiv:nucl-th/9711022 [nucl-th]].
  * (3) J. A. Oller and U.-G. Meißner, Phys. Lett. B 500, 263 (2001),
  * (4) M. Bando, T. Kugo, S. Uehara, K. Yamawaki and T. Yanagida, Phys. Rev. Lett. 54, 1215 (1985).
  * (5) M. Bando, T. Kugo and K. Yamawaki, Phys. Rept. 164, 217-314 (1988).
  * (6) U.-G. Meißner, Phys. Rept. 161, 213 (1988).
  * (7) H. Nagahiro, L. Roca, A. Hosaka and E. Oset, Phys. Rev. D 79, 014015 (2009) [arXiv:0809.0943 [hep-ph]].
  * (8) J. J. Wu, R. Molina, E. Oset and B. S. Zou, Phys. Rev. Lett. 105, 232001 (2010) [arXiv:1007.0573 [nucl-th]].
  * (9) J. J. Wu, R. Molina, E. Oset and B. S. Zou, Phys. Rev. C 84, 015202 (2011) [arXiv:1011.2399 [nucl-th]].
  * (10) W. L. Wang, F. Huang, Z. Y. Zhang and B. S. Zou, Phys. Rev. C 84, 015203 (2011). [arXiv:1101.0453 [nucl-th]].
  * (11) Z. C. Yang, Z. F. Sun, J. He, X. Liu and S. L. Zhu, Chin. Phys. C 36, 6 (2012) [arXiv:1105.2901 [hep-ph]].
  * (12) S. G. Yuan, K. W. Wei, J. He, H. S. Xu and B. S. Zou, Eur. Phys. J. A 48, 61 (2012). [arXiv:1201.0807 [nucl-th]].
  * (13) J. J. Wu, T.-S. H. Lee and B. S. Zou, Phys. Rev. C 85, 044002 (2012) [arXiv:1202.1036 [nucl-th]].
  * (14) C. García-Recio, J. Nieves, O. Romanets, L. L. Salcedo and L. Tolos, Phys. Rev. D 87, 074034 (2013) [arXiv:1302.6938 [hep-ph]].
  * (15) C. W. Xiao, J. Nieves and E. Oset, Phys. Rev. D 88, 056012 (2013) [arXiv:1304.5368 [hep-ph]].
  * (16) T. Uchino, W. H. Liang and E. Oset, Eur. Phys. J. A 52, no. 3, 43 (2016). [arXiv:1504.05726 [hep-ph]].
  * (17) M. Karliner and J. L. Rosner, Phys. Rev. Lett. 115, no. 12, 122001 (2015). [arXiv:1506.06386 [hep-ph]].
  * (18) R. Aaij et al. [LHCb], Phys. Rev. Lett. 115, 072001 (2015) [arXiv:1507.03414 [hep-ex]].
  * (19) R. Aaij et al. [LHCb], Chin. Phys. C 40, no.1, 011001 (2016) [arXiv:1509.00292 [hep-ex]].
  * (20) R. Aaij et al. [LHCb], Phys. Rev. Lett. 117, no.8, 082002 (2016) [arXiv:1604.05708 [hep-ex]].
  * (21) R. Aaij et al. [LHCb], Phys. Rev. Lett. 117, no.8, 082003 (2016) [arXiv:1606.06999 [hep-ex]].
  * (22) R. Aaij et al. [LHCb Collaboration], Phys. Rev. Lett. 122, no. 22, 222001 (2019) [arXiv:1904.03947 [hep-ex]].
  * (23) A. Ozpineci, C. W. Xiao and E. Oset, Phys. Rev. D 88, 034018 (2013) [arXiv:1306.3154 [hep-ph]].
  * (24) C. W. Xiao, J. X. Lu, J. J. Wu and L. S. Geng, Phys. Rev. D 102, no. 5, 056018 (2020) [arXiv:2007.12106 [hep-ph]].
  * (25) C. W. Xiao, J. Nieves and E. Oset, Phys. Rev. D 100, no. 1, 014021 (2019) [arXiv:1904.01296 [hep-ph]].
  * (26) N. Isgur and M. B. Wise, Phys. Lett. B 232, 113 (1989).
  * (27) M. Neubert, Phys. Rept. 245, 259 (1994) [hep-ph/9306320].
  * (28) A.V. Manohar and M.B. Wise, Heavy Quark Physics, Cambridge Monographs on Particle Physics, Nuclear Physics and Cosmology, vol. 10 (2000).
  * (29) C. W. Xiao, J. Nieves and E. Oset, Phys. Lett. B 799, 135051 (2019) [arXiv:1906.09010 [hep-ph]].
  * (30) H. X. Chen, L. S. Geng, W. H. Liang, E. Oset, E. Wang and J. J. Xie, Phys. Rev. C 93, no.6, 065203 (2016) [arXiv:1510.01803 [hep-ph]].
  * (31) E. Santopinto and A. Giachino, Phys. Rev. D 96, no.1, 014014 (2017) [arXiv:1604.03769 [hep-ph]].
  * (32) A. Feijoo, V. K. Magas, A. Ramos and E. Oset, Eur. Phys. J. C 76, no.8, 446 (2016) [arXiv:1512.08152 [hep-ph]].
  * (33) J. X. Lu, E. Wang, J. J. Xie, L. S. Geng and E. Oset, Phys. Rev. D 93, 094009 (2016) [arXiv:1601.00075 [hep-ph]].
  * (34) C. W. Shen, J. J. Wu and B. S. Zou, Phys. Rev. D 100, no.5, 056006 (2019) [arXiv:1906.03896 [hep-ph]].
  * (35) R. Chen, J. He and X. Liu, Chin. Phys. C 41, no.10, 103105 (2017) [arXiv:1609.03235 [hep-ph]].
  * (36) B. Wang, L. Meng and S. L. Zhu, Phys. Rev. D 101, no.3, 034018 (2020) [arXiv:1912.12592 [hep-ph]].
  * (37) R. Aaij et al. [LHCb], [arXiv:2012.10380 [hep-ex]].
  * (38) H. X. Chen, W. Chen, X. Liu and X. H. Liu, [arXiv:2011.01079 [hep-ph]].
  * (39) Z. G. Wang, [arXiv:2011.05102 [hep-ph]].
  * (40) K. Azizi, Y. Sarac and H. Sundu, arXiv:2101.07850 [hep-ph].
  * (41) U. Özdem, [arXiv:2102.01996 [hep-ph]].
  * (42) F. Z. Peng, M. J. Yan, M. Sánchez Sánchez and M. P. Valderrama, [arXiv:2011.01915 [hep-ph]].
  * (43) R. Chen, [arXiv:2011.07214 [hep-ph]].
  * (44) R. Chen, [arXiv:2101.10614 [hep-ph]].
  * (45) M. Z. Liu, Y. W. Pan and L. S. Geng, [arXiv:2011.07935 [hep-ph]].
  * (46) J. T. Zhu, L. Q. Song and J. He, [arXiv:2101.12441 [hep-ph]].
  * (47) W. Y. Liu, W. Hao, G. Y. Wang, Y. Y. Wang, E. Wang and D. M. Li, [arXiv:2012.01804 [hep-ph]].
  * (48) F. K. Guo, C. Hanhart, U.-G. Meißner, Q. Wang, Q. Zhao and B. S. Zou, Rev. Mod. Phys. 90, no.1, 015004 (2018) [arXiv:1705.00141 [hep-ph]].
  * (49) H. X. Chen, W. Chen, X. Liu and S. L. Zhu, Phys. Rept. 639, 1-121 (2016) [arXiv:1601.02092 [hep-ph]].
  * (50) X. K. Dong, F. K. Guo and B. S. Zou, [arXiv:2101.01021 [hep-ph]].
  * (51) X. K. Dong, F. K. Guo and B. S. Zou, [arXiv:2011.14517 [hep-ph]].
  * (52) J. J. Wu, L. Zhao and B. S. Zou, Phys. Lett. B 709, 70-76 (2012) [arXiv:1011.5743 [hep-ph]].
  * (53) T. Sekihara and T. Hyodo, Phys. Rev. C 87, no.4, 045202 (2013) [arXiv:1209.0577 [nucl-th]].
  * (54) H. A. Ahmed and C. W. Xiao, Phys. Rev. D 101, no. 9, 094034 (2020) [arXiv:2001.08141 [hep-ph]].


# Controllable Text Generation via Probability Density Estimation in the
Latent Space

Yuxuan Gu†,Xiaocheng Feng†‡,Sicheng Ma†,Lingyuan Zhang†,Heng Gong†, Bing Qin†‡
†Harbin Institute of Technology ‡ Peng Cheng Laboratory
<EMAIL_ADDRESS>

###### Abstract

Previous work on controllable text generation has explored the idea of control
from the latent space, such as optimizing a representation with attribute-
related classifiers or sampling a representation from relevant discrete
samples. However, they are not effective enough in modeling both the latent
space and the control, leaving controlled text with low quality and diversity.
In this work, we propose a novel control framework using probability density
estimation in the latent space. Our method utilizes an invertible
transformation function, the Normalizing Flow, that maps the complex
distributions in the latent space to simple Gaussian distributions in the
prior space. Thus, we can perform sophisticated and flexible control in the
prior space and feed the control effects back into the latent space owing to
the one-one-mapping property of invertible transformations. Experiments on
single-attribute controls and multi-attribute control reveal that our method
outperforms several strong baselines on attribute relevance and text quality
and achieves the SOTA. Further analysis of control strength adjustment
demonstrates the flexibility of our control strategy111 Work in
progress.222https://github.com/HappyGu0524/MultiControl.

##  1 Introduction

Controllable text generation, a fundamental issue in natural language
generation, refers to generating fluent and attractive sentences conditioned
on target attributes Zhang et al. (2022a). With the development of pre-trained
language models Radford et al. (2019), early work explores converting
generative language models to conditional models by altering their parameters
using fine-tuning Ziegler et al. (2019); Keskar et al. (2019) or reinforcement
learning Khalifa et al. (2020). Due to the high cost of modifying parameters
Brown et al. (2020); Zhang et al. (2022b), there exist control approaches that
leave language models fixed Dathathri et al. (2020); Krause et al. (2021).

Figure 1: Illustration of methods controlling in Latent Space. Orange
background denotes the latent space. Blue and red represent two attributes.
Prefix-Tuning represents attributes with points in the manifold and composes
them by interpolation. LatentOps uses classifiers to continuously estimate
distributions of attributes and control by optimizing in the manifold.
Discrete maps sentences to discrete samples in latent space and controls with
direct searching. Our method deploys probability density estimation by
transforming complex latent space into well-formed prior space, where common
control strategies can be more effective.

Recent studies achieve satisfying control effects by influencing the fixed
language model from the latent space Yu et al. (2021); Qian et al. (2022) with
prefix-tuning Li and Liang (2021). However, modeling the complex latent space
remains a problem that plagues control performance. As in the left part of
Figure 1, Gu et al. (2022b) provide a distributional perspective to reveal
that distributions of attributes in high dimensional latent space are usually
asymmetric and even non-convex, making simple control strategies such as
interpolation Qian et al. (2022) and optimization Liu et al. (2022)
inefficient. For example, interpolation may often exceed the support set of
distributions, making generated sentences unable to acquire desired
attributes. Besides, the optimization process can stuck in the saddle or local
optimal points. Although mitigating the problem by discrete modeling and
direct searching, Gu et al. (2022b) accidentally leave the control process
more complicated that searching for intersection of attributes suffers from
the high-dimensionality of space and noise in samples.

In this paper, we are dedicated to better modeling the latent space and
control methods. As in right of Figure 1, we propose probability density
estimation in latent space by invertible transformation. Complex distributions
of attributes in latent space are mapped (bijection between continuous spaces)
to simple ones, such as Gaussian distributions, in prior space. Thus,
traditional control strategies such as interpolation can be tractable and
explainable in this normalized prior space. Then we explore the relationship
between these two spaces and attempt to prove under what circumstances the
control in the prior space can be effectively fed back to the latent space.
Finally, our controllable text generation process becomes: control attributes
in prior space, activate the language model in latent space, and take effect
on sentences in language space.

We conduct experiments on single-attribute control and multi-attribute
control. Datasets we used are IMDb movie reviews Maas et al. (2011) for
Sentiment, AGNews Zhang et al. (2015) for Topic, and Jigsaw Toxic Comment
Classification Challenge Dataset for Detoxification. We measure the control
ability of our method using the correlation of generated sentences with each
attribute. For generation quality, we evaluate sentences with perplexity and
distinctness concerning fluency and diversity. Results show that our method
can significantly outperform baseline models and analytical experiments on
control strength adjustment reveal our flexibility. The main contributions of
our work are summarized as follows:

  * •

We propose a novel framework for controllable text generation that introduces
a well-formed prior space for tractable and flexible control via invertible
transformation.

  * •

We theoretically explore approaches to exploit invertibility to feed control
in prior space back into latent space.

  * •

We experimentally reveal the effectiveness of our method compared to strong
baselines and achieve the SOTA.

##  2 Related Work

###  2.1 Controllable Text Generation

Variational autoencoders are often used for controllable text generation Hu et
al. (2017); Duan et al. (2020); Mai et al. (2020) before the prosperity of
large-scale pre-trained language models Radford et al. (2019). Traditional
control approaches like fine-tuning Ficler and Goldberg (2017); Ziegler et al.
(2019); Keskar et al. (2019) and reinforcement learning Khalifa et al. (2020)
gradually become infeasible with the rapid increase of language models’
parameters. Recent methods investigate control with fixed language models,
including biasing the token distribution during decoding Dathathri et al.
(2020); Krause et al. (2021); Yang and Klein (2021); Liu et al. (2021a); Gu et
al. (2022a); Meng et al. (2022), optimization in the language space Kumar et
al. (2021); Qin et al. (2022); Mireshghallah et al. (2022); Kumar et al.
(2022), and optimization in the latent space Yu et al. (2021); Qian et al.
(2022); Carlsson et al. (2022); Yang et al. (2022); Liu et al. (2022); Lu et
al. (2022); Zhang and Song (2022); Gu et al. (2022b). There is also work that
trains a diffusion language model before controlling sentence attributes in
the denoising process Li et al. (2022).

###  2.2 Normalizing Flow

The Normalizing Flow Dinh et al. (2014, 2016); Kingma and Dhariwal (2018);
Kingma et al. (2016); Papamakarios et al. (2017), consisting of a sequence of
invertible transformations for continuous variables, is a powerful deep
generative model Kingma and Welling (2013); Goodfellow et al. (2020); Ho et
al. (2020) that enables capturing the inner probabilistic distribution of
complex and high-dimensional data Oussidi and Elhassouny (2018), including
images and text. In natural language processing, Normalizing Flows are often
used as enhanced prior distributions in VAE structures Ma et al. (2019); Ding
and Gimpel (2021) or as deep generative language models Tran et al. (2019);
Ziegler and Rush (2019); Tang et al. (2021). Besides, Wu et al. (2022) uses
the Normalizing Flow as prefix-tuning for controllable image generation.
However, previous work usually treats Normalizing Flow as an ordinary
generative model, easily replaced by stronger models like the denoising
diffusion model Ho et al. (2020), while ignoring its invertible property. In
this work, we will explore the potential for the flexible application of the
Normalizing Flow’s invertible feature in controllable text generation.

Figure 2: An overview of our framework. Part  1 represents the process of
estimating latent space, which trains an autoencoder framework by prefix-
tuning on the fixed language model. Part  2 denotes the invertible
transformation from latent space to prior space at the training stage. Part  3
consists of two steps: we first operate in prior space for control and then
feed the effect back into latent space to activate the language model.

##  3 Methodology

As illustrated in Figure 2, our framework is divided into three parts, where
the former two are training phases, and the latter is the generation phase.

###  3.1 Estimating the Latent Space

Given sentence and attribute pairs $\\{(s_{i},a_{i})\\}$, we use a learnable
encoder to map each sentence to a sample point $x_{i}\in\mathbb{R}^{n\times
1}$, which can activate the fixed language model to reconstruct the same
sentence afterward via prefix-tuning. We denote the training loss of this
reconstruction target as:

$\displaystyle\mathcal{L}_{R}$ $\displaystyle=\sum\nolimits_{i}\log
p_{\text{LM}}(s_{i}|\text{Prefix}_{i})$ $\displaystyle\text{Prefix}_{i}$
$\displaystyle=\text{MLP}_{\phi}(x_{i})$ $\displaystyle x_{i}$
$\displaystyle=\text{Encode}_{\phi}(s_{i}),$

where we can regard each point $x_{i}$ as being sampled from a continuous
Latent Space. It’s worth noting that estimating the Latent Space can be a pre-
training or a pre-processing phase, in which we can directly use the
parameters and mapped points Gu et al. (2022b) provide.

###  3.2 Invertible Transformation

The invertible transformation, denoting as $z=f_{K}\circ\cdots\circ
f_{1}(x)=\mathcal{F}_{\theta}(x)$, maps points $x_{i}$ in a complex
distribution to ones $z_{i}\in\mathbb{R}^{n\times 1}$ in a simple
distribution, such as the Gaussian distribution, with a series of invertible
functions $\\{f_{i}(\cdot)\\}$. The probability density function of $p(x)$ can
be derived as
$p(x)=\pi(z)\left|\text{det}\frac{\mathrm{d}\mathcal{F}_{\theta}(x)}{\mathrm{d}x}\right|$
and the corresponding training target is: $\mathcal{L}=\sum_{x}\log
p(x)=\sum_{x}\left[\log\pi(\mathcal{F}_{\theta}(x))+\log\left|\text{det}\frac{\mathrm{d}\mathcal{F}_{\theta}(x)}{\mathrm{d}x}\right|\right]$.

For controllable text generation, we have to model the conditional probability
$p(x|a)$. Therefore, we can decompose the probability as:

$\displaystyle p(x)$ $\displaystyle=\sum\nolimits_{a}p(x|a)p(a)$
$\displaystyle\pi(z)$ $\displaystyle=\sum\nolimits_{a}\pi(z|a)p(a),$

where
$\sum\limits_{a}p(x|a)p(a)\\!=\\!\sum\limits_{a}\pi(z|a)p(a)\left|\text{det}\frac{\mathrm{d}\mathcal{F}_{\theta}(x)}{\mathrm{d}x}\right|$.
This means distributions of different attributes in Latent Space are mapped to
the distributions in Prior Space through the same invertible transformation.
When all sentences possess labels in every attribute, which is an ideal
supervised situation, we can obtain attribute distributions and their
correlations. However, we usually encounter a semi-supervised situation where
each sentence only has a single attribute label. As a result, we set a
stricter transformation constraint for semi-supervision that
$p(x|a)=\pi(z|a)\left|\text{det}\frac{\mathrm{d}\mathcal{F}_{\theta}(x)}{\mathrm{d}x}\right|$.
Our target becomes:

$\displaystyle\mathcal{L}=\sum\limits_{(x,a)}\log p(x|a)$
$\displaystyle=\sum\limits_{(x,a)}\left[\log\pi(\mathcal{F}_{\theta}(x)|a)+\log\left|\text{det}\frac{\mathrm{d}\mathcal{F}_{\theta}(x)}{\mathrm{d}x}\right|\right].$

In this case, we train each attribute independently under the same spatial
mapping, where attribute correlations in Latent Space can still be revealed by
operation in Prior Space. It’s worth noting that the amount of training data
for different attributes should be consistent as possible to ensure the
balance of the transformation. Besides, for the convenience of control, we set
covariance matrices $\mathbf{\Sigma}\in\mathbb{R}^{n\times n}$ of prior
distributions as diagonal matrices $\sigma^{2}=\sigma\sigma^{T}\mathbf{I}$,
where $\pi(z|a)=\mathcal{N}(\mu_{a},\sigma_{a}^{2})$.

###  3.3 Control in the Prior Space

In this part, we will first prove three significant properties that bridge the
prior and latent spaces, and then we will introduce how to conduct flexible
control in the prior space and feed the control effect back into the latent
space.

####  3.3.1 Theoretical Support for Control

##### Attribute Preservation

We define $z$ possesses the attribute $a$ as in the support set of $\pi(z|a)$,
$z\in\text{supp}(\pi_{a})$, which equals to $\forall z,\pi(z|a)\\!>\\!0$.
Thus:

$\displaystyle\exists
x,x=\mathcal{F}_{\theta}^{-1}(z),z\in\text{supp}(\pi_{a})$
$\displaystyle\Rightarrow
p(x|a)=\pi(z|a)\left|\text{det}\frac{\mathrm{d}\mathcal{F}_{\theta}^{-1}(z)}{\mathrm{d}z}\right|^{-1}>0$
$\displaystyle\Rightarrow x\in\text{supp}(x),$

which means that sampling in Prior Space contributes to sampling in Latent
Space.

##### Intersection Invertibility

The intersection area of multiple attributes
${a_{1},\cdots,a_{d}},d\\!\leq\\!n+1$, can be defined as the overlapping of
their probability density functions
$\\{z|\min\left\\{\pi(z|a_{1}),\\!\cdots\\!,\pi(z|a_{d})\right\\}\\!>\\!0\\}$.
In addition, the point where attributes are most tightly combined is
considered center of the intersection:
$z^{*}\\!=\\!\text{argmax}_{z}\min\\{\pi(z|a_{1}),\\!\cdots\\!,\pi(z|a_{d})\\}$.
Though there does not necessarily exist a mapping from $z^{*}$ to the
intersection center in Latent Space, we can restrict the region of this
mapping to an upper bound. Since $z^{*}$ lies in the $n\text{-}d\text{+}1$
dimensional subspace
$\mathcal{I}\\!=\\!\\{z|\pi(z|a_{1})\\!=\\!\cdots\\!=\\!\pi(z|a_{d})\\}$,
named as Intersection Subspace, we can have:

$\displaystyle\forall\hat{z}$
$\displaystyle\in\mathcal{I},\exists\hat{x}=\mathcal{F}_{\theta}^{-1}(\hat{z}),$
$\displaystyle p(\hat{x}|a_{i})$
$\displaystyle=\pi(\hat{z}|a_{i})\left|\text{det}\frac{\mathrm{d}\mathcal{F}_{\theta}^{-1}(\hat{z})}{\mathrm{d}\hat{z}}\right|^{-1}$
$\displaystyle=\pi(\hat{z}|a_{j})\left|\text{det}\frac{\mathrm{d}\mathcal{F}_{\theta}^{-1}(\hat{z})}{\mathrm{d}\hat{z}}\right|^{-1}$
$\displaystyle=p(\hat{x}|a_{j}),1\\!\leq\\!i\\!\leq\\!d,1\\!\leq\\!j\\!\leq\\!d,$

which means that the intersection subspace in Prior Space corresponds to the
subspace in Latent Space.

##### Inequality Maintenance

We define the discrepancy between two attributes concerning the control
strength as $\mathcal{D}_{a_{1}||a_{2}}(x)=p(x|a_{1})-p(x|a_{2})$, measuring
the degree of their mutual exclusion. Thus:

$\displaystyle\mathcal{D}_{a_{1}||a_{2}}(x)=p(x|a_{1})-p(x|a_{2})$
$\displaystyle\ \
=(\pi(z|a_{1})-\pi(z|a_{2}))\left|\text{det}\frac{\mathrm{d}\mathcal{F}_{\theta}^{-1}(z)}{\mathrm{d}z}\right|^{-1}$
$\displaystyle\ \
=\mathcal{D}_{a_{1}||a_{2}}(z)\left|\text{det}\frac{\mathrm{d}\mathcal{F}_{\theta}^{-1}(z)}{\mathrm{d}z}\right|^{-1}$
$\displaystyle\Rightarrow\exists\hat{z},\mathcal{D}_{a_{1}||a_{2}}(\hat{z})=0$
$\displaystyle\left\\{\begin{aligned} &\forall
z\\!>\\!\hat{z},\mathcal{D}_{a_{1}||a_{2}}(z)>0,\mathcal{D}_{a_{1}||a_{2}}(x)>0\\\
&\forall
z\\!<\\!\hat{z},\mathcal{D}_{a_{1}||a_{2}}(z)<0,\mathcal{D}_{a_{1}||a_{2}}(x)<0\end{aligned}\right.,$

which means inequality of two attributes in Prior Space is also true in Latent
Space.

####  3.3.2 Details for Control

Figure 3: Intersection of two distributions. In the case of isotropy, the
intersection of two Gaussian distributions is also a Gaussian distribution,
where interpolation of two distribution centers can lie at the intersection
center.

Given the Attribute Preservation property above, sampling a point $x_{a}$
related to attribute $a$ in the Latent Space is equivalent to sampling in
Prior Space $z_{a}\sim\mathcal{N}(\mu_{a},\sigma_{a}^{2})$ and then
transforming as $x_{a}=\mathcal{F}_{\theta}^{-1}(z_{a})$. For single-attribute
control, we convert the sampling strategy to
$z_{a}=\mu_{a}+\sigma_{a}\epsilon,\epsilon\sim\mathcal{N}(\mathbf{0},\lambda^{2}\mathbf{I})$,
where $\lambda$ is a hyperparameter333We will discuss how $\lambda$ influences
control strength in §5.1..

##### Control Strength Adjustment

Given two mutually exclusive attributes, such as positive $a$ and negative
$\bar{a}$ sentiment, sampling an $\alpha$-weighted interpolated point
$\tilde{z}$ in Prior Space is $\tilde{z}=\alpha
z_{a}+\bar{\alpha}z_{\bar{a}}$, where $\alpha+\bar{\alpha}=\mathbf{1}$. This
linear combination is:

$\displaystyle\tilde{z}$
$\displaystyle=(\alpha\mu_{a}+\bar{\alpha}\mu_{\bar{a}})+(\alpha\sigma_{a}\epsilon_{a}+\bar{\alpha}\sigma_{\bar{a}}\epsilon_{\bar{a}})$
$\displaystyle=(\alpha\mu_{a}+\bar{\alpha}\mu_{\bar{a}})+\sqrt{(\alpha\sigma_{a})^{2}+(\bar{\alpha}\sigma_{\bar{a}})^{2}}\cdot\epsilon,$

which is
$\tilde{z}\sim\mathcal{N}((\alpha\mu_{a}+\bar{\alpha}\mu_{\bar{a}}),(\alpha\sigma_{a})^{2}+(\bar{\alpha}\sigma_{\bar{a}})^{2}\mathbf{I})$.
As illustrated in the upper left of Figure 3, interpolation between $\mu_{a}$
and $\mu_{\bar{a}}$ equals a line ($1$-dimensional subspace) in Prior Space
that passes through the Intersection Subspace, where the intersection point is
$\hat{z}\\!=\\!\alpha^{*}\mu_{a}\\!+\\!\bar{\alpha}^{*}\mu_{\bar{a}}$.444See
Appendix A for the calculation of $\hat{z}$. Therefore, sampling with
$\hat{z}$ as the center has a great opportunity to sample from the
Intersection Subspace in Prior Space, approximate to sampling from the
Intersection Subspace in Latent Space based on Intersection Invertibility. It
is worth noting that when distributions are isotropic, there is
$\hat{z}\\!=\\!z^{*}$ as in Figure 3, which improves the effect of
interpolation. The Inequality Maintenance further ensures that
$\alpha\\!>\\!\alpha^{*}\\!\iff\\!p(\mathcal{F}_{\theta}^{-1}(z^{*})|a)\\!>\\!p(\mathcal{F}_{\theta}^{-1}(z^{*})|\bar{a})$,
which means that positive sentiment is guaranteed to be more powerful than
negative as long as our weight is larger than $\alpha^{*}$. However, the
monotonicity of control can not be theoretically guaranteed, which is
different from intuition. Fortunately, our experiment in §5.1 demonstrates
that the control strength can be monotonic at a coarse granularity. When
trading off control strength between two polarities, $\alpha$ is usually
ranging from $0$ to $1$. We can even make $\alpha$ slightly larger than $1$,
which means extending the control strength, as long as it can be guaranteed
that the points sampled are still within their distribution.

##### Multi-Attribute Control

Due to the spatial symmetry of the Gaussian distribution, when we change the
covariance matrix to a diagonal matrix, the trained distribution is even
approximately isotropic555See Appendix B for approximately isotropic.. This
means we can simply deploy the interpolation of each attributes as:

$\displaystyle z_{i}$
$\displaystyle=\mu_{i}+\sigma_{i}\epsilon_{i},\sum\nolimits_{i}\alpha_{i}=1,1\\!\leq\\!i\\!\leq\\!d$
$\displaystyle\tilde{z}$
$\displaystyle=\sum\nolimits_{i}(\alpha_{i}\mu_{i})+\sum\nolimits_{i}(\alpha_{i}\sigma_{i}\epsilon_{i})$
$\displaystyle=\sum\nolimits_{i}(\alpha_{i}\mu_{i})+\sqrt{\sum\nolimits_{i}(\alpha_{i}\sigma_{i})^{2}}\cdot\epsilon$
$\displaystyle=\mathcal{N}((\sum\nolimits_{i}(\alpha_{i}\mu_{i}),\sum\nolimits_{i}(\alpha_{i}\sigma_{i})^{2}\mathbf{I})$

##  4 Experimtents

Methods |  Sentiment↑ (%) |  Topic↑ (%) |  Detox.↑ | PPL.↓ | Dist.-1/2/3↑
---|---|---|---|---|---
Avg. | Neg. | Pos. | Avg. | W. | S. | B. | T. | (%)
Biasing during Decoding |  |  |  |  |  |
PPLM | 80.0 | 97.2 | 62.7 | 70.6 | 74.9 | 46.5 | 62.4 | 98.6 | 93.2 | 63.2 | 31.1 / 70.9 / 85.9
GeDi | 82.3 | 93.9 | 70.7 | 83.2 | 73.4 | 85.7 | 75.7 | 98.0 | 94.9 | 81.6 | 38.1 / 74.0 / 78.4
GeDi raw | 88.4 | 96.6 | 80.2 | 90.8 | 84.3 | 92.6 | 87.1 | 99.2 | 95.4 | 134.1 | 47.5 / 88.9 / 93.0
Optimization in the Language Space |  |  |  |  |  |
MUCOCO | 75.4 | 95.5 | 55.3 | 73.5 | 56.9 | 67.3 | 72.3 | 97.5 | 94.8 | 381.7 | 22.5 / 49.9 / 64.3
Mix&Match | 82.8 | 99.2 | 63.3 | 75.6 | 79.5 | 57.4 | 69.6 | 99.3 | 96.9 | 65.2 | 31.5 / 74.8 / 88.8
Optimization in the Latent Space |  |  |  |  |  |
Prefix | 81.6 | 86.8 | 76.4 | 82.4 | 72.2 | 81.1 | 84.9 | 91.5 | 88.3 | 20.8 | 16.3 / 43.8 / 67.5
Con. Prefix | 89.5 | 88.4 | 90.6 | 86.7 | 74.5 | 85.3 | 93.5 | 93.6 | 93.8 | 37.7 | 17.3 / 47.0 / 71.1
Discrete | 92.5 | 99.1 | 85.9 | 89.4 | 80.9 | 94.7 | 84.6 | 97.5 | 90.1 | 46.1 | 37.2 / 77.5 / 87.2
PriorControl | 97.1 | 99.9 | 94.3 | 95.9 | 95.5 | 99.3 | 90.2 | 98.7 | 90.7 | 54.3 | 29.1 / 70.1 / 86.9
\+ extend | 99.7 | 99.9 | 99.5 | 97.8 | 97.9 | 99.4 | 94.0 | 99.8 | 95.7 | 54.6 | 29.8 / 70.5 / 86.8
Table 1: Automatic Results on Single-Attribute Control. We control on
Sentiment (Negative and Positive), Topic (World, Sports, Business, and
Science/Technology), and Detoxification independently.

###  4.1 Tasks and Baselines

##### Tasks

All our experimental setups, including datasets, evaluation metrics, and
generation configurations, follow Gu et al. (2022b) for fair comparisons.
There are IMDb movie reviews Maas et al. (2011), AGNews dataset Zhang et al.
(2015), and Jigsaw Toxic Comment Classification Challenge
Dataset666https://www.kaggle.com/c/jigsaw-toxic-comment-classification-
challenge/ for $2$ sentiments, $4$ topics, and $1$ detoxification,
respectively. It’s worth noting that Gu et al. (2022b) randomly samples 10k
sentences from each dataset, constituting a minor subset, to balance the data
scale for the latent space construction. And we directly use this latent space
to make a fair comparison. To evaluate the attribute relevance, we use
classifiers trained by Gu et al. (2022b) for sentiment and topic, and we
utilize the Google Perspective API777https://www.perspectiveapi.com for
detoxification. We also measure text quality with Perplexity and
DistinctnessLi et al. (2016). There are $35$ prompts used for text generation,
as in PPLM Dathathri et al. (2020). For single-attribute control, models will
generate $5$ completions for each attribute and each prompt, which are
$35\times(2+4+1)\times 5=1225$ sentences. For multi-attribute control, each
model generates $35\times(2\times 4\times 1)\times 5=1400$ sentences.

##### Baselines

(I) Biasing during Decoding: PPLM Dathathri et al. (2020) accumulates
gradients from classifiers as bias signals to influence the language model.
GeDi Krause et al. (2021) biases the decoding process with small conditional
generative models.  (II) Optimization in Language Space: MUCOCO Kumar et al.
(2021) converts the decoding process to multi-objective optimization in
language space. Mix&Match Mireshghallah et al. (2022) discretely optimizes the
sentence in language space by token-level masking and resampling.  (III)
Optimization in Latent Space: Prefix Liu et al. (2021b) is the original
Prefix-Tuning method which activates the language model to generate attribute-
relevant sentences with tunable prefixes. Contrastive Prefix Qian et al.
(2022) enhances the prefixes through contrastive learning. Discrete Gu et al.
(2022b) uses discrete samples to represent the distribution of attributes in
latent space and controls the generation by sampling in relevant areas.

###  4.2 Single-Attribute Control

We demonstrate the automatic evaluation results on single-attribute contol in
Table 1. In addition to the degree of each independent attribute relevance, we
compute their average for Sentiment and Topic. Models are grouped with their
types of approaches.

We mainly compare the control methods in the latent space, and the other two
technical routes serve as supplementary references. Biasing methods can
achieve decent control at the cost of some fluency. The diversity of their
generated sentences is almost the same as the language model, owing to their
plug-and-play property during decoding. Besides, we illustrate the raw GeDi
without retraining, which is trained on the superset of our dataset. Results
show that its performance is affected by the amount of data to some extent.
Optimization methods in language space, elegant in theory, are often troubled
by high dimensionality when implemented, making it difficult to obtain
favorable control effects and text fluency. Optimization in latent space is a
compromise strategy where the space dimension is relatively reduced, making
the control process more robust. However, they also suffer from low diversity
and dumb control.

Figure 4: The probability density function of two exclusive attributes.
$\pi(z|a)=\mathcal{N}(0,1)$, $\pi(z|\bar{a})=\mathcal{N}(1.5,1)$, and
$\hat{\pi}(z|\bar{a})=\mathcal{N}(\text{-}0.2,1)$. Above:
$P(\mathcal{D}_{a||\bar{a}}(z)\\!>\\!0)\\!\approx\\!0.773$,
$E_{a||\bar{a}}(z^{*})\\!\approx\\!0.161$; Below:
$P(\mathcal{D}_{a||\bar{a}}(z)\\!>\\!0)\\!\approx\\!0.829$.
$E_{a||\bar{a}}(z^{*})\\!\approx\\!0.171$.

Our method not only enhances the existing latent space optimization method at
the level of control strength, with at least 5.0% and 7.3% significant
improvement over baselines on sentiment and topic. For text quality, our
model, sampling points from a Gaussian distribution, can also exceed the
original prefix tuning method by 20.5 in the average distinctness. Our method
performs mediocre on detoxification because we directly use Discrete’s latent
space, which is not good at this task. Compared with Discrete, which assigns
the same weight to different sample points, our method can be seen as sampling
from the area where sample points possess higher weights. This means we can
sample points with higher control strength. However, we will be relatively
concentrated on the semantic level compared with totally random sampling,
making the diversity slightly inferior to Discrete.

Fortunately, our performance can be further improved by some sophisticatedly
extended control. We can achieve opposite control, as in contrastive learning,
by using negative weights when interpolating. Figure 4 denotes a typical
situation where we sample blue points with their probability density function.
One reason for existing methods’ suboptimal control effect is that exclusive
attributes, denoted as the red distribution, interfere with desired ones, the
blue. We can use the probability of blue surpassing red
$P(\mathcal{D}_{a||\bar{a}}(z)\\!>\\!0)\\!=\\!\int_{\text{-}\infty}^{z^{*}}\pi(z|a)\mathrm{d}z$
and the expectation of the difference between blue and red
$E_{a||\bar{a}}(z^{*})\\!=\\!\int_{\text{-}\infty}^{z^{*}}\pi(z|a)(\pi(z|a)\\!-\\!\pi(z|\bar{a}))\mathrm{d}z$
to measure the anti-interference ability in the sampling process. Above of
Figure 4 shows sampling according to its original Gaussian distribution
$\mathcal{N}(0,1)$, where the surpassing probability is about 0.773 and the
expectation of difference is about 0.161. When our new blue sampling
distribution, $\hat{\pi}(z|a)\\!=\\!\mathcal{N}(\text{-}0.2,1)$, is slightly
away from the red by a distance of 0.2, as in below of Figure 4, the
surpassing probability
$\int_{\text{-}\infty}^{z^{*}}\hat{\pi}(z|a)\mathrm{d}z$ and expectation of
difference
$\int_{\text{-}\infty}^{z^{*}}\hat{\pi}(z|a)(\pi(z|a)\\!-\\!\pi(z|\bar{a}))\mathrm{d}z$
increase to 0.829 and 0.171, respectively. This means that the sampling center
farther away from interference sources possesses better confidence. Results of
this extended control feeding back to the attribute relevance are 2.6, 1.9,
and 5.0 improvements on Sentiment, Topic, and Detoxification, respectively.
It’s worth mentioning that this offset needs to be balanced between staying
away from interference and maintaining the original sampling area.

###  4.3 Multi-Attribute Control

Methods |  Average↑ (%) |  Sentiment↑ (%) |  Topic↑ (%) |  Detoxification↑ (%) |  PPL.↓ |  Dist.↑
---|---|---|---|---|---|---
Biasing during Decoding |
PPLM | 71.0 $\pm$ 21.4 | 64.7 $\pm$ 24.8 | 63.5 $\pm$ 22.7 | 84.9 $\pm$ 6.5 | 62.6 | 62.0
GeDi | 81.4 $\pm$ 14.7 | 76.1 $\pm$ 17.2 | 73.8 $\pm$ 11.3 | 94.2 $\pm$ 1.9 | 116.6 | 75.1
Optimization in the Language Space |
MUCOCO | 73.9 $\pm$ 24.1 | 65.0 $\pm$ 33.7 | 67.2 $\pm$ 18.3 | 89.5 $\pm$ 3.5 | 405.6 | 49.7
Mix&Match | 79.7 $\pm$ 21.8 | 73.5 $\pm$ 25.9 | 69.9 $\pm$ 21.1 |  95.8 $\pm$ 1.9 | 63.0 | 61.8
Optimization in the Latent Space |
Contrastive Prefix |  |  |  |  |  |
concatenation | 77.2 $\pm$ 18.5 | 67.3 $\pm$ 20.7 | 71.8 $\pm$ 16.5 | 92.6 $\pm$ 2.9 | 54.6 | 39.9
semi-supervised | 81.3 $\pm$ 16.5 | 74.4 $\pm$ 19.6 | 76.9 $\pm$ 16.7 | 92.7 $\pm$ 3.5 | 31.9 | 43.3
Discrete |  87.4 $\pm$ 10.9 |  86.7 $\pm$ 10.5 |  84.8 $\pm$ 14.2 | 90.7 $\pm$ 7.4 | 28.4 | 49.5
PriorControl |  89.9 $\pm$ 08.7 |  88.0 $\pm$ 10.6 |  87.4 $\pm$ 08.5 |  94.3 $\pm$ 3.2 | 34.7 | 55.5
Table 2: Automatic Results on Multi-Aspect Control.

Automatic evaluation results on multi-attribute control are demonstrated in
Table 2. We group methods in the same way as single-attribute control, and we
add an extra average score for all control combinations. Besides, we
demonstrate their standard deviations, which denote the stability of models
among different attribute combinations.

Compared to single-attribute control, multi-attribute control is more
challenging since all models suffer a drop in overall performance. There are
at least 6.3% and 5.1% drops in the attribute relevance for Sentiment and
Topic. There is little drop in detoxification because this attribute is
generally compatible with others. On one hand, biasing models such as GeDi
suffer from a drop not only in control strength, but also in the fluency of
the generated text, as multiple biasing signals may conflict. On the other
hand, latent optimizing approaches undergo an extra loss in diversity, even
including our model, since we have to shrink the variance of the sampling to
cut down the decline of the control effect. As observed in Discrete Gu et al.
(2022b), this gap between single-attribute control and multi-attribute control
is reasonable because different attributes usually combine at sparse edges of
their distributions, which is in line with our intuition. It can also be
observed in our mapped prior space that the probability density of the
attribute combination region is relatively small.

Compared with Discrete, whose estimated latent space we are following, our
model improves at the sentiment, topic, and detoxification by $1.5\%$,
$3.1\%$, and $4.0\%$. Besides, we can achieve better stability according to
lower standard deviation, $8.7$, on average. It is worth noting that we
outperform the Discrete in diversity for multi-attribute control because they
can only obtain a small number of points in intersection regions, while we can
sample from this continuous region.

##  5 Analysis

###  5.1 Influence of $\lambda$

During the sampling stage
$\epsilon\sim\mathcal{N}(\mathbf{0},\lambda^{2}\mathbf{I})$, we often
anticipate that the obtained points have a higher probability density, which
is influenced by $\lambda$. As mentioned in Figure 4, exclusive attributes can
interfere with the control effect, and decreasing $\lambda$ is another
optional strategy to reduce the interference. We plot the probability density
function for $\lambda=0.8$ in Figure 5. The probability of blue surpassing red
$\int_{\text{-}\infty}^{z^{*}}\tilde{\pi}(z|a)\mathrm{d}z$ is $0.826$ and the
expectation of the difference between blue and red
$\int_{\text{-}\infty}^{z^{*}}\tilde{\pi}(z|a)(\pi(z|a)\\!-\\!\pi(z|\bar{a}))\mathrm{d}z$
is $0.181$, which is larger than original scores.

Figure 5: The probability density function of two exclusive attributes when $\lambda=0.8$. $\tilde{\pi}(z|a)\\!=\\!\mathcal{N}(0,0.8^{2})$. $P(\mathcal{D}_{a||\bar{a}}(x)\\!>\\!0)\\!\approx\\!0.826$. $E_{a||\bar{a}}(z^{*})\\!\approx\\!0.181$. Control on Sentiment |  |  |
---|---|---|---
$\lambda$ | $P(\mathcal{D}\\!>\\!0)$ | $E(z^{*})$ |  Neg./Pos. | PPL. | Dist.
1.0 | 0.773 | 0.161 | 99.1 / 78.7 | 85.0 | 64.9
0.9 | 0.798 | 0.171 | 99.4 / 83.0 | 74.7 | 64.6
0.8 | 0.826 | 0.181 | 99.4 / 88.5 | 64.9 | 64.2
0.7 | 0.858 | 0.192 | 99.4 / 92.7 | 59.9 | 63.1
0.6 | 0.894 | 0.205 | 99.9 / 94.3 | 53.9 | 62.0
0.5 | 0.933 | 0.218 | 99.9 / 97.4 | 49.5 | 61.3
0.4 | 0.970 | 0.232 | 99.9 / 99.0 | 45.1 | 60.0
0.3 | 0.994 | 0.246 | 99.9 / 99.0 | 40.3 | 58.2
0.2 | 0.999 | 0.259 | 99.9 / 99.0 | 37.1 | 54.9
0.1 | 1.000 | 0.267 | 99.9 / 99.9 | 34.8 | 52.3
0.0 | 1.000 | 0.269 | 99.9 / 99.9 | 34.3 | 49.9
Table 3: Results on the $\lambda$’s influence.

Next, we analyze the effect of $\lambda$ adjustment fed back into the latent
space. As the mean is $0$, we can have
$P(\mathcal{D}_{a||\bar{a}}(z)\\!>\\!0)\\!=\\!\int_{\text{-}\infty}^{z^{*}}\tilde{\pi}(z|a)\mathrm{d}z\\!=\\!\int_{\text{-}\infty}^{z^{*}}\frac{\pi(\frac{z}{\lambda}|a)}{\lambda}\mathrm{d}z\\!=\\!\int_{\text{-}\infty}^{\frac{z^{*}}{\lambda}}\pi(z|a)\mathrm{d}z\\!=\\!\int_{\text{-}\infty}^{z^{*}}\pi(z|a)\mathrm{d}z\\!+\\!\int_{z^{*}}^{\frac{z^{*}}{\lambda}}\pi(z|a)\mathrm{d}z>\int_{\text{-}\infty}^{z^{*}}\pi(z|a)\mathrm{d}z$,
which means the probability of blue surpassing red is monotonic as the
function of $\lambda$. Besides, Inequality Maintenance we proved before
ensures that there exists no $x\\!<\\!x^{*}$ that can make
$\mathcal{F}_{\theta}(x)\\!>\\!\mathcal{F}_{\theta}(x^{*})$. Therefore, we
have
$\int_{\text{-}\infty}^{x^{*}}p(x|a)\mathrm{d}x\\!=\\!\int_{\text{-}\infty}^{z^{*}}\pi(z|a)\frac{\mathrm{d}\mathcal{F}_{\theta}(x)}{\mathrm{d}x}\mathrm{d}x\\!=\\!\int_{\text{-}\infty}^{z^{*}}\pi(z|a)\mathrm{d}z$.
As a result, decreasing $\lambda$ means enhancing control strength in latent
space as:
$\int_{\text{-}\infty}^{z^{*}}\tilde{\pi}(z|a)\mathrm{d}z\\!=\\!\int_{\text{-}\infty}^{x^{*}}p(x|a)\mathrm{d}x\\!+\\!\int_{z^{*}}^{\frac{z^{*}}{\lambda}}\pi(z|a)\mathrm{d}z\\!>\\!\int_{\text{-}\infty}^{x^{*}}p(x|a)\mathrm{d}x$.
It’s worth noting that the interval $[z^{*},\frac{z^{*}}{\lambda}]$ is not
guaranteed to correspond to the interval
$[x^{*},\mathcal{F}_{\theta}^{\text{-}1}\left(\frac{z^{*}}{\lambda}\right)]$.
For experimental results, we demonstrate the influence of $\lambda$ on
Sentiment in Table 3. Consistent with intuition, as $\lambda$ decreases, both
attribute relevance and text fluency increase while diversity declines.

###  5.2 Control Strength Adjustment

Control strength adjustment can be directly achieved with interpolation over
distribution centers under the approximately isotropic situation. We show the
effect of $\alpha$ adjustment in Table 4. Although the change of $\alpha$ at
the coarse-grained level, in the form of a Sigmoid-like curve, is
approximately continuous and monotonic, we cannot theoretically prove its
monotonicity like $\lambda$. Because the training target of invertible
transformation is the combination of probability density and Jacobian
determinant, which means higher probability in latent space will also tend to
be mapped to a higher probability in prior space. However, this tendency is
not always guaranteed, as the Jacobian determinant can compensate for some
loss in probability to obtain a better form of the mapped distribution.

$\alpha$ |  Neg $\rightarrow$ Pos (%) |  Toxic $\rightarrow$ NonTox (%)
---|---|---
1.0 | 99.2 / 00.8 | 79.5 / 20.5
0.9 | 97.1 / 02.9 | 69.4 / 30.6
0.8 | 93.5 / 06.5 | 56.2 / 43.8
0.7 | 88.5 / 11.5 | 49.3 / 50.7
0.6 | 77.7 / 22.3 | 44.2 / 55.8
0.5 | 66.4 / 33.6 | 31.7 / 68.3
0.4 | 49.7 / 50.3 | 23.6 / 76.4
0.3 | 38.1 / 61.9 | 22.0 / 78.0
0.2 | 26.5 / 73.5 | 13.1 / 86.9
0.1 | 19.1 / 80.9 |  09.7 / 90.3
0.0 | 13.7 / 86.3 |  07.4 / 92.6
Table 4: Control Strength Adjustment ($\lambda\\!=\\!1$).

##  6 Conclusion

In this work, we present a novel control framework by introducing a well-
formed prior space converted from latent space via invertible transformation.
Experimental results confirm the superiority of our model on control
effectiveness, control flexibility, and generation quality. We further provide
some theoretical support to feed the control effect from the prior space back
to the latent space. In the future, we can explore the coupling degree between
different attributes for a better combination, and we also can increase the
scalability of the prior space for customized control strategies.

## References

  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language models are few-shot learners.  In _Advances in Neural Information Processing Systems_ , volume 33, pages 1877–1901. Curran Associates, Inc.
  * Carlsson et al. (2022) Fredrik Carlsson, Joey Öhman, Fangyu Liu, Severine Verlinden, Joakim Nivre, and Magnus Sahlgren. 2022.  Fine-grained controllable text generation using non-residual prompting.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 6837–6857, Dublin, Ireland. Association for Computational Linguistics.
  * Dathathri et al. (2020) Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020.  Plug and play language models: A simple approach to controlled text generation.  In _International Conference on Learning Representations_.
  * Ding and Gimpel (2021) Xiaoan Ding and Kevin Gimpel. 2021.  FlowPrior: Learning expressive priors for latent variable sentence models.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 3242–3258, Online. Association for Computational Linguistics.
  * Dinh et al. (2014) Laurent Dinh, David Krueger, and Yoshua Bengio. 2014.  Nice: Non-linear independent components estimation.  _arXiv preprint arXiv:1410.8516_.
  * Dinh et al. (2016) Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. 2016.  Density estimation using real nvp.  _arXiv preprint arXiv:1605.08803_.
  * Duan et al. (2020) Yu Duan, Canwen Xu, Jiaxin Pei, Jialong Han, and Chenliang Li. 2020.  Pre-train and plug-in: Flexible conditional text generation with variational auto-encoders.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 253–262, Online. Association for Computational Linguistics.
  * Ficler and Goldberg (2017) Jessica Ficler and Yoav Goldberg. 2017.  Controlling linguistic style aspects in neural language generation.  In _Proceedings of the Workshop on Stylistic Variation_ , pages 94–104, Copenhagen, Denmark. Association for Computational Linguistics.
  * Goodfellow et al. (2020) Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020.  Generative adversarial networks.  _Communications of the ACM_ , 63(11):139–144.
  * Gu et al. (2022a) Yuxuan Gu, Xiaocheng Feng, Sicheng Ma, Jiaming Wu, Heng Gong, and Bing Qin. 2022a.  Improving controllable text generation with position-aware weighted decoding.  In _Findings of the Association for Computational Linguistics: ACL 2022_ , pages 3449–3467, Dublin, Ireland. Association for Computational Linguistics.
  * Gu et al. (2022b) Yuxuan Gu, Xiaocheng Feng, Sicheng Ma, Lingyuan Zhang, Heng Gong, and Bing Qin. 2022b.  A distributional lens for multi-aspect controllable text generation.  _arXiv preprint arXiv:2210.02889_.
  * Ho et al. (2020) Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020.  Denoising diffusion probabilistic models.  In _Advances in Neural Information Processing Systems_ , volume 33, pages 6840–6851. Curran Associates, Inc.
  * Hu et al. (2017) Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. 2017\.  Toward controlled generation of text.  In _Proceedings of the 34th International Conference on Machine Learning - Volume 70_ , ICML’17, page 1587–1596. JMLR.org.
  * Keskar et al. (2019) Nitish Shirish Keskar, Bryan McCann, Lav Varshney, Caiming Xiong, and Richard Socher. 2019.  CTRL - A Conditional Transformer Language Model for Controllable Generation.  _arXiv preprint arXiv:1909.05858_.
  * Khalifa et al. (2020) Muhammad Khalifa, Hady Elsahar, and Marc Dymetman. 2020.  A distributional approach to controlled text generation.  In _International Conference on Learning Representations_.
  * Kingma and Welling (2013) Diederik P Kingma and Max Welling. 2013.  Auto-encoding variational bayes.  _arXiv preprint arXiv:1312.6114_.
  * Kingma and Dhariwal (2018) Durk P Kingma and Prafulla Dhariwal. 2018.  Glow: Generative flow with invertible 1x1 convolutions.  In _Advances in Neural Information Processing Systems_ , volume 31. Curran Associates, Inc.
  * Kingma et al. (2016) Durk P Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, and Max Welling. 2016.  Improved variational inference with inverse autoregressive flow.  In _Advances in Neural Information Processing Systems_ , volume 29. Curran Associates, Inc.
  * Krause et al. (2021) Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, and Nazneen Fatema Rajani. 2021.  GeDi: Generative discriminator guided sequence generation.  In _Findings of the Association for Computational Linguistics: EMNLP 2021_ , pages 4929–4952, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Kumar et al. (2021) Sachin Kumar, Eric Malmi, Aliaksei Severyn, and Yulia Tsvetkov. 2021.  Controlled text generation as continuous optimization with multiple constraints.  _Advances in Neural Information Processing Systems_ , 34.
  * Kumar et al. (2022) Sachin Kumar, Biswajit Paria, and Yulia Tsvetkov. 2022.  Constrained sampling from language models via langevin dynamics in embedding spaces.  _arXiv preprint arXiv:2205.12558_.
  * Li et al. (2016) Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016.  A diversity-promoting objective function for neural conversation models.  In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 110–119, San Diego, California. Association for Computational Linguistics.
  * Li and Liang (2021) Xiang Lisa Li and Percy Liang. 2021.  Prefix-tuning: Optimizing continuous prompts for generation.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 4582–4597, Online. Association for Computational Linguistics.
  * Li et al. (2022) Xiang Lisa Li, John Thickstun, Ishaan Gulrajani, Percy Liang, and Tatsunori B Hashimoto. 2022.  Diffusion-lm improves controllable text generation.  _arXiv preprint arXiv:2205.14217_.
  * Liu et al. (2021a) Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, and Yejin Choi. 2021a.  DExperts: Decoding-time controlled text generation with experts and anti-experts.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 6691–6706, Online. Association for Computational Linguistics.
  * Liu et al. (2022) Guangyi Liu, Zeyu Feng, Yuan Gao, Zichao Yang, Xiaodan Liang, Junwei Bao, Xiaodong He, Shuguang Cui, Zhen Li, and Zhiting Hu. 2022.  Composable text controls in latent space with odes.  _arXiv preprint arXiv:2208.00638_.
  * Liu et al. (2021b) Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2021b.  Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing.  _arXiv preprint arXiv:2107.13586_.
  * Lu et al. (2022) Ximing Lu, Sean Welleck, Liwei Jiang, Jack Hessel, Lianhui Qin, Peter West, Prithviraj Ammanabrolu, and Yejin Choi. 2022.  Quark: Controllable text generation with reinforced unlearning.  _arXiv preprint arXiv:2205.13636_.
  * Ma et al. (2019) Xuezhe Ma, Chunting Zhou, Xian Li, Graham Neubig, and Eduard Hovy. 2019.  FlowSeq: Non-autoregressive conditional sequence generation with generative flow.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 4282–4292, Hong Kong, China. Association for Computational Linguistics.
  * Maas et al. (2011) Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011.  Learning word vectors for sentiment analysis.  In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_ , pages 142–150, Portland, Oregon, USA. Association for Computational Linguistics.
  * Mai et al. (2020) Florian Mai, Nikolaos Pappas, Ivan Montero, Noah A. Smith, and James Henderson. 2020\.  Plug and play autoencoders for conditional text generation.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 6076–6092, Online. Association for Computational Linguistics.
  * Meng et al. (2022) Tao Meng, Sidi Lu, Nanyun Peng, and Kai-Wei Chang. 2022.  Controllable text generation with neurally-decomposed oracle.  _arXiv preprint arXiv:2205.14219_.
  * Mireshghallah et al. (2022) Fatemehsadat Mireshghallah, Kartik Goyal, and Taylor Berg-Kirkpatrick. 2022.  Mix and match: Learning-free controllable text generationusing energy language models.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 401–415, Dublin, Ireland. Association for Computational Linguistics.
  * Oussidi and Elhassouny (2018) Achraf Oussidi and Azeddine Elhassouny. 2018.  Deep generative models: Survey.  In _2018 International Conference on Intelligent Systems and Computer Vision (ISCV)_ , pages 1–8.
  * Papamakarios et al. (2017) George Papamakarios, Theo Pavlakou, and Iain Murray. 2017.  Masked autoregressive flow for density estimation.  In _Advances in Neural Information Processing Systems_ , volume 30. Curran Associates, Inc.
  * Qian et al. (2022) Jing Qian, Li Dong, Yelong Shen, Furu Wei, and Weizhu Chen. 2022.  Controllable natural language generation with contrastive prefixes.  In _Findings of the Association for Computational Linguistics: ACL 2022_ , pages 2912–2924, Dublin, Ireland. Association for Computational Linguistics.
  * Qin et al. (2022) Lianhui Qin, Sean Welleck, Daniel Khashabi, and Yejin Choi. 2022.  Cold decoding: Energy-based constrained text generation with langevin dynamics.  _arXiv preprint arXiv:2202.11705_.
  * Radford et al. (2019) Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019.  Language models are unsupervised multitask learners.
  * Tang et al. (2021) Zineng Tang, Shiyue Zhang, Hyounghun Kim, and Mohit Bansal. 2021.  Continuous language generative flow.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 4609–4622, Online. Association for Computational Linguistics.
  * Tran et al. (2019) Dustin Tran, Keyon Vafa, Kumar Agrawal, Laurent Dinh, and Ben Poole. 2019.  Discrete flows: Invertible generative models of discrete data.  In _Advances in Neural Information Processing Systems_ , volume 32. Curran Associates, Inc.
  * Wu et al. (2022) Chen Henry Wu, Saman Motamed, Shaunak Srivastava, and Fernando De la Torre. 2022\.  Generative visual prompt: Unifying distributional control of pre-trained generative models.  _arXiv preprint arXiv:2209.06970_.
  * Yang and Klein (2021) Kevin Yang and Dan Klein. 2021.  FUDGE: Controlled text generation with future discriminators.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 3511–3535, Online. Association for Computational Linguistics.
  * Yang et al. (2022) Kexin Yang, Dayiheng Liu, Wenqiang Lei, Baosong Yang, Mingfeng Xue, Boxing Chen, and Jun Xie. 2022.  Tailor: A prompt-based approach to attribute-based controlled text generation.  _arXiv preprint arXiv:2204.13362_.
  * Yu et al. (2021) Dian Yu, Zhou Yu, and Kenji Sagae. 2021.  Attribute alignment: Controlling text generation from pre-trained language models.  In _Findings of the Association for Computational Linguistics: EMNLP 2021_ , pages 2251–2268, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Zhang and Song (2022) Hanqing Zhang and Dawei Song. 2022.  Discup: Discriminator cooperative unlikelihood prompt-tuning for controllable text generation.  _arXiv preprint arXiv:2210.09551_.
  * Zhang et al. (2022a) Hanqing Zhang, Haolin Song, Shaoyu Li, Ming Zhou, and Dawei Song. 2022a.  A survey of controllable text generation using transformer-based pre-trained language models.  _arXiv preprint arXiv:2201.05337_.
  * Zhang et al. (2022b) Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022b.  Opt: Open pre-trained transformer language models.  _arXiv preprint arXiv:2205.01068_.
  * Zhang et al. (2015) Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015.  Character-level convolutional networks for text classification.  In _Advances in Neural Information Processing Systems_ , volume 28. Curran Associates, Inc.
  * Ziegler et al. (2019) Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. 2019.  Fine-tuning language models from human preferences.  _arXiv preprint arXiv:1909.08593_.
  * Ziegler and Rush (2019) Zachary Ziegler and Alexander Rush. 2019.  Latent normalizing flows for discrete sequences.  In _Proceedings of the 36th International Conference on Machine Learning_ , volume 97 of _Proceedings of Machine Learning Research_ , pages 7673–7682. PMLR.

##  Appendix A Calculation of $\hat{z}$

Interpolation of two distribution centers is a line (one-dimensional subspace)
where the probability density functions in this subspace are still two
Gaussian distributions. That is: Our target goes back to solve the equation
$\pi(\hat{z}|a)=\pi(\hat{z}|\bar{a})$ under one-dimensional situation.

$\displaystyle\textit{Given: }\pi(\hat{z}|a)=\pi(\hat{z}|\bar{a})$
$\displaystyle\Rightarrow$
$\displaystyle\mathcal{N}(\hat{z};\mu_{a},\sigma^{2}_{a})=\mathcal{N}(\hat{z};\mu_{\bar{a}},\sigma_{\bar{a}}^{2})$
$\displaystyle\Rightarrow$
$\displaystyle\log\frac{\exp(-\frac{(\hat{z}-\mu_{a})^{2}}{2\sigma_{a}^{2}})}{\sqrt{2\pi}\sigma_{a}}=\log\frac{\exp(-\frac{(\hat{z}-\mu_{\bar{a}})^{2}}{2\sigma_{\bar{a}}^{2}})}{\sqrt{2\pi}\sigma_{\bar{a}}}$
$\displaystyle\Rightarrow$
$\displaystyle\frac{1}{2}\left[\log(\frac{\sigma^{2}_{a}}{\sigma^{2}_{\bar{a}}})-\frac{(\hat{z}-\mu_{a})^{2}}{\sigma^{2}_{a}}+\frac{(\hat{z}-\mu_{\bar{a}})^{2}}{\sigma^{2}_{\bar{a}}}\right]=0$
$\displaystyle\Rightarrow$ $\displaystyle\left\\{\begin{aligned}
&A(\hat{z})^{2}+B\hat{z}+C=0\\\
&A=-\frac{1}{\sigma_{a}^{2}}+\frac{1}{\sigma_{\bar{a}}^{2}}\\\
&B=2(\frac{\mu_{a}}{\sigma_{a}^{2}}-\frac{\mu_{\bar{a}}}{\sigma_{\bar{a}}^{2}})\\\
&C=\log(\frac{\sigma_{\bar{a}}^{2}}{\sigma_{a}^{2}})-\frac{\mu_{a}^{2}}{\sigma_{a}^{2}}+\frac{\mu_{\bar{a}}^{2}}{\sigma_{\bar{a}}^{2}}\end{aligned}\right.$
$\displaystyle\Rightarrow$ $\displaystyle\begin{aligned} \Delta&=B^{2}-4AC\\\
&=\frac{4}{\sigma_{a}^{2}\sigma_{\bar{a}}^{2}}\left[(\mu_{a}-\mu_{\bar{a}})^{2}+(\sigma_{a}^{2}-\sigma_{\bar{a}}^{2})\log(\frac{\sigma_{a}^{2}}{\sigma_{\bar{a}}^{2}})\right]\\\
&\geq 0\end{aligned}$ $\displaystyle\Rightarrow$
$\displaystyle\left\\{\begin{aligned} &\textit{if
}\sigma_{a}=\sigma_{\bar{a}},\hat{z}=-\frac{C}{B}=\frac{\mu_{a}+\mu_{\bar{a}}}{2}\\\
&\textit{if
}\sigma_{a}\neq\sigma_{\bar{a}},\hat{z}=\frac{-B\pm\sqrt{\Delta}}{2A}\end{aligned}\right.$

According to the derivation above and Figure 6, when
$\sigma_{a}=\sigma_{\bar{a}}$, the $\hat{z}$ is simply the midpoint of
$\mu_{a}$ and $\mu_{\bar{a}}$. When $\sigma_{a}\neq\sigma_{\bar{a}}$, there
are usually two solutions for $\hat{z}$, and the one we expect needs to be in
the interval $\min(\mu_{a},\mu_{\bar{a}})$ to $\max(\mu_{a},\mu_{\bar{a}})$.
It is worth noting that there may be cases where solutions of $\hat{z}$ are
both outside this interval, which is caused by the distance between $\mu_{a}$
and $\mu_{\bar{a}}$ being too small. In this case, the interval of the two
solutions of $\hat{z}$ becomes the region where two attributes intersect.

Figure 6: Different cases of intersection in one-dimensional subspace.

As illustrated in Figure 6, it is complicated to accurately calculate the
point where two attributes intersect, even in a one-dimensional case.
Fortunately, we can observe that $\hat{z}$ is always between $\mu_{a}$ and
$\mu_{\bar{a}}$. This means we can find an approximate intersection point by
adjusting the interpolation parameter in practical use.

##  Appendix B Approximately Isotropic

We analyze the standard deviation $\sigma$ of different attribute’s Gaussian
distribution in Table 5. We demonstrate the maximum, minimum, average, and
standard deviation values among all dimensions for each $\sigma$. The maximum
differences of $\sigma$s are around $1$, and the standard deviations are all
less than $0.02$, in which case we consider the distributions to be
approximately isotropic.

Attribute | $\sigma_{i}$
---|---
$\max$ | $\min$ | $\mathop{\text{avg}}$ | std
Negative | 0.886 | 0.756 | 0.800 | 0.018
Positive | 0.889 | 0.760 | 0.801 | 0.018
World | 0.848 | 0.737 | 0.782 | 0.018
Sports | 0.837 | 0.728 | 0.776 | 0.018
Business | 0.851 | 0.738 | 0.783 | 0.018
Sci./Tech. | 0.853 | 0.737 | 0.784 | 0.018
Toxic | 0.853 | 0.740 | 0.783 | 0.017
Non-Tox. | 0.853 | 0.747 | 0.790 | 0.017
Table 5: Statistics for Distributions