# Maximum Approximate Bernstein Likelihood Estimation of Densities in a Two-
sample Semiparametric Model

Zhong Guan
Department of Mathematical Sciences
Indiana University South Bend

###### Abstract

Maximum likelihood estimators are proposed for the parameters and the
densities in a semiparametric density ratio model in which the nonparametric
baseline density is approximated by the Bernstein polynomial model. The EM
algorithm is used to obtain the maximum approximate Bernstein likelihood
estimates. Simulation study shows that the performance of the proposed method
is much better than the existing ones. The proposed method is illustrated by
real data examples. Some asymptotic results are also presented and proved.

Keywords: Bernstein polynomial model, Beta mixture model, Case-control data,
Density estimation, Exponential tilting, Kernel density, Logistic regression.

##  1 Introduction

Nonparametric density estimation is a difficult task in statistics. It is even
more difficult for small sample data. For each $x$ in the support of a density
$f$ in a nonparametric model, the information for this one-dimensional
parameter $f(x)$ is zero (see Bickel et al. (1993)). Ibragimov and Khasminskii
(1983) also showed that there is no nonparametric model for which this
information is positive. Properly reducing the infinite dimensional parameter
to a finite dimensional one is necessary. To estimate an unknown smooth
function as the nonparametric component of a non- and semi-parametric model,
as we have done in empirical likelihood we usually approximate it by a step-
function and parameterize it using the jump sizes of the step-function. This
approach gives an efficient estimate of the underlying cumulative distribution
function. Because this estimate is a step-function, we have to use kernel or
other method to smooth it to obtain a density estimate. However, kernel
density is actually the convolution of the scaled kernel and the underlying
distribution to be estimated. There is always trade-off between the bias and
variance. In semiparametric problems, the roughness of the step-function
approximation could also affect the finite sample performance of the estimates
of the parametric components. Instead of approximating the underlying
distribution function by a step-function and then smoothing the discretized
estimation, Guan (2016) proposed to use a Bernstein polynomial approximation
and to directly and smoothly estimate the underlying distribution using a
maximum approximate Bernstein likelihood method. Guan (2016)’s method
parameterizes the underlying distribution by the coefficients of the Bernstein
polynomial and differs from other Bernstein polynomial smoothing methods which
was initiated by Vitale (1975) and use empirical distribution to estimate
these coefficients. The maximum approximate Bernstein likelihood method has
been successfully applied to grouped, contaminated, multivariate, and interval
censored data (Guan, 2017, 2021a; Wang and Guan, 2019; Guan, 2021b). In
application to the Cox’s proportional hazards regression model, not only a
smooth estimate of the survival function but also improved estimates of
regression coefficients can be resulted, due to a better approximation of the
unknown underlying baseline density function.

In applications of statistics especially in biostatistics, independent two-
sample data from case-control study for instance are common. If the two
nonparametric underlying distributions are linked in a certain parametric way,
then we can find better estimates of the distributions by efficiently
combining the two independent samples. Examples of such linked models are two-
sample proportional odds model (Dabrowska and Doksum, 1988), two-sample
proportional hazard model (Cox, 1972), two-sample density ratio (DR) model
(see for example, Qin and Zhang, 1997, 2005; Cheng and Chu, 2004), and so on.
Suppose that the densities $f_{0}$ and $f_{1}$ of “control” data $X_{0}$ and
“case” data $X_{1}$, respectively, satisfy the following density ratio model

$f_{1}(x)=f(x;\bm{\alpha})=f_{0}(x)\exp\\{\bm{\alpha}^{\mathrm{\scriptscriptstyle\top}}\tilde{r}(x)\\},$
(1)

where
$\bm{\alpha}=(\alpha_{0},\ldots,\alpha_{d})^{\mathrm{\scriptscriptstyle\top}}\in\mathcal{A}\subset
R^{d+1}$, and
$\tilde{r}(x)=(1,r^{\mathrm{\scriptscriptstyle\top}}(x))^{\mathrm{\scriptscriptstyle\top}}$.
In this model $f_{0}$ is also called “baseline” density. Let $D$ be a binary
response variable, $\pi_{j}=\mathrm{P}(D=j)$, $j=0,1$. Define
$f_{i}(x)=f_{X|D}(x|D=i)$, $j=0,1$. By Bayes’ theorem, the two-sample DR model
is equivalent to the following logistic regression model (Qin and Zhang, 1997)

$\log\left\\{\frac{P(D=1|X=x)}{P(D=0|X=x)}\right\\}={\bm{\alpha}}^{*\mathrm{\scriptscriptstyle\top}}\tilde{r}(x),$
(2)

where $\alpha_{0}^{*}=\alpha_{0}-\log(\pi_{0}/\pi_{1})$ and
$\alpha_{i}^{*}=\alpha_{i}$, $i\geq 1$. Model (1) is appropriate because the
right-hand side of (2) can be a good approximation of the log odds function.
The goodness-of-fit of this model is also testable (Qin and Zhang, 1997). An
advantage of this model is that one can also choose $f_{1}$ as the baseline
density, that is,
$f_{0}(x)=f_{1}(x)\exp\\{-\bm{\alpha}^{\mathrm{\scriptscriptstyle\top}}\tilde{r}(x)\\}$.
For transformed data $Y=h(X)$ we have
$g_{1}(y)=g_{0}(y)\exp\\{\bm{\alpha}^{\mathrm{\scriptscriptstyle\top}}\tilde{r}[h^{-1}(y)]\\}$,
where $h^{-1}(\cdot)$ is the inverse of $h(\cdot)$ and $g_{i}(y)$ is the
density of $Y$ given $D=i$, $i=0,1$. Model (1) was also used for one-sample
density estimation by Efron and Tibshirani (1996) in which $f_{0}$ is a
carrier density and $r(x)$ is a known $d$-dimensional sufficient statistic.

Parametrizing the infinite dimensional parameter $f_{0}$ in (1) using the
multinomial model with unknown probabilities at the observations results in
the maximum empirical likelihood estimator (MELE) (Qin and Zhang, 1997)
$\tilde{\bm{\alpha}}$ of $\bm{\alpha}$ and step-function estimator of $f_{0}$.
The MELE $\tilde{\bm{\alpha}}$ can also be obtained by fitting the data
$(X,D)$ with the logistic regression (2). This method works well when $f_{0}$
is a nuisance parameter. However in many applications, both $\bm{\alpha}$ and
$f_{0}$ are of interest. A jagged step-function estimate of $f_{0}$ is
unsatisfactory especially when sample sizes are small. Smooth and efficient
estimator is desirable. Qin and Zhang (2005) proposed to smooth the discrete
empirical density estimates of $f_{0}$ and $f_{1}$ using kernel method. As a
smoothing technique kernel density does not target the unknown density but its
convolution with the scaled kernel function for any positive bandwidth. Good
density estimation is key to solve many difficult statistical problems such as
the goodness-of-fit test (Cheng and Chu, 2004) and the estimation of the
receiver operating characteristic curve when result of diagnostic test is
continuous (Zou et al., 1997) and sample size is small. In this paper, we
shall investigate the estimation of densities and the parameters under model
(1) using approximate Bernstein likelihood method.

The nonparametric component $f_{0}$ in the semiparametric model is totally
unspecified. If we have no information about the support of $f_{0}$, we can
only estimate $f_{0}$ as a density with support $[z_{(1)},z_{(n)}]$, where
$z_{(1)}$ and $z_{(n)}$ are, respectively, the minimum and maximum order
statistics of a pooled sample of size $n$ from $f_{0}$ and $f_{1}$. If the
density $f_{i}$ of $X_{i}$ has support $[a,b]$, $i=0,1$, and
$f_{1}(x)=f_{0}(x)\exp\\{\bm{\alpha}^{\mathrm{\scriptscriptstyle\top}}\tilde{r}(x)\\}$,
then the desnity of $Y_{i}=(X_{i}-a)/(b-a)$ is $g_{i}(y)=(b-a)f_{i}[a+(b-a)y]$
which have support $[0,1]$ and satisfy
$g_{1}(y)=g_{0}(y)\exp\\{\bm{\alpha}^{\mathrm{\scriptscriptstyle\top}}\tilde{r}[a+(b-a)y]\\}$.
Without loss of generality we will assume that both $f_{0}$ and $f_{1}$ have
support $[0,1]$.

The paper is organized as follows. The approximate Bernstein polynomial model
for DR model is introduced and is proved to be nested in Section 2. The EM
algorithm for finding the maximum approximate Bernstein likelihood estimates
of the mixture proportions and the regression coefficients, the methods for
determining a lower bound for the model degree $m$ based on sample mean and
variance and for choosing the optimal degree $m$ are also given in this
section. The proposed methods are illustrated by some real datasets in Section
3 and compared with some existing competitors through Monte Carlo experiments
in Section 4. Some asymptotic results about the convergence rate of the
proposed estimators are presented in Section 5. Some concluding remarks are
given in Section 6. The proofs of the theoretical results are relegated to the
Appendix.

##  2 Methodology

###  2.1 Approximate Bernstein Polynomial Model

Let $\bm{x}_{n_{i}}=\\{x_{i1},\ldots,x_{in_{i}}\\}$ be independent
observations of $X_{i}$, $i=0,1$. The true loglikelihood is
$\ell(\bm{\alpha},f_{0})=\ell(\bm{\alpha},f_{0};\bm{z}_{n})=\sum_{i=1}^{n}\log
f_{0}(z_{i})+\bm{\alpha}^{\mathrm{\scriptscriptstyle\top}}\sum_{j=1}^{n_{1}}\tilde{r}(x_{1j}),$
where $\bm{z}_{n}=\\{z_{1},\ldots,z_{n}\\}=\\{x_{01},\ldots,x_{0n_{0}}$;
$x_{11},\ldots,x_{1n_{1}}\\}$, $n=n_{0}+n_{1}$. Define simplex
$\mathbb{S}_{m}=\\{(u_{0},\ldots,u_{m}):u_{i}\geq 0,\sum_{i=0}^{m}u_{i}=1\\}$.
Instead of discretizing baseline density $f_{0}$ with finite support
$\bm{z}_{n}$ as in Qin and Zhang (1997), we use Bernstein polynomial
approximation (Guan, 2016) $f_{0}(x)\approx
f_{m}(x;\bm{0},\bm{p})=f_{m}(x;\bm{p})=\sum_{j=0}^{m}p_{j}\beta_{mj}(x),$
where $\bm{p}\in\mathbb{S}_{m}$, and $\beta_{mj}(x)=(m+1){m\choose
j}x^{j}(1-x)^{m-j}$ is the density of beta distribution with shape parameters
$(j+1,m-j+1)$, $j\in\mathbb{I}_{0}^{m}$. Here and in what follows
$\mathbb{I}_{m}^{n}=\\{m,\ldots,n\\}$ for any integers $m\leq n$. Therefore
$f(x;\bm{\alpha})$ can be approximated by $f_{m}(x;\bm{\alpha},\bm{p})\equiv
f_{m}(x;\bm{p})\exp\\{\bm{\alpha}^{\mathrm{\scriptscriptstyle\top}}\tilde{r}(x)\\}$.
The cumulative distribution function of $f_{m}(x;\bm{\alpha},\bm{p})$ is
$F_{m}(x;\bm{\alpha},\bm{p})=\sum_{j=0}^{m}p_{j}B_{mj}(x;\bm{\alpha})$, where
$B_{mj}(x;\bm{\alpha})=\int_{0}^{x}\beta_{mj}(y)\exp\\{\bm{\alpha}^{\mathrm{\scriptscriptstyle\top}}\tilde{r}(y)\\}dy.$
The approximate loglikelihood is then

$\ell_{m}(\bm{\alpha},\bm{p})=\sum_{i=1}^{n}\log
f_{m}(z_{i};\bm{p})+\bm{\alpha}^{\mathrm{\scriptscriptstyle\top}}\sum_{j=1}^{n_{1}}\tilde{r}(x_{1j}),$
(3)

with constraint

$(\bm{\alpha},\bm{p})\in\Theta_{m}(\mathcal{A})\equiv\biggr{\\{}(\bm{\alpha},\bm{p})\in\mathcal{A}\times\mathbb{S}_{m}:\sum_{i=0}^{m}p_{i}w_{mi}(\bm{\alpha})=1\biggr{\\}},$
(4)

where $w_{mj}(\bm{\alpha})=B_{mj}(1;\bm{\alpha})$, $j\in\mathbb{I}_{0}^{m}.$
Under constraint (4) the approximate density $f_{m}(x;\bm{\alpha},\bm{p})$ is
mixture of
$\beta_{mj}(x;\bm{\alpha})=\beta_{mj}(x)\exp\\{\bm{\alpha}^{\mathrm{\scriptscriptstyle\top}}\tilde{r}(x)\\}/w_{mj}(\bm{\alpha})$
with mixing proportions $\tilde{p}_{j}(\bm{\alpha})=p_{j}w_{mj}(\bm{\alpha})$,
$j\in\mathbb{I}_{0}^{m}$.

For the given $r(\cdot)$ and $\mathcal{A}$, let $\mathcal{D}_{m}(\mathcal{A})$
be the family of all functions
$f_{m}(x;\bm{\alpha},\bm{p}_{m})=f_{m}(x;\bm{p}_{m})\exp\\{\bm{\alpha}^{\mathrm{\scriptscriptstyle\top}}\tilde{r}(x)\\}$,
$(\bm{\alpha},\bm{p}_{m})\in\Theta_{m}(\mathcal{A})$. The following
proposition implies that the models $\mathcal{D}_{m}(\mathcal{A})$ are nested.

######  Proposition 1.

For the given regressor vector $r(\cdot)$ and parameter space $\mathcal{A}$,
$\mathcal{D}_{m}(\mathcal{A})\subset\mathcal{D}_{m+1}(\mathcal{A})$, for all
positive integers $m$.

The maximizer $\hat{\bm{\theta}}=(\hat{\bm{\alpha}},\hat{\bm{p}})$ of
$\ell_{m}(\bm{\alpha},\bm{p})$ subject to constraint (4) for an optimal degree
$m$ is called the maximum approximate Bernstein likelihood estimate (MABLE) of
$\bm{\theta}=(\bm{\alpha},\bm{p})$. Then $f_{i}$ and $F_{i}$, respectively,
can be estimated by $\hat{f}_{i}(x)=f_{m}(x;i\hat{\bm{\alpha}},\hat{\bm{p}})$
and $\hat{F}_{i}(x)=F_{m}(x;i\hat{\bm{\alpha}},\hat{\bm{p}})$, $i=0,1$.

For densities $f_{i}$ on $[a,b]$ which satisfy (1), we can obtain
$(\hat{\bm{\alpha}},\hat{\bm{p}})$ based on transformed data
$(x_{ij}-a)/(b-a)$ with $r(x)$ replaced by $r[a+(b-a)x]$. Then we have
estimates of $f_{i}$ and $F_{i}$, respectively,

$\displaystyle\hat{f}_{i}(x)=$
$\displaystyle~{}\frac{1}{b-a}f_{m}\Big{(}\frac{x-a}{b-a};i\hat{\bm{\alpha}},\hat{\bm{p}}\Big{)}=\frac{\exp\\{i\hat{\bm{\alpha}}^{\mathrm{\scriptscriptstyle\top}}\tilde{r}(x)\\}}{b-a}\sum_{j=0}^{m}\hat{p}_{j}\beta_{mj}\Big{(}\frac{x-a}{b-a}\Big{)},$
(5) $\displaystyle\hat{F}_{i}(x)=$
$\displaystyle~{}F_{m}\Big{(}\frac{x-a}{b-a};i\hat{\bm{\alpha}},\hat{\bm{p}}\Big{)}=\sum_{j=0}^{m}\hat{p}_{j}B_{mj}\Big{(}\frac{x-a}{b-a};i\hat{\bm{\alpha}}\Big{)},~{}\mbox{$i=0,1$,}$
(6)

where
$B_{mj}(x;{\bm{\alpha}})=\int_{0}^{x}\beta_{mj}(u)\exp\\{{\bm{\alpha}}^{\mathrm{\scriptscriptstyle\top}}\tilde{r}[a+(b-a)u]\\}du$,
$x\in[0,1]$, $j\in\mathbb{I}_{0}^{m}$.

To find maximum likelihood estimates of the parameters $(\bm{\alpha},\bm{p})$
we first introduce some notations. For any function $\varphi(\bm{\alpha})$
which may also depend upon the data, its first and second derivatives with
respect to $\bm{\alpha}$ are denoted by
$\dot{\varphi}(\bm{\alpha})=\frac{\partial\varphi(\bm{\alpha})}{\partial\bm{\alpha}}$
and
$\ddot{\varphi}(\bm{\alpha})=\frac{\partial^{2}\varphi(\bm{\alpha})}{\partial\bm{\alpha}\partial\bm{\alpha}^{\mathrm{\scriptscriptstyle\top}}}.$
The entries are denoted by
$[\dot{\varphi}(\bm{\alpha})]_{i}=\frac{\partial\varphi(\bm{\alpha})}{\partial\alpha_{i}}$
and
$[\ddot{\varphi}(\bm{\alpha})]_{ij}=\frac{\partial^{2}\varphi(\bm{\alpha})}{\partial\alpha_{i}\partial\alpha_{j}}$,
$i,j\in\mathbb{I}_{0}^{d}.$ For example, the derivatives of
$w_{mj}(\bm{\alpha})$, $j\in\mathbb{I}_{0}^{m}$, are
$\dot{w}_{mj}(\bm{\alpha})=\int_{0}^{1}\tilde{r}(x)\beta_{mj}(x)\exp\\{\bm{\alpha}^{\mathrm{\scriptscriptstyle\top}}\tilde{r}(x)\\}dx$
and
$\ddot{w}_{mj}(\bm{\alpha})=\int_{0}^{1}\tilde{r}(x)\tilde{r}^{\mathrm{\scriptscriptstyle\top}}(x)\beta_{mj}(x)\exp\\{\bm{\alpha}^{\mathrm{\scriptscriptstyle\top}}\tilde{r}(x)\\}dx$.
Note
$\tilde{r}(x)=(1,r^{\mathrm{\scriptscriptstyle\top}}(x))^{\mathrm{\scriptscriptstyle\top}}$,
$[\dot{w}_{mj}(\bm{\alpha})]_{0}=w_{mj}(\bm{\alpha})$,
$[\ddot{w}_{mj}(\bm{\alpha})]_{00}=w_{mj}(\bm{\alpha})$, and
$[\ddot{w}_{mj}(\bm{\alpha})]_{0i}=[\dot{w}_{mj}(\bm{\alpha})]_{i}$,
$i\in\mathbb{I}_{0}^{d}$.

The standard EM algorithm combined with method of Lagrange multipliers leads
to the following algorithm.

#### Algorithm for finding $(\hat{\bm{\alpha}},\hat{\bm{p}})$ with a given
$m$:

  *     * Step 0.

Choose small numbers $\epsilon_{1},\epsilon_{2}>0$ and large integers $N_{1}$
and $N_{2}$.

    * Step 1.

Use the logistic regression to find an MELE
$\bm{\alpha}^{(0)}=\tilde{\bm{\alpha}}$. Choose a uniform initial
$p^{(0)}=\bm{1}^{\mathrm{\scriptscriptstyle\top}}/(m+1)$ for $\bm{p}$. If
vanishing boundary contraints $f_{0}(0)=0$ and/or $f_{0}(1)=0$ are available,
choose $p_{0}^{(0)}=0$ and/or $p_{m}^{(0)}=0$ accordingly and set the other
$p_{i}$’s uniformly.

    * Step 2.

Set $s=0$, $\bm{\theta}^{(s)}=(\bm{\alpha}^{(s)},\bm{p}^{(s)})$. Calculate
$\ell_{m}^{(s)}=\ell_{m}(\bm{\theta}^{(s)})=\ell_{m}(\bm{\alpha}^{(s)},\bm{p}^{(s)})$.

    * Step 3.

Set $t=0$, $\bm{\alpha}^{\langle t\rangle}=\tilde{\bm{\alpha}}$. Run the
Newton-Raphson iteration $\bm{\alpha}^{\langle
t+1\rangle}=\bm{\alpha}^{\langle t\rangle}-J_{s}^{-1}(\bm{\alpha}^{\langle
t\rangle})\bm{H}_{s}(\bm{\alpha}^{\langle t\rangle})$, $t=0,1,2,\ldots,$ until
$|\bm{\alpha}^{\langle t+1\rangle}-\bm{\alpha}^{\langle
t\rangle}|<\epsilon_{1}$ or $t>N_{1}$ to obtain
$\bm{\alpha}^{(s+1)}=\bm{\alpha}^{\langle t+1\rangle}$, where

$\displaystyle H_{s}(\bm{\alpha})$
$\displaystyle=\sum_{j=1}^{n_{1}}\tilde{r}(x_{1j})-n_{1}\sum_{k=0}^{m}\frac{T_{k}(\bm{\theta}^{(s)})\dot{w}_{mk}(\bm{\alpha})}{n_{0}+n_{1}w_{mk}(\bm{\alpha})},$
(7) $\displaystyle J_{s}(\bm{\alpha})$
$\displaystyle=-n_{1}\sum_{k=0}^{m}\frac{[n_{0}+n_{1}w_{k}(\bm{\alpha})]\ddot{w}_{k}(\bm{\alpha})-n_{1}\dot{w}_{k}(\bm{\alpha})\dot{w}_{k}^{\mathrm{\scriptscriptstyle\top}}(\bm{\alpha})}{[n_{0}+n_{1}w_{mk}(\bm{\alpha})]^{2}}T_{k}(\bm{\theta}^{(s)}),\;\;\;$
(8) $\displaystyle T_{k}(\bm{\theta}^{(s)})$
$\displaystyle=\sum_{i=0}^{1}\sum_{j=1}^{n_{i}}\frac{p_{k}\beta_{mk}(x_{ij})}{f_{m}(x_{ij};\bm{p})},\quad
k\in\mathbb{I}_{0}^{m}.$ (9)

    * Step 4.

set $p_{k}^{(s+1)}=p_{k}(\bm{\alpha}^{(s+1)},\bm{\theta}^{(s)})$,
$k\in\mathbb{I}_{0}^{m}$, where

$p_{k}(\bm{\alpha},\bm{\theta}^{(s)})=\frac{T_{k}(\bm{\theta}^{(s)})}{n_{0}+n_{1}w_{mk}(\bm{\alpha})},\quad
k\in\mathbb{I}_{0}^{m}.$ (10)

    * Step 5.

Set $s=s+1$. Calculate
$\ell_{m}^{(s)}=\ell_{m}(\bm{\alpha}^{(s)},\bm{p}^{(s)})$.

    * Step 6.

If $\ell_{m}^{(s)}-\ell_{m}^{(s-1)}<\epsilon_{2}$ or $s>N_{2}$ then set
$\hat{\bm{\theta}}=(\hat{\bm{\alpha}},\hat{\bm{p}})=(\bm{\alpha}^{(s)},\bm{p}^{(s)})$
and stop. Otherwise go to Step 3.

Bootstrap method can be used to approximate the standard error of
$\hat{\bm{\alpha}}$: Generate $x_{i1}^{*},\ldots,x_{in_{i}}^{*}$ from
$\hat{f}_{i}(x)$, $i=0,1$, and fit the bootstrap samples by the proposed model
with $m=\hat{m}$ or $\tilde{m}$ to obtain $\hat{\bm{\alpha}}^{*}$. Repeat the
boostrap run a large number of times and estimate the standard error of
$\hat{\bm{\alpha}}$ by the sample standard deviation of
$\hat{\bm{\alpha}}^{*}$.

###  2.2 Choice of baseline and the model degree

Let
$\hat{m}_{b}^{(i)}=\max\\{\left\lceil\bar{x}_{i}(1-\bar{x}_{i})/s^{2}_{i}-3\right\rceil,1\\}$
be the estimated lower bound for $m$ based $x_{ij}$, $j=1,\ldots,n_{i}$ as in
Guan (2016, 2017). If $\hat{m}_{b}^{(1)}<\hat{m}_{b}^{(0)}$ we switch “case”
and “control” data and take $f_{1}$ as baseline so that the estimated lower
bound for the model degree of the two-sample density ratio model is
$\hat{m}_{b}=\min\\{\hat{m}_{b}^{(0)},\hat{m}_{b}^{(1)}\\}$. Proposition 1
implies that $\ell_{m}(\hat{\bm{\alpha}},\hat{\bm{p}})$ is nondecreasing in
$m$. Applying the change-point method of Guan (2016) to
$\ell_{m}(\hat{\bm{\alpha}},\hat{\bm{p}})$ one can obtain an optimal degree
$\hat{m}$. In many cases an optimal degree is very close to $\hat{m}_{b}$. The
search of an optimal degree starts at some $m_{0}<\hat{m}_{b}$. Approximating
$\ell_{m}(\hat{\bm{\alpha}},\hat{\bm{p}})$ by
$\ell_{m}(\tilde{\bm{\alpha}},\tilde{\bm{p}})=\max_{\bm{p}\in\Theta_{m}(\\{\tilde{\bm{\alpha}}\\})}\ell_{m}(\tilde{\bm{\alpha}},\bm{p})$,
where $\tilde{\bm{\alpha}}$ is the MELE of $\bm{\alpha}$, can reduce the cost
of EM computation and results in an optimal degree $\tilde{m}$. One can obtain
$\tilde{\bm{p}}$ by iteration

$\tilde{p}_{k}^{(s+1)}=\frac{T_{k}(\tilde{\bm{\alpha}},\tilde{\bm{p}}^{(s)})}{n+\lambda^{(s)}(\tilde{\bm{\alpha}})[w_{mk}(\tilde{\bm{\alpha}})-1]},\quad
k\in\mathbb{I}_{0}^{m},\quad s\in\mathbb{I}_{0}^{\infty},$ (11)

where $T_{k}(\bm{\alpha},\bm{p})$ is given by (9) and
$\lambda=\lambda^{(s)}(\tilde{\bm{\alpha}})$ can be obtained by Newton-Raphson
iteration $\lambda^{\langle t+1\rangle}=\lambda^{\langle
t\rangle}-{\psi(\lambda^{\langle t\rangle})}/{\psi^{\prime}(\lambda^{\langle
t\rangle})}$, $t\in\mathbb{I}_{0}^{\infty}$, where

$\displaystyle\psi(\lambda)$
$\displaystyle=\sum_{k=0}^{m}p_{k}(\tilde{\bm{\alpha}},\bm{p}^{(s)})\left[w_{mk}(\tilde{\bm{\alpha}})-1\right]=\sum_{k=0}^{m}\frac{T_{k}(\tilde{\bm{\alpha}},\bm{p}^{(s)})\left[w_{mk}(\tilde{\bm{\alpha}})-1\right]}{n+\lambda\left[w_{mk}(\tilde{\bm{\alpha}})-1\right]},$
$\displaystyle\psi^{\prime}(\lambda)$
$\displaystyle=-\sum_{k=0}^{m}\frac{T_{k}(\tilde{\bm{\alpha}},\bm{p}^{(s)})\left[w_{mk}(\tilde{\bm{\alpha}})-1\right]^{2}}{\\{n+\lambda\left[w_{mk}(\tilde{\bm{\alpha}})-1\right]\\}^{2}}.$

The proposal is implemented in R as a component of package mable (Guan, 2019)
which is publically available.

##  3 Real Data Application

###  3.1 Coronary Heart Disease Data

Hosmer and Lemeshow (1989) analyzed the relationship between age and the
status of coronary heart disease (CHD) based on 100 subjects participated in a
study. The data set contains $n_{0}=57$ ages from control group and $n_{1}=43$
ages from case group: $y_{0}=$(20, 23, 24, 25, 26, 26, 28, 28, 29, 30, 30, 30,
30, 30, 32, 32, 33, 33, 34, 34, 34, 34, 35, 35, 36, 36, 37, 37, 38, 38, 39,
40, 41, 41, 42, 42, 42, 43, 43, 44, 44, 45, 46, 47, 47, 48, 49, 49, 50, 51,
52, 55, 57, 57, 58, 60, 64) and $y_{1}=$(25, 30, 34, 36, 37, 39, 40, 42, 43,
44, 44, 45, 46, 47, 48, 48, 49, 50, 52, 53, 53, 54, 55, 55, 56, 56, 56, 57,
57, 57, 57, 58, 58, 59, 59, 60, 61, 62, 62, 63, 64, 65, 69). The extreme
sample statistics are $z_{(1)}=20$ and $z_{(n)}=69$. We choose truncation
interval $[a,b]=[20,70]$, $r(y)=y$, and transform $y_{i}$’s to
$x_{i}=(y_{i}-a)/(b-a)$, $i=0,1$. The control is selected as baseline and
$\hat{m}_{b}=3$. Using $M=\mathbb{I}_{1}^{20}$ as a candidate set we obtained
$\tilde{m}=\hat{m}=3$ (see the upper panel of Figure 1). The MABLE’s of
$f_{i}$ and $F_{1}$ are given by (5) and (6) with
$\hat{\bm{p}}=(0.09686,0.89834,0.00000,0.004796)^{\mathrm{\scriptscriptstyle\top}}$
and $\hat{\bm{\alpha}}=(-5.040,0.111)^{\mathrm{\scriptscriptstyle\top}}$ with
SE $(0.945,0.020)^{\mathrm{\scriptscriptstyle\top}}$ based 1000 bootstrap
runs. This is very close to the MELE
$\tilde{\bm{\alpha}}=(-5.02760,0.11092)^{\mathrm{\scriptscriptstyle\top}}$
with SE $(1.134,0.024)^{\mathrm{\scriptscriptstyle\top}}$ (Hosmer and
Lemeshow, 1989; Qin and Zhang, 2005).

The lower panel of Figure 1 also shows the proposed density estimates, the
semiparametric estimates of Qin and Zhang (2005) based on two-sample empirical
likelihood method with Gaussian kernel and the kernel density estimates using
Gaussian kernel based on one sample only. We can see that the proposed method
gives a smoother density estimate. From Figure 1 we see that the MABLE’s
$\hat{f}_{i}$ differs from the other two density estimates of $f_{i}$
especially especially for the case data. The $\hat{f}_{0}$ leans a little bit
more to the left. All estimates of $f_{1}$ show strong evidence supporting the
observation that individuals at age between 45 and 60 are more likely to have
CHD.

Figure 1: Coronary Heart Disease Data. Upper left panel: log-likelihood of the
data; upper right panel: likelihood ratio for change-point estimate. Lower
panel: histograms(light gray), the MABLE $\hat{f}_{i}$, the semparametric
kernel density estimate $\tilde{f}_{i\mathrm{S}}$, and the one-sample
nonparametric kernel density estimate $\tilde{f}_{i\mathrm{N}}$ of $f_{i}$,
$i=0,1$.

###  3.2 Pancreatic Cancer Data

We apply the proposed method to the Pancreatic cancer diagnostic marker data
in which sera from $n_{0}=51$ control patients with pancreatitis and
$n_{1}=90$ case patients with pancreatic cancer were studied at the Mayo
Clinic with a cancer antigen, CA-125, and with a carbohydrate antigen, CA19-9.
Wieand et al. (1989) showed that CA19-9 has higher sensitivity to Pancreatic
cancer. Let $y_{ij}$, $j=1,\ldots,n_{i}$, $i=0,1$, denote the logarithm of the
observed value of CA19-9 for the $j$th subject of control group ($i=0$) and
case group ($i=1$). The combined sample is $\\{z_{1},\ldots,z_{n}\\}$,
$n=n_{0}+n_{1}$. Qin and Zhang (2003) considered the measurement $y$ on CA19-9
and obtained $p$-value 0.769 of the Kolmogorov–Smirnov–test for the density
ratio model with $r(y)=(y,y^{2})^{\mathrm{\scriptscriptstyle\top}}$. Qin and
Zhang (2003)’s MELE is
$\tilde{\bm{\alpha}}=(0.56,-1.91,0.45)^{\mathrm{\scriptscriptstyle\top}}$ with
SE $(1.66,1.22,0.21)^{\mathrm{\scriptscriptstyle\top}}$.

We choose $a=z_{(1)}=0.8754687$ and $b=z_{(n)}=10.08581$. The estimated lower
bounds for $m$ based on “control” and “case” data are, respectively,
$\hat{m}_{b}^{(0)}=19$ and $\hat{m}_{b}^{(1)}=2$. We chose “case” as baseline.
Based on the transformed data $x_{ij}=(y_{ij}-a)/(b-a)$, we obtain an optimal
degree $\tilde{m}=\hat{m}=3$ and estimates $\hat{f}_{i}$, $i=0,1$, as given by
(5) with $m=3$, where
$\hat{\bm{\alpha}}=(0.045,-1.677,0.434)^{\mathrm{\scriptscriptstyle\top}}$
with SE $(1.35,0.91,0.15)^{\mathrm{\scriptscriptstyle\top}}$ based 1000
bootstrap runs, and
$(\hat{p}_{0},\ldots,\hat{p}_{3})=(0.09747,0.42829,0.38557,0.08867)$. The case
density estimates agree each other. These results show that healthy people
have lower logarithmic level of CA 19-9 in their blood while logarithmic
levels of CA 19-9 for pancreatic cancer patients are nearly uniform.

Figure 2: Pancreatic cancer CA 19-9 data. Histograms(light gray) and density
estimates of log CA 19-9 level without pancreatic cancer (left panel) and with
pancreatic cancer (right panel): the MABLE $\hat{f}_{i}$, the semparametric
kernel density estimate $\tilde{f}_{i\mathrm{S}}$, and the one-sample
nonparametric kernel density estimate $\tilde{f}_{i\mathrm{N}}$ of $f_{i}$,
$i=0,1$.

###  3.3 Melanoma Data

Venkatraman and Begg (1996) compared two systems which can be used to evaluate
suspicious lesions of being a melanoma based on paired data. The two systems
are the clinical score system given by doctors and the dermoscope. Qin and
Zhang (2003) suggest the density ratio model with $r(x)=x$. The MELE of
$\bm{\alpha}$ is $\tilde{\bm{\alpha}}=(0.887,1.000)$ with SE
$(0.37,0.23)^{\mathrm{\scriptscriptstyle\top}}$.

Using the proposed method with $a=z_{(1)}=-6.5$ and $b=5.0$ we have model
degree $\hat{m}=\tilde{m}=10$. We obtained the MABLE
$\hat{\bm{\alpha}}=(0.881,1.018)^{\mathrm{\scriptscriptstyle\top}}$ of
$\bm{\alpha}$ with SE $(0.75,0.77)^{\mathrm{\scriptscriptstyle\top}}$ based
1000 bootstrap runs, $\hat{p}_{i}<10^{-5}$, $i\notin\mathbb{I}_{2}^{5}$, and
$(\hat{p}_{2},\cdots,\hat{p}_{5})=(.30153,.14619,.55226,.00002)$. From Figure
3 we see that the clinical scores have different distributions with a small
overlap for people with and without melanoma.

Figure 3: Melanoma data. Histograms(light gray) and density estimates of
clinical scores without melanoma (left panel) and with melanoma (right panel):
the MABLE $\hat{f}_{i}$, the semparametric kernel density estimate
$\tilde{f}_{i\mathrm{S}}$, and the one-sample nonparametric kernel density
estimate $\tilde{f}_{i\mathrm{N}}$ of $f_{i}$, $i=0,1$.

##  4 Simulation

In this section we compare the performances of the proposed estimator
$\hat{f}_{i}$ with the one-sample parametric MLE $\hat{f}_{i\mathrm{P}}$, the
two-sample semiparametric estimator $\tilde{f}_{i\mathrm{S}}$ of Qin and Zhang
(2005), and the one-sample kernel density estimator $\tilde{f}_{i\mathrm{N}}$
by examining the point-wise mean squared error (pMSE) $\mathrm{mse}_{j}$ at
$t_{j}=a+j(b-a)/N$, $j\in\mathbb{I}_{0}^{N}$, $N=512$) and approximate mean
integrated squared error (MISE)
$\mathrm{mise}=N^{-1}\sum_{j=1}^{N}\mathrm{mse}_{j}$ for $i=0,1$. For
convenience and fair comparison, we used same setups as in Qin and Zhang
(2005). The sample were generated using the models below. In all the
simulations, the sample sizes are $(n_{0},n_{1})=(50,50),(100,100)$ and the
number of Monte Carlo runs is 1000.

Model 1: Normal distributions $X_{0}\sim N(0,1)$, $X_{1}\sim N(\mu,1)$,
$\tilde{r}(x)=(1,x)^{\mathrm{\scriptscriptstyle\top}}$,
$f_{0}(x)=\frac{1}{\sqrt{2\pi}}e^{-x^{2}/2}$,
$\bm{\alpha}=(-\mu^{2}/2,\;\mu)^{\mathrm{\scriptscriptstyle\top}}$, and
$\mu=0.25(0.25)2.00$. We choose $a=\min(-4,\mu-4)$ and $b=\max(4,\mu+4)$. In
this model, the bandwidths for $\tilde{f}_{i\mathrm{S}}$ and
$\tilde{f}_{i\mathrm{N}}$ are those suggested by Qin and Zhang (2005). The
parametric MLE is
$\hat{f}_{i\mathrm{P}}(x)=f_{0}[(x-\bar{x}_{i})/s_{i}]/s_{i}$, where
$\bar{x}_{i}$ and $s_{i}$ are, respectively, the sample mean and sample
standard deviation of $x_{i1},\ldots,x_{in_{i}}$, $i=0,1$.

Figure 4: Simulated pointwise mean squared error of the MABLE $\hat{f}_{0}$, the one-sample parametric MLE $\hat{f}_{0\mathrm{P}}$, the semparametric kernel density estimate $\tilde{f}_{0\mathrm{S}}$, and the one-sample nonparametric kernel density estimate $\tilde{f}_{0\mathrm{N}}$ of $f_{0}$ based 1000 datasets generated from normal distributions with $n_{0}=n_{1}=50$. Table 1: Simulation results based on $B=1000$ Monte Carlo runs and samples of sizes $(n_{0},n_{1})$ from normal distributions $\mathrm{N}(0,1)$ and $\mathrm{N}(\mu,1)$ using optimal degree $\hat{m}$. |  |  | mse $\times 10^{2}$ |  | mise $\times 10^{4}$
---|---|---|---|---|---
$\mu$ | $E(\hat{m})$ | $\sigma(\hat{m})$ | $\hat{\alpha}_{0}$ | $\hat{\alpha}_{1}$ | $\tilde{\alpha}_{0}$ | $\tilde{\alpha}_{1}$ |  | $\hat{f}_{0\mathrm{P}}$ | $\hat{f}_{0}$ | $\tilde{f}_{0\mathrm{S}}$ | $\tilde{f}_{0\mathrm{N}}$
| $n_{0}=n_{1}=50$ |
0.25 | 15.00 | 2.77 |  0.16 |  4.22 |  0.18 |  4.61 |  | 6.46 | 6.24 | 16.22 | 24.93
0.50 | 16.25 | 2.92 |  0.54 |  4.44 |  0.65 |  5.07 |  | 6.15 | 5.81 | 14.92 | 21.27
0.75 | 17.29 | 2.91 |  1.29 |  5.06 |  1.54 |  6.06 |  | 5.94 | 5.57 | 15.86 | 21.31
1.00 | 18.29 | 3.06 |  2.52 |  5.95 |  3.07 |  7.37 |  | 5.52 | 5.14 | 16.11 | 21.66
1.25 | 19.31 | 3.28 |  4.40 |  7.24 |  5.73 |  9.49 |  | 5.68 | 5.65 | 14.87 | 19.65
1.50 | 20.45 | 3.25 |  6.99 |  8.97 | 10.37 | 13.57 |  | 5.47 | 5.82 | 16.88 | 20.99
1.75 | 21.47 | 3.40 | 10.46 | 10.37 | 15.60 | 16.19 |  | 5.48 | 5.69 | 17.19 | 19.66
2.00 | 22.83 | 3.47 | 15.48 | 12.51 | 28.62 | 23.05 |  | 5.15 | 5.94 | 18.56 | 19.88
| $n_{0}=n_{1}=100$ |
0.25 | 15.20 | 2.16 | 0.07 | 2.23 |  0.08 | 2.32 |  | 3.05 | 3.52 |  8.76 | 12.56
0.50 | 16.28 | 2.49 | 0.28 | 2.33 |  0.30 | 2.53 |  | 3.09 | 3.55 |  8.18 | 11.87
0.75 | 17.27 | 2.33 | 0.65 | 2.61 |  0.72 | 2.77 |  | 2.84 | 3.26 |  8.49 | 12.26
1.00 | 18.42 | 2.58 | 1.20 | 3.03 |  1.32 | 3.37 |  | 2.74 | 3.17 |  9.30 | 12.26
1.25 | 19.51 | 2.60 | 2.08 | 3.37 |  2.34 | 3.84 |  | 2.78 | 3.11 |  9.56 | 12.12
1.50 | 20.60 | 2.75 | 3.69 | 4.49 |  4.52 | 5.62 |  | 2.64 | 2.83 |  9.22 | 11.56
1.75 | 21.61 | 2.80 | 5.47 | 5.32 |  6.73 | 6.60 |  | 2.54 | 2.93 |  9.67 | 11.29
2.00 | 23.02 | 2.94 | 8.17 | 6.02 | 11.15 | 8.32 |  | 2.45 | 3.01 | 10.09 | 11.61

Model 2: Exponential distributions $X_{0}$ is exponential with density
$f_{0}(x)=e^{-x}$, $x>0$. $X_{1}$ is exponential with density
$f_{1}(x)=\mu^{-1}e^{-x/\mu}=f_{0}(x)e^{-\log\mu+(1-1/\mu)x}$, $x>0$, where
$\mu=1.25(0.25)3.00$ as in Qin and Zhang (2005). We choose $a=0$, $b=5\mu$, In
this model, the bandwidths for $\tilde{f}_{i\mathrm{S}}$ and
$\tilde{f}_{i\mathrm{N}}$ in Table 2 are those suggested by Qin and Zhang
(2005). The parametric MLE is
$\hat{f}_{i\mathrm{P}}(x)=f_{0}(x/\bar{x}_{i})/\bar{x}_{i}$, where
$\bar{x}_{i}$ is the sample mean of $x_{i1},\ldots,x_{in_{i}}$, $i=0,1$.

Figure 5: Simulated pointwise mean squared error of the MABLE $\hat{f}_{0}$, the one-sample parametric MLE $\hat{f}_{0\mathrm{P}}$, the semparametric kernel density estimate $\tilde{f}_{0\mathrm{S}}$, and the one-sample nonparametric kernel density estimate $\tilde{f}_{0\mathrm{N}}$ of $f_{0}$ based on 1000 datasets generated from exponential distributions with $n_{0}=n_{1}=50$. Table 2: Simulation results based on $B=1000$ Monte Carlo runs and samples of sizes $(n_{0},n_{1})$ from exponential distributions $\mathrm{Exp}(1)$ and $\mathrm{Exp}(\mu)$ using optimal degree $m=\tilde{m}$. |  |  | mse $\times 10^{2}$ |  | mise $\times 10^{4}$
---|---|---|---|---|---
$\mu$ | $E(\tilde{m})$ | $\sigma(\tilde{m})$ | $\hat{\alpha}_{0}$ | $\hat{\alpha}_{1}$ | $\tilde{\alpha}_{0}$ | $\tilde{\alpha}_{1}$ |  | $\hat{f}_{0\mathrm{P}}$ | $\hat{f}_{0}$ | $\tilde{f}_{0\mathrm{S}}$ | $\tilde{f}_{0\mathrm{N}}$
| $n_{0}=n_{1}=50$ |
1.25 | 5.04 | 2.45 | 5.01 | 4.46 | 4.93 | 4.41 |  | 8.39 | 11.61 | 91.98 | 99.95
1.50 | 4.87 | 1.96 | 5.07 | 4.01 | 5.29 | 4.10 |  | 7.00 |  8.54 | 75.84 | 82.96
1.75 | 4.72 | 1.81 | 4.94 | 3.42 | 5.81 | 3.79 |  | 5.59 |  6.69 | 66.23 | 72.76
2.00 | 4.68 | 2.25 | 5.03 | 3.53 | 6.33 | 4.06 |  | 5.82 |  6.75 | 58.51 | 63.84
2.25 | 4.70 | 2.54 | 4.64 | 3.06 | 6.46 | 3.75 |  | 4.82 |  5.45 | 52.16 | 56.51
2.50 | 4.66 | 2.81 | 4.71 | 3.20 | 7.46 | 4.16 |  | 4.42 |  4.89 | 48.76 | 53.04
2.75 | 4.54 | 2.20 | 4.54 | 3.13 | 7.28 | 3.99 |  | 4.21 |  4.59 | 44.03 | 47.72
3.00 | 4.58 | 2.87 | 5.25 | 2.96 | 8.99 | 4.14 |  | 3.39 |  3.68 | 40.62 | 43.90
| $n_{0}=n_{1}=100$ |
1.25 | 4.61 | 1.12 | 2.60 | 2.26 | 2.47 | 2.15 |  | 4.19 | 5.99 | 62.32 | 90.99
1.50 | 4.46 | 1.05 | 2.32 | 1.87 | 2.48 | 1.88 |  | 3.62 | 4.55 | 58.10 | 80.45
1.75 | 4.33 | 0.83 | 2.29 | 1.67 | 2.66 | 1.79 |  | 2.92 | 3.62 | 49.12 | 65.99
2.00 | 4.22 | 0.75 | 2.36 | 1.66 | 3.00 | 1.86 |  | 2.73 | 3.09 | 44.31 | 58.99
2.25 | 4.18 | 0.69 | 2.01 | 1.41 | 2.94 | 1.69 |  | 2.31 | 2.62 | 41.40 | 52.25
2.50 | 4.15 | 0.37 | 2.40 | 1.64 | 3.50 | 1.95 |  | 2.34 | 2.44 | 38.76 | 48.49
2.75 | 4.18 | 0.56 | 2.09 | 1.32 | 3.27 | 1.66 |  | 1.86 | 1.98 | 34.80 | 41.90
3.00 | 4.22 | 0.76 | 2.45 | 1.53 | 4.01 | 2.03 |  | 1.97 | 2.05 | 33.19 | 40.12

The kernel density estimates suffers from serious boundary effect for a
densities like expontial distribution. In the simulation presented in Figure 5
both $\tilde{f}_{i\mathrm{N}}$ and $\tilde{f}_{i\mathrm{S}}$ used the same
bandwidth selected by the default method of R function “density()” which seems
a little better than those selected by the method of Qin and Zhang (2005).

From the above simulation results we observe the folloowing. (i) The optimal
degree increases slowly as sample sizes increase; (ii) As sample sizes
increase the variation of the optimal degree decreases; (iii) The larger
$\alpha_{1}$ is the more eficient the proposed estimator $\hat{\alpha}_{1}$ is
than $\tilde{\alpha}_{1}$; (iv) The proposed estimator $\hat{f}_{0}$ is very
similar to the parametric one but is much better than the semiparamtric and
the nonparametric ones.

##  5 Large Sample Properties

We denote the chi-squared divergence($\chi^{2}$-distance) between densities
$\varphi$ and $\psi$ by

$\chi^{2}(\varphi\|\psi)=\int_{-\infty}^{\infty}\frac{[\varphi(y)-\psi(y)]^{2}}{\psi(y)}dy\equiv\int_{-\infty}^{\infty}\Big{[}\frac{\varphi}{\psi}(y)-1\Big{]}^{2}\psi(y)dy.$

We need the following assumptions for the asymptotic properties of
$\hat{f}_{\mathrm{B}}$ which will be proved in the appendix:

######  (A.1).

There exists $\bm{p}_{0}\in\mathbb{S}_{m}$ and $k>0$ such that
$[{f_{m}(x;\bm{p}_{0})-f_{0}(x)}]/{f_{0}(x)}=\mathcal{O}(m^{-k/2})$, uniformly
in $x\in[0,1]$, and thus
$\chi^{2}(f_{m}(\cdot;\bm{p}_{0})\|f_{0})=\mathcal{O}(m^{-k})$.

######  (A.2).

Assume that the zero vector $\bm{0}\in\mathcal{A}$ and that the components of
$\tilde{r}(x)$ are linearly independent.

Let $C^{(r)}[0,1]$ be the class of functions which have $r$th continuous
derivative $f^{(r)}$ on $[0,1]$. If $f_{0}\in C^{(r)}[0,1]$, and $f_{0}(x)\geq
b_{0}>0$, $x\in[0,1]$, then Assumption (A.1) is fulfilled with $k=r$ (Lorentz,
1963).

A weaker sufficient condition can assure Assumption (A.1). A function $f$ is
said to be $\gamma$–Hölder continuous with $\gamma\in(0,1]$ if
$|f(x)-f(y)|\leq C|x-y|^{\gamma}$ for some constant $C>0$. The following
(Lemma 3.1 of Wang and Guan, 2019) is a generalization of the result of
Lorentz (1963) which requires a positive lower bound for $f_{0}$.

######  Lemma 1.

Suppose that $f_{0}(x)=x^{a}(1-x)^{b}\varphi_{0}(x)$ is a density on $[0,1]$,
$a$ and $b$ are nonnegative real numbers, $\varphi_{0}\in C^{(r)}[0,1]$,
$r\geq 0$, $\varphi_{0}(x)\geq b_{0}>0$, and $\varphi_{0}^{(r)}$ is
$\gamma$-Hölder continuous with $\gamma\in(0,1]$. Then Assumption (A.1) is
fulfilled with $k=r+\gamma$.

We have the following asymptotic results in terms of distances
$D^{2}_{0}(\bm{\alpha},\bm{p})=\chi^{2}(f_{m}(\cdot;\bm{p})\|f_{0})$ and
$D^{2}_{1}(\bm{\alpha},\bm{p})=\chi^{2}(f_{m}(\cdot;\bm{\alpha},\bm{p})\|f_{1})$.

######  Theorem 1.

Under the density ratio model (1) and the assumptions (A.1) with $k>0$, and
(A.2), as $n\to\infty$, with probability one the maximum value of
$\ell_{m}(\bm{\alpha},\bm{p})$ with $m={\cal O}(n^{1/k})$ is attained at
$(\hat{\bm{\alpha}},\hat{\bm{p}})$ in the interior of
$\mathbb{B}_{m}(r_{n})=\\{(\bm{\alpha},\bm{p})\in\Theta_{m}(\mathcal{A})\,:\,D^{2}_{i}(\bm{\alpha},\bm{p})\leq
r_{n},\,i=0,1\\},$ where $r_{n}=n^{-1}\log n$. Thus the mean
$\chi^{2}$-distance between $f_{m}(\cdot;i\hat{\bm{\alpha}},\hat{\bm{p}})$ and
$f_{i}(\cdot)$ satisfies

$\mathrm{E}[D^{2}_{i}(\hat{\bm{\alpha}},\hat{\bm{p}})]=\mathrm{E}\int\frac{[f_{m}(x;i\hat{\bm{\alpha}},\hat{\bm{p}}_{m})-f_{i}(x)]^{2}}{f_{i}(x)}dx=\mathcal{O}\left(\frac{\log
n}{n}\right),\;i=0,1.$ (12)

Moreover, almost surely,
$\|\hat{\bm{\alpha}}-\bm{\alpha}_{0}\|^{2}=\mathcal{O}(\log n/n)$.

######  Remark 1.

Theorem 1 implies that $|\hat{F}_{i}(x)-F_{i}(x)|^{2}=\mathcal{O}(\log n/n)$,
uniformly on $[0,1]$, a.s., $i=0,1$.

##  6 Concluding Remark

Unlike the empirical likelihood method of Qin and Zhang (2003, 2005) in which
an estimate of a discrete probability mass function is obtained first then
smoothed using kernel method, the proposed method produces smooth estimates of
density and distribution functions directly. From the simulation study we also
conclude that the proposed method does not only simply smooth the estimation
but also gives more accurate estimates. The improvement over the existing
methods is significant especially for small samples. The proposed method also
gives better estimates of coefficients of logistic regression for
retrospective sampling data especially for samll sample data. Although the
optimal model degree is large for some data the effective degrees of freedom,
the number of nonzero mixing proportions $\hat{p}_{i}$, is usually much
smaller. Instead of the exponential tilting model (1), we can consider an even
more general weighted model $f_{1}(x)=f_{0}(x)w(x;\bm{\alpha}),$ where
$w(x;\bm{\alpha})$ is a known nonnegative weight with unknown parameter
$\bm{\alpha}$ and satisfies $\int w(x;\bm{\alpha})f_{0}(x)dx=1$ and
$w(x;\bm{0})=1$.

## Appendix

###  6.1 Proof of Proposition 1

###### Proof.

For any $f_{m}(x;\bm{\alpha},\bm{p}_{m})\in\mathcal{D}_{m}(\mathcal{A})$, we
have $f_{m}(x;\bm{p}_{m})=\sum_{j=0}^{m}p_{mj}\beta_{mj}(x),$ and

$\sum_{i=0}^{m}p_{mi}w_{mi}(\bm{\alpha})=\int_{0}^{1}f_{m}(x;\bm{p}_{m})\exp\\{\bm{\alpha}^{\mathrm{\scriptscriptstyle\top}}\tilde{r}(x)\\}dx=1,$

so that
$f_{m}(x;\bm{\alpha},\bm{p}_{m})=f_{m}(x;\bm{p}_{m})\exp\\{\bm{\alpha}^{\mathrm{\scriptscriptstyle\top}}\tilde{r}(x)\\}$.
By Property 3.1. of Wang and Ghosh (2012) or Lemma 2.2 of Guan (2017) we also
have that
$f_{m}(x;\bm{p}_{m})=f_{m+1}(x;\bm{p}_{m+1})=\sum_{j=0}^{m+1}p_{m+1,j}\beta_{m+1,j}(x)$
with $p_{m+1,0}=(m+1)p_{m0}/(m+2)$, $p_{m+1,m+1}=(m+1)p_{mm}/(m+2)$, and
$p_{m+1,j}=\left[{j}p_{m,j-1}+{(m-j+1)}p_{mj}\right]/({m+2})$,
$j\in\mathbb{I}_{1}^{m}.$ Thus we have
$f_{m}(x;\bm{\alpha},\bm{p}_{m})=f_{m+1}(x;\bm{p}_{m+1})\exp\\{\bm{\alpha}^{\mathrm{\scriptscriptstyle\top}}\tilde{r}(x)\\}=f_{m+1}(x;\bm{\alpha},\bm{p}_{m+1})$
and $\sum_{i=0}^{m+1}p_{m+1,i}w_{m+1,i}(\bm{\alpha})=1$. Hence
$f_{m}(x;\bm{\alpha},\bm{p}_{m})\in\mathcal{D}_{m+1}(\mathcal{A})$. ∎

###  6.2 Proof of Theorem 1

###### Proof.

Let
$\bm{\alpha}_{0}=(\alpha_{00},\ldots,\alpha_{0d})^{\mathrm{\scriptscriptstyle\top}}$
be the true value of $\bm{\alpha}$ so that
$\int_{0}^{1}f_{0}(x)\exp\\{\bm{\alpha}_{0}^{\mathrm{\scriptscriptstyle\top}}\tilde{r}(x)\\}dx=1.$
By Assumption (A.1), we have

$f_{m}(x;i\bm{\alpha}_{0},\bm{p}_{0})=f_{i}(x)+R_{m}(x)\exp\\{i\bm{\alpha}_{0}^{\mathrm{\scriptscriptstyle\top}}\tilde{r}(x)\\},\quad
i=0,1,$ (13)

where $R_{m}(x)=f_{0}(x)\mathcal{O}(m^{-r/2})$. Thus
$\int_{0}^{1}f_{m}(x;\bm{\alpha}_{0},\bm{p}_{0})dx=1+\int_{0}^{1}R_{m}(x)\exp\\{i\bm{\alpha}_{0}^{\mathrm{\scriptscriptstyle\top}}\tilde{r}(x)\\}dx=1+\rho_{m},$
where $\rho_{m}=\mathcal{O}(m^{-r/2})$. If we define
$\tilde{\bm{\alpha}}_{0}=\tilde{\bm{\alpha}}_{0}{(m)}=(\tilde{\alpha}_{00},\alpha_{01},\ldots,\alpha_{0d})^{\mathrm{\scriptscriptstyle\top}}$
with $\tilde{\alpha}_{00}=\alpha_{00}-\log(1+\rho_{m})$, then we have
$|\tilde{\bm{\alpha}}_{0}{(m)}-\bm{\alpha}_{0}|=|\log(1+\rho_{m})|=\mathcal{O}(m^{-r/2})$,
$\int_{0}^{1}f_{m}(x;\tilde{\bm{\alpha}}_{0},\bm{p}_{0})dx=\sum_{j=0}^{m}p_{0j}w_{mj}(\tilde{\bm{\alpha}}_{0})=1$,
and

$\frac{f_{m}(x;\tilde{\bm{\alpha}}_{0},\bm{p}_{0})-f_{1}(x)}{f_{1}(x)}=\frac{R_{m}(x)}{(1+\rho_{m})f_{0}(x)}-\frac{\rho_{m}}{1+\rho_{m}}=\mathcal{O}(m^{-r/2}).$

Define the log-likelihood ratio
$\mathcal{R}(\bm{\alpha},\bm{p})=\ell(\bm{\alpha}_{0},f_{0})-\ell_{m}(\bm{\alpha},\bm{p})$.
Thus we have

$\displaystyle\mathcal{R}(\bm{\alpha},\bm{p})$ $\displaystyle=$
$\displaystyle-\sum_{i=0}^{1}\sum_{j=1}^{n_{i}}\log[f_{m}(x_{ij};i\bm{\alpha},\bm{p})/f_{i}(x_{ij})].$
(14)

Consider subsets

$\Theta(\epsilon_{0})=\\{(\bm{\alpha},\bm{p})\in\Theta_{m}(\mathcal{A}):\mbox{$\forall$
$x\in[0,1]$, $i=0,1$,
$|f_{m}(x;i\bm{\alpha},\bm{p})/f_{i}(x)-1|\leq\epsilon_{0}$}\\},$

$0<\epsilon_{0}<1$. Clearly, by (A.1) and (13), $\Theta(\epsilon_{0})$ is
nonempty if $m$ is large enough.

By Taylor expansion we have, for
$(\bm{\alpha},\bm{p})\in\Theta(\epsilon_{0})$, $0<\epsilon_{0}<1$, and large
$m$,

$\displaystyle\mathcal{R}(\bm{\alpha},\bm{p})$ $\displaystyle=$
$\displaystyle\sum_{i=0}^{1}\left\\{\sum_{j=1}^{n_{i}}\Big{[}\frac{1}{2}U_{ij}^{2}(\bm{\alpha},\bm{p})-U_{ij}(\bm{\alpha},\bm{p})\Big{]}+\mathcal{O}(R_{mi}(\bm{\alpha},\bm{p}))\right\\},\,a.s.,$

where
$U_{ij}(\bm{\alpha},\bm{p})=[{f_{m}(x_{ij};i\bm{\alpha},\bm{p})-f_{i}(x_{ij})}]/{f_{i}(x_{ij})}$,
$j\in\mathbb{I}_{1}^{n_{i}}$, and
$R_{mi}(\bm{\alpha},\bm{p})=\sum_{j=1}^{n_{i}}U_{ij}^{2}(\bm{\alpha},\bm{p})$,
$i=0,1.$ Since $\mathrm{E}[U_{ij}(\bm{\alpha},\bm{p})]=0$,
$\sigma^{2}[U_{ij}(\bm{\alpha},\bm{p})]=\mathrm{E}[U_{ij}^{2}(\bm{\alpha},\bm{p})]=D^{2}_{i}(\bm{\alpha},\bm{p})$,
by the LIL we have
$\sum_{j=1}^{n}U_{ij}(\bm{\alpha},\bm{p})/\sigma[U_{ij}(\bm{\alpha},\bm{p})]=\mathcal{O}(\sqrt{n\log\log
n})$, a.s.. By the strong law of large numbers we have, a.s.,

$\mathcal{R}(\bm{\alpha},\bm{p})=\sum_{i=0}^{1}\left\\{\frac{n}{2}D^{2}_{i}(\bm{\alpha},\bm{p})-\mathcal{O}(D_{i}(\bm{\alpha},\bm{p})\sqrt{n\log\log
n})+o(nD^{2}_{i}(\bm{\alpha},\bm{p}))\right\\}.$ (15)

If $D^{2}_{i}(\bm{\alpha},\bm{p})=r_{n}=\log n/n$, then, by (15), there is an
$\eta>0$ such that $\mathcal{R}(\bm{\alpha},\bm{p})\geq\eta\log n,\,a.s..$ If
$(\bm{\alpha},\bm{p})=(\tilde{\bm{\alpha}}_{0},\bm{p}_{0})$ and $m=Cn^{1/k}$,
we have
$D^{2}_{i}(\tilde{\bm{\alpha}}_{0},\bm{p}_{0})=\mathcal{O}(m^{-k})=\mathcal{O}(n^{-1})$.
By (15) again we have
$\mathcal{R}(\tilde{\bm{\alpha}}_{0},\bm{p}_{0})=\mathcal{O}(\sqrt{\log\log
n})$, a.s.. Therefore, similar to the proof of Lemma 1 of Qin and Lawless
(1994), we have

$D^{2}_{i}(\hat{\bm{\alpha}},\hat{\bm{p}})=\int_{0}^{1}\frac{[f_{m}(x;i\hat{\bm{\alpha}},\hat{\bm{p}})-f_{i}(x)]^{2}}{f_{i}(x)}dx\leq\frac{\log
n}{n},a.s.,$ (16)

and $(\hat{\bm{\alpha}},\hat{\bm{p}})\in\Theta(\epsilon_{0})$. Thus (12)
follows. Define

$\psi({\bm{\alpha}},g)=\int_{0}^{1}\frac{g^{2}(x;{\bm{p}})}{f_{0}^{2}(x)}\left[\frac{w(x;{\bm{\alpha}})}{w(x;{\bm{\alpha}}_{0})}-1\right]^{2}f_{1}(x)dx,$

where
$w(x;{\bm{\alpha}})=\exp\\{\bm{\alpha}^{\mathrm{\scriptscriptstyle\top}}\tilde{r}(x)\\}$
and $g$ is a density on $[0,1]$. Then we have

$\displaystyle\psi(\hat{\bm{\alpha}},\hat{f}_{0})$ $\displaystyle\leq$
$\displaystyle
2D^{2}_{1}(\hat{\bm{\alpha}},\hat{\bm{p}})+2\int_{0}^{1}\left[\frac{f_{m}(x;\hat{\bm{p}})}{f_{0}(x)}-1\right]^{2}f_{1}(x)dx$
$\displaystyle\leq$ $\displaystyle
2D^{2}_{1}(\hat{\bm{\alpha}},\hat{\bm{p}})+2CD^{2}_{0}(\hat{\bm{\alpha}},\hat{\bm{p}})$
$\displaystyle=$ $\displaystyle\mathcal{O}(\log n/n),$

where $C=\max_{x\in[0,1]}w(x;{\bm{\alpha}}_{0})$. It is clear that
$\psi(\bm{\alpha}_{0},g)=0$, $\dot{\psi}(\bm{\alpha}_{0},g)=0$ and

$\ddot{\psi}(\bm{\alpha}_{0},f_{0})=2\int_{0}^{1}\tilde{r}(x)\tilde{r}^{\mathrm{\scriptscriptstyle\top}}(x)f_{1}(x)dx\equiv
2J(\bm{\alpha}_{0}).$

By Taylor expansion and (16) we have
$\psi(\hat{\bm{\alpha}},\hat{f}_{m})=(\hat{\bm{\alpha}}-\bm{\alpha}_{0})^{\mathrm{\scriptscriptstyle\top}}J(\bm{\alpha}_{0})(\hat{\bm{\alpha}}-\bm{\alpha}_{0})+o(R_{n}),$
where $R_{n}=\|\hat{\bm{\alpha}}-\bm{\alpha}_{0}\|^{2}+\mathcal{O}(\log n/n).$
Then we have
$(\hat{\bm{\alpha}}-\bm{\alpha}_{0})^{\mathrm{\scriptscriptstyle\top}}J(\hat{\bm{\alpha}}-\bm{\alpha}_{0})+o(\|\hat{\bm{\alpha}}-\bm{\alpha}_{0}\|^{2})=\psi(\hat{\bm{\alpha}},\hat{f}_{m})+o(\log
n/n)$ and thus
$(\lambda_{0}+o(1))\|\hat{\bm{\alpha}}-\bm{\alpha}_{0}\|^{2}\leq\mathcal{O}(\log
n/n)$, where $\lambda_{0}$ is the minimum eigenvalue of $J(\bm{\alpha}_{0})$.
Because the components of
$\tilde{r}(x)=(1,r^{\mathrm{\scriptscriptstyle\top}}(x))^{\mathrm{\scriptscriptstyle\top}}$
are linearly independent, $J(\bm{\alpha}_{0})$ is positive definite. Thus
$\lambda_{0}>0$ and we have
$\|\hat{\bm{\alpha}}-\bm{\alpha}_{0}\|^{2}=\mathcal{O}(\log n/n)$, a.s.. The
proof is complete. ∎

## References

  * Bickel et al. (1993) Bickel, P. J., Klaassen, C. A. J., Ritov, Y., and Wellner, J. A. (1993), Efficient and adaptive estimation for semiparametric models, Johns Hopkins Series in the Mathematical Sciences, Johns Hopkins University Press, Baltimore, MD.
  * Cheng and Chu (2004) Cheng, K. F., and Chu, C. K. (2004), “Semiparametric density estimation under a two-sample density ratio model,” Bernoulli, 10, 583–604.
  * Cox (1972) Cox, D. R. (1972), “Regression models and life-tables,” J. Roy. Statist. Soc. Ser. B, 34, 187–220.
  * Dabrowska and Doksum (1988) Dabrowska, D. M., and Doksum, K. A. (1988), “Estimation and testing in a two-sample generalized odds-rate model,” J. Amer. Statist. Assoc., 83, 744–749.
  * Efron and Tibshirani (1996) Efron, B., and Tibshirani, R. (1996), “Using specially designed exponential families for density estimation,” Annals of Statistics, 24, 2431–2461.
  * Guan (2016) Guan, Z. (2016), “Efficient and robust density estimation using Bernstein type polynomials,” Journal of Nonparametric Statistics, 28, 250–271.
  * Guan (2017) — (2017), “Bernstein Polynomial Model for Grouped Continuous Data,” Journal of Nonparametric Statistics, 29, 831–848.
  * Guan (2019) — (2019), mable: Maximum Approximate Bernstein/Beta Likelihood Estimation, r package version 3.0.
  * Guan (2021a) — (2021a), “Fast Nonparametric Maximum Likelihood Density Deconvolution Using Bernstein Polynomials,” Statistica Sinica, 31, to appear.
  * Guan (2021b) — (2021b), “Maximum approximate Bernstein likelihood estimation in proportional hazard model for interval-censored data,” Statistics in Medicine, 40, 758–778.
  * Hosmer and Lemeshow (1989) Hosmer, D. J., and Lemeshow, S. (1989), Applied logistic regression, New York: John Wiley & Sons Inc.
  * Ibragimov and Khasminskii (1983) Ibragimov, I., and Khasminskii, R. (1983), “Estimation of Distribution Density Belonging to a Class of Entire Functions,” Theory of Probability & Its Applications, 27, 551–562.
  * Lorentz (1963) Lorentz, G. G. (1963), “The degree of approximation by polynomials with positive coefficients,” Mathematische Annalen, 151, 239–251.
  * Qin and Lawless (1994) Qin, J., and Lawless, J. (1994), “Empirical likelihood and general estimating equations,” Ann. Statist., 22, 300–325.
  * Qin and Zhang (1997) Qin, J., and Zhang, B. (1997), “A goodness-of-fit test for logistic regression models based on case-control data,” Biometrika, 84, 609–618.
  * Qin and Zhang (2003) Qin, J.— (2003), “Using logistic regression procedures for estimating receiver operating characteristic curves,” Biometrika, 90, 585–596.
  * Qin and Zhang (2005) Qin, J.— (2005), “Density estimation under a two-sample semiparametric model,” Journal of Nonparametric Statistics, 17, 665–683.
  * Venkatraman and Begg (1996) Venkatraman, E. S., and Begg, C. B. (1996), “A distribution-free procedure for comparing receiver operating characteristic curves from a paired experiment,” Biometrika, 83, 835–848.
  * Vitale (1975) Vitale, R. A. (1975), “Bernstein polynomial approach to density function estimation,” in Statistical Inference and Related Topics (Proc. Summer Res. Inst. Statist. Inference for Stochastic Processes, Indiana Univ., Bloomington, Ind., 1974, Vol. 2; dedicated to Z. W. Birnbaum), New York: Academic Press, pp. 87–99.
  * Wang and Ghosh (2012) Wang, J., and Ghosh, S. K. (2012), “Shape restricted nonparametric regression with Bernstein polynomials,” Computational Statistics & Data Analysis, 56, 2729–2841.
  * Wang and Guan (2019) Wang, T., and Guan, Z. (2019), “Bernstein polynomial model for nonparametric multivariate density,” Statistics, 53, 321–338.
  * Wieand et al. (1989) Wieand, S., Gail, M. H., James, B. R., and James, K. L. (1989), “A family of nonparametric statistics for comparing diagnostic markers with paired or unpaired data,” Biometrika, 76, 585–592.
  * Zou et al. (1997) Zou, K. H., Hall, W. J., and Shapiro, D. E. (1997), “Smooth non-parametric receiver operating characteristic (ROC) curves for continuous diagnostic tests,” Statistics in Medicine, 16, 2143–2156.


# Music2Video: Automatic Generation of Music Video with fusion of audio and
text

###### Abstract

Creation of images using generative adversarial networks has been widely
adapted into multi-modal regime with the advent of multi-modal representation
models pre-trained on large corpus. Various modalities sharing a common
representation space could be utilized to guide the generative models to
create images from text or even from audio source. Departing from the previous
methods that solely rely on either text or audio, we exploit the
expressiveness of both modality. Based on the fusion of text and audio, we
create video whose content is consistent with the distinct modalities that are
provided. A simple approach to automatically segment the video into variable
length intervals and maintain time consistency in generated video is part of
our method. Our proposed framework for generating music video shows promising
results in application level where users can interactively feed in music
source and text source to create artistic music videos. Our code is available
at this https URL.

##  1 Introduction

The rapid development in deep learning has brought the visual generative
models to not only create realistic images but to creatively render fictional
images. Generative adversarial network models such as BigGAN, StyleGAN, and
VQGAN exert their creative ability from randomly generated latent codes [2,
10, 11, 7]. Since a single latent code corresponds to a particular image,
modification in the latent space of GAN naturally creates a different image.
In order to find meaningful manipulative directions in latent space of GAN,
unsupervised methods such as singular value decomposition on sampled latent
codes has been used.

On the other hand, common representation space of language and vision [13, 15,
22, 3] allows iterative update on the latent code of GAN to create a intended
image which is given in a text format. For illustration, a popular multi-modal
representation, CLIP [19], which can encode the image and text into the common
representation space is widely adapted with generative models. Since CLIP is a
pre-trained model that can measure similarity between image and text in a
shared latent space, variety of tasks that employ both image and text relies
on the representational ability of CLIP. CLIP encodes the generated image and
text guidance into a common vector space and compute the cosine similarity
between the image encoding and text encoding to determine if they are similar.

Recent work extends the representational ability of CLIP into audio using
diffusion [27]. The shared representation space of image, text, and audio
enables the latent code of GAN to generate an intended image when both text
and audio given. However, naively integrating the audio representation into
the text-based generation process tends to impedes the visualization of text
representation. The conflicting behavior of the two modalities is the major
obstacle in automatic music video generation. Moreover, it is vital to
consider the property of music. Since music consists of varying themes which
change irregularly, the created music video should correspondingly change its
theme accordingly which is hard to capture when the music is segmented with
fixed time interval.

The proposed method of this paper has two contributions.

  * •

We propose a method to seamlessly integrate the audio guidance and text
guidance to create a music video of high quality.

  * •

We also show how to determine the change of scenery in the video based on the
musical statistics.

##  2 Related Works

###  2.1 Common Representation of Diverse Modalities

Learning common representation of image and text using contrastive objective
has gained great attention for its versatile usage in text-guided generation
and manipulation of image. Multi-modal representation learning such as CLIP
and its variants [19, 27] consist of two separate encoder networks each for
different modality. For a pair of positive multi-modal samples, contrastive
learning framework attempts to minimize the distance between the different
modalities while pushing away the unrelated samples. The pre-trained model
could be used for comparing the representation of image, text and audio. Works
that optimize image generating process generally minimize the distance between
the generated image representation and text prompt representation.

Figure 1: A graphical description of overall framework of music2video

###  2.2 Vector Quantized GAN

Convolutional neural network [12] has been widely used for learning the
representation of image since it captures the local patterns of a given image
using kernels. However, CNN models fail to encode the long-range interaction
between local features. In order to recognize the long-term dependency of
high-level features using CNN, vector quantization [23, 7] summarizes the
local features into a code book. The discretized code book enables transformer
model to compute attention between the local features. Using the encoder
network that maps the original image into the latent space, VQ-GAN
additionally uses the vector quantization process to represent the latent
space using code book. Using the discretized latent space, generator
reconstructs the latent code into realistic images using adversarial loss
between generator and discriminator.

###  2.3 CLIP Based Image Generation

Linear manipulation in the latent space of GAN corresponds to a different
generated image. Such interpretable directions can found in both supervised
[20, 1] and unsupervised [8, 4, 24, 21, 26, 9, 18] way. Recent methods [18, 8,
4] find such directions with the aid of multi-modal representation of CLIP.
Text based image generation methods are built on off-shelf image generative
models such as VQ-VAE [5] and Big-GAN [14].

###  2.4 Visualizing Audio

The extension of CLIP which is originally a visual-language model allows
audio-based image generation. A method that is built on the pretrained Big-GAN
model utilizes the intensity controlling property of the noise vector [2]
which adapts naturally to frame per second velocity based on the pitch of
audio [16]. On the other hand, VQ-GAN is a unconditioned GAN which is capable
of generating images out of ImageNET classes. Therefore, VQ-GAN based image
generation could be widely adapted to express various themes [17].

(a) L1 Loss: frame 1 - Campfire under sky full of stars

(b) L1 Loss: frame 2 - Crowd of people laughing

(c) Add prompts: frame 1 - Campfire under sky full of stars

(d) Add prompts: frame 2 - Crowd of people laughing

Figure 2: Snapshots of a video sample with the corresponding text prompts.

##  3 Our Framework

We discuss how to automatically find a time stamp where scene transition
should take place in Section 3.1. Based on the variable length segments of
music, we demonstrate how to fuse the distinct guidance of audio and text
domain to create sequence of images in Section 3.2. Based on the created
sequence of images, we generate a video that maintains context from previous
frame which is shown in detail in Section 3.3. The flow of overall framework
is described in Figure 1.

###  3.1 Video Scenery Transition based on Music

In order to create a music video which changes the scenery accordingly with
the music, we determine the time stamp where the musical statistics change
drastically based on the strength of onset [6]. We use the measured beat of
music to split the music into dynamic intervals. The music used for
demonstration is ‘Be my Weasel’ [25] composed by Yannic Kilcher. The music is
divided into roughly 500 segments whose average time interval is 0.6 seconds.

Given the segments of music, we determine frame-per-second based on the
average intensity of the music segment. The given music is transformed into
mel-spectrogram which shows intensity of each frequency in decibel for a given
time interval. We compute average intensity of each segment then regularize
the intensity of each segment to range between 0 and 1. The regularized
intensity value corresponds to frame-per-second. Since each iteration of VQ-
GAN generation process creates one image, frame-per-second is equivalent to
the repeated number of image generation and latent code optimization process.

###  3.2 Iterative Audio and Text Guided Optimization

The naive approach of iteratively optimizing the generated image with audio
and text fails to produce quality images. In this section we present details
of optimizing generation process of VQ-GAN using both audio and text prompts.
Given a segmented sections of music as illustrated in Section 3.1, we provide
a single guidance for multiple frames in a single section. Since multiple
frames correspond to the same number of iterations, each iteration will have
the same audio or text guidance for an extended period. To illustrate further,
each section is given an segmented audio prompt encoded into CLIP vector and
only if the section consists of the time stamp given with the lyric, then the
lyric encoded into CLIP vector is given instead. Unlike the iterative approach
which alternates audio and text guidance for every iteration, this approach
successfully represent both modalities of music and lyrics.

###  3.3 Time Consistency in Video Generation

Since every frame in a video should be consistent with the previous frame, we
explain how to maintain time consistency. We use two methods, L1
regularization of latent vectors and addition of guiding prompt vectors.

The first approach we experiment is to regularize the latent vector of VQ-GAN
at iteration $t$ which is denoted as $z_{t}$ to have minimized l1 norm with
the previous latent vector as in Equation (1).

$min||z_{t}-z_{t-1}||$ (1)

This method fails to show consistency of frames therefore we use the second
approach Equation (2) adding the two distinct CLIP prompts for different
frames. Providing a new guidance of adding the two CLIP vectors showed
consistency in frames such as remaining objects from the previous frame as
shown in Figure 2.

$\frac{CLIP(p_{t-1})+CLIP(p_{t})}{2}$ (2)

(a) be my weasel

(b) what’s inside this wooden chest

(c) be my pig

(d) and bring them all to my whiskey jug

Figure 3: Snapshots of a music video sample with the corresponding lyrics.

## References

  * [1] Rameen Abdal, Peihao Zhu, Niloy J Mitra, and Peter Wonka.  Styleflow: Attribute-conditioned exploration of stylegan-generated images using conditional continuous normalizing flows.  TOG, 2021.
  * [2] Andrew Brock, Jeff Donahue, and Karen Simonyan.  Large scale gan training for high fidelity natural image synthesis, 2019\.
  * [3] Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu.  Uniter: Universal image-text representation learning, 2020.
  * [4] Edo Collins, Raja Bala, Bob Price, and Sabine Susstrunk.  Editing in style: Uncovering the local semantics of gans.  In CVPR, 2020.
  * [5] Ming Ding, Zhuoyi Yang, Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, and Jie Tang.  Cogview: Mastering text-to-image generation via transformers, 2021.
  * [6] Daniel PW Ellis.  Beat tracking by dynamic programming.  Journal of New Music Research, 36(1):51–60, 2007.
  * [7] Patrick Esser, Robin Rombach, and Björn Ommer.  Taming transformers for high-resolution image synthesis, 2020.
  * [8] Erik Härkönen, Aaron Hertzmann, Jaakko Lehtinen, and Sylvain Paris.  Ganspace: Discovering interpretable gan controls.  In NeurIPS, 2020.
  * [9] Ali Jahanian*, Lucy Chai*, and Phillip Isola.  On the ”steerability” of generative adversarial networks.  In ICLR, 2020.
  * [10] Tero Karras, Samuli Laine, and Timo Aila.  A style-based generator architecture for generative adversarial networks, 2019.
  * [11] Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila.  Analyzing and improving the image quality of stylegan, 2020.
  * [12] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton.  Imagenet classification with deep convolutional neural networks.  In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 25. Curran Associates, Inc., 2012.
  * [13] Angeliki Lazaridou, Nghia The Pham, and Marco Baroni.  Combining language and vision with a multimodal skip-gram model, 2015\.
  * [14] Xingchao Liu, Chengyue Gong, Lemeng Wu, Shujian Zhang, Hao Su, and Qiang Liu.  Fusedream: Training-free text-to-image generation with improved clip+gan space optimization, 2021.
  * [15] Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee.  Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks, 2019.
  * [16] msieg.  Deep music visualizer.  https://github.com/msieg/deep-music-visualizere, 2019.
  * [17] nerdyrodent.  Vqgan-clip.  https://github.com/nerdyrodent/VQGAN-CLIP, 2021.
  * [18] Or Patashnik, Zongze Wu, Eli Shechtman, Daniel Cohen-Or, and Dani Lischinski.  Styleclip: Text-driven manipulation of stylegan imagery.  In ICCV, 2021.
  * [19] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever.  Learning transferable visual models from natural language supervision, 2021.
  * [20] Yujun Shen, Jinjin Gu, Xiaoou Tang, and Bolei Zhou.  Interpreting the latent space of gans for semantic face editing.  In CVPR, 2020.
  * [21] Yujun Shen and Bolei Zhou.  Closed-form factorization of latent semantics in gans.  In CVPR, 2021.
  * [22] Hao Tan and Mohit Bansal.  Lxmert: Learning cross-modality encoder representations from transformers, 2019.
  * [23] Aäron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu.  Neural discrete representation learning.  CoRR, abs/1711.00937, 2017.
  * [24] Andrey Voynov and Artem Babenko.  Unsupervised discovery of interpretable directions in the gan latent space.  In ICML, 2020.
  * [25] VQGAN+CLIP.  Clip music video.  https://github.com/yk/clip_music_video, 2021.
  * [26] Binxu Wang and Carlos R Ponce.  A geometric analysis of deep generative image models and its applications.  In ICLR, 2021.
  * [27] Ho-Hsiang Wu, Prem Seetharaman, Kundan Kumar, and Juan Pablo Bello.  Wav2clip: Learning robust audio representations from clip, 2021.


Online Handbook of Argumentation for AI Volume 1 Edited by

Federico Castagna
Francesca Mosca
Jack Mumford
Stefan Sarkadi
Andreas Xydis

June 2020

Preface

This volume contains revised versions of the papers selected for the first
volume of the Online Handbook of Argumentation for AI (OHAAI). Previously,
formal theories of argument and argument interaction have been proposed and
studied, and this has led to the more recent study of computational models of
argument. Argumentation, as a field within artificial intelligence (AI), is
highly relevant for researchers interested in symbolic representations of
knowledge and defeasible reasoning. The purpose of this handbook is to provide
an open access and curated anthology for the argumentation research community.
OHAAI is designed to serve as a research hub to keep track of the latest and
upcoming PhD-driven research on the theory and application of argumentation in
all areas related to AI. The handbook’s goals are to:

  1. 1.

Encourage collaboration and knowledge discovery between members of the
argumentation community.

  2. 2.

Provide a platform for PhD students to have their work published in a citable
peer-reviewed venue.

  3. 3.

Present an indication of the scope and quality of PhD research involving
argumentation for AI.

The papers in this volume are those selected for inclusion in OHAAI Vol.1
following a back-and-forth peer-review process undertaken by the editors of
OHAAI Vol.1. The volume thus presents a strong representation of the current
state of the art research of argumentation in AI that has been strictly
undertaken during PhD studies. Papers in this volume are listed alphabetically
by author. We hope that you will enjoy reading this handbook.

---
Editors
Federico Castagna
Francesca Mosca
Jack Mumford
Stefan Sarkadi
Andreas Xydis
June 2020

Acknowledgements

We thank the senior researchers in the area of Argumentation and Artificial
Intelligence for their efforts in spreading the word about the OHAAI project
with early-career researchers.

We are also grateful to ArXiv for their willingness to publish this handbook.

We are especially thankful to Costanza Hardouin for designing the OHAAI logo.

We owe many thanks to Sanjay Modgil for helping to form the motivation for the
handbook, and to Elizabeth Black and Simon Parsons for their advice and
guidance that enabled the OHAAI project to come to fruition.

We owe special thanks to the contributing authors: Federico Castagna,
Timotheus Kampik, Atefeh Keshavarzi Zafarghandi, Mickaël Lafages, Jack
Mumford, Christos T. Rodosthenous, Samy Sá, Stefan Sarkadi, Joseph Singleton,
Kenneth Skiba, Andreas Xydis. Thank you for making the world of argumentation
greater!

See pages - of castagna.pdf

See pages - of kampik.pdf

See pages - of keshavarzi.pdf

See pages - of lafages.pdf

See pages - of mumford.pdf

See pages - of rodosthenous.pdf

See pages - of sa.pdf

See pages - of sarkadi.pdf

See pages - of singleton.pdf

See pages - of skiba.pdf

See pages - of xydis.pdf


# Thermodynamics of interacting tachyonic scalar field

S. D<EMAIL_ADDRESS>M. M<EMAIL_ADDRESS>and
Shiyuan<EMAIL_ADDRESS>
(a) School of Physics, Shandong University, Jinan 250100, China.
(b) Department of Physics, University of Lucknow, Lucknow 274505 India.

###### Abstract

In this paper we discuss the laws of thermodynamics for interacting tachyonic
scalar field. The components of the tachyonic scalar field in the universe are
taken to exist in the state of non-equilibrium initially, but due to
interaction they undergo a transition towards the equilibrium state. We show
that the zeroth law of thermodynamics demands interaction among the components
of cosmic field. The second law of thermodynamics is governing dynamics in
transfer of energy among the three components of the proposed field with local
violation of conservation of energy for individual components. .

##  1 Introduction

The accelerated expansion of the universe reveal by a large number of
cosmological observations [1] and can be understood by introduction of
repulsive gravity, although the other alternatives [2] may also explain the
accelerated expansion. The repulsive gravity demands some different types of
entities which have negative pressure. Considering its repulsive character
such type of entity is called dark energy.

A class of scalar fields is one of the promising candidates of dark energy.
Among itself tachyonic scalar field appearing in the context of string theory
[3] is logically more appealing than its counterpart quintessence due to its
relativistic Lagrangian as analogue of particles. Cosmological relevance of
this field has been studied by several authors during last few decades [4].
Action and Lagrangian of the cosmological tachyonic scalar field is given by

$\displaystyle\mathcal{A}=\int d^{4}x\sqrt{-g}\left(\frac{R}{16\pi
G}-V(\phi)\sqrt{1-\partial^{i}\phi\partial_{i}\phi}\right)$ (1)

with Lagrangian $L=-V(\phi)\sqrt{1-\partial_{i}\phi\partial^{i}\phi}$ and the
equation of motion of the field is found by varying action as

$\frac{\ddot{\phi}}{\dot{\phi}}+\frac{(1-\dot{\phi}^{2})V^{\prime}(\phi)}{\dot{\phi}V(\phi)}+3H(1-\dot{\phi}^{2})=0$
(2)

The stress energy tensor for this Lagrangian

$\displaystyle
T^{ik}=\frac{\partial{L}}{\partial{(\partial_{i}\phi)}}\partial^{k}\phi-g^{ik}L$
(3)

This gives energy density and pressure for spatially homogeneous field as

$\displaystyle\rho=\frac{V(\phi)}{\sqrt{1-\dot{\phi}^{2}}};\qquad
P=-V(\phi)\sqrt{1-\dot{\phi}^{2}}$ (4)

respectively.

##  2 Components of Field

We assume that radiation with equation of state $w_{r}=1/3$ also exists as one
inherent component of same tachyonic scalar field. Due to some physical
mechanism not known in detail at present to us, the cosmic tachyonic scalar
field may be decomposed into several components with the assumption that the
field is spatially homogeneous. we can write the expressions for energy
density and pressure as

$\displaystyle
P=-\frac{V(\phi)}{\sqrt{1-\dot{\phi}^{2}}}+\frac{\dot{\phi}^{2}V(\phi)}{\sqrt{1-\dot{\phi}^{2}}}+0$
(5)

$\displaystyle\rho=\frac{V(\phi)}{\sqrt{1-\dot{\phi}^{2}}}+\frac{3\dot{\phi}^{2}V(\phi)}{\sqrt{1-\dot{\phi}^{2}}}-\frac{3\dot{\phi}^{2}V(\phi)}{\sqrt{1-\dot{\phi}^{2}}}.$
(6)

From (5) and (6) it is seen that when we include radiation in tachyonic scalar
field then one new exotic component also appears (say, exotic matter since its
energy density is negative) with zero pressure. Thus, the tachyonic scalar
field resolves into three components say $a$, $b$ and $c$. The pressure and
energy density of $a$ is given as

$P_{a}=-\frac{V(\phi)}{\sqrt{1-\dot{\phi}^{2}}},\quad\rho_{a}=\frac{V(\phi)}{\sqrt{1-\dot{\phi}^{2}}}\Rightarrow
w_{a}=-1=w_{\lambda}.$

This is nothing but the ‘true’ cosmological constant because of its equation
of state being $w_{\lambda}=-1$.

The second component with

$P_{b}=\frac{\dot{\phi}^{2}V(\phi)}{\sqrt{1-\dot{\phi}^{2}}},\quad\rho_{b}=\frac{3\dot{\phi}^{2}V(\phi)}{\sqrt{1-\dot{\phi}^{2}}}\Rightarrow
w_{b}=1/3$

can be identified as radiation with $w_{r}=1/3$. The last component is
characterised by

$P_{c}=0,\quad\rho_{c}=-\frac{3\dot{\phi}^{2}V(\phi)}{\sqrt{1-\dot{\phi}^{2}}}\Rightarrow
w_{c}=0.$

This component mimics dust matter but has negative energy density. The exotic
matter may include the Dirac fermions as well as the Majorana fermions whence
the negative energy states turn into the positive energy states[5, 6]. In our
earlier work [7] we allowed a small time dependent perturbation in the
equation of state(EoS) of the cosmological constant with
$\bar{w}_{\lambda}=-1+\varepsilon(t)$. Thus, with the perturbed EoS, the true
cosmological constant becomes a shifted cosmological parameter. This has a
bearing upon the EoS of radiation and exotic matter, both. Therefore, these
two entities turn into shifted radiation and shifted exotic matter
respectively. With fixed energy density of field components, the expressions
for the energy density and pressure of each component are given as below. For
the shifted cosmological constant one has

$\displaystyle\bar{\rho}_{\lambda}=\frac{V(\phi)}{\sqrt{1-\dot{\phi}^{2}}};\qquad\bar{p}_{\lambda}=\frac{-V(\phi)}{\sqrt{1-\dot{\phi}^{2}}}+\frac{\varepsilon
V(\phi)}{\sqrt{1-\dot{\phi}^{2}}}$ (7)

and $\bar{w}_{\lambda}=-1+\varepsilon(t)$. For shifted radiation, we have

$\displaystyle\bar{\rho}_{r}=\frac{3\dot{\phi}^{2}V(\phi)}{\sqrt{1-\dot{\phi}^{2}}};\qquad\bar{p}_{r}=\frac{(1+3\varepsilon)\dot{\phi}^{2}V(\phi)}{\sqrt{1-\dot{\phi}^{2}}}$
(8)

In presence of perturbation the zero pressure of exotic matter turns into
negative non-zero pressure due to shifted exotic matter which would also
accelerate the universe like dark energy. Thus, the energy density and
pressure for shifted exotic matter are now, respectively, given as

$\displaystyle\bar{\rho}_{m}=\frac{-3\dot{\phi}^{2}V(\phi)}{\sqrt{1-\dot{\phi}^{2}}};\qquad\bar{p}_{m}=p_{\phi}-\bar{p}_{\lambda}-\bar{p}_{r}=\frac{-\varepsilon(1+3\dot{\phi}^{2})V(\phi)}{\sqrt{1-\dot{\phi}^{2}}}$
(9)

with $\bar{w}_{m}=\frac{\varepsilon(1+3\dot{\phi}^{2})}{3\dot{\phi}^{2}}$.

##  3 Thermodynamics Laws for interacting components

Why must the components of cosmic field interact? This is one of the most
interesting question about interaction. Interaction might be justify by
thermodynamics [8]. As shown by obvious observations the cosmic field must
include at least three components representing matter, dark energy and
radiation (many other components are also possible) and behave as an ensemble
of three interacting thermodynamic systems. We apply the Zeroth law of the
thermodynamics to these three systems called as shifted cosmological parameter
(SCP), shifted radiation (SR) and shifted exotic matter (SEM) each. _If SCP is
in equilibrium with SR and SR is in equilibrium with SEM then SEM should be
also in equilibrium with SCP_. This zeroth law demands interaction among
cosmic field components whenever equilibrium gets perturbed for any reasons.
If equilibrium is disturbed and the components are in non-equilibrium
(thermal, mechanical or else), then to re-attain the equilibrium the
components must interact mutually. If the components are in equilibrium then
due to interaction perturbation in equilibrium state reacts trying to restore
its state or achieve a new one (Le Châtelier-Braun principle) [9, 10]. If all
components of tachyonic scalar field are in non-equilibrium, then, to achieve
an equilibrium state they must fall into mutual interaction. This motivation
provides one justification to study the interaction of these components. Here,
we assuming that even though the total energy of the perturbed field
(spatially homogeneous) is kept conserved (First law of thermodynamics) yet
during interaction it can get reasonably violated for individual components.
We consider the three components of cosmic tachyonic scalar field the
components are shifted cosmological parameter (SCP), shifted radiation (SR)
and shifted exotic matter (SEM). The individual equations of energy
conservation for SCP, SR and SEM are respectively given as,

$\displaystyle\dot{\bar{\rho}}_{\lambda}+3H(1+\bar{w}_{\lambda})\bar{\rho}_{\lambda}=-Q_{1}$
(10) $\displaystyle\dot{\bar{\rho}}_{r}+3H(1+\bar{w}_{r})\bar{\rho}_{r}=Q_{2}$
(11)
$\displaystyle\dot{\bar{\rho}}_{m}+3H(1+\bar{w}_{m})\bar{\rho}_{m}=Q_{1}-Q_{2}$
(12)

where $Q_{1}$ and $Q_{2}$ are the interaction strengths and $H$ is Hubble
parameter. Second Law of Thermodynamics is the governing dynamics of
interaction and sign of $Q_{1}$ and $Q_{2}$ shows the direction of flow of
energy density during interaction among components. The positivity of the
quantity $Q_{1}-Q_{2}$ implies that $Q_{1}$ should be large and positive. For
if $Q_{1}$ had been large and negative, then the second law of thermodynamics
would have been violated and the SCP (as the dark energy candidate) would have
dominated much earlier withholding the structure formation against the present
observations. Also, $Q_{2}$ should be positive and small since if it is
negative and large then conservation of energy of tachyonic field is violated.

##  4 Interaction among the components

The interacting dark energy models have been recently proposed by several
authors [11]. We study the interaction of these three components assuming that
even though the total energy of the perturbed field (spatially homogeneous) is
kept conserved , yet during interaction it can get reasonably violated for
individual components.

In the above expressions (10),(11) and (12) the following broad conditions
must govern the dynamics.

Condition(I) $\mid Q_{1}\mid>\mid Q_{2}\mid$. This corresponds to the
following cases,

(i)

If $Q_{1}>0$, $Q_{2}>0$ then the right hand side of (10) is negative while
(11) and (12) are positive, respectively. This means that there is energy
transfer from shifted cosmological parameter to shifted radiation and shifted
exotic matter, respectively. Thermodynamics allows this kind of transfer of
energy.

(ii)

$Q_{1}<0$, $Q_{2}<0$ implies that there is an energy transfer to shifted
cosmological parameter from shifted radiation and shifted exotic matter.

Condition(II) $\mid Q_{2}\mid>\mid Q_{1}\mid$.

(i)

$Q_{2}>0$, $Q_{1}>0$ would make the right hand side of (11) as positive and
(10) and (12) as negative. This shows that there is an energy transfer to
shifted radiation from shifted cosmological parameter and shifted exotic
matter. Thermodynamics again does not allow this kind interaction.

(ii)

$Q_{1}<0$, $Q_{2}<0$ makes way for the energy transfer from shifted radiation
to shifted exotic matter and shifted cosmological parameter.

Condition(III) If $Q_{2}=Q_{1}=Q$ then we have the following possibility

(i)

$Q>0$ leads to an energy transfer to shifted radiation from shifted
cosmological parameter, while the shifted exotic matter remains free from
interaction with its energy density held conserved. This type of interaction
holds compatibility with the laws of thermodynamics.

(ii)

If $Q<0$, energy would flow from shifted radiation to shifted cosmological
parameter, whereas the shifted exotic matter does not get involved in
interaction mechanism. Thus, the conservation of energy for shifted exotic
matter holds good.

(iii)

As an alternative, $Q=0$ would pull the components of tachyonic scalar field
out of mutual interaction like the standard $\Lambda$CDM model.

The second case of condition (I) and condition (II) violate the laws of
thermodynamics, therefore, we are not interested in these types of
interactions. The interaction of type condition (iii) has been discussed for
two components of tachyonic scalar field in our earlier work [12]. Due to the
lack of information regarding the exact nature of dark matter and dark energy
(as the cosmological constant or else) we present the form of interaction term
heuristically as function of time rate of change in energy densities as

$\displaystyle Q_{1}=\alpha\dot{\bar{\rho_{\lambda}}};\quad
Q_{2}=\beta\dot{\bar{\rho_{r}}}$ (13)

where $\alpha,\beta$ are proportionality constant. While several authors have
proposed different forms of $Q$ [11].

From (10), (11) and (12), for the specific dynamical form of interaction
strengths (13) one can found the functional form of energy density with
redshift $z$ as,

$\displaystyle\bar{\rho}_{\lambda}=\bar{\rho}^{0}_{\lambda}x^{3\varepsilon/1+\alpha}$
(14)

where $\frac{a_{0}}{a}=1+z=x$

$\displaystyle\bar{\rho}_{r}=\bar{\rho}^{0}_{r}x^{4+3\varepsilon/1-\beta}$
(15)

and

$\displaystyle\bar{\rho}_{m}=\bar{\rho}^{0}_{m}x^{\eta}+\left(\frac{3\varepsilon\alpha\bar{\rho}^{0}_{\lambda}}{3\varepsilon-\eta-\eta\alpha}\right)[x^{3\varepsilon/1+\alpha}-x^{\eta}]-\left(\frac{\beta\bar{\rho}^{0}_{r}(4+3\varepsilon)}{4+3\varepsilon-\eta+\eta\beta}\right)[x^{4+3\varepsilon/1-\beta}-x^{\eta}]$
(16)

where $\eta$ assuming constant (with $\dot{\phi}^{2}\approx$ constant) is
defined as

$\displaystyle\eta=\frac{3\dot{\phi}^{2}(1+\varepsilon)+\varepsilon}{\dot{\phi}^{2}}.$
(17)

Thus the cosmic expansion history of the universe is given by Hubble parameter
with interaction as,

$H^{2}=\frac{\kappa^{2}}{3}[\bar{\rho}_{\lambda}+\bar{\rho}_{r}+\bar{\rho}_{m}]$
(18)

where $\kappa^{2}=8\pi G$.

##  5 Conclusion

Having the motivation for the relativistic (tachyonic) scalar field, in
contrast to quintessence, the single tachyonic scalar field which splits due
to some unknown mechanism into three components (cosmological constant,
radiation and dust matter). Due to consideration of radiation in this field
the dust matter appears with negative energy. A small perturbation allowed in
EoS of cosmological constant changes its status from a true cosmological
constant to a shifted cosmological parameter (SCP). Similarly, status of
radiation and dust matter changes to shifted radiation (SR) and shifted exotic
matter (SEM). Thermodynamics laws (Zeroth, First and Second) might be
responsible for interaction among components of single cosmic field. We
consider the components of the field as thermodynamic systems and they
interact to achieve a thermodynamical equilibrium. Particularly Zeroth Law
invite interaction among components to maintain thermodynamical equilibrium or
get new one, First Law demands the total energy of field stays conserved but
the field components mutually interact with interaction strength parameter $Q$
resulting in local violation of energy conservation and Second Law decide the
direction of flow of energy during interaction. The entire evolution of the
universe arises from this process of interaction.

##  6 Acknowledgement

We thank the members of the theoretical physics group of Shandong University
at Jinan, China for helpful discussions. MMV also thanks Edward Rocky Kolb for
the kind hospitality at Kavli Institute of Cosmological Physics, the
University of Chicago, USA.

## References

  * [1] A. G. Riess et al. Astron. J. 116, 1009 (1998); S. Perlmutter et al. Astrophys. J. 517, 565 (1999); D. N. Spergel et al. Astrophys. J. Suppl. 148, 175 (2003); E. Komatsu et al. ApJS 192, 18 (2011); P. A. R. Ade et al. A & A 571, A16 (2014).
  * [2] K. Tomita, Astrophys. J. 529, 38 (2000); K. Tomita, Mon. Not. Roy. Astron. Soc. 326, 287 (2001); M. N. Celerier, Astron. Astrophys 353, 63 (2000); H. Iguchi, T. Nakamura and K. I. Nakao, Prog. Theor. Phys 108, 809 (2002).
  * [3] A.Sen, JHEP 0204, 048(2002); A.Sen, JHEP 0207, 065 (2002); A.Sen, Mod.Phys.Lett. A17, 1797 (2002).
  * [4] G.W. Gibbons, Phys. Lett. B 537, 1 (2002); T. Padmanabhan, Phys. Rev. D 66, 021301(2002); A. Frolov, L. Kofman and A. Starobinsky, Phys. Lett. B 545, 8 (2002); Huiquan Li, JCAP, 07, 018 (2012); D. Choudhury, D. Ghoshal, D.P. Jatkar and S. Panda, Phys. Lett. B, 544 231(2002); D.A. Steer and F. Vernizzi, Phys. Rev. D 70 043527 (2004); G. Shiu and I. Wasserman, Phys. Lett. B 541, 6 (2002).
  * [5] E. S. Reich, NATURE 483, 132 (2012).
  * [6] R. M. Lutchyn, J. D. Sau and S. D. Sarma, Phys. Rev. Lett. 105, 077001 (2010).
  * [7] M. M. Verma and S. D. Pathak, Astrophys. Space Sci. 344, 505(2013).
  * [8] S. D. Pathak and M. M. Verma, Journal of Physics: Conference Series 481, 012015 (2014).
  * [9] H. B. Callen, “Equilibrium Thermodynamics”, Wiley. NY(1960).
  * [10] J. Keizer, “Statistical Thermodynamics of Non-equilibrium Processes”, Springer. NY(1987).
  * [11] M. S. Berger and H. Shojaei, Phys. Rev. D 77, 123504 (2008); H. Farajollahi, A. Ravanpak and G. F. Fadakar, Phy. Let. B 711, 225 (2012); S. Cao, N. Liang and Z. H. Zhu, Mon. Not. R. Astron. Soc. 416, 1099 (2011); M. M. Verma and S. D. Pathak, Int. J. Mod. Phys. D 23, No.9 1450075 (2014); J. Valiviita, R. Maartens and E. Majerotto, Mon. Not. R. Astron. Soc. 402, 2355 (2010); C. Feng, B. Wang, E. Abdalla and R. Su, Phys. Lett. B 665, 111 (2008); M. Shahalam et al. Eur. Phys. J. C. 75, 395 (2015); A. Aviles and J. L. C. Cota, Phys. Rev. D 84, 083515 (2011); T. Clemson et al. Phys. Rev. D 85, 043007 (2012); B. Wang et al. Nucl. Phys. B 778, 69 (2007); H. Kodama and M. Sasaki, Prog. Theor. Phys. Suppl. 78, 1 (1984); L. Amendola, C. Quercellini, D. T. Valentini and A. Pasqui, Astrophys. J. Lett. 583, L53(2003); V. Pettorino and C. Baccigalupi, Phys. Rev. D 77, 103003 (2008).
  * [12] M. M. Verma and S. D. Pathak, Int. J. Theor. Phys. 10773 (2012).