# Equivalence of the Initialized Riemann-Liouville Derivatives and the
Initialized Caputo Derivatives

Jian Yuan  College of Mathematic and Information Science, Shandong Technology
and Business University, Yantai 264005, P.R.China  Guozhong Xiu, Bao Shi
Institute of System Science and Mathematics, Naval Aeronautical University,
Yantai 264001, P.R.China

###### Abstract

Initialization of fractional differential equations remains an ongoing
problem. In recent years, the initialization function approach and the
infinite state approach provide two effective ways to deal with this problem.
The purpose of this paper is to prove the equivalence of the initialized
Riemann-Liouville derivatives and the initialized Caputo derivatives with
arbitrary orders. By synthesizing the above two initialization theories, the
diffusive representations of the two initialized derivatives with arbitrary
orders are derived. Laplace transforms of the two initialized derivatives are
shown to be equal. As a result, the two most commonly used derivatives are
proved to be equivalent when initial conditions are properly imposed.

###### keywords:

Fractional calculus , initialization problems , diffusive representation ,
equivalence of fractional derivatives

##  1 Introduction

Fractional calculus provides a powerful tool to model real-world phenomena
exhibiting memory and hereditary properties [1]. Initial conditions of
fractional differential equations characterize the past histories of dynamical
systems from the starting time to the initial time. As a result, initial
conditions of fractional differential equations should be imposed in a
different way from that of integer-order differential equations. However,
proper initialization of fractional-order systems remains an ongoing problem
[2-4]. This issue can date back as far as Riemann’s complementary function
theory, in which many mathematicians were made confused, including Liouville,
Peacoch, Cayley, and Riemann himself [3].

In recent years, the initialization function approach and the infinite state
approach provide two effective ways to impose physically coherent initial
conditions to fractional systems. In the initialization function theory [3,
5], initialization functions are required to initialize fractional
differential equations. The initialization function is a time-varying function
and a generalization of the constant of integration required for the order-one
integral. The Riemann-Liouville fractional derivatives and the Caputo
derivatives can be initialized using the initialization function approach . In
the infinite state theory, the Riemann-Liouville fractional integral with
order between 0 and 1 is viewed a linear system which is characterized by the
impulse function and excited by the integrand function. The linear system is
termed as the fractional integrator, which is also an exactly equivalent
frequency distributed differential system. As a result, initial conditions of
fractional systems can be represented by the distributed initial conditions of
the fractional integrator [6-8]. Equivalence and compatibility of the above
initialization theories have been proved in [9, 10].

The Riemann-Liouville derivative and the Caputo derivative are the two most
commonly used derivatives in the real-world modeling of factional systems.
However, which derivative to choose is a trial and error process. In many
applications, the two derivatives are expected to be equivalent as long as the
initial conditions are properly taken into account [11, 12]. In [3] and [13],
Laplace transforms of the Riemann-Liouville derivative and the Caputo
derivative have been derived and shown to be equal in a special case where the
history function is a constant and the fractional order lies between 0 and 1.
In this paper, we present the equivalence and compatibility of the initialized
Riemann-Liouville derivatives and the initialized Caputo derivatives with
arbitrary fractional orders. By synthesizing the initialization function
approach and the infinite state approach, the diffusive representations of the
initialized Riemann-Liouville derivatives and the initialized Caputo
derivatives are derived. Laplace transforms of the two initialized derivatives
are shown to be equal. Consequently, the Riemann-Liouville derivatives and the
Caputo derivatives are proved to equivalent when initial conditions are
properly imposed.

The rest of this paper is organized as follows. Section 2 revisits the
diffusive representation for the initialized Riemann-Liouville fractional
integral. Section 3 presents the equivalence of the initialized Riemann-
Liouville derivatives and the initialized Caputo derivatives with order
between 0 and 1. Section 4 shows the equivalence of the two initialized
derivatives with order between 1 and 2. Section 5 proves the equivalence of
the two initialized derivatives with arbitrary orders. Finally, the paper is
concluded in Section 6.

##  2 Preliminaries

###  2.1 Diffusive model of the fractional integrator [7]

The Riemann-Liouville fractional integral of a function $f\left(t\right)$with
order $0<\alpha<1$ is defined as

${}_{{{t}_{0}}}I_{t}^{\alpha}f\left(t\right)=\frac{1}{\Gamma\left(\alpha\right)}\int_{{{t}_{0}}}^{t}{{{\left(t-\tau\right)}^{\alpha-1}}}f\left(\tau\right)d\tau$
(1)

Eq.(1)can also be viewed as a convolution between $f\left(t\right)$ and the
impulse response ${{h}_{\alpha}}\left(t\right)=\frac{{{t}^{\alpha-1}}}{\text{
}\\!\\!\Gamma\\!\\!\text{ }\left(\alpha\right)}$, i.e.,

${}_{{{t}_{0}}}I_{t}^{\alpha}f(t)={{h}_{\alpha}}\left(t\right)*f\left(t\right).$

From this viewpoint, the fractional integral can be obtained as the output of
a linear system, which is characterized by the impulse response
${{h}_{\alpha}}\left(t\right)$ and excited by $f\left(t\right)$:

$x(t)={{h}_{\alpha}}\left(t\right)*f\left(t\right).$

This linear system is called the fractional integrator. The frequency
distributed state $z(\omega,t)$of the fractional integrator verifies the
following ordinary differential equation:

$\frac{\partial z(\omega,t)}{\partial t}=-\omega z(\omega,t)+f(t)$ (2)

and the output of the fractional integrator is

$x(t)=\int_{0}^{+\infty}{{{\mu}_{\alpha}}\left(\omega\right)z(\omega,t)}d\omega,$
(3)

where the elementary frequency $\omega$ is ranging from 0 to $\infty$, and

${{\mu}_{\alpha}}(\omega)=\frac{sin\alpha\pi}{\pi}{{\omega}^{-\alpha}}.$

The relations (2) and (3) are called the frequency distributed model or
diffusive model of the fractional integrator.

###  2.2 The initialized fractional integrals [14, 15]

In the initialization theory for fractional calculus, the integrand of
Riemann-Liouville fractional integral is represented as a composite function

$v(t)=\left\\{\begin{aligned} &0,\text{ }t<-a\\\ &{{f}_{in}}(t),\text{ }-a\leq
t\leq 0\\\ &f(t),\text{ }t>0\\\ \end{aligned}\right.$

where $t=-a$ is the starting time of integral, $t=0$ is the initial time,
${{f}_{i}}(t)$ is the function describing the behavior during the
initialization period $\left[-a,0\right]$, $f(t)$ is the function of primary
interest after the initial time $t=0$.

The initialized Riemann-Liouville fractional integral of order $\alpha$ is
defined as

${}_{0}D_{t}^{-\alpha}f\left(t\right)=\frac{1}{\Gamma\left(\alpha\right)}\int_{0}^{t}{{{\left(t-\tau\right)}^{\alpha-1}}}f\left(\tau\right)d\tau+\psi\left(t\right),\text{
}t>0$

or

${}_{0}D_{t}^{-\alpha}f\left(t\right)={}_{0}d_{t}^{-\alpha}f\left(t\right)+\psi\left(t\right),\text{
}t>0.$

where ${}_{0}D_{t}^{-\alpha}f\left(t\right)$ is called the initialized
fractional integral, ${}_{0}d_{t}^{-\alpha}f\left(t\right)$ is called the
uninitialized fractional integral, $\psi\left(t\right)$ is called the
initialization function describing the hereditary effect of the past.

###  2.3 Diffusive representation of the initialized Riemann-Liouville
fractional integrals [10]

###### Lemma 1

The uninitialized Riemann-Liouville fractional integral
${}_{0}d_{t}^{-\alpha}f(t)$ with order $0<\alpha<1$ is equivalent to the
diffusive model with zero initial condition:

$\left\\{\begin{aligned} &\frac{\partial z(\omega,t)}{\partial t}=-\omega
z(\omega,t)+f(t)\\\ &z(\omega,{{t}_{0}})=0\\\ \end{aligned}\right.$

and

${}_{0}d_{t}^{-\alpha}f(t)=\int_{0}^{+\infty}{{{\mu}_{\alpha}}\left(\omega\right)z(\omega,t)}d\omega.$

###### Lemma 2

The initialized Riemann-Liouville fractional integral
${}_{0}D_{t}^{-\alpha}f(t)$ with order $0<\alpha<1$ is equivalent to the
diffusive model with distributed initial condition:

$\left\\{\begin{aligned} &\frac{\partial z(\omega,t)}{\partial t}=-\omega
z(\omega,t)+f(t),\text{ }\\\
&z(\omega,0)=\int_{-a}^{0}{{{e}^{-\omega\tau}}{{f}_{in}}(\tau)d\tau}\\\
\end{aligned}\right.$

and

${}_{0}D_{t}^{-\alpha}f(t)=\int_{0}^{+\infty}{{{\mu}_{\alpha}}(\omega)z(\omega,t)d\omega}.$

##  3 Equivalence of the two Derivatives with order between 0 and 1

###  3.1 Diffusive model for the initialized Riemann-Liouville derivatives
with order between 0 and 1

The initialized Riemann-Liouville fractional derivative
${}_{0}D_{t}^{\alpha}f(t)$ with order $0<\alpha<1$ is defined in [14] as

${}_{0}^{RL}D_{t}^{\alpha}f\left(t\right)=\frac{d}{dt}{}_{0}D_{t}^{\alpha-1}f\left(t\right).$

By virtue of Lemma 2, the diffusive representation of the initialized Riemann-
Liouville fractional integral ${}_{0}D_{t}^{\alpha-1}f(t)$ is

${}_{0}D_{t}^{\alpha-1}f\left(t\right)=\int_{0}^{\infty}{{{\mu}_{1\text{-}\alpha}}(\omega){{z}_{RL}}\left(\omega,t\right)d\omega}$

where ${{z}_{RL}}\left(\omega,t\right)$ satisfies

$\begin{cases}\frac{\partial{{z}_{RL}}(\omega,t)}{\partial
t}=-\omega{{z}_{RL}}(\omega,t)+f(t)\\\
{{z}_{RL}}(\omega,0)=\int_{-a}^{0}{{{e}^{-\omega\tau}}{{f}_{in}}(\tau)d\tau}\\\
\end{cases}$ (4)

As a result, the diffusive model for the initialized Riemann-Liouville
derivatives is

${}_{0}^{RL}D_{t}^{\alpha}f\left(t\right)=\frac{d}{dt}\int_{0}^{\infty}{{{\mu}_{1\text{-}\alpha}}(\omega){{z}_{RL}}\left(\omega,t\right)d\omega}$
(5)

Taking Laplace transform of the first equation of Eq.(4), we have

${{Z}_{RL}}\left(\omega,t\right)=\frac{{{z}_{RL}}\left(\omega,0\right)+F\left(s\right)}{s+\omega}$
(6)

Taking Laplace transform of the initialized Riemann-Liouville derivative from
Eq.(5), we have

$\begin{split}L\left\\{{}_{0}^{RL}D_{t}^{\alpha}f\left(t\right)\right\\}&=L\left\\{\frac{d}{dt}\int_{0}^{\infty}{{{\mu}_{1\text{-}\alpha}}(\omega){{z}_{RL}}\left(\omega,t\right)d\omega}\right\\}\\\
&=s\int_{0}^{\infty}{{{\mu}_{1\text{-}\alpha}}(\omega){{Z}_{RL}}\left(\omega,t\right)d\omega}-\int_{0}^{\infty}{{{\mu}_{1\text{-}\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)d\omega}\\\
\end{split}$ (7)

Substituting Eq. (6) into Eq.(7), we obtain

$\begin{split}L\left\\{{}_{0}^{RL}D_{t}^{\alpha}f\left(t\right)\right\\}&=sF\left(s\right)\int_{0}^{\infty}{\frac{{{\mu}_{1\text{-}\alpha}}(\omega)}{s+\omega}d\omega}+s\int_{0}^{\infty}{\frac{{{\mu}_{1\text{-}\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)}{s+\omega}d\omega}\\\
&\quad-\int_{0}^{\infty}{{{\mu}_{1\text{-}\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)d\omega}\\\
\end{split}$ (8)

Because

$\int_{0}^{\infty}{\frac{{{\mu}_{\alpha}}(\omega)}{s+\omega}d\omega}={{s}^{-\alpha}},$

Eq.(8) becomes

$\begin{split}L\left\\{{}_{0}^{RL}D_{t}^{\alpha}f\left(t\right)\right\\}&={{s}^{\alpha}}F\left(s\right)+s\int_{0}^{\infty}{\frac{{{\mu}_{1\text{-}\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)}{s+\omega}d\omega}\\\
&\quad-\int_{0}^{\infty}{{{\mu}_{1\text{-}\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)d\omega}\\\
&={{s}^{\alpha}}F\left(s\right)+\int_{0}^{\infty}{\left(\frac{s}{s+\omega}-1\right){{\mu}_{1\text{-}\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)d\omega}\\\
&={{s}^{\alpha}}F\left(s\right)-\int_{0}^{\infty}{\frac{\omega{{\mu}_{1\text{-}\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)}{s+\omega}d\omega}\\\
\end{split}$ (9)

###  3.2 Diffusive model for initialized Caputo derivatives with order between
0 and 1

The initialized Caputo fractional derivative ${}_{0}D_{t}^{\alpha}f(t)$ with
order $0<\alpha<1$ is defined in [14] as

${}_{0}^{C}D_{t}^{\alpha}f\left(t\right)={}_{0}D_{t}^{\alpha-1}{f}^{\prime}\left(t\right).$

By virtue of Lemma 2, the diffusive representation of the initialized Caputo
fractional derivative is

${}_{0}^{C}D_{t}^{\alpha}f\left(t\right)=\int_{0}^{\infty}{{{\mu}_{1\text{-}\alpha}}(\omega){{z}_{C}}\left(\omega,t\right)d\omega}$
(10)

where ${{z}_{C}}\left(\omega,t\right)$ satisfies

$\begin{cases}\frac{\partial{{z}_{C}}(\omega,t)}{\partial
t}=-\omega{{z}_{C}}(\omega,t)+f(t)\\\
{{z}_{C}}(\omega,0)=\int_{-a}^{0}{{{e}^{-\omega\tau}}{{f}_{in}}^{\prime}(\tau)d\tau}\\\
\end{cases}$ (11)

Taking Laplace transform of Eq.(11), we have

${{Z}_{C}}\left(\omega,t\right)=\frac{sF\left(s\right)-f\left(0\right)+{{z}_{C}}\left(\omega,0\right)}{s+\omega}$
(12)

Taking Laplace transform of Eq.(10), we have

$\begin{split}L\left\\{{}_{0}^{C}D_{t}^{\alpha}f\left(t\right)\right\\}&=L\left\\{\int_{0}^{\infty}{{{\mu}_{1\text{-}\alpha}}(\omega){{z}_{C}}\left(\omega,t\right)d\omega}\right\\}\\\
&=\int_{0}^{\infty}{{{\mu}_{1\text{-}\alpha}}(\omega){{Z}_{C}}\left(\omega,t\right)d\omega}\\\
\end{split}$ (13)

Substituting Eq.(12) into Eq.(13), we obtain

$\begin{split}L\left\\{{}_{0}^{C}D_{t}^{\alpha}f\left(t\right)\right\\}&=sF\left(s\right)\int_{0}^{\infty}{\frac{{{\mu}_{1\text{-}\alpha}}(\omega)}{s+\omega}d\omega}+\int_{0}^{\infty}{\frac{{{\mu}_{1\text{-}\alpha}}(\omega){{z}_{C}}\left(\omega,0\right)}{s+\omega}d\omega}\\\
&\quad-f\left(0\right)\int_{0}^{\infty}{\frac{{{\mu}_{1\text{-}\alpha}}(\omega)}{s+\omega}d\omega}\\\
&={{s}^{\alpha}}F\left(s\right)+\int_{0}^{\infty}{\frac{{{\mu}_{1\text{-}\alpha}}(\omega){{z}_{C}}\left(\omega,0\right)}{s+\omega}d\omega}-{{s}^{\alpha-1}}f\left(0\right)\\\
\end{split}$ (14)

In terms of Eq.(11) and integrating by parts, we have

${{z}_{C}}(\omega,0)=\int_{-a}^{0}{{{e}^{-\omega\tau}}{{f}_{in}}^{\prime}(\tau)d\tau}=\left.{{e}^{-\omega\tau}}{{f}_{in}}(\tau)\right|_{\tau=-a}^{\tau=0}-\omega\int_{-a}^{0}{{{e}^{-\omega\tau}}{{f}_{in}}(\tau)d\tau}$

In terms of Eq.(4), we have

${{z}_{C}}(\omega,0)=f\left(0\right)-\omega{{z}_{Rl}}(\omega,0)$ (15)

Then

$\begin{split}\int_{0}^{\infty}{\frac{{{\mu}_{1\text{-}\alpha}}(\omega){{z}_{C}}\left(\omega,0\right)}{s+\omega}d\omega}&=f\left(0\right)\int_{0}^{\infty}{\frac{{{\mu}_{1-\alpha}}(\omega)}{s+\omega}d\omega}-\int_{0}^{\infty}{\frac{\omega{{\mu}_{1\text{-}\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)}{s+\omega}d\omega}\\\
&={{s}^{\alpha-1}}f\left(0\right)-\int_{0}^{\infty}{\frac{{{\mu}_{1\text{-}\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)}{s+\omega}d\omega}\\\
\end{split}$ (16)

Substituting Eq.(16) into Eq.(14), we get

$L\left\\{{}_{0}^{C}D_{t}^{\alpha}f\left(t\right)\right\\}={{s}^{\alpha}}F\left(s\right)-\int_{0}^{\infty}{\frac{\omega{{\mu}_{1\text{-}\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)}{s+\omega}d\omega}$
(17)

From Eq.(9) and Eq.(17), it is clear that

$L\left\\{{}_{0}^{RL}D_{t}^{\alpha}f\left(t\right)\right\\}=L\left\\{{}_{0}^{C}D_{t}^{\alpha}f\left(t\right)\right\\}.$
(18)

Thus

${}_{0}^{RL}D_{t}^{\alpha}f\left(t\right)\text{=}{}_{0}^{C}D_{t}^{\alpha}f\left(t\right).$
(19)

Eq.(19) shows the equivalence of the initialized Riemann-Liouville derivatives
and Caputo derivatives with order between 0 and 1.

##  4 Equivalence of the two Derivatives with order between 1 and 2

###  4.1 Diffusive model for the initialized Riemann-Liouville derivatives
with order between 1 and 2

The initialized Riemann-Liouville fractional derivative
${}_{0}D_{t}^{\alpha}f(t)$ with order $1<\alpha<2$ is defined as

${}_{0}^{RL}D_{t}^{\alpha}f\left(t\right)=\frac{{{d}^{2}}}{d{{t}^{2}}}{}_{0}D_{t}^{\alpha-2}f\left(t\right).$

By virtue of Lemma 2, the diffusive representation of the initialized Riemann-
Liouville fractional integral ${}_{0}D_{t}^{\alpha-2}f(t)$ is

${}_{0}D_{t}^{\alpha-2}f\left(t\right)=\int_{0}^{\infty}{{{\mu}_{2-\alpha}}(\omega){{z}_{RL}}\left(\omega,t\right)d\omega}$

where ${{z}_{RL}}\left(\omega,t\right)$ satisfies Eq.(4). As a result, the
diffusive model for the initialized Riemann-Liouville derivatives is

${}_{0}^{RL}D_{t}^{\alpha}f\left(t\right)=\frac{{{d}^{2}}}{d{{t}^{2}}}\int_{0}^{\infty}{{{\mu}_{1\text{-}\alpha}}(\omega){{z}_{RL}}\left(\omega,t\right)d\omega}$
(20)

Taking Laplace transform of Eq.(20), we have

$\begin{split}L\left\\{{}_{0}^{RL}D_{t}^{\alpha}f\left(t\right)\right\\}&=L\left\\{\frac{{{d}^{2}}}{d{{t}^{2}}}\int_{0}^{\infty}{{{\mu}_{1\text{-}\alpha}}(\omega){{z}_{RL}}\left(\omega,t\right)d\omega}\right\\}\\\
&={{s}^{2}}\int_{0}^{\infty}{{{\mu}_{\text{2-}\alpha}}(\omega){{Z}_{RL}}\left(\omega,t\right)d\omega}-s\int_{0}^{\infty}{{{\mu}_{\text{2-}\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)d\omega}\\\
&\quad-\int_{0}^{\infty}{{{\mu}_{\text{2-}\alpha}}(\omega){{\left.\frac{\partial{{z}_{RL}}(\omega,t)}{\partial
t}\right|}_{t=0}}d\omega}\\\ \end{split}$ (21)

Substituting Eq.(6) into Eq.(21), we have

$\begin{split}L\left\\{{}_{0}^{RL}D_{t}^{\alpha}f\left(t\right)\right\\}&=sF\left(s\right)\int_{0}^{\infty}{\frac{{{\mu}_{\text{2-}\alpha}}(\omega)}{s+\omega}d\omega}+{{s}^{2}}\int_{0}^{\infty}{\frac{{{\mu}_{\text{2-}\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)}{s+\omega}d\omega}\\\
&\quad-s\int_{0}^{\infty}{{{\mu}_{\text{2-}\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)d\omega-\int_{0}^{\infty}{{{\mu}_{2-\alpha}}(\omega){{\left.\frac{\partial{{z}_{RL}}(\omega,t)}{\partial
t}\right|}_{t=0}}d\omega}}\\\ \end{split}$ (22)

Because

$\int_{0}^{\infty}{\frac{{{\mu}_{2-\alpha}}(\omega)}{s+\omega}d\omega}={{s}^{\alpha-2}},$

By substituting the first equation of Eq.(4) into Eq.(22), we obtain

$\begin{split}L\left\\{{}_{0}^{RL}D_{t}^{\alpha}f\left(t\right)\right\\}&={{s}^{\alpha}}F\left(s\right)+\int_{0}^{\infty}{\left(\frac{{{s}^{2}}}{s+\omega}-s+\omega\right){{\mu}_{\text{2-}\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)d\omega}\\\
&\quad-f\left(0\right)\int_{0}^{\infty}{{{\mu}_{2-\alpha}}(\omega)d\omega}\\\
&={{s}^{\alpha}}F\left(s\right)+\int_{0}^{\infty}{\frac{{{\omega}^{2}}{{\mu}_{\text{2-}\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)}{s+\omega}d\omega}\\\
&\quad-f\left(0\right)\int_{0}^{\infty}{{{\mu}_{\text{2-}\alpha}}(\omega)d\omega}\\\
\end{split}$ (23)

###  4.2 Diffusive model for the initialized Caputo derivatives with order
between 1 and 2

The initialized Caputo fractional derivative ${}_{0}D_{t}^{\alpha}f(t)$ with
order $1<\alpha<2$ is defined as

${}_{0}^{C}D_{t}^{\alpha}f\left(t\right)={}_{0}D_{t}^{\alpha-2}{f}^{\prime\prime}\left(t\right)$

By virtue of Lemma 2, the diffusive representation of the initialized Caputo
fractional derivative is

${}_{0}^{C}D_{t}^{\alpha}f\left(t\right)=\int_{0}^{\infty}{{{\mu}_{\text{2-}\alpha}}(\omega){{z}_{C}}\left(\omega,t\right)d\omega}$
(24)

where ${{z}_{C}}\left(\omega,t\right)$ satisfies

$\begin{cases}\frac{\partial{{z}_{C}}(\omega,t)}{\partial
t}=-\omega{{z}_{C}}(\omega,t)+{f}^{\prime\prime}\left(t\right)\\\
{{z}_{C}}(\omega,0)=\int_{-a}^{0}{{{e}^{-\omega\tau}}{{f}_{in}}^{\prime\prime}(\tau)d\tau}\\\
\end{cases}$ (25)

Taking Laplace transform of the first equation in Eq.(25), we have

${{Z}_{C}}\left(\omega,t\right)=\frac{{{s}^{2}}F\left(s\right)-sf\left(0\right)-{f}^{\prime}\left(0\right)+{{z}_{C}}\left(\omega,0\right)}{s+\omega}.$
(26)

In terms of the initial conditions of Eq.(25),

$\begin{split}{{z}_{C}}(\omega,0)&=\int_{-a}^{0}{{{e}^{-\omega\tau}}{{f}_{in}}^{\prime\prime}(\tau)d\tau}\\\
&=\left.{{e}^{-\omega\tau}}{{f}_{in}}^{\prime}(\tau)\right|_{\tau=-a}^{\tau=0}-\omega\int_{-a}^{0}{{{e}^{-\omega\tau}}{{f}_{in}}^{\prime}(\tau)d\tau}\\\
&={f}^{\prime}\left(0\right)-\omega\int_{-a}^{0}{{{e}^{-\omega\tau}}{{f}_{in}}^{\prime}(\tau)d\tau}\\\
&={f}^{\prime}\left(0\right)-\omega
f\left(0\right)+{{\omega}^{2}}{{z}_{R}}(\omega,0)\\\ \end{split}$ (27)

Substituting Eq.(27) into Eq.(26), we have

${{Z}_{C}}\left(\omega,t\right)=\frac{{{s}^{2}}F\left(s\right)-\left(s+\omega\right)f\left(0\right)+{{\omega}^{2}}{{z}_{RL}}\left(\omega,0\right)}{s+\omega}.$
(28)

Taking Laplace transform of Eq.(24)and substituting Eq.(28) into it, we obtain

$\begin{split}L\left\\{{}_{0}^{C}D_{t}^{\alpha}f\left(t\right)\right\\}&=L\left\\{\int_{0}^{\infty}{{{\mu}_{\text{2-}\alpha}}(\omega){{z}_{C}}\left(\omega,t\right)d\omega}\right\\}\\\
&=\int_{0}^{\infty}{{{\mu}_{\text{2-}\alpha}}(\omega){{Z}_{C}}\left(\omega,t\right)d\omega}\\\
&={{s}^{2}}F\left(s\right)\int_{0}^{\infty}{\frac{{{\mu}_{2-\alpha}}(\omega)}{s+\omega}d\omega}+\int_{0}^{\infty}{\frac{{{\omega}^{2}}{{\mu}_{\text{2-}\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)}{s+\omega}d\omega}\\\
&\quad-f\left(0\right)\int_{0}^{\infty}{{{\mu}_{\text{2-}\alpha}}(\omega)d\omega}\\\
&={{s}^{\alpha}}F\left(s\right)+\int_{0}^{\infty}{\frac{{{\omega}^{2}}{{\mu}_{\text{2-}\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)}{s+\omega}d\omega}\\\
&\quad-f\left(0\right)\int_{0}^{\infty}{{{\mu}_{\text{2-}\alpha}}(\omega)d\omega}\\\
\end{split}$ (29)

From Eq.(23) and Eq.(29), it is clear that

$L\left\\{{}_{0}^{RL}D_{t}^{\alpha}f\left(t\right)\right\\}=L\left\\{{}_{0}^{C}D_{t}^{\alpha}f\left(t\right)\right\\}.$
(30)

Thus,

${}_{0}^{RL}D_{t}^{\alpha}f\left(t\right)\text{=}{}_{0}^{C}D_{t}^{\alpha}f\left(t\right).$
(31)

Eq.(31) shows the equivalence of the initialized Riemann-Liouville derivatives
and Caputo derivatives with order between 1 and 2.

##  5 Equivalence of the two Derivatives with arbitrary orders

###  5.1 Diffusive model for initialized Riemann-Liouville derivatives with
order between $n$ and $n-1$

The initialized Riemann-Liouville fractional derivative
${}_{0}D_{t}^{\alpha}f(t)$ with order $n<\alpha<n-1$ is defined as

${}_{0}^{RL}D_{t}^{\alpha}f\left(t\right)=\frac{{{d}^{n}}}{d{{t}^{n}}}{}_{0}D_{t}^{\alpha-n}f\left(t\right).$

By virtue of Lemma 2, the diffusive representation of the initialized Riemann-
Liouville fractional integral ${}_{0}D_{t}^{\alpha-n}f\left(t\right)$ is

${}_{0}D_{t}^{\alpha-n}f\left(t\right)=\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega){{z}_{RL}}\left(\omega,t\right)d\omega}$

where ${{z}_{RL}}\left(\omega,t\right)$ satisfies Eq.(4).

As a result, the diffusive model for the initialized Riemann-Liouville
derivative is

${}_{0}^{RL}D_{t}^{\alpha}f\left(t\right)=\frac{{{d}^{n}}}{d{{t}^{n}}}\int_{0}^{\infty}{{{\mu}_{\text{n-}\alpha}}(\omega){{z}_{RL}}\left(\omega,t\right)d\omega}$
(32)

Taking Laplace transform of Eq.(32), we have

$\begin{split}L\left\\{{}_{0}^{RL}D_{t}^{\alpha}f\left(t\right)\right\\}&=L\left\\{\frac{{{d}^{n}}}{d{{t}^{n}}}\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega){{z}_{RL}}\left(\omega,t\right)d\omega}\right\\}\\\
&={{s}^{n}}\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega){{Z}_{RL}}\left(\omega,t\right)d\omega}-{{s}^{n-1}}\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)d\omega}\\\
&\quad-{{s}^{n-2}}\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega){{{{z}^{\prime}}}_{RL}}(\omega,0)d\omega}\\\
&\quad-{{s}^{n-3}}\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega){{{{z}^{\prime\prime}}}_{RL}}(\omega,0)d\omega}\\\
&\quad-\cdots-\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega){{z}^{\left(n-1\right)}}_{RL}(\omega,0)d\omega}\\\
\end{split}$ (33)

Substituting Eq.(6) into Eq.(33), and applying
$\int_{0}^{\infty}{\frac{{{\mu}_{2-\alpha}}(\omega)}{s+\omega}d\omega}={{s}^{\alpha-2}}$,
we have

$\begin{split}L\left\\{{}_{0}^{RL}D_{t}^{\alpha}f\left(t\right)\right\\}&={{s}^{\alpha}}F\left(s\right)+{{s}^{n}}\int_{0}^{\infty}{\frac{{{\mu}_{n-\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)}{s+\omega}d\omega}\\\
&\quad-{{s}^{n-1}}\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)d\omega}\\\
&\quad-{{s}^{n-2}}\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega){{{{z}^{\prime}}}_{RL}}\left(\omega,0\right)d\omega}\\\
&\quad-{{s}^{n-3}}\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega){{{{z}^{\prime\prime}}}_{RL}}\left(\omega,0\right)d\omega}\\\
&\quad-\cdots-\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega)z_{RL}^{{}^{\left(n-1\right)}}\left(\omega,0\right)d\omega}\\\
\end{split}$ (34)

From Eq.(4), we can calculate high-order derivatives of
${{z}_{RL}}\left(\omega,t\right)$:

$\frac{{{\partial}^{2}}{{z}_{RL}}\left(\omega,t\right)}{\partial{{t}^{2}}}={{\omega}^{2}}{{z}_{RL}}\left(\omega,t\right)-\omega
f\left(t\right)+{f}^{\prime\prime}\left(t\right)$
$\frac{{{\partial}^{3}}{{z}_{RL}}\left(\omega,t\right)}{\partial{{t}^{3}}}=-{{\omega}^{3}}{{z}_{RL}}\left(\omega,t\right)+{{\omega}^{2}}f\left(t\right)-\omega{f}^{\prime}\left(t\right)+{f}^{\prime\prime}\left(t\right)$
$\cdots\cdots$
$\begin{split}\frac{{{\partial}^{n-1}}{{z}_{RL}}\left(\omega,t\right)}{\partial{{t}^{n-1}}}&={{\left(-\omega\right)}^{n-1}}{{z}_{RL}}\left(\omega,t\right)+{{\left(-\omega\right)}^{n-2}}f\left(t\right)+{{\left(-\omega\right)}^{n-3}}{f}^{\prime}\left(t\right)\\\
&\quad+\cdots+{{f}^{\left(n-2\right)}}\left(t\right)\end{split}$

Substituting the above derivatives into Eq.(34), we obtain

$\begin{split}L\left\\{{}_{0}^{RL}D_{t}^{\alpha}f\left(t\right)\right\\}&={{s}^{\alpha}}F\left(s\right)+{{s}^{n}}\int_{0}^{\infty}{\frac{{{\mu}_{n-\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)}{s+\omega}d\omega}\\\
&\quad-{{s}^{n-1}}\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)d\omega}\\\
&\quad-{{s}^{n-2}}\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega)\left[-\omega{{z}_{RL}}(\omega,0)+f(0)\right]d\omega}\\\
&\quad-{{s}^{n-3}}\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega)\left[{{\omega}^{2}}{{z}_{RL}}\left(\omega,0\right)-\omega
f\left(0\right)+{f}^{\prime\prime}\left(0\right)\right]d\omega}\\\
&\quad-\cdots\cdots\\\
&\quad-\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega)\left[{{\left(-\omega\right)}^{n-1}}{{z}_{RL}}\left(\omega,0\right)+{{\left(-\omega\right)}^{n-2}}f\left(0\right)\right.}\\\
&\quad\left.+{{\left(-\omega\right)}^{n-3}}{f}^{\prime}\left(0\right)+\cdots+{{f}^{\left(n-2\right)}}\left(0\right)\right]d\omega\\\
&={{s}^{\alpha}}F\left(s\right)+\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)\left[\frac{{{s}^{n}}}{s+\omega}-{{s}^{n-1}}+\omega{{s}^{n-2}}\right.}\\\
&\quad\left.-{{\omega}^{2}}{{s}^{n-3}}+\cdots+{{\left(-\omega\right)}^{n-2}}s+{{\left(-\omega\right)}^{n-1}}\right]d\omega\\\
&\quad+\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega)\left\\{{{\Delta}_{1}}\right\\}d\omega}\\\
&={{s}^{\alpha}}F\left(s\right)+\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)\frac{{{\left(-\omega\right)}^{n}}}{s+\omega}d\omega}\\\
&\quad-\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega)\left\\{{{\Delta}_{1}}\right\\}d\omega}\\\
\end{split}$ (35)

where

$\begin{split}{{\Delta}_{1}}&=\left[{{s}^{n-2}}+\left(-\omega\right){{s}^{n-3}}+{{\left(-\omega\right)}^{2}}{{s}^{n-4}}+\cdots{{\left(-\omega\right)}^{n-2}}\right]f\left(0\right)\\\
&\quad+\left[{{s}^{n-3}}+\left(-\omega\right){{s}^{n-4}}+\cdots{{\left(-\omega\right)}^{n-3}}\right]{f}^{\prime}\left(0\right)+\cdots+{{f}^{\left(n-2\right)}}\left(0\right)\\\
\end{split}$ (36)

###  5.2 Diffusive model for initialized Caputo derivatives with order between
$n$ and $n-1$

The initialized Caputo fractional derivative ${}_{0}D_{t}^{\alpha}f(t)$ with
order $n-1<\alpha<n$ is defined as

${}_{0}^{C}D_{t}^{\alpha}f\left(t\right)={}_{0}D_{t}^{\alpha-n}{{f}^{\left(n\right)}}\left(t\right).$

By virtue of Lemma 2, the diffusive representation of the initialized Caputo
fractional derivative is

${}_{0}^{C}D_{t}^{\alpha}f\left(t\right)=\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega){{z}_{C}}\left(\omega,t\right)d\omega}$
(37)

where ${{z}_{C}}\left(\omega,t\right)$ satisfies

$\begin{cases}\frac{\partial{{z}_{C}}(\omega,t)}{\partial
t}=-\omega{{z}_{C}}(\omega,t)+{{f}^{\left(n\right)}}\left(t\right)\\\
{{z}_{C}}(\omega,0)=\int_{-a}^{0}{{{e}^{-\omega\tau}}f_{in}^{\left(n\right)}(\tau)d\tau}\\\
\end{cases}$ (38)

Taking Laplace transform of Eq.(38), we have

$\begin{split}{{Z}_{C}}\left(\omega,s\right)&=\frac{1}{s+\omega}\left[{{z}_{C}}\left(\omega,0\right)+{{s}^{n}}F\left(s\right)-{{s}^{n-1}}f\left(0\right)-{{s}^{n-1}}{f}^{\prime}\left(0\right)-\cdots\right.\\\
&\quad\left.-s{{f}^{\left(n-2\right)}}\left(0\right)-{{f}^{\left(n-1\right)}}\left(0\right)\right]\\\
\end{split}$ (39)

In terms of the initial condition of Eq.(38) and integrating by parts, we have

$\begin{split}{{z}_{C}}(\omega,0)&={{f}^{\left(n-1\right)}}\left(0\right)+\left(-\omega\right){{f}^{\left(n-2\right)}}\left(0\right)+{{\left(-\omega\right)}^{2}}{{f}^{\left(n-3\right)}}\left(0\right)+{{\left(-\omega\right)}^{3}}{{f}^{\left(n-4\right)}}\left(0\right)\\\
&\quad+\cdots+{{\left(-\omega\right)}^{n-2}}{f}^{\prime}\left(0\right)+{{\left(-\omega\right)}^{n-1}}f\left(0\right)+{{\left(-\omega\right)}^{n}}{{z}_{RL}}(\omega,0)\\\
\end{split}$ (40)

Substituting Eq.(40) into Eq.(39), we have

$\begin{split}{{Z}_{C}}\left(\omega,s\right)&=\frac{1}{s+\omega}\left\\{{{s}^{n}}F\left(s\right)+\left[{{\left(-\omega\right)}^{n-1}}-{{s}^{n-1}}\right]f\left(0\right)\right.\\\
&\quad+\left[{{\left(-\omega\right)}^{n-2}}-{{s}^{n-2}}\right]{f}^{\prime}\left(0\right)+\cdots\\\
&\quad\left.+\left[\left(-\omega\right)-s\right]{{f}^{\left(n-2\right)}}\left(0\right)+{{\left(-\omega\right)}^{n}}{{Z}_{RL}}\left(\omega,0\right)\right\\}\\\
\end{split}$ (41)

Taking Laplace transform of Eq.(37) and substituting Eq.(41) into it, we
obtain

$\begin{split}L\left\\{{}_{0}^{C}D_{t}^{\alpha}f\left(t\right)\right\\}&=L\left\\{\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega){{z}_{C}}\left(\omega,t\right)d\omega}\right\\}\\\
&=\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega){{Z}_{C}}\left(\omega,t\right)d\omega}\\\
&={{s}^{\alpha}}F\left(s\right)+\int_{0}^{\infty}{\frac{{{\left(-\omega\right)}^{n}}}{s+\omega}{{\mu}_{n-\alpha}}(\omega){{z}_{RL}}\left(\omega,0\right)d\omega}\\\
&\quad-\int_{0}^{\infty}{{{\mu}_{n-\alpha}}(\omega)\left\\{{{\Delta}_{2}}\right\\}d\omega}\\\
\end{split}$ (42)

where

${{\Delta}_{2}}\text{=}\left[\frac{{{s}^{n-1}}-{{\left(-\omega\right)}^{n-1}}}{s+\omega}\right]f\left(0\right)+\left[\frac{{{s}^{n-2}}-{{\left(-\omega\right)}^{n-2}}}{s+\omega}\right]{f}^{\prime}\left(0\right)+\cdots+{{f}^{\left(n-2\right)}}\left(0\right).$
(43)

From Eqs. (36)and(43), it is clear that ${{\Delta}_{1}}={{\Delta}_{2}}$, and

$L\left\\{{}_{0}^{RL}D_{t}^{\alpha}f\left(t\right)\right\\}=L\left\\{{}_{0}^{C}D_{t}^{\alpha}f\left(t\right)\right\\}.$
(44)

Thus,

${}_{0}^{RL}D_{t}^{\alpha}f\left(t\right)={}_{0}^{C}D_{t}^{\alpha}f\left(t\right).$
(45)

Eq.(45) shows the equivalence of the initialized Riemann-Liouville derivatives
and Caputo derivatives with arbitrary orders $\alpha$.

##  6 Conclusions

This paper has presented the equivalence of the initialized Riemann-Liouville
derivatives and the initialized Caputo derivatives with arbitrary orders. By
synthesizing the initialization function theory and the infinite state theory,
the diffusive representations of the two initialized derivatives have been
obtained. Laplace transforms of the two initialized derivatives with arbitrary
orders have been shown to be equal from Eqs.(18) (30) and (44). As a result,
the two most commonly used derivatives have been shown equivalent from
Eqs.(19) (31) and (45).
Acknowledgements All the authors acknowledge the valuable suggestions from the
peer reviewers. This work was supported by the National Natural Science
Foundation of China (Grant No. 11802338).

## References

  * [1] Podlubny, I. Fractional Differential Equations. in Mathematics in Science and Engineering. 1999.
  * [2] Fukunaga, M. and N.J.N.D. Shimizu, Role of Prehistories in the Initial Value Problems of Fractional Viscoelastic Equations. Nonlinear Dynamics, 2004. 38(1-4): p. 207-220.
  * [3] Lorenzo, C.F. and T.T. Hartley, Initialization of Fractional-Order Operators and Fractional Differential Equations. Journal of Computational and Nonlinear Dynamics, 2008. 3(2): p. 021101.
  * [4] Trigeassou, J.C. and N. Maamri, Initial conditions and initialization of linear fractional differential equations. Signal Processing, 2011. 91(3): p. 427-436.
  * [5] Lorenzo, C.F. and T.T. Hartley, Dynamics and Control of Initialized Fractional-Order Systems. 2002. 29(1-4): p. 201-233.
  * [6] Trigeassou, J.C., N. Maamri, and A. Oustaloup, The infinite state approach: Origin and necessity. Computers & Mathematics with Applications, 2013. 66(5): p. 892-907.
  * [7] Trigeassou, J. C., Maamri, N., Sabatier, J., Oustaloup, A., State variables and transients of fractional order differential systems. Computers & Mathematics with Applications, 2011. 64(10): p.3117-3140.
  * [8] Trigeassou, J. C., Maamri, N., Sabatier, J., Oustaloup, A., A lyapunov approach to the stability of fractional differential equations. Signal Processing, 2011. 91(3): p.437-445.
  * [9] T.T. Hartley, Lorenzo, C.F., Trigeassou, J. C., Maamri, N., Equivalence of History-Function Based and Infinite-Dimensional-State Initializations for Fractional-Order Operators. Journal of Computational and Nonlinear Dynamics, 2013. 8(4): p. 041014 1-7.
  * [10] Yuan, J., Zhang, Y., Liu, J., Shi, B., Equivalence of initialized fractional integrals and the diffusive model. Journal of Computational and Nonlinear Dynamics, 2018. 13(3): p. 034501 1-4.
  * [11] Paola, M. D., Fiore, V., Pinnola, F. P., Valenza, A., On the influence of the initial ramp for a correct definition of the parameters of fractional viscoelastic materials. Mechanics of Materials, 2014. 69(1): p.63-70.
  * [12] Bagley, R., On the equivalence of the Riemann-Liouville and the Caputo fractional order derivatives in modeling of linear viscoelastic materials. Fractional Calculus and Applied Analysis, 2007. 10(2): p. 123-126.
  * [13] Lorenzo, C.F. and T.T. Hartley, Time-varying initialization and Laplace transform of the Caputo derivative : with order between zero and one. ASME 2011 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. 2011.
  * [14] Lorenzo, C.F. and T.T. Hartley, Initialized fractional calculus. 2000, NASA/TP-2000-209943.
  * [15] Lorenzo, C.F. and T.T. Hartley, Initialization, conceptualization, and application in the generalized (fractional) calculus. Critical Reviews in Biomedical Engineering, 2007, 35(6): p. 447-553.


# UKARA 1.0 Challenge Track 1:
Automatic Short-Answer Scoring in Bahasa Indonesia

Ali Akbar Septiandri
Airy
Jakarta, Indonesia
<EMAIL_ADDRESS>
Yosef Ardhito Winatmoko
Jheronimus Academy of Data Science
’s-Hertogenbosch, The Netherlands
<EMAIL_ADDRESS>

###### Abstract

We describe our third-place solution to the UKARA 1.0 challenge on automated
essay scoring. The task consists of a binary classification problem on two
datasets — answers from two different questions. We ended up using two
different models for the two datasets. For task A, we applied a random forest
algorithm on features extracted using unigram with latent semantic analysis
(LSA). On the other hand, for task B, we only used logistic regression on TF-
IDF features. Our model results in F1 score of 0.812.

##  1 Introduction

Automated essay scoring is the application of computers technologies to assist
human grader in evaluating the score of written answers (Dikli, 2006). The
first track of UKARA 1.0 is the binary classification version of essay
scoring, where participants are expected to develop a model that can
distinguish right and wrong answers in free text format. The organizer
published the questions, the responses with the labels, and the guideline on
how to determine whether an answer is acceptable.

During a period of five weeks, the training set and the development set were
available and we could validate our model through the score of the development
set in a leaderboard. Subsequently, the test set was released, which consists
of roughly four times the size of the development set. We are required to
submit predicted labels based on the model that we have developed and the
winner was determined by the F1 score of the submitted prediction.

Our final submission to this task consists of feature extraction, such as
n-grams and TF-IDF, and classical machine learning algorithms, namely logistic
regression and random forest. We did not use deep learning at all in our
submission. The details of the dataset and our approach will be discussed in
the following sections.

##  2 Datasets

The dataset consists of two questions and the respective responses collected
by the organizer of the challenge. All questions and responses are in
Indonesian. The first question, from now on will be referred as task A, asked
about the consequence of the climate change. Concretely, what are the
potential problems faced by a climate refugee when they have to migrate to a
new place. The second question, referred as task B, is based on an experiment.
There were potential customers, initially wanted to buy clothes, who prefer to
donate the money instead when they are presented with videos of the clothes
manufacturing worker condition before paying. The respondents were required to
give their opinion on why do people decided to change their mind. The
statistics of the responses for both tasks are shown in Table 1.

| Task A | Task B
---|---|---
#Positive Train | $191(71\%)$ | $168(55\%)$
#Negative Train | $77(29\%)$ | $137(45\%)$
Avg. #Char | $87.23$ | $97.33$
#Dev | $215$ | $244$
#Test | $855$ | $974$
Table 1: Summary statistics of the dataset.

##  3 Methodology

###  3.1 Preprocessing

For the preprocessing steps, we first tokenized and lemmatized the text using
bahasa Indonesia tokenizer provided by spaCy (Honnibal and Montani, 2017). We
then extracted the features using bag-of-words or TF-IDF. Since the resulting
matrix from this feature extraction method tends to be sparse, and to encode
token relations, we applied Latent Semantic Analysis (LSA) using Singular
Value Decomposition (SVD) (Deerwester et al., 1990) on the matrix.

Based on our observation, we noticed that the labels of the provided training
set are highly inconsistent. Some responses are clearly labelled incorrectly.
For illustration, in task A we found ”untuk pindah ke daerah yang aman” (to
move to a safe place) labelled as 1 (correct) while clearly it does not fit
the criteria based on the guideline. The mislabeling was even more prominent
in task B: ”karana dengan menyumbang kita bisa membuat produksi pakaian
menjadi lebih beretika” (By donating, we can make clothes production becomes
more ethical) is considered wrong while ”agar upaya untuk membuat produksi
pakaian menjadi lebih beretika.” (As an effort to make clothes production
becomes more ethical) is approved. To approach this problem, we decided to
prepare a separate training set with manually corrected labels based on our
own judgment. The correction result is shown in Table 2.

Finally, as the responses contain a lot of typo and slang words, we also
experimented with simple typo corrector using python difflib package and
Indonesian colloquial dictionary (Salsabila et al., 2018). We tried every
possible combination of preprocessing steps and whether to use altered version
of the training set with a parameter optimization library described in the
following subsection.

|  | Original
---
Label
| Corrected
---
Label
Count
Task A | $0$ | $1$ | $10$
$1$ | $0$ | $4$
Task B | $0$ | $1$ | $46$
$1$ | $0$ | $13$
Table 2: Corrected labels of the training set.

###  3.2 The Winning Approach

After trying several machine learning algorithms, such as k-Nearest Neighbors,
Naïve Bayes, logistic regression, and random forest, we found that random
forest was the best model for task A. This corroborates what was found by
Fernández-Delgado et al. (2014) in their comprehensive comparisons among
several machine learning algorithms on different datasets. On the other hand,
logistic regression with L2 regularization was the best for task B. The
machine learning library used in this study is scikit-learn (Pedregosa et al.,
2011). Since the dataset is quite small, we used 5-fold cross validation on
the training set to avoid overfitting. For our winning approach, we set the
n_estimators parameter for random forest to 200 and keep the default values
for other parameters. We also found that it is the best to keep the default
parameter values for the logistic regression model.

###  3.3 Alternative Approaches

In parallel, we also experimented with optimization using
hyperopt111http://hyperopt.github.io/hyperopt/ library, which utilizes
sequential model-based optimization (Bergstra et al., 2011). We tried to
optimize the hyperparameters, including the preprocessing steps, of four
different machine learning algorithms: logistic regression, random forest,
gradient boosting tree, and support vector machine. We trained separate models
for task A and task B. In addition, we tested a voting-based ensemble model by
combining all the optimized model of each algorithm. The evaluation metric
used for the optimization, including for the voting-ensemble model, is F1
score. The results of the experiments are presented in the next section along
with the discussion.

##  4 Results and Discussion

We found that choosing different preprocessing methods resulted in different
performances in the two tasks. Therefore, we varied the use of unigram or TF-
IDF, and whether we should apply SVD to the resulting matrix. On the other
hand, we found that it is always better to use the lemmatizer built on top of
spaCy in this task. Moreover, removing stopwords did not contribute much to
the performance on the training set. Table of local CV results can be seen in
Table 3 and Table 4.

| Precision | Recall | F1
---|---|---|---
1-gram+RF | $0.845\pm 0.057$ | $0.921\pm 0.037$ | $0.881\pm 0.035$
1-gram+logreg | $0.857\pm 0.074$ | $0.869\pm 0.067$ | $0.862\pm 0.065$
1-gram+SVD+RF | $0.794\pm 0.025$ | $0.984\pm 0.014$ | $0.879\pm 0.014$
1-gram+SVD+logreg | $\mathbf{0.859\pm 0.069}$ | $0.874\pm 0.047$ | $0.866\pm 0.053$
TF-IDF+RF | $0.847\pm 0.054$ | $0.942\pm 0.039$ | $\mathbf{0.891\pm 0.036}$
TF-IDF+logreg | $0.748\pm 0.019$ | $0.979\pm 0.034$ | $0.848\pm 0.022$
TF-IDF+SVD+RF | $0.772\pm 0.025$ | $\mathbf{0.990\pm 0.014}$ | $0.867\pm 0.014$
TF-IDF+SVD+logreg | $0.751\pm 0.025$ | $0.979\pm 0.034$ | $0.850\pm 0.025$
Table 3: 5-fold cross validation results from task A | Precision | Recall | F1
---|---|---|---
1-gram+RF | $0.731\pm 0.066$ | $0.744\pm 0.054$ | $0.736\pm 0.047$
1-gram+logreg | $\mathbf{0.735\pm 0.046}$ | $0.727\pm 0.080$ | $0.730\pm 0.059$
1-gram+SVD+RF | $0.686\pm 0.035$ | $0.762\pm 0.046$ | $0.721\pm 0.032$
1-gram+SVD+logreg | $0.722\pm 0.067$ | $0.726\pm 0.089$ | $0.723\pm 0.074$
TF-IDF+RF | $0.709\pm 0.036$ | $0.750\pm 0.049$ | $0.728\pm 0.034$
TF-IDF+logreg | $0.725\pm 0.035$ | $0.810\pm 0.060$ | $\mathbf{0.764\pm 0.035}$
TF-IDF+SVD+RF | $0.637\pm 0.026$ | $\mathbf{0.834\pm 0.061}$ | $0.721\pm 0.036$
TF-IDF+SVD+logreg | $0.705\pm 0.023$ | $0.809\pm 0.063$ | $0.753\pm 0.036$
Table 4: 5-fold cross validation results from task B

For this challenge, we need to optimize the F1 score. Therefore, it is clear
from Table 4 that we should use the model from TF-IDF with random forest
algorithm for task B. From what we can see ini Table 3, we should also go with
the same method for task A. However, we decided to be more pessimistic by
looking at the largest F1 score after subtracting 1 standard deviation from
the mean F1. Thus, we chose 1-gram + SVD + random forest for task A.

Table 5 shows the performance of our best single models compared with two
alternative approaches. First, we used the optimized ensemble model trained on
the original training set (Ens+Ori). Second, we also use a similar method but
with the label-corrected training set (Ens+Upd). While we can see in Table 5
that the F1 score on task B is higher on the label-corrected training set and
got a similar result for the development set, the test score is lower than the
best single models. The test set labels were most likely as noisy as the
training set, thus making the training score with modified label not
representative.

| Train A | Train B | Dev | Test
---|---|---|---|---
Best | $0.879$ | $0.764$ | $0.810$ | $0.812$
Ens+Ori | $0.885$ | $0.764$ | $0.799$ | $0.801$
Ens+Upd | $0.898$ | $0.831$ | $0.810$ | $0.803$
Table 5: F1 score comparison with the alternative Ensemble model.

To analyze how hard it is to separate the right from the wrong answers, we
reduced the dimensionality of the data into 2D using 1-gram, SVD, and t-SNE
(Maaten and Hinton, 2008). Figure 1 suggests that it is harder to separate the
two answers in task B. Since most of the answers are short, we also cannot see
“islands” from applying t-SNE to the data.

Figure 1: t-SNE visualisation

A similar problem was also shown in Figure 2 and Figure 3. We can see a lot
more data points in the 0.4-0.6 prediction range in Figure 3. This suggests
more uncertainty in the model. We argue that this is potentially because of
the incorrect labels in the original training set for task B.

Figure 2: Best model prediction (random forest) with probability on Task A
Figure 3: Best model prediction (logistic regression) with probability on Task
B

##  5 Conclusions

In this report, we describe our winning approach for UKARA 1.0 Challenge Track
1. During the competition, we experimented with single models and ensemble
models to predict which answer is correct given an open-ended question. We
also tried to re-label the training set and pre-process with text correction
which gave a boost in our local cross validation. However, we found that
single models with less pre-processing performed better in the test set. The
best single model for task A is random forest with unigram+SVD and for task B
is logistic regression with TF-IDF. The prediction of both model achieved an
overall F1 score of 0.812, which was enough to get us the third position in
the final leaderboard.

## References

  * Bergstra et al. (2011) James S Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. 2011\.  Algorithms for hyper-parameter optimization.  In _Advances in neural information processing systems_ , pages 2546–2554.
  * Deerwester et al. (1990) Scott Deerwester, Susan T Dumais, George W Furnas, Thomas K Landauer, and Richard Harshman. 1990.  Indexing by latent semantic analysis.  _Journal of the American society for information science_ , 41(6):391–407.
  * Dikli (2006) Semire Dikli. 2006.  An overview of automated scoring of essays.  _The Journal of Technology, Learning and Assessment_ , 5(1).
  * Fernández-Delgado et al. (2014) Manuel Fernández-Delgado, Eva Cernadas, Senén Barro, and Dinani Amorim. 2014\.  Do we need hundreds of classifiers to solve real world classification problems?  _The Journal of Machine Learning Research_ , 15(1):3133–3181.
  * Honnibal and Montani (2017) Matthew Honnibal and Ines Montani. 2017.  spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing.  To appear.
  * Maaten and Hinton (2008) Laurens van der Maaten and Geoffrey Hinton. 2008.  Visualizing data using t-sne.  _Journal of machine learning research_ , 9(Nov):2579–2605.
  * Pedregosa et al. (2011) F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011.  Scikit-learn: Machine learning in Python.  _Journal of Machine Learning Research_ , 12:2825–2830.
  * Salsabila et al. (2018) Nikmatun Aliyah Salsabila, Yosef Ardhito Winatmoko, Ali Akbar Septiandri, and Ade Jamal. 2018.  Colloquial indonesian lexicon.  In _2018 International Conference on Asian Language Processing (IALP)_ , pages 226–229. IEEE.

##  Appendix A Supplemental Material

The code for this analysis can be seen on https://github.com/aliakbars/ukara/.