# A non-local problem for the fractional order Rayleigh-Stokes equation

Ravshan Ashurov, Oqila Mukhiddinova, Sabir Umarov  Ravshan Ashurov
V.I. Romanovskiy Institute of Mathematics,
Uzbekistan Academy of Science,
University str.,9, Olmazor district,
Tashkent, 100174, Uzbekistan<EMAIL_ADDRESS>Oqila Mukhiddinova
Tashkent University of Information Technologies,
108 Amir Temur Avenue,
Tashkent, 100200, Uzbekistan

<EMAIL_ADDRESS>Sabir Umarov
University of New Haven,
Department of Mathematics
300 Boston Post Road
West Haven, CT 06516, USA
<EMAIL_ADDRESS>
###### Abstract.

A nonlocal boundary value problem for the fractional version of the well known
in fluid dynamics Rayleigh-Stokes equation is studied. Namely, the condition
$u(x,T)=\beta u(x,0)+\varphi(x)$, where $\beta$ is an arbitrary real number,
is proposed instead of the initial condition. If $\beta=0$, then we get the
inverse problem in time, called the backward problem. It is well known that
the backward problem is ill-posed in the sense of Hadamard. If $\beta=1$, then
the corresponding non-local problem becomes well-posed in the sense of
Hadamard, and moreover, in this case a coercive estimate for the solution can
be established. The aim of this work is to find values of the parameter
$\beta$, which separates two types of behavior of the semi-backward problem
under consideration. We prove the following statements: if $\beta\geq 1,$ or
$\beta<0$, then the problem is well-posed; if $\beta\in(0,1)$, then depending
on the eigenvalues of the elliptic part of the equation, for the existence of
a solution an additional condition on orthogonality of the right-hand side of
the equation and the boundary function to some eigenfunctions of the
corresponding elliptic operator may emerge.

###### Key words and phrases:

The Rayleigh-Stokes problem, non-local problem, fractional derivative, Mittag-
Leffler function, Fourier method

##  1\. Introduction

Fractional derivatives serve as an essential tools in modeling of complex
processes. The concept of fractional derivatives arose simultaneously with
derivatives of integer order. Starting with the work of Abel (see, e.g. [1]),
the concept of fractional derivatives began to be widely used in various
fields, such as electrochemistry, neuron models in biology, applied
mathematics, fluid dynamics, viscoelasticity and fluid mechanics [2]. Models
with fractional derivatives are used to analyze viscoelasticity, for example,
of polymers during glass transition and in the glassy state [3], the
theoretical base of which is the well-known Rayleygh-Stokes equation. A
fractional model of a generalized second-class fluid flow can be represented
as the Rayleigh-Stokes problem with a time-fractional derivative [4]:

$\left\\{\begin{aligned}
&\partial_{t}u(x,t)-(1+\gamma\,\partial_{t}^{\alpha})\Delta
u(x,t)=f(x,t),\quad x\in\Omega,\quad 0<t\leq T;\\\ &u(x,t)=0,\quad
x\in\partial\Omega,\quad 0<t\leq T;\\\ &u(x,0)=\varphi(x),\quad
x\in\Omega,\end{aligned}\right.$ (1.1)

where $1/\gamma>0$ is the fluid density, a fixed constant, the source term
$f(x,t)$ and the initial data $\varphi$ are given functions,
$\partial_{t}=\partial/\partial t$, and $\partial_{t}^{\alpha}$ is the
Riemann-Liouville fractional derivative of order $\alpha\in(0,1)$ defined by
(see, e.g. [1]):

$\partial_{t}^{\alpha}h(t)=\frac{d}{dt}\int\limits_{0}^{t}\omega_{1-\alpha}(t-s)h(s)ds,\quad\omega_{\alpha}(t)=\frac{t^{\alpha-1}}{\Gamma(\alpha)}.$
(1.2)

Here $\Gamma(\cdot)$ is Euler’s gamma function. Usually problem (1.1) is
considered in the domain $\Omega\subset R^{N}$, $N=1,2,3$, and for $N>1$ it is
assumed that the boundary $\partial\Omega$ of $\Omega$ is sufficiently smooth.

When $\alpha=1$ the equation in (1.1) is also called the Haller equation. This
equation is a mathematical model of water movement in capillary-porous media,
including the soil. In this case, $u$ is humidity in fractions of a unit, $x$
is a point inside the soil, $t$ is time (see, for example, in [5], formulas
(1.4) and (1.84) on p. 137 and 158, [6], formula (9.6.4) on p. 255, and [7],
formula (2.6.1) on p. 59). See also works [8], [9], where on the base of
modified Darcy’s law for a viscoelastic fluid, the first Stokes problem was
extended to the problem for an Oldroyd-B fluid in a porous half-space, and
equation (1.1) was obtained as a mathematical model. Recall that usually
Stokes’ first problem describes flows caused by a suddenly accelerated plate
for homogeneous incompressible isotropic fluids with pressure-dependent
viscosity.

The Rayleigh-Stokes problem (1.1) plays an important role in the study of the
behavior of some non-Newtonian fluids, as well. A non-Newtonian fluid is a
fluid that has a constant viscosity independent of stress, i.e. does not obey
Newton’s law of viscosity. The fractional derivative $\partial_{t}^{\alpha}$
is used in the equation (1.1) to describe the viscoelastic flow behavior (see,
for example, [10], [11]).

In recent years, the Rayleigh-Stokes problem (1.1) has received much attention
due to its importance for applications (see, for example, [8] \- [18]). An
overview of work in this direction can be found in Bazhlekova et al. [4] (see
also [18]). Properties of the solution of this model were studied by a number
of authors applying various methods; see e.g. [10], [11], [12]. The authors of
the work by Bazhlekova et al. [4] proved the Sobolev regularity of the
homogeneous Rayleigh-Stokes problem for both smooth and non-smooth initial
data $\varphi(x)$, including $\varphi(x)\in L_{2}(\Omega)$. A number of works
are devoted to the development of efficient and accurate numerical algorithms
for solving the problem (1.1). A survey of works in this direction is
contained in the above-mentioned paper [4]. See also recent articles [13],
[14], and references therein.

The study of the inverse problem of determining the right-hand side of the
Rayleigh-Stokes equation is the subject of many works (see, for example, [15],
[16], [17] and the bibliography cited there). Since this inverse problem is
ill-posed in the sense of Hadamard, various regularization methods are
considered in the above works, as well as numerical methods for finding the
right-hand side of the equation are proposed. We note also that the inverse
problem of determining the right-hand side of the equation is also ill-posed
for the subdiffusion equation (see, for example, [19], [20], [21]).

If the initial condition $u(x,0)=\varphi(x)$ in the problem (1.1) is replaced
by $u(x,T)=\varphi(x)$, then one gets the, so-called, _backward problem_. This
problem is not well-posed, i.e. a small change in $u(x,T)$ leads to a large
change of the solution. In papers [22], [23] (see also references therein)
various regularization methods are proposed, accompanied by verification of
these methods using numerical experiments. We emphasize that in these papers
$N<4,$ and this is connected with the method used there. Namely, if the
dimension of the space is less than four, then for the eigenvalues
$\lambda_{k}$ of the Laplace operator with the Direchlet condition, the series

$\sum_{k}\lambda_{k}^{-2}$

converges.

Let us focus, in more detail, on the recently published work [18]. In this
paper, along with other questions, problem (1.1) is investigated by taking the
non-local condition $u(x,T)=\beta u(x,0)+\varphi(x)$ instead of the initial
condition. The authors considered only the cases $\beta=0$ and $\beta=1$: if
$\beta=0$ then we have the backward problem (note that here the dimension $N$
is arbitrary). The authors proved that if $\beta=0$, then the solution exists
and is unique, but there is no stability. If $\beta=1$, then the problem is
well-posed in the sense of Hadamard, i.e. the unique solution exists, and the
solution continuously depends on the initial data and on the right-hand side
of the equation.

The question naturally arises: what happens if $\beta$ takes other values than
$0$ and $1$? In the present paper we consider a more general non-local
condition $u(x,t_{0})=\beta u(x,0)+\varphi(x)$, $t_{0}\in(0,T]$ and provide a
definitive answer to this question. The main results of the current work can
be formulated as follows:

1) If $\beta\geq 1$, or $\beta<0$, then the problem is well-posed in the sense
of Hadamard: the solution exists, it is unique and stable;

2) case $\beta=0$ is considered in [18]: in this case there is a unique
solution, but it is not stable;

3) if $\beta\in(0,1)$, then the well-posedness of the problem depends on the
location of the spectrum (i.e., the eigenvalues $\lambda_{k}$) of the Laplace
operator with the Direchlet condition. If the inequality
$B_{\alpha}(\lambda_{k},t_{0})\neq\beta$ (the definition of this function is
given in Section 3) holds for all $k=1,2,\cdots$, then the problem is well-
posed in the sense of Hadamard. If $B_{\alpha}(\lambda_{k},t_{0})=\beta$ for
some $k\in K_{0}$ (it is proved in the paper that the set $K_{0}$ contains
only a finite number of points), then a necessary and sufficient condition for
the existence of a solution is found. However, in this case there is no unique
solution.

In what concerns the non-local condition

$u(x,T)=\beta u(x,0)+\varphi(x),$

in the variable $t,$ the corresponding problem with the parameter $\beta=1$
for the classical diffusion equation was first considered in [24], [25] and
[26]. In [27] and [28], this problem with an arbitrary parameter $\beta$ was
studied in detail for subdiffusion equations with Riemann-Liouville and Caputo
derivatives correspondingly. In a recent paper [29], the authors considered
the subdiffusion equation with the Caputo-Fabrizio derivative on an
$N$-dimensional torus with a non-local condition

$\varepsilon u(T)=u(0)+\varphi.$

In these works the cases $\varepsilon=0$ and $\varepsilon>0$ are studied
separately. The authors also studied solution limit at $\varepsilon\to 0$.
Note, in this paper if $\varepsilon=0$, then we have the Cauchy problem,
whereas in our case we have the backward problem.

The present paper consists of five sections. Section 2 provides precise
formulations of the problems studied in this paper. In Section 3, we introduce
the standard Hilbert space of ”smooth” functions via the power of an elliptic
operator and give some well-known properties of the function
$B_{\alpha}(\lambda,t)$ introduced in [4]. Here we prove an important lemma
used for solution of the non-local problem in the variable $t.$ Section 4 is
devoted to the study of the main non-local problem with operator $A$
generalizing the Laplace operator.

##  2\. Problem formulations

Let $H$ be a separable Hilbert space. Denote by $(\cdot,\cdot)$ be the inner
product and by $||\cdot||$ the norm in $H$. Consider an arbitrary unbounded
positive self-adjoint operator $A$ with a dense domain in $H$. We assume that
$A$ has a complete in $H$ system of orthonormal eigenvectors (eigenfunctions)
$\\{v_{k}\\}$ and a countable set of positive eigenvalues

$\lambda_{k}:\quad
0<\lambda_{1}\leq\lambda_{2}\cdot\cdot\cdot\rightarrow+\infty.$

We also assume that the set $\\{\lambda_{k}\\}$ does not have a finite limit
point.

For a vector-valued functions $h:\mathbb{R}_{+}\rightarrow H$, we define the
Riemann-Liouville fractional derivative of order $0<\alpha<1$ in the same way
as (1.2) (see, e.g. [30]). Finally, let $C((a,b);H)$ denote the set of
functions $u(t)$ continuous in $t\in(a,b)$ with values in $H$.

Consider the following non-local problem for the abstract Rayleigh-Stokes
equation accepting the integral (in the definition of the fractional
derivative) in the sense of Bochner:

$\left\\{\begin{aligned}
&\partial_{t}u(t)+(1+\gamma\,\partial_{t}^{\alpha})Au(t)=f(t),\quad 0<t\leq
T;\\\ &u(t_{0})=\beta u(0)+\varphi,\end{aligned}\right.$ (2.1)

where $\gamma>0$ and $t_{0}\in(0,T]$ are fixed constants, $\varphi\in H,$
$f(t)\in C((0,T];H),$ and $\beta$ is an arbitrary real number. If $\beta=0,$
then this problem is called the backward problem.

######  Definition 2.1.

A function $u(t)\in C([0,T];H)$ is called a solution of the non-local
Rayleigh-Stokes problem (2.1), if

$\partial_{t}u(t),\,Au(t),\,\partial_{t}^{\alpha}Au(t)\in C((0,T);H),$

and it satisfies conditions (2.1) for all $t\in(0,T].$

######  Remark 2.2.

As an example of the operator $A$ one can take, for example, the Laplace
operator with the Dirichlet condition in an arbitrary $N$ (not only $\leq
3$)-dimensional bounded domain with a sufficiently smooth boundary. This
operator has all the properties listed above.

##  3\. Preliminaries

For a given real number $\tau$, we define the operator $A^{\tau}$ by

$A^{\tau}h=\sum\limits_{k=1}^{\infty}\lambda_{k}^{\tau}h_{k}v_{k}.$

Note, that the operator $A$ is positive and therefore $\lambda_{k}>0$ for all
$k.$ Here and everywhere for the vector $h\in H$, the symbol $h_{k}$ will
denote the Fourier coefficients of this vector: $h_{k}=(h,v_{k})$. The domain
of definition of the operator $A^{\tau}$ is

$D(A^{\tau})=\\{h\in
H:\sum\limits_{k=1}^{\infty}\lambda_{k}^{2\tau}|h_{k}|^{2}<\infty\\}.$

For elements of $h,g\in D(A^{\tau})$ we introduce the norm

$||h||^{2}_{\tau}=\sum\limits_{k=1}^{\infty}\lambda_{k}^{2\tau}|h_{k}|^{2}=||A^{\tau}h||^{2},$

and the inner product

$(h,g)_{\tau}=\sum_{k=1}^{\infty}\lambda_{k}^{2\tau}h_{k}g_{k}=(A^{\tau}h,A^{\tau}g).$

With this inner product, the linear-vector space $D(A^{\tau})$ becomes a
Hilbert space. Further, let $B_{\alpha}(\lambda,t)$ be a solution of the
following Cauchy problem

$Ly(t)\equiv
y^{\prime}(t)+\lambda(1+\gamma\partial_{t}^{\alpha})y(t)=0,\,\,t>0,\,\,\lambda>0,\,\,y(0)=1.$
(3.1)

The solution of this problem can be expressed in terms of the generalized
Wright function (see e.g. A.A. Kilbas et. al [1], Example 5.3, p. 289 and
[31]). The function $B_{\alpha}(\lambda,t)$ is studied in detail in
Bazhlekova, Jin, Lazarov, and Zhou [4]. See also Luc, Tuan, Kirane, Thanh
[22], where important lower bounds are obtained.

The authors of [4], in particular, proved the following lemma.

######  Lemma 3.1.

Let $B_{\alpha}(\lambda,t)$ be a solution of the Cauchy problem (3.1). Then

  1. (1)

$B_{\alpha}(\lambda,0)=1,\,\,0<B_{\alpha}(\lambda,t)<1,\,\,t>0$,

  2. (2)

$\partial_{t}B_{\alpha}(\lambda,t)<0,\,\,t\geq 0$,

  3. (3)

$\lambda B_{\alpha}(\lambda,t)<C\min\\{t^{-1},t^{\alpha-1}\\},\,\,t>0$,

  4. (4)

$\int\limits_{0}^{T}B_{\alpha}(\lambda,t)dt\leq\frac{1}{\lambda},\,\,T>0.$

The function $B_{\alpha}(\lambda,t)$ has the representation [4]

$B_{\alpha}(\lambda,t)=\int\limits_{0}^{\infty}e^{-rt}b_{\alpha}(\lambda,r)dr,$
(3.2)

where

$b_{\alpha}(\lambda,r)=\frac{\gamma}{\pi}\frac{\lambda
r^{\alpha}\sin\alpha\pi}{(-r+\lambda\gamma
r^{\alpha}\cos\alpha\pi+\lambda)^{2}+(\lambda\gamma
r^{\alpha}\sin\alpha\pi)^{2}}.$ (3.3)

######  Lemma 3.2.

([4, 18]) The Cauchy problem

$y^{\prime}(t)+\lambda(1+\gamma\partial_{t}^{\alpha})y(t)=f(t),\,\,t>0,\,\,\lambda>0,\,\,y(0)=y_{0},$
(3.4)

has a unique solution, which has a representation

$y(t)=y_{0}B_{\alpha}(\lambda,t)+\int\limits_{0}^{t}B_{\alpha}(\lambda,t-\tau)f(\tau)d\tau.$
(3.5)

We will also need an estimate obtained in [18] for the derivative of the
function $B_{\alpha}(\lambda,t).$ In view of the importance of this assertion
for our further considerations, we present it with a brief proof.

######  Lemma 3.3.

There is a constant $C>0$, such that

$|\partial_{t}B_{\alpha}(\lambda,t)|\leq\frac{C}{\lambda\,t^{2-\alpha}},\,\,t>0.$

###### Proof.

Differentiating the function $B_{\alpha}(\lambda,t)$ defined in (3.2), we have

$\partial_{t}B_{\alpha}(\lambda,t)=-\int\limits_{0}^{\infty}re^{-rt}b_{\alpha}(\lambda,r)dr.$

Therefore, in accordance with the definition of $b_{\alpha}(\lambda,r)$ in
(3.3)

$\displaystyle|\partial_{t}B_{\alpha}(\lambda,t)|$
$\displaystyle\leq\frac{\gamma}{\pi}\int\limits_{0}^{\infty}\frac{\lambda
r^{\alpha}\sin\alpha\pi}{(\lambda\gamma
r^{\alpha}\sin\alpha\pi)^{2}}\,re^{-rt}dr$
$\displaystyle=\frac{1}{\gamma\pi\lambda\sin\alpha\pi}\int\limits_{0}^{\infty}r^{1-\alpha}e^{-rt}dr.$

Now the change of the variable $\tau=rt$ implies

$\displaystyle|\partial_{t}B_{\alpha}(\lambda,t)|$
$\displaystyle\leq\frac{t^{\alpha-2}}{\gamma\pi\lambda\sin\alpha\pi}\int\limits_{0}^{\infty}\tau^{1-\alpha}e^{-\tau}d\tau$
$\displaystyle=\frac{t^{\alpha-2}}{\gamma\pi\lambda\sin\alpha\pi}\Gamma(2-\alpha)=\frac{C}{\lambda\,t^{2-\alpha}}.$

∎

In what follows, $\lambda$ will be replaced by eigenvalues $\lambda_{k}$ of
the operator $A$. The following important lower bound for
$B_{\alpha}(\lambda_{k},t)$ was obtained in Luc, N.H., Tuan, N.H., Kirane, M.,
Thanh, D.D.X [22].

######  Lemma 3.4.

The following estimate holds for all $t\in[0,T]$ and $k\geq 1$:

$B_{\alpha}(\lambda_{k},t)\geq\frac{C(\alpha,\gamma,\lambda_{1})}{\lambda_{k}},$

where

$C(\alpha,\gamma,\lambda_{1})=\frac{\gamma\sin\alpha\pi}{4}\int\limits_{0}^{\infty}\frac{r^{\alpha}e^{-rT}}{\frac{r^{2}}{\lambda_{1}^{2}}+\gamma^{2}r^{2\alpha}+1}dr.$

Next, we estimate the derivative $\partial_{\lambda}B_{\alpha}(\lambda,t_{0})$
from above when $\lambda\geq\lambda_{1}>0$, where $\lambda_{1}$ is the first
eigenvalue of the operator $A$.

######  Lemma 3.5.

Let $0<t_{0}\leq T$, $\gamma>0$, $\alpha\in(0,1)$ be given numbers. There
exists a positive number
$\Lambda_{0}=\Lambda_{0}(t_{0},\gamma,\alpha,\lambda_{1})>0$ such that for any
$\lambda\geq\Lambda_{0}$ the inequality

$\partial_{\lambda}B_{\alpha}(\lambda,t_{0})<0$ (3.6)

holds.

###### Proof.

We rewrite the function $B_{\alpha}(\lambda,t_{0})$ in the form

$B_{\alpha}(\lambda,t_{0})=\frac{1}{\lambda}\int\limits_{0}^{\infty}e^{-t_{0}r}b_{\alpha,1}(\lambda,r)dr,$

where

$b_{\alpha,1}(\lambda,r)=\frac{\gamma}{\pi}\,\,\frac{r^{\alpha}\sin\alpha\pi}{(-\frac{r}{\lambda}+\gamma
r^{\alpha}\cos\alpha\pi+1)^{2}+(\gamma r^{\alpha}\sin\alpha\pi)^{2}},$

Now differentiating with respect to $\lambda$ we have

$\displaystyle\partial_{\lambda}B_{\alpha}(\lambda,t_{0})$
$\displaystyle=-\frac{1}{\lambda^{2}}\int\limits_{0}^{\infty}e^{-t_{0}r}b_{\alpha,1}(\lambda,r)dr$
$\displaystyle+\frac{2}{\lambda^{3}}\int\limits_{0}^{\infty}e^{-t_{0}r}b_{\alpha,1}(\lambda,r)\,\,\frac{r\big{[}\frac{r}{\lambda}-\gamma
r^{\alpha}\cos\alpha\pi-1\big{]}}{(-\frac{r}{\lambda}+\gamma
r^{\alpha}\cos\alpha\pi+1)^{2}+(\gamma r^{\alpha}\sin\alpha\pi)^{2}}dr.$ (3.7)

We estimate each term in the latter separately. For the first integral, taking
into account the inequality $(a+b+c)^{2}\leq 3(a^{2}+b^{2}+c^{2})$, we have

$\displaystyle(-\frac{r}{\lambda}+\gamma
r^{\alpha}\cos\alpha\pi+1)^{2}+(\gamma r^{\alpha}\sin\alpha\pi)^{2}$
$\displaystyle\leq
3(\frac{r^{2}}{\lambda_{1}^{2}}+\gamma^{2}r^{2\alpha}+1)+\gamma^{2}r^{2\alpha}$
$\displaystyle\leq\left\\{\begin{aligned} &c,\,\,r<1;\\\ &c\,r^{2},\,\,r\geq
1.\end{aligned}\right.$ (3.8)

We note that here and elsewhere in this section, $c$ denotes a constant (not
necessarily the same one), depending on the fixed parameters $\lambda_{1}$,
$\alpha$ and $\gamma$. Therefore

$\int\limits_{0}^{1}e^{-t_{0}r}b_{\alpha,1}(\lambda,r)dr\geq
c\int\limits_{0}^{1}r^{\alpha}e^{-rt_{0}}dr=c\,t_{0}^{-1-\alpha}\,I_{1}(t_{0}),$
(3.9)

where

$I_{1}(t_{0})=\int\limits_{0}^{t_{0}}\xi^{\alpha}e^{-\xi}d\xi,$ (3.10)

and

$\int\limits_{1}^{\infty}e^{-t_{0}r}b_{\alpha,1}(\lambda,r)dr\geq
c\int\limits_{1}^{\infty}r^{\alpha-2}e^{-rt_{0}}dr=c\,t_{0}^{1-\alpha}\,I_{2}(t_{0}).$
(3.11)

where

$I_{2}(t_{0})=\int\limits_{t_{0}}^{\infty}\xi^{\alpha-2}e^{-\xi}d\xi.$ (3.12)

It follows from estimates (3.9) and (3.11) that

$-\frac{1}{\lambda^{2}}\int\limits_{0}^{\infty}e^{-t_{0}r}b_{\alpha,1}(\lambda,r)dr\leq-\frac{C_{0}(t_{0})}{\lambda^{2}},$
(3.13)

where
$C_{0}(t_{0})=c\,\left(t_{0}^{-1-\alpha}\,I_{1}(t_{0})+t_{0}^{1-\alpha}\,I_{2}(t_{0})\right)$.

Let us now estimate the second term in (3.7) from above. Denote

$R(r)=-\frac{r}{\lambda}+\gamma r^{\alpha}\cos\alpha\pi+1.$

Then $R(0)=1$ and we can choose a positive number $r_{0}$ such that for all
$r\in(0,r_{0})$ one has $R(r)\geq\varepsilon$ with some $\varepsilon>0$. For
these $r$ we have

$b_{\alpha,1}(\lambda,r)\leq\frac{\gamma}{\pi}\,\,\frac{r^{\alpha}\sin\alpha\pi}{(-\frac{r}{\lambda}+\gamma
r^{\alpha}\cos\alpha\pi+1)^{2}}\leq\frac{\gamma}{\varepsilon^{2}\pi}\,\,r^{\alpha}\sin\alpha\pi.$

Taking this into account, we obtain

$\displaystyle\int\limits_{0}^{r_{0}}$ $\displaystyle
e^{-t_{0}r}b_{\alpha,1}(\lambda,r)\,\,\frac{r\big{[}\frac{r}{\lambda}-\gamma
r^{\alpha}\cos\alpha\pi-1\big{]}}{(-\frac{r}{\lambda}+\gamma
r^{\alpha}\cos\alpha\pi+1)^{2}+(\gamma r^{\alpha}\sin\alpha\pi)^{2}}dr$
$\displaystyle\leq\frac{\gamma}{\varepsilon^{3}\pi}\sin\alpha\pi\int\limits_{0}^{r_{0}}e^{-t_{0}r}r^{\alpha+1}dr$
$\displaystyle=\frac{c}{t_{0}^{\alpha+2}}\int\limits_{0}^{r_{0}t_{0}}e^{-\xi}\xi^{\alpha+1}dr\leq\frac{c\,\Gamma(\alpha+2)}{t_{0}^{\alpha+2}}.$
(3.14)

Now let $r\geq r_{0}$. Then, evidently,

$b_{\alpha,1}(\lambda,r)\leq\frac{1}{\gamma\pi}\,\,\frac{1}{r^{\alpha}\sin\alpha\pi},$

and, moreover

$\displaystyle\frac{r\big{|}\frac{r}{\lambda}-\gamma
r^{\alpha}\cos\alpha\pi-1\big{|}}{(-\frac{r}{\lambda}+\gamma
r^{\alpha}\cos\alpha\pi+1)^{2}+(\gamma r^{\alpha}\sin\alpha\pi)^{2}}$
$\displaystyle\leq\frac{r^{1-2\alpha}}{(\gamma\sin\alpha\pi)^{2}}\,\big{|}\frac{r}{\lambda}-\gamma
r^{\alpha}\cos\alpha\pi-1\big{|}\leq\frac{Cr^{2-2\alpha}}{(\gamma\sin\alpha\pi)^{2}},$

where $C=\lambda_{1}^{-1}+r_{0}^{\alpha-1}\gamma^{-1}+r_{0}^{-1}$. Hence

$\displaystyle\int\limits_{r_{0}}^{\infty}$ $\displaystyle
e^{-t_{0}r}b_{\alpha,1}(\lambda,r)\,\,\frac{r\big{|}\frac{r}{\lambda}-\gamma
r^{\alpha}\cos\alpha\pi-1\big{|}}{(-\frac{r}{\lambda}+\gamma
r^{\alpha}\cos\alpha\pi+1)^{2}+(\gamma r^{\alpha}\sin\alpha\pi)^{2}}dr$
$\displaystyle\leq\frac{C}{\pi(\gamma\sin\alpha\pi)^{3}}\,\,\int\limits_{r_{0}}^{\infty}e^{-t_{0}r}r^{2-3\alpha}dr\leq\frac{C}{\pi(\gamma\,r_{0}^{\alpha}\sin\alpha\pi)^{3}}\,\,\int\limits_{0}^{\infty}e^{-t_{0}r}r^{2}dr$
$\displaystyle=\frac{C\Gamma(3)}{\pi(\gamma\,r_{0}^{\alpha}\sin\alpha\pi)^{3}}\,\cdot\frac{1}{t_{0}^{3}}.$

It follows from the latter and estimate (3.14)) that the second term in (3.7)
is estimated from above by the quantity

$\frac{c}{\lambda^{3}}\cdot\left[\frac{1}{t_{0}^{\alpha+2}}+\frac{1}{t_{0}^{3}}\right].$
(3.15)

Finally estimates (3.13) and (3.15) imply

$\partial_{\lambda}B_{\alpha}(\lambda,t_{0})\leq-\frac{1}{\lambda^{2}}\left[C_{0}(t_{0})-\frac{c}{\lambda}\left(\frac{1}{t_{0}^{\alpha+2}}+\frac{1}{t_{0}^{3}}\right)\right]<0.$
(3.16)

The lemma is proved. ∎

Let $\\{\lambda_{k}\\}$ be the set of eigenvalues of the operator $A$. Recall
that this set has no finite limit points. In particular, the multiplicity of
any eigenvalue is finite. Let $\beta\in(0,1)$. In our further analysis of non-
local problem (2.1) we will face with the solution of the following equation

$B_{\alpha}(\lambda,t_{0})=\beta$ (3.17)

with respect to $\lambda$. If $\lambda_{0}$ is a root of equation (3.17), then
the set of all $k$ for which $\lambda_{k}=\lambda_{0}$ will be denoted by
$K_{0}$. If there is no an eigenvalue $\lambda_{k}$ equal to $\lambda_{0}$,
then evidently the set $K_{0}$ is empty.

######  Remark 3.6.

According to Lemma 3.5, starting from some number $k$, the function
$B_{\alpha}(\lambda_{k},t_{0})$ decreases strictly with respect to
$\lambda_{k}$ (if the multiplicity of the eigenvalue $\lambda_{k}$ is not
taken into account). Therefore, the set $K_{0}$ is always finite.

Thus, Lemma 3.5 states that only a finite number of eigenvalues $\lambda_{k}$
can be solutions of equation (3.17). It can also be proved that if $t_{0}$ is
large enough, then there can be only one such an eigenvalue. Indeed, the
following statement is true:

######  Lemma 3.7.

Let $\gamma>0$ and $\alpha\in(0,1)$ be given numbers. There exists a positive
number $T_{0}=T_{0}(\gamma,\alpha,\lambda_{1})\geq 1$ such that for any
$t_{0}$ in the interval $T_{0}\leq t_{0}\leq T$ the inequality

$\partial_{\lambda}B_{\alpha}(\lambda,t_{0})<0,\quad\lambda\geq\lambda_{1}.$
(3.18)

holds.

###### Proof.

It is easy to see that in estimates (3.13) and (3.15) the parameter $\lambda$
can be replaced by its smallest value $\lambda_{1}$ and the corresponding
estimates remain valid. Then estimate (3.16) takes the form:

$\partial_{\lambda}B_{\alpha}(\lambda,t_{0})\leq-\frac{1}{\lambda_{1}^{2}}\left[C_{0}(t_{0})-\frac{c}{\lambda_{1}}\left(\frac{1}{t_{0}^{\alpha+2}}+\frac{1}{t_{0}^{3}}\right)\right].$
(3.19)

Now suppose that $t_{0}\geq 1$ and estimate $C_{0}(t_{0})$ from below. For the
integral $I_{1}(t_{0}),$ defined in (3.10), we have

$I_{1}(t_{0})\geq\int\limits_{0}^{1}\xi^{\alpha}e^{-\xi}d\xi\geq\frac{1}{(\alpha+1)\,e}.$

Similarly for the integral $I_{2}(t_{0})$, defined in (3.12), after
integration by parts twice, we obtain

$\displaystyle I_{2}(t_{0})$
$\displaystyle=\frac{t_{0}^{\alpha-1}e^{-t_{0}}}{1-\alpha}+\frac{t_{0}^{\alpha}e^{-t_{0}}}{(1-\alpha)\alpha}+\frac{1}{(1-\alpha)\alpha}\int\limits_{t_{0}}^{\infty}\xi^{\alpha}e^{-\xi}d\xi$
$\displaystyle\geq
e^{-t_{0}}\left[\frac{t_{0}^{\alpha-1}}{1-\alpha}+\frac{t_{0}^{\alpha}}{(1-\alpha)\alpha}+\frac{t_{0}^{\alpha+1}}{(1-\alpha)\alpha(\alpha+1)}\right].$

Therefore, for sufficiently large $t_{0}$ we have

$C_{0}(t_{0})\geq c\,t_{0}^{-1-\alpha}.$

Consequently, estimate (3.19) takes the form

$\partial_{\lambda}B_{\alpha}(\lambda,t_{0})\leq-\frac{c}{\lambda_{1}^{2}\,t_{0}^{1+\alpha}}\left[1-\frac{1}{\lambda_{1}}\left(\frac{1}{t_{0}}+\frac{1}{t_{0}^{2-\alpha}}\right)\right].$

This implies the assertion of the lemma. ∎

######  Remark 3.8.

Under the conditions of this lemma, only one eigenvalue $\lambda_{k_{0}}$ may
satisfy equation (3.17). Let the multiplicity of $\lambda_{k_{0}}$ be equal to
$p$. Then $K_{0}=\\{k_{0},k_{0}+1,\cdots,k_{0}+p-1\\}$.

We also note that in Lemma 3.5 $\lambda$ is a sufficiently large and $t_{0}$
is an arbitrary positive number, in Lemma 3.7, on the contrary, $t_{0}$ is
sufficiently large and $\lambda\geq\lambda_{1}$ is an arbitrary number.

##  4\. Existence of a solution of the non-local problem (2.1)

To solve the non-local problem (2.1), we divide it into two auxiliary
problems:

$\left\\{\begin{aligned}
&\partial_{t}\omega(t)+(1+\gamma\,\partial_{t}^{\alpha})A\omega(t)=f(t),\quad
0<t\leq T,\\\ &\omega(0)=0,\end{aligned}\right.$ (4.1)

and

$\left\\{\begin{aligned}
&\partial_{t}w(t)+(1+\gamma\,\partial_{t}^{\alpha})Aw(t)=0,\quad 0<t\leq T,\\\
&w(t_{0})=\beta w(0)+\psi,\end{aligned}\right.$ (4.2)

where $\psi\in H$ is a given element and $t_{0}$ is any fixed number from
$(0,T]$.

Problems (4.1) and (4.2) are a special cases of problem (2.1), so solutions to
problems (4.1) and (4.2) are defined similarly to Definition 2.1.

######  Lemma 4.1.

Let $\psi$ in (4.2) have the form $\psi=\varphi-\omega(t_{0}),$ where
$\varphi$ is a function in non-local problem (2.1). If $\omega(t)$ and $w(t)$
are solutions of problems (4.1) and (4.2) correspondingly, then the solution
to problem (2.1) has the form $u(t)=\omega(t)+w(t)$ .

Proof of this lemma is standard, and therefore, we omit it.

Auxiliary problem (4.1) is solved in [18]. Let us formulate the corresponding
result:

######  Theorem 4.2.

Let $\varepsilon\in(0,1)$ and $f(t)\in C([0,T];D(A^{\varepsilon}))$. Then
problem (4.1) has a unique solution

$\omega(t)=\sum\limits_{k=1}^{\infty}\left[\int\limits_{0}^{t}B_{\alpha}(\lambda_{k},t-\tau)f_{k}(\tau)d\tau\right]v_{k}.$
(4.3)

Moreover, there is a constant $C_{\varepsilon}>0$, such that

$||\partial_{t}\omega(t)||^{2}+||\partial_{t}^{\alpha}A\omega(t)||^{2}\leq
C_{\varepsilon}\max\limits_{t\in[0,T]}||f||_{\varepsilon}^{2},\quad 0\leq
t\leq T.$ (4.4)

Now let us consider non-local problem (4.2). We will seek the solution of this
problem in the form of a generalized Fourier series

$w(t)=\sum\limits_{k=1}^{\infty}T_{k}(t)v_{k},$

where $v_{k}$ are the eigenvectors of the operator $A$ and $T_{k}(t)$, $k\geq
1$, are solutions of the following non-local problems:

$\left\\{\begin{aligned}
&T^{\prime}_{k}(t)+\lambda_{k}(1+\gamma\partial_{t}^{\alpha})T_{k}(t)=0,\quad
0<t\leq T;\\\ &T_{k}(t_{0})=\beta T_{k}(0)+\psi_{k},\end{aligned}\right.$
(4.5)

where $k\geq 1,$ $t_{0}\in(0,T]$ is a fixed point and $\psi_{k}$ is the
Fourier coefficients of the element $\psi\in H$. Denote
$h_{k}=T_{k}(0),k=1,2,\dots.$ Then the unique solution to problem (4.5) has
the form (see Lemma 3.2)

$T_{k}(t)=h_{k}B_{\alpha}(\lambda_{k},t).$ (4.6)

To find the unknown numbers $h_{k},$ we use the non-local conditions of (4.5.
Namely,

$h_{k}B_{\alpha}(\lambda_{k},t_{0})=\beta h_{k}+\psi_{k}.$

or, the same,

$h_{k}(B_{\alpha}(\lambda_{k},t_{0})-\beta)=\psi_{k}.$ (4.7)

If $\beta\geq 1,$ or $\beta<0$ (note, $t_{0}>0$ and $\lambda_{k}>0$), then
$B_{\alpha}(\lambda_{k},t_{0})\neq\beta$ due to Lemma 3.1. Therefore, in these
cases it follows from (4.7) that

$h_{k}=\frac{\psi_{k}}{B_{\alpha}(\lambda_{k},t_{0})-\beta},$ (4.8)

and

$|h_{k}|\leq C_{\beta}|\psi_{k}|,\,\,k\geq 1,$ (4.9)

where $C_{\beta}$ is a constant depending on $\beta$. If $\beta=0$, then
$B_{\alpha}(\lambda_{k},t_{0})\neq 0$, however, in accordance with Lemma 3.1,
the function $B_{\alpha}(\lambda_{k},t_{0})$ asymptotically tends to zero, as
$k\to\infty.$ Therefore, by Lemma 3.4 one has:

$h_{k}=\frac{\psi_{k}}{B_{\alpha}(\lambda_{k},t_{0})},\,\,C_{1}\lambda_{k}|\psi_{k}|\leq|h_{k}|\leq
C_{2}\lambda_{k}|\psi_{k}|.$

Here constants $C_{j},j=1,2,$ may depend on $\alpha,\gamma,\lambda_{1}$ and
$t_{0}$. As noted above, this case has been studied in detail in [18].
Therefore, we will not consider it further.

Now, let $0<\beta<1$ and consider equation (3.17). In accordance with Remark
3.6, there are two possible cases: the set $K_{0}$ is empty and it is not
empty. If $K_{0}$ is empty, then since the set $\\{\lambda_{k}\\}$ does not
have a finite limit point, the estimate in (4.8) holds with some constant
$C_{\beta}>0$ for all $k$.

Thus, if $\beta\in(0,1)$ and $K_{0}$ is empty, then the formal solution of
problem (4.2) still has the form

$w(t)=\sum\limits_{k=1}^{\infty}\frac{\psi_{k}}{B_{\alpha}(\lambda_{k},t_{0})-\beta}B_{\alpha}(\lambda_{k},t)v_{k}.$
(4.10)

Finally, let us assume that $0<\beta<1$ and the set $K_{0}$ is not empty. In
this case, due to equation (4.7), non-local problem (4.5) has a solution if
and only if the following orthogonality conditions are verified:

$\psi_{k}=(\psi,v_{k})=0,\,\,k\in K_{0}.$ (4.11)

Moreover, for the values $k\in K_{0}$ arbitrary numbers $h_{k}$ are solutions
of equation (4.7). For all other $k$ we have

$h_{k}=\frac{\psi_{k}}{B_{\alpha}(\lambda_{k},t_{0})-\beta},\quad|h_{k}|\leq
C_{\beta}|\psi_{k}|,\quad k\notin K_{0}.$ (4.12)

Thus, the formal solution of problem (4.2) in this case has the form

$w(t)=\sum\limits_{k\notin
K_{0}}\frac{\psi_{k}}{B_{\alpha}(\lambda_{k},t_{0})-\beta}B_{\alpha}(\lambda_{k},t)v_{k}+\sum\limits_{k\in
K_{0}}h_{k}B_{\alpha}(\lambda_{k},t)v_{k}.$ (4.13)

Now let us show that the series (4.10) and (4.13) indeed define solutions to
non-local problem (4.2). According to Definition 2.1, it suffices to show the
applicability of the operators $\partial_{t}$ and $\partial_{t}^{\alpha}A$
term by term to these series and $w(t)\in C([0,T];H)$,
$\partial_{t}w(t),Aw(t),\partial_{t}^{\alpha}Aw(t)\in C((0,T);H)$. We
demonstrate this with the solution in (4.10). In what concerns the solution in
(4.13), then it is treated in exactly the same way.

Let $S_{j}(t),\ j\geq 1,$ be the sequence of partial sums of series (4.10).
Applying Parseval’s equality, estimate (4.8), and the first assertion of Lemma
3.1, we have

$||S_{j}(t)||^{2}=\sum\limits_{k=1}^{j}\bigg{|}\frac{\psi_{k}}{B_{\alpha}(\lambda_{k},t_{0})-\beta}B_{\alpha}(\lambda_{k},t)\bigg{|}^{2}\leq
C_{\beta}||\psi||^{2},$

Letting $j\to\infty,$ it follows from the latter that $w(t)\in C([0,T];H)$.
Further, we have

$AS_{j}(t)=\sum\limits_{k=1}^{j}\frac{\lambda_{k}\psi_{k}}{B_{\alpha}(\lambda_{k},t_{0})-\beta}B_{\alpha}(\lambda_{k},t)v_{k}.$

Using the same reasoning (using the third assertion of Lemma 3.1) as above, we
get

$||AS_{j}(t)||^{2}=\sum\limits_{k=1}^{j}\bigg{|}\frac{\lambda_{k}\psi_{k}}{B_{\alpha}(\lambda_{k},t_{0})-\beta}B_{\alpha}(\lambda_{k},t)\bigg{|}^{2}\leq\frac{C_{\beta}}{t^{2(1-\alpha)}}||\psi||^{2},$
(4.14)

which implies that $Aw(t)\in C((0,T);H)$. Next, applying Lemma 3.3, we have
the following estimate:

$||\partial_{t}S_{j}(t)||^{2}=\sum\limits_{k=1}^{j}\left|\frac{\psi_{k}}{B_{\alpha}(\lambda_{k},t_{0})-\beta}\partial_{t}B_{\alpha}(\lambda_{k},t)\right|^{2}\leq\frac{C_{\beta}}{\lambda_{1}t^{2(2-\alpha)}}||\psi||^{2}.$
(4.15)

The latter implies $\partial_{t}w(t)\in C((0,T];H)$. Equation (4.2)written in
the form $\partial_{t}w(t)=-Aw(t)-\gamma\,\partial_{t}^{\alpha}Aw(t)$, $t>0$,
and the estimates obtained above imply

$||\partial_{t}^{\alpha}Aw(t)||^{2}\leq\frac{C_{\beta}}{t^{2(2-\alpha)}}||\psi||^{2},$
(4.16)

Hence, $\partial_{t}^{\alpha}Aw(t)\in C((0,T];H),$ as well.

Thus, if $\beta\notin(0,1)$ or $\beta\in(0,1)$, but $K_{0}$ is an empty set,
then the function (4.10) is indeed a solution to problem (4.2).

Finally we show the uniqueness of the solution to problem (4.2). Suppose we
have two solutions: $w_{1}(t)$, $w_{2}(t)$ and set $w(t)=w_{1}(t)-w_{2}(t)$.
Then we have

$\left\\{\begin{aligned}
&\partial_{t}w(t)+(1+\gamma\,\partial_{t}^{\alpha})Aw(t)=0,\quad 0<t\leq T;\\\
&w(t_{0})=\beta w(0).\end{aligned}\right.$

Let $w(t)$ be any solution to this problem and let $w_{k}(t)=(w(t),v_{k})$.
Since the operator $A$ is self-adjoint, one has

$\displaystyle\partial_{t}^{\rho}w_{k}(t)$
$\displaystyle=(\partial_{t}^{\rho}w(t),v_{k})=-(Aw(t),v_{k})-\gamma(\partial_{t}^{\alpha}w,v_{k})$
$\displaystyle=-\lambda_{k}(1+\gamma\partial_{t}^{\alpha})w_{k}(t)$

or

$\partial_{t}^{\rho}w_{k}(t)+\lambda_{k}(1+\gamma\partial_{t}^{\alpha})w_{k}(t)=0.$
(4.17)

It follows from the nonlocal condition that

$w_{k}(t_{0})=\beta w_{k}(0).$ (4.18)

Let us denote $w_{k}(0)=h_{k}$. Then the unique solution to the differential
equation (4.17) with this initial condition has the form
$w_{k}(t)=h_{k}B_{\alpha}(\lambda_{k},t)$ (see Lemma 3.2). Using condition
(4.18) we obtain the following equation for unknown numbers $h_{k}$:

$h_{k}B_{\alpha}(\lambda_{k},t_{0})=\beta h_{k}.$ (4.19)

Let $\beta\notin(0,1)$ or $\beta\in(0,1)$, but $K_{0}$ is an empty set. Then
$B_{\alpha}(\lambda_{k},t)\neq\beta$ for all $k$. Consequently, in this case
all $h_{k}$ are equal to zero. Ttherefore $w_{k}(t)=0$, and by virtue of
completeness of the set of eigenfunctions $\\{v_{k}\\}$, we conclude that
$w(t)\equiv 0$. Thus, problem (4.2) in this case has a unique solution.

Now suppose that $\beta\in(0,1)$ and $K_{0}$ is not empty. Then
$B_{\alpha}(\lambda_{k},t)=\beta$, $k\in K_{0}$ and therefore equation (4.19)
has the following solution: $h_{k}=0$ if $k\notin K_{0}$ and $h_{k}$ is an
arbitrary number for $k\in K_{0}$. Thus, in this case, there is no uniqueness
of the solution to problem (4.2). We note that the non-local problem under
consideration has a finite-dimensional kernel

$Ker=\\{h\in H:h=\sum_{k\in K_{0}}h_{k}v_{k}\\}.$

in this case.

Thus we obtain the following statement:

######  Theorem 4.3.

Let $\psi\in H$. If $\beta\notin[0,1)$ or $\beta\in(0,1)$, but $K_{0}$ is
empty, then problem (4.2) has a unique solution and this solution has the form
(4.10). If $\beta\in(0,1)$ and $K_{0}$ is not empty, then a solution to
problem (4.2) exists if and only if the orthogonality conditions (4.11) are
satisfied. The solution of problem (4.2) has the form (4.13) with arbitrary
coefficients $h_{k}$, $k\in K_{0}$. Moreover, there is a constant
$C_{\beta}>0$ such that the following coercive estimate holds:

$||\partial_{t}w(t)||^{2}+||Aw(t)||^{2}+||\partial_{t}^{\alpha}Aw(t)||^{2}\leq
C_{\beta}t^{-2(1-\alpha)}||\psi||^{2},\quad 0<t\leq T.$ (4.20)

Note that the proof of the coercive inequality (4.20) follows from the
estimates (4.14), (4.15) and (4.16).

Now we are ready to solve the main problem in (2.1). Let $\varphi\in H$ and
$f(t)\in C([0,T];D(A^{\varepsilon}))$ for some $\varepsilon\in(0,1)$. If we
put $\psi=\varphi-\omega(t_{0})\in H$ and $\omega(t)$ and $w(t)$ are the
corresponding solutions of problems (4.1) and (4.2), then the function
$u(t)=\omega(t)+w(t)$ is a solution to problem (2.1). Therefore, if
$\beta\notin(0,1)$ or $\beta\in(0,1)$, but $K_{0}$ is empty, then

$u(t)=\sum\limits_{k=1}^{\infty}\left[\frac{\varphi_{k}-\omega_{k}(t_{0})}{B_{\alpha}(\lambda_{k},t_{0})-\beta}\,B_{\alpha}(\lambda_{k},t)+\omega_{k}(t)\right]v_{k},$
(4.21)

where

$\omega_{k}(t)=\int\limits_{0}^{t}B_{\alpha}(\lambda_{k},\eta)f_{k}(t-\eta)d\eta.$

The uniqueness of the solution $u(t)$ of problem (2.1) follows from the
uniqueness of the solutions $\omega(t)$ and $w(t)$.

If $\beta\in(0,1)$ and $K_{0}$ is not empty, then

$u(t)=\sum\limits_{k\notin
K_{0}}\left[\frac{\varphi_{k}-\omega_{k}(t_{0})}{B_{\alpha}(\lambda_{k},t_{0})-\beta}\,B_{\alpha}(\lambda_{k},t)+\omega_{k}(t)\right]v_{k}+\sum\limits_{k\in
K_{0}}h_{k}B_{\alpha}(\lambda_{k},t)v_{k},$ (4.22)

where $h_{k}$ are arbitrary numbers. The corresponding orthogonality
conditions have the form

$(\varphi,v_{k})=(\omega(t_{0}),v_{k}),\,\,k\in K_{0}.$ (4.23)

In particular, if

$(\varphi,v_{k})=0,\,\,(f(t),v_{k})=0,\,\,\text{for all}\,\,0\leq t\leq
t_{0},\,\,k\in K_{0},$ (4.24)

then the orthogonality conditions (4.23) are satisfied.

Thus we proved the following statement.

######  Theorem 4.4.

Let $\varphi\in H$ and $f(t)\in C([0,T];D(A^{\varepsilon}))$ for some
$\varepsilon\in(0,1)$. If $\beta\notin[0,1)$ or $\beta\in(0,1)$, but $K_{0}$
is empty, then problem (2.1) has a unique solution and this solution has the
form (4.21). If $\beta\in(0,1)$ and $K_{0}$ is not empty, then we assume that
the orthogonality conditions (4.24) are satisfied. The solution of problem
(2.1) has the form (4.22) with arbitrary coefficients $h_{k}$, $k\in K_{0}$.
Moreover, there are constants $C_{\beta}>0$ and $C_{\varepsilon}>0$ such that
the following coercive estimate holds:

$||\partial_{t}w(t)||^{2}+||Aw(t)||^{2}+||\partial_{t}^{\alpha}Aw(t)||^{2}\leq
C_{\beta}t^{-2(2-\alpha)}||\varphi||^{2}+C_{\varepsilon}\max\limits_{t\in[0,T]}||f||_{\varepsilon}^{2},$
$0<t\leq T.$

Note that the validity of the assertions in Theorem 4.3 and Theorem 4.4
requires that the orthogonality conditions (4.11) and (4.24) be satisfied,
respectively. In light of these conditions a natural question arises: how
restrictive are these orthogonality conditions. To answer this question,
consider the following example.

Let a bounded domain $\Omega\subset\mathbb{R}^{N}$ have sufficiently smooth
boundary $\partial\Omega$. Consider the operator $A_{0}$, defined in
$L_{2}(\Omega)$ with domain of definition $D(A_{0})=\\{f\in C^{2}(\Omega)\cap
C(\overline{\Omega}):\,f(x)=0,\,x\in\partial\Omega\\}$ and acting as
$A_{0}f(x)=-\triangle f(x)$. Then, as is known (see, e.g. [32]), $A_{0}$ has a
complete in $L_{2}(\Omega)$ system of orthonormal eigenfunctions
$\\{v_{k}(x)\\}$ and a countable set of nonnegative eigenvalues $\lambda_{k}$
($\rightarrow+\infty$), and $\lambda_{1}=\lambda_{1}(\Omega)>0$.

Denote by $A$ the operator, acting as $Af(x)=\sum\lambda_{k}f_{k}v_{k}(x)$
with the domain of definition $D(A)=\\{f\in
L_{2}(\Omega):\,\sum\lambda^{2}_{k}f^{2}_{k}<\infty\\}$. Then it is not hard
to verify, that $A$ is a positive self-adjoint extension in $L_{2}(\Omega)$ of
operator $A_{0}$. Therefore, one can apply Theorems 4.3 and 4.4 to the
problem:

$\left\\{\begin{aligned}
&\partial_{t}w(x,t)-(1+\gamma\partial_{t}^{\alpha})\triangle w(x,t)=0,\quad
x\in\Omega,\quad 0<t\leq T,\,\,0<\alpha<1;\\\ &w(x,t_{0})=\beta
w(x,0)+\psi(x),\,\,x\in\Omega,\quad 0<t_{0}\leq T;\\\ &w(x,t)=0,\quad
x\in\partial\Omega,\quad 0<t\leq T,\end{aligned}\right.$ (4.25)

Suppose $\beta\in(0,1)$ and $t_{0}\in(0,T]$ satisfies conditions of Lemma
3.18. Then, according to Lemma 3.18, only one eigenvalue can satisfy equation
(3.17). Let this number be $\lambda_{1}$, i.e.
$B_{\alpha}(\lambda_{1},t_{0})=\beta.$ We note also that the multiplicity
$\lambda_{1}$ is equal to one.

Therefore, applying Theorem 4.3 we have that problem (4.25) has a solution for
any function $\psi\in L_{2}(\Omega)$, if and only if

$\psi_{1}=\int\limits_{\Omega}\psi(x)v_{1}(x)dx=0.$

In other words the first Fourier coefficient of $\psi(x)$ must be zero. In
this case, the solution of the problem is not unique and all solutions can be
represented in the series form

$w(t)=\sum\limits_{k=2}^{\infty}\frac{\psi_{k}}{B_{\alpha}(\lambda_{k},t_{0})-\beta}B_{\alpha}(\lambda_{k},t)v_{k}+hB_{\alpha}(\lambda_{1},t)v_{1},$

that converges in the norm of $L_{2}(\Omega)$ uniformly in $t\in[0,T].$ Here
$h$ is an arbitrary real number.

##  5\. Conclusion

In this paper, for the Rayleigh-Stokes equation, we study a new time-nonlocal
problem, i.e., in the problem (1.1), instead of the initial condition
$u(x,0)=\varphi(x)$, we consider the nonlocal condition $u(x,t_{0})=\beta
u(x,0)+\varphi(x)$, $0<t_{0}\leq T$. Instead of the Laplace operator
$(-\Delta)$, we take an arbitrary positive self-adjoint operator $A$, and the
naturally obtained results are also valid for an equation with a Laplace
operator with the Dirichlet condition.

Cases $\beta=0$ and $\beta=1$ were studied earlier: if $\beta=0$, then we get
a well-known time backward problem that has a unique solution, but the
solution is not stable. If $\beta=1$, then the problem becomes ”good”, i.e.
there is a unique solution and it is stable (see [18]).

Naturally, the question arises: starting from what value $\beta$ does the task
deteriorate? This paper provides a comprehensive answer to this question. It
turns out that the critical values of the parameter $\beta$ lie on the half-
interval $[0,1)$. If $\beta\notin[0,1)$, then the problem is well-posed
according to Hadamard: there is a unique solution and it continuously depends
on the data of the problem; if $\beta\in(0,1)$ (case $\beta=0$ is considered
in [18]), then the well-posedness of the problem depends on the location of
the eigenvalues of the Laplace operator, namely, if the set $K_{0}$, defined
above, is empty, then the problem is again well-posed according to Hadamard .
If $K_{0}$ is not empty, then necessary and sufficient conditions are found in
the paper, the fulfillment of which guarantees the existence of a solution,
but in this case the solution is not unique.

## References

  * [1] A. Kilbas, H. Srivastava, J. Trujillo, Theory and Applications of Fractional Differential Equations. Elsevier, Amsterdam, 2006.
  * [2] V. Kulish, J. Luis Lage, Application of fractional calculus to fluid mechanics, J. Fluids Engg. 124, No 3, 413-442, 2002.
  * [3] L. Debnath, Recent applications of fractional calculus to science and engineering, Int. J. Math. Sci. 203, No 54, 3413-3442, 2003.
  * [4] E. Bazhlekova, B. Jin, R. Lazarov, Z. Zhou, An analysis of the Rayleigh-Stokes problem for a generalized second-grade fluid. Numer. Math. 131, 1-31, 2015.
  * [5] A. F. Chudnovsky, Thermal physics of soils. Nauka, 1976. [In Russian].
  * [6] A. M. Nakhushev, Problems with displacement for partial differential equations. Nauka, 2006. [In Russian].
  * [7] A. M. Nakhushev, Loaded equations and their application. Nauka, 2012. [In Russian].
  * [8] W. C. Tan, T. Masuoka, Stokes’ first problem for a second grade fluid in a porous half-space with heated boundary. Int. J. Non-Linear Mech., 40, 515-522, 2005.
  * [9] W. C. Tan, T. Masuoka, Stokes’ first problem for an Oldroyd-B fluid in a porous half-space. Phys. Fluid, 17, 023101-7, 2005.
  * [10] C. Fetecau, M. Jamil, C. Fetecau, D. Vieru, The Rayleigh-Stokes problem for an edge in a generalized Oldroyd-B fluid. Z. Angew. Math. Phys., 60, No 5, 921-933, 2009.
  * [11] F. Shen, W. Tan, Y. Zhao, T. Masuoka, The Rayleigh-Stokes problem for a heated generalized second grade fluid with fractional derivative model. Nonlinear Anal. Real World Appl., 7, No 5, 1072-1080, 2006.
  * [12] C. Zhao, C. Yang, Exact solutions for electro-osmotic flow of viscoelastic fluids in rectangular micro-channels. Appl. Math. Comp., 211, No 2, 502-509, 2009.
  * [13] L. D. Long, B. Moradi, O. Nikan, Z. Avazzadeh, A. M. Lopes, Numerical Approximation of the Fractional Rayleigh-Stokes Problem Arising in a Generalised Maxwell Fluid. Fractal Fract., 6, 377, 2022.
  * [14] D. D. Dai, T. T. Ban, Y. L. Wang, W. Zhang, T. Hang, The piecewise reproducing kernel method for the time variable fractional order advection-reaction-diffusion equations. Thermal science, 25, 1261-1268, 2021.
  * [15] T. T. Binh, D. Baleanu, N. H. Luc, N. Can, Determination of source term for the fractional Rayleigh-Stokes equation with random data. Journal of Inequalities and Applications, 308, 1-16, 2019.
  * [16] T. T. Binh, H. K. Nashine, L. D. Long, N. H. Luc, N. Can, Identification of source term for the ill-posed Rayleigh-Stokes problem by Tikhonov regularization method. Advances in Difference Equations, 331, 1-20, 2019.
  * [17] P. N. Duc, H. D. Binh, L. D. Long, T. V. Kim, Reconstructing the right-hand side of the Rayleigh-Stokes problem with non-local in time condition. Advances in Difference Equations, 470, 1-18, 2021.
  * [18] R. Ashurov, N. Vaisova, Backward and Non-Local Problems for the Rayleigh-Stokes Equation. Fractal Fract., 6, No 10, 587, 2022.
  * [19] M. Kirane, A. S. Malik, Determination of an unknown source term and the temperature distribution for the linear heat equation involving fractional derivative in time. Applied Mathematics and Computation, 218, 163-170, 2011.
  * [20] M. Kirane, B. Samet, B. T. Torebek, Determination of an unknown source term and the temperature distribution for the subdiffusion equation at the initial and final data. Electronic Journal of Differential Equations, 217, 1-13, 2017.
  * [21] R. R. Ashurov, A. T. Mukhiddinova, Inverse problem of determining the heat source density for the subdiffusion equation. Differential Equations, 56, 1550-1563, 2020.
  * [22] H. L. Nguyen, H. T. Nguyen, K. Mokhtar, X. T. Duong Dang, Identifying initial condition of the Rayleigh-Stokes problem with random noise. Wiley, 6, 1-11, 2018.
  * [23] H. L. Nguyen, L. N. Huynh, O. D. Regan, N. H. Can, Regularization of the fractional Rayleigh-Stokes equation using a fractional Landweber method. Advances in Difference Equations, 459, 1-23, 2020.
  * [24] A. O. Ashyralyev, Nonlocal boundary-value problems for abstract parabolic equations: well-posedness in Bochner spaces. Journal of Evolution Equations, 6, 1-28, 2006.
  * [25] A. O. Ashyralyev, A. Hanalyev, P. E. Sobolevskii, Coercive solvability of nonlocal boundary value problem for parabolic equations. Abstract and Applied Analysis, 6, 53-61, 2001.
  * [26] A. O. Ashyralyev, P. E. Sobolevskii, Coercive stability of a multidimensional difference elliptic equation of 2m-th order with variable coefficients. Investigations in the Theory of Differential Equations, Minvuz Turkmen. SSR, Ashkhabad, 31-43, 1987. [in Russian].
  * [27] R. R. Ashurov, Yu. Fayziev, On the nonlocal problems in time for time-fractional subdiffusion equations. Fractal and Fractional , 6, No 41, 1-21, 2022.
  * [28] R. R. Ashurov, Yu. Fayziev, On the nonlocal problems in time for subdiffusion equations with the Riemann-Liouville derivatives. Bulletin of the Karaganda University, 106, No 2, 18-37, 2022.
  * [29] T. T. Phong, L. D. Long, Well-posed results for nonlocal fractional parabolic equation involving Caputo-Fabrizio operator. J. Math. Computer Sci., 26, 357-367,2022.
  * [30] C. Lizama, Abstract linear fractional evolution equations. Handbook of Fractional Calculus with Applications, 2, 465-497, 2019.
  * [31] A. V. Pskhu, Initial problem for linear ordinary differential fractional order equations. Mat. Sb., 4, 111-122, 2011. [In Russian].
  * [32] V. A. Il’in, On the solvability of mixed problems for hyperbolic and parabolic equations, Math. Surveys, 15, 85-142, 1960.[in Russian]


# Zero-Shot Cost Models for Distributed Stream Processing

Roman Heinrich  DHBW Mannheim ,  Manisha Luthra  Technical University
of Darmstadt ,  Harald Kornmayer  DHBW Mannheim and  Carsten Binnig  Technical
University of Darmstadt & DFKI

(2022)

###### Abstract.

This paper proposes a learned cost estimation model for Distributed Stream
Processing Systems (DSPS) with an aim to provide accurate cost predictions of
executing queries. A major premise of this work is that the proposed learned
model can generalize to the dynamics of streaming workloads _out-of-the-box_.
This means a model once trained can accurately predict performance metrics
such as _latency_ and _throughput_ even if the characteristics of the data and
workload or the deployment of operators to hardware changes at runtime. That
way, the model can be used to solve tasks such as optimizing the placement of
operators to minimize the end-to-end latency of a streaming query or maximize
its throughput even under varying conditions. Our evaluation on a well-known
DSPS, Apache Storm, shows that the model can predict accurately for unseen
workloads and queries while generalizing across real-world benchmarks.

Stream processing, Cost Models, Zero-shot learning

††journalyear: 2022††copyright: acmcopyright††conference: The 16th ACM
International Conference on Distributed and Event-based Systems; June 27–30,
2022; Copenhagen, Denmark††booktitle: The 16th ACM International Conference on
Distributed and Event-based Systems (DEBS ’22), June 27–30, 2022, Copenhagen,
Denmark††price: 15.00††doi: 10.1145/3524860.3539639††isbn:
978-1-4503-9308-9/22/06††ccs: Information systems Stream management

##  1\. Introduction

#### Motivation

Distributed Stream Processing System (DSPS) correlates and analyzes data
streams from multiple data sources to derive higher-level information for a
wide range of applications. At the core, a DSPS takes a continuous query that
represents one or more tasks for the given application and processes the query
in a distributed way using multiple hardware resources (cf. Figure 1). While
doing so, for many applications, a DSPS has to provide guarantees in terms of
one or more quality-of-service (QoS) cost metrics such as latency and
throughput. For instance, in Facebook, queries for click stream analytics have
to provide a very high throughput to process input event streams at around 9
GiB/s and at the same time also ensure low latency (shao2011, ).

Typically, a DSPS provides QoS guarantees using optimization mechanisms such
as _operator placement_ that usually monitors the costs to decide on the
mapping of operators to hardware as shown in Figure 1 (nardelli2019, ).
Moreover, frequent reconfigurations of the operator placement are required
based on the observed changes of the workload (e.g., during peaks in the input
stream). Further, to provide QoS guarantees, a DSPS uses multiple other
techniques such as _elasticity_ or adaption of _operator parallelism_
(russo2019, ). (hirzel2014, ) provides an extensive list of other DSPS
optimization mechanisms.

Figure 1. A DSPS has to provide guarantees in terms of one or more quality-of-
service (QoS) cost metrics such as latency and throughput. The challenge is
that DSPS serve a wide range of workloads on potentially diverse hardware,
which makes the cost estimation harder.

However, many of these optimization mechanisms classically rely on heuristics
(nardelli2019, ; eskandari2021, ) or other analytical approaches like queueing
theory (mencagli2018, ) that approximates the effects of placement decisions,
the degree of parallelism, etc., on the actual latency and throughput. As a
result, these techniques often make simplifying assumptions on the estimation
of the required QoS and thus take non-optimal decisions (e.g., they decide on
a non-optimal placement).

More recently, machine learning (ML) has been used in solving these
optimization tasks such as to place operators (luthra2021, ; li2018, ), to
auto-scale DSPS (russo2019, ) or even to estimate resource utilization
(wang2017, ), which has shown promising initial results (ahmad2021, ).
However, the existing ML-based approaches cannot be used out-of-the-box as
these often are specialized either for the given optimization task (e.g.,
elasticity) or support only a restricted set of workloads and hardware
resources.

While, in this paper, we argue that learned cost models should be used for
DSPS to allow better decisions for a broad set of optimization tasks. For
example, the cost model can be used in advance before actually placing
operators to different resources to know the expected throughput and latency.
The same observations hold for other optimization tasks, such as determining
the degree of parallelism for an operator. Recently, learned cost models have
also been explored in database systems to estimate costs such as query runtime
and thus solve problems related to query optimization or scheduling. Yet these
learned models are highly _workload-driven_ (li2021, ) – meaning that they
make strong assumptions about what data and queries as well as hardware should
be supported.

Thus, a core problem of existing learned cost models is that they can hardly
generalize to the changes in the workload (queries and data) or even the
placement on different hardware platforms at runtime. Consequently, either a
separate model must be trained for a combination of workload and hardware or
the existing one must be constantly retrained, which not only causes high
overhead but also does not allow the DSPS to react instantly to changes.

#### Contributions

In this paper, therefore, we propose a new approach for _learned cost models_
for DSPSs based on the concept of _zero-shot learning_ (hilprecht2022, ;
hilprecht2022cidr, ). The key idea is to train a model on a broad spectrum of
different streaming workloads and hardware resources to enable generalization.
As such, a zero-shot cost model for DSPS can provide accurate estimates even
under the changes in the streaming workload or for different hardware
platforms that are used for placing operators. Moreover, the zero-shot cost
model can even adapt to an entirely _new_ workload _out-of-the-box_ , with
queries and data characteristics the model has not seen during the training.
Thus, our approach comes as a huge benefit for DSPSs since they are known to
be highly dynamic in nature (e.g., fluctuations in input stream). That way,
using such a zero-shot cost model, several DSPS optimization tasks can be
performed (e.g., operator placement) in a highly accurate manner such that the
desired QoS guarantees are satisfied even under dynamics in streaming
workload.

A key to enable a zero-shot cost model is a novel model architecture that
represents data, queries and hardware as input for our cost model. At the core
of our model architecture is a so-called _transferable feature representation_
that allows the learned cost model to generalize to a broader set of workloads
or even be used across hardware platforms. For instance, to make predictions
under changing workloads, the transferable feature representation relies on
general information such as _event rate at the source_ and other information
such as _tuple width_ of data streams or _window size_ of operators. Another
important aspect of the transferable feature representation is that we include
so-called _data characteristics_ of the queries and data streams as features,
such as selectivity of an operator, so that the zero-shot model can learn the
runtime behavior of a query.

In summary, this work makes the following contributions:
_(i)_ We discuss the training and inference procedure of the zero-shot model.
For training, we provide a broad spectrum of workloads and hardware platforms
to a zero-shot cost model for DSPS such that it can learn to generalize to
unseen streaming workloads and provide estimates across different hardware
platforms.
_(ii)_ We present a new model architecture for learning such a zero-shot cost
model for DSPSs that can generalize to unseen workloads by using a
transferable feature representation.
_(iii)_ We provide an evaluation of our model on Apache Storm, a well-known
DSPS. The results show that our model accurately predicts the performance of
DSPS queries on Storm, even for unseen operator and data stream parameters and
generalizes across different real-world benchmarks without explicitly training
for them.

#### Outline

In Section 2, we provide an overview of our approach before we explain the
details of the model architecture and its feature representation in Section 3.
Afterwards, we present the experiments in Section 4, related work in Section
5, and conclude in Section 6.

##  2\. Overview of Approach

In the following, we provide a high-level overview of our approach. We first
start with the idea of how a zero-shot cost model is being trained for DSPS
before we then explain its usage at runtime to estimate the cost for
guaranteeing QoS.

#### Training a Zero-Shot Cost Model

The proposed zero-shot cost model learns from previous query executions and
their observed costs in a supervised manner. To allow the model to generalize,
we generate and train our model with a broad spectrum of queries and streaming
data as well as different observed cost metrics (i.e., latency and throughput)
that are induced by these queries. To be more precise, to enable a high
variety in the training data, we represent standard query structures for DSPS
and vary them in terms of complexity and operators properties such as _window
size_ (cf. Section 4 to see our training range). Similarly, we diversify the
input data streams by training for several _event rates_ , and we capture many
different streaming workloads. Moreover, during training, operators of queries
are also deployed on different hardware platforms. While this would seem like
a huge effort, it is a one-time training effort in contrast to state-of-the-
art learned approaches that need to train a model per streaming workload (data
and query). The main idea to enable generalization across streaming workloads
and hardware platforms is our model architecture that relies on the
aforementioned transferable feature representation of workloads and hardware
(cf. Section 3).

#### Using a Zero-Shot Cost Model

Once a zero-shot model is trained, it can be used at runtime to predict cost
metrics for an unseen query across different hardware platforms. In
particular, our model can infer costs accurately for an unseen query with an
entirely different data distribution of input data stream and extrapolate for
unseen operator properties, e.g., window size.

Consequently, we envision zero-shot cost models as a foundation for complex
optimization tasks like the operator placement problem. For instance, cost
predictions of operator placement on different hardware platforms can be used
to find a hardware resource with an objective to satisfy a certain latency
constraint or demands on throughput. Moreover, such cost predictions can also
serve as a basis for other optimization tasks, such as to determine the right
parallelization degree or the number of resources for deployment. While the
combination of zero-shot models with these optimization tasks is clearly an
interesting direction, in this paper, we focus only on how to enable the cost
estimation using zero-shot models.

##  3\. Zero-Shot Cost Models for DSPS

In Section 3.1, we first describe the cost metrics and the proposed model
architecture. Afterwards, we discuss our transferable feature representation
in Section 3.2 and conclude with the training and inference procedure in
Section 3.3.

###  3.1. QoS Metrics and Model Architecture

Our approach draws inspiration from the zero-shot cost models for databases
(hilprecht2022, ) that aims to predict query runtimes across unseen relational
databases. However, (hilprecht2022, ; hilprecht2022cidr, ) is not easily
extensible for DSPS because the characteristics of databases and their queries
differ largely from that of a DSPS, as discussed in the following.

First, different from databases, a query is continuous in a DSPS and is
composed of a logical set of streaming operators ($\Omega$), i.e., operators
that operate on unbounded data streams ($D$) instead of tables. Streaming
queries are composed of multiple operators in a so-called operator graph
($G$). In this graph, each vertex represents an operator ($\omega\in\Omega$),
and the edge between them represents the data stream $D$. Hence, the operator
graph describes the logical flow of data streams from one or multiple
_sources_ (data producer) to the _sink_ (data consumer). The data stream $D$
that flows through the operator graph represents an unbounded set of tuples.

Other differences from databases are: _(i)_ The operator graph in a DSPS
contains very different operators, e.g., window operator is typically used to
bound the data stream. _(ii)_ For DSPS, different QoS metrics need to be
predicted by a cost model for continuous queries that varies over the time.
For this paper, we consider _latency_ and _throughput_ for streaming queries
in DSPSs that need to be predicted by our learned cost model. In the
following, we now discuss the definitions for those metrics, which we use in
this paper since several different definitions have been used in the
literature (karimov2018, ; li2018, ; carbone2015, ). Afterwards, we then
explain the details of our model architecture.

#### QoS Metrics

While throughput is well-defined for DSPS, there is no unique definition of
latency (nardelli2019, ). The reason is that there are several stages of an
event tuple (production, ingestion and processing) at which it is timestamped
and thus different classes of latency exist (carbone2015, ). In this paper, we
only consider the so-called _end-to-end latency_ that includes all stages of
an event.

###### Definition 3.1.

_End-to-end latency_ : For each output tuple $d_{O}$, the end-to-end latency
is the interval between the time at which the oldest input event tuple $d_{I}$
involved in producing the output tuple $d_{O}$ is generated at the source and
the time that $d_{O}$ arrives at the sink. In the paper, we use the average
end-to-end latency of all the output tuples that arrive at the sink for a
given query.

###### Definition 3.2.

_Throughput_ : We define the second metric _throughput_ for our cost models
inline with the literature (karimov2018, ). For the execution of a given
query, throughput is the number of output tuples that arrive at the sink per
time unit.

Figure 2. Zero-shot cost estimation for DSPS. For predicting costs a streaming
query \raisebox{-.9pt} {1}⃝ is transformed into a graph structure
\raisebox{-.9pt} {2}⃝ which uses a novel transferable representation
\raisebox{-.9pt} {3}⃝. For cost predictions \raisebox{-.9pt} {4}⃝, we use the
graph-based structure where features of every operator are encoded in a
separate multi-layer perceptron (MLP) and propagated along the graph using the
order of operators in the query. A final MLP predicts throughput and latency.

#### Model Architecture

For realizing zero-shot models for DSPS that can predict these QoS metrics, a
core question we ask is, _“what can be learned from a given query and data
stream that can be generalized for most of the streaming workloads?”_. We
answer this by using a new model architecture that aims to represent streaming
queries using a graph representation with so-called transferable features
(Section 3.2). The overall approach of the model architecture is explained by
using an example as shown in Figure 2. \raisebox{-.9pt} {1}⃝ Given a DSPS
query that computes whether an average temperature from a data stream
TemperatureStream exceeds a threshold, \raisebox{-.9pt} {2}⃝ we represent each
operator of the query as a _node_ in our graph structure. For instance,
source, windowed aggregation and filter operators are represented as nodes in
the model shown in the example. \raisebox{-.9pt} {3}⃝ For each graph node, we
use transferable features that describe operator properties (e.g., filter
predicate), data characteristics (e.g., selectivity) and hardware properties
(e.g., instance size). \raisebox{-.9pt} {4}⃝ Once the graph representation is
constructed, the model can predict the cost (e.g., throughput and end-to-end
latency). The training and inference procedure is explained in detail in
Section 3.3. Overall, the generic representation based on graphs allows us to
make predictions for different query structures.

###  3.2. Transferable Feature Representation

Query execution costs like latency and throughput depend heavily on the
operator parameters, the data characteristics of the streams and the operator
placement. To describe these aspects and make them usable for learning, we
propose transferable features. For this, we derive nodes from the operator
graph $G$ (excluding the sink) and collect transferable operator-related,
data-related, and hardware-related input features for each of them. By
transferable, we mean that these features can be applied for any arbitrary
streaming workload (data and query). This representation targets commonly used
operators for DSPS (e.g. filter, join, aggregation, window). A listing of all
nodes with the features we use for our cost model is presented in Table 1 and
explained as follows.

Node |  Category |  Feature | Description
---|---|---|---
all |  hardware |  instance size | Properties of the hardware
data |  tuple width in | Averaged incoming tuple width
data |  tuple width out | Outgoing tuple width
source |  data |  input event rate | Event rate emitted by the source
data |  tuple data type | Data type for each value in tuple
filter |  operator |  filter function | Comparison function
operator |  literal data type | Data type of comparison literal
data |  selectivity |  see Definition 3.3
join |  operator |  join-key data type | Data type of the join key
data |  selectivity |  see Definition 3.4
agg. |  operator |  agg. function | Aggregation function
operator |  group-by data type | Data type of group-by attribute
operator |  agg. data type | Data type of each value to aggregate
|  data |  selectivity |  see Definition 3.5
window |  operator |  window type | Shifting strategy (sliding/tumbling)
operator |  window policy | Counting mode (count/time-based)
operator |  window size | Size of the window
operator |  slide size | Size of the sliding interval
Table 1. Transferable features categorized as operator-, data- and hardware-
related. The zero-shot model can learn from them and can be applied to
different streaming workloads (data and queries).

#### Operator-related features

The main idea behind operator-related features is to only include properties
as features that describe the operator logic and are transferable for _any_
query and DSPS. So, instead of using non-transferable features of operators,
such as _literals_ of a filter operator (e.g. tuple temperature = 5), which
might have a very different meaning for different data streams, we use generic
features such as data type of literal and the function of a filter predicate
(e.g., an equality predicate or a range predicate).

#### Data-related features

A major aspect of zero-shot models is that it is agnostic of the underlying
data distribution of the streaming workload. Thus, instead of using features
like _attribute names_ to encode the semantics of the data, we learn from data
characteristics (DCs) such as tuple width and event rate that can be
transferred to any workload. While DCs like tuple width are easily derived,
other DCs, such as selectivity of operators, are harder to derive or are even
not well-defined for operators like windowed joins (cugola2012, ). In the
following, we thus first define the selectivity for the streaming operators
shown in Table 1:

###### Definition 3.3.

_Filter selectivity_ : The selectivity $sel({\omega_{\sigma}})$ of a filter
operator $\omega_{\sigma}$ is the ratio of the number of outgoing to incoming
tuples in the input stream $D$, formally as:

(1)
$sel({\omega_{\sigma}})=\frac{|f_{\omega_{\sigma}}(D)|}{|D|},\quad\text{with
}0\leq sel({\omega_{\sigma}})\leq 1.$

###### Definition 3.4.

_Join selectivity_ : The selectivity $sel(\omega_{\Join})$ of a _windowed
join_ operator that considers tuples from windows $W_{d_{1}}$ and $W_{d_{2}}$
over two input streams $d_{1}$ and $d_{2}$ is the ratio of qualifying join
partners to the cartesian product for all tuples in the input windows:

(2) $sel(\omega_{\Join})=\frac{|W_{d_{1}}\Join
W_{d_{2}}|}{|W_{d_{1}}|\times|W_{d_{2}}|},\quad\text{with }0\leq
sel(\omega_{\Join})\leq 1.$

###### Definition 3.5.

_Aggregation selectivity_ : The selectivity $sel(\omega_{\xi})$ of a _windowed
aggregation_ operator that considers tuples in a window $W$ from an input
stream $D$, is the ratio of distinct group-by values in the window over the
window length:

(3) $sel(\omega_{\xi})=\frac{|\text{\emph{group-by
}}(W_{D})|}{|W_{D}|},\quad\text{with }0\leq sel(\omega_{\xi})\leq 1.$

In general, selectivities, as well as other DCs (e.g., tuple width) can be
derived during the training phase since queries anyways are executed to
collect observed metrics (throughput and latency) as training data. However,
in the inference phase, this is obviously a challenge since for some
optimizations tasks (e.g., the initial placement of operators), a query has
not been executed and the DCs are unknown. Yet the cost model can clearly be
used at runtime to re-optimize a query in case it does not meet the desired
QoS metrics since then DCs are known from the execution before. We think that
this approach is justified for DSPS, as queries are anyways long-running. In
future, we aim to work on learned approaches such as (hilprecht2020deepdb, )
that can be used to predict DCs for DSPS before execution and thus allow our
model also to be used for optimizations before a query starts (e.g., an
initial placement decision).

#### Hardware-related features

Hardware characteristics have also a profound impact on the performance of the
query in a DSPS, and thereby also on tasks such as placement decisions. In our
featurization, we thus include common properties of hardware such as CPU
cores, RAM and disk size. These hardware properties are clustered as instance
size (small, medium and large), similar to the categorization of major cloud
providers. These features are also encoded in the graph node to describe the
placement characteristic of the operator. Clearly, more properties such as
network bandwidth can be added in future, which is out of scope for this
paper.

###  3.3. Training and Inference Procedure

We train the zero-shot models in a supervised way using Graph Neural Networks
(GNNs) to learn from the transferable features. All categorical transferable
features are encoded using a so-called one-hot encoding per operator, while
numerical features are normalized.

During training, the encoded transferable features are used as input to the
nodes of the GNN. Particularly, the features of each node are encoded by a
corresponding node-specific multi-later perceptron (MLP) into a fixed vector
(i.e., the hidden state). These hidden states of nodes are then combined along
the graph, using a bottom-up message passing phase by adding up the hidden
vectors from the children nodes and combining them with the parent node of the
graph. Lastly, the hidden state of the parent node is fed into a final MLP
that predicts cost metrics at inference time. In the training phase, instead,
the predictions are compared to the real costs and the MLPs are updated
accordingly.

Feature | Training data range
---|---
instance size | small, medium, large
input event rate |  [0.25, 0.75, 0.5, 1, 1.5, 2.5] $\times 10^{3}$ e/s
tuple data type |  [1…5] $\times$ [int, string, double]
filter function |  <,>,<=, >=, !=, startswith, endswith
literal data type | int, string, double
window type | sliding, tumbling
window policy | count-based, time-based
window size | [0.25, 0.5, 1, 2, 3] sec; [3, 5, 10, 25, 50, 75, 100] tuples
slide size |  [0.3 … 0.7] $\times$ window length
join-key data type | int, string, double
agg. function | min, max, mean, avg
group-by data type | int, string, double, none
Table 2. Operator-related features for training data generation using
synthetic data.

##  4\. Experimental Evaluation

In this section, we discuss the results of our experimental evaluation. We
first explain the generation of data that is used for training and the
evaluation setup. Afterwards, we demonstrate the accuracy and robustness of
our cost model by carrying out various experiments.

###  4.1. Data Generation and Setup

#### Training Data Generation:

We created a synthetic training data set with randomly generated queries by
enumerating over (i) operator parameters, (ii) different data streams and
(iii) distinct query structures. Additionally, we evaluated our model against
the existing streaming benchmarks (bordin2020, ). The different operator
parameters and streaming workload configurations (e.g. event rate, data types)
are described in Table 2. A variety of queries are evaluated by the use of
three query structures as depicted in Figure 3. They are composed of widely
used streaming operators that we instantiated with different parameters. As a
result, a high number of possible combinations of operator properties,
streaming workloads and query structures are covered in the training set.

Figure 3. Basic query structures used for training data generation. _Filter_
operators and _group-by_ are optional.

VM size |  CPU |  RAM (GB) |  Disk (GB)
---|---|---|---
small |  1 |  1 |  10
medium |  2 |  2 |  20
large |  8 |  8 |  80

Table 3. Hardware-related parameters of virtual instances used for training.

#### Setup and Metrics

To obtain sufficient training data, 15.000 training queries (5000 per query
structure) were generated and executed for 1.5 min over Storm clusters with
ten virtual machines of different sizes (small, medium, and large — S, M, L).
We used the Stream-API of Apache Storm and the standard round-robin scheduler
for placement. The worker nodes have properties as specified in Table 3, which
is used as instance size in the zero-shot model training. For queries that did
not receive an output tuple at the sink during their runtime due to
unfavorable random operator properties, no costs could be determined.
Consequently, 11.175 queries remained that were divided into training (80%),
validation (10%) and test (10%) sets. The model was trained with the training
data as explained in Section 3.3. We report the Q-error metric $q(c,\hat{c})$
to describe the relative deviation of the true cost metric value $c$ (latency
or throughput) and its prediction $\hat{c}$, which is a standard metric for
the quality of cost models, defined as:
$q(c,\hat{c})=max\left(\frac{c}{\hat{c}},\frac{\hat{c}}{c}\right)$, with
$q\geq 1$ (hilprecht2022, ).

###  4.2. Experimental Results

In the following evaluation, we investigate the accuracy of the zero-shot cost
models on different workloads. In the experiments, we mainly focus on the
generalization ability to predict _(i)_ for an unseen test set but using
parameters inside the defined training range, _(ii)_ for workloads with
parameters outside the defined training range, _(iii)_ for unseen workloads
from existing streaming benchmarks and _(iv)_ for workloads under varying
hardware heterogeneity.

#### Q1: How accurate is the model for an unseen combination of operator and
workload properties (inside the training range)?

At first, we provide the observed Q-errors in predictions for the test set.
These are queries and data streams not considered during the training, but use
values from the same ranges as shown in Table 2. It can be seen in in Table 4
that the median and the $95$th percentile of the Q-error of our model is
highly accurate for the given query structures. Typically, cost models based
on heuristics have much higher Q-errors of up to $300$ for the $95$th
percentile or more (hilprecht2020deepdb, ).

Query structure | Latency | Throughput
---|---|---
| median | 95th | median | 95th
Linear query | 1.09 | 2.14 | 1.15 | 2.68
2-way-join | 1.13 | 2.69 | 1.14 | 2.91
3-way-join | 1.21 | 5.25 | 1.22 | 5.59
Overall | 1.13 | 3.19 | 1.16 | 3.50
Table 4. Q-errors (median and 95th percentile) for unseen combinations of
operator and workload properties inside the training range.

#### Q2: How accurate is the model for completely unseen workloads (outside
the training range)?

To answer this, we generated queries that are within the known parameter space
except for one property that exceeds or falls below the training value range.
For each property, we generated 50 queries (for all three query structures).
The median Q-error from all estimations is reported in Figure 4. To be more
precise, we tested tuple widths larger than the training set (A) as well as
lower and higher event rates (B). To investigate extrapolation for unseen
operator properties, we applied bigger time-based (C) and count-based (D)
window sizes as well. In almost all cases, a fairly low median Q-error could
be achieved. Intuitively, the Q-error increases with the distance to the
training value range as more extreme properties will lead to more extreme
costs, which are hard to predict precisely; still way better than existing
heuristics (hilprecht2020deepdb, ). In a second set of experiments, we
observed the accuracy for unseen query structures; i.e., we use 4-way and
5-way-joins instead of only up to 3-way joins that were used for training.
Moreover, for linear queries, we added 2 to 4 additional filter operators
after the windowed aggregate operator (cf. Table 5). While the median Q-errors
for those are also promising, the tail accuracy of the unseen query structures
increases with the increasing query complexity. We aim to tackle this in a
follow-up work by incorporating extreme unseen properties in an additional
fine-tuning phase (called _few-shot learning_), where only a few unseen
queries are used for retraining.

Figure 4. Median Q-errors for queries with unseen tuple widths (A), event rates (B), time-based and count-based window lengths (C, D). The gray shaded area marks the training range. The model extrapolates for unseen queries and workloads (white shaded areas), often with very high accuracy. Query structure | Latency | Throughput
---|---|---
| median | 95th | median | 95th
2-filter-chain | 1.14 | 2.41 | 1.59 | 3.65
3-filter-chain | 2.67 | 46.34 | 2.82 | 27.78
4-filter-chain | 7.33 | 54.68 | 3.94 | 59.73
4-way-join | 1.95 | 24.30 | 1.33 | 20.79
5-way-join | 1.91 | 26.76 | 1.35 | 21.87
Table 5. Q-errors for unseen query structures. Although the model has never
seen these query structures during training, it is able to make accurate cost
predictions for them.

#### Q3: How accurate is the model for existing benchmark workloads?

To evaluate the generalization capability of the model, we applied it on a
selection of three different existing benchmark workloads (bordin2020, ).
_(i)_ The _Advertisement Analysis_ is a benchmark that computes the ratio of
aggregates of two incoming streams (i.e. clicks, impressions), grouped by two
attributes. In our evaluation, we used two sub-queries with a windowed
aggregation for both of these streams and a third sub-query that joins both
streams. _(ii)_ The _Spike Detection_ benchmark is derived from an IoT use
case that computes and compares the moving average of sensor values. _(iii)_
The _Smart Grid_ benchmark targets the computation of energy consumption in
households from the DEBS Grand Challenge 2014. Its _outlier detection_ task is
decomposed into three sub-queries as presented in (koliousis2016, ). We
consider two sub-queries that contain a large sliding window of 60 min (and
60s sliding length) and build average over the local and the global energy
consumption. As our model was trained with window lengths up to 3 secs, we
reduce the benchmark window length to 5 min (and 5s sliding length), which is
still is $100\times$ higher than our training range. As no event rates were
specified in the benchmark, we execute the queries five times with different
rates each and report the results in Table 6. This shows that the zero-shot
model can adapt to streaming workloads of existing benchmark out-of-the-box
and estimates the costs very accurately.

Benchmark | Latency | Throughput
---|---|---
| median | 95th | median | 95th
Advertisement (clicks) | 1.51 | 1.53 | 1.38 | 1.39
Advertisement (impressions) | 1.51 | 1.52 | 1.38 | 1.39
Advertisement (join) | 1.99 | 2.06 | 1.55 | 2.16
Spike Detection | 1.01 | 1.04 | 1.73 | 1.94
Smart Grid (local) | 1.21 | 1.23 | 1.92 | 1.92
Smart Grid (global) | 1.20 | 1.66 | 1.91 | 1.91
Table 6. Q-errors for benchmarks from (bordin2020, ). Each query has been
executed 5 times with different event rates. This shows our model is able to
accurately predict out-of-the-box for these queries.

#### Q4: Does the model predict appropriately for different hardware types?

As streaming operators are placed on distinct instances (cf. Table 3), we
expect the model to take the underlying node performance into account during
the prediction. To investigate that, we take all queries from the test set and
obtain the predictions under the assumption that all operators are either
located on a small, medium or large instance and report the mean of the
predictions. Given the linear query, for instance, the model predicts for
larger instances higher throughput (31.17/36.14/39.98 ev/sec for S/M/L,
resp.). Hence, the zero-shot model correctly learns the effect of instance
size on the query performance. In further studies, we aim to model and
evaluate hardware characteristics more fine-grained.

##  5\. Related Work

We classify the existing work on cost estimation into two main categories:
_(i)_ analytical approaches, heuristics and _(ii)_ learning-based approaches
as stated below.

#### Analytical Approaches and Heuristics

Several analytical approaches have been applied to predict resource costs in
DSPS, such as using queuing theory (mencagli2018, ; dematteis2017, ) for
different DSPS optimization tasks. However, analytical approaches often make
assumptions about the data distribution of incoming streaming workload that
renders the estimates either inaccurate or ill-suited for DSPS due to the
inherently dynamic nature of the streaming workload. Many works propose
heuristics to optimize tasks like placement, namely, greedy approaches
(aniello2013, ), meta heuristics (chandramouli2011, ) and custom heuristics
(eskandari2021, ). Storm (toshnival2014, ) also adopts heuristics for
placement (nardelli2019, ). However, heuristics often also make simplifying
assumptions leading to sub-optimal decisions for the optimization tasks. Flink
DSPS (carbone2015, ) uses a query optimizer from Stratosphere (alexandrov2014,
), however, it explicitly targets only user-defined operators in the cost
modelling.

#### Learned Stream Processing

Approaches related to resource cost estimation are the closest to this work.
Regression models have been used extensively for estimating performance
metrics such as latency (li2018, ) and throughput (imai2017, ; wang2017, ) in
DSPS. However, these works only consider a limited set of features and largely
ignores featurization related to queries that have a major impact on the
performance. Different learning methods have been applied to optimize DSPS
tasks (ahmad2021, ), namely for elasticity (russo2019, ) or operator placement
(li2018, ; luthra2021, ). Similar to this work, (lombardi2018, ) employs
neural networks to model the performance of DSPS; however, they stumble around
the challenge of re-training for a large number of heterogeneous queries and
workloads. In contrast, we introduce a data and query agnostic approach that
can support a variety of stream processing optimization tasks that are
dependent on the cost estimation of resources _without_ any cost of additional
training.

##  6\. Conclusion

In this paper, we show that zero-shot cost models are highly effective in
predicting costs even in the presence of unseen workload conditions. The model
can robustly predict QoS metrics for a wide range of queries out of a large
set of possible configurations. As such, we believe that the cost estimations
are readily usable for optimization tasks of DSPS, such as the operator
placement. In the future, an interesting direction is to solve optimization
tasks like placement _out-of-the-box_ and more explicitly modelling hardware
and network properties in the proposed graph structure.

Acknowledgments. This work has been supported by the IPF program and the Cloud
Computing Competence Centre of DHBW Mannheim, the Collaborative Research
Center (CRC) 1053 MAKI funded by the German Research Foundation (DFG), the
NHR4CES Program, hessian.AI at TU Darmstadt as well as DFKI Darmstadt.

## References

  * (1) Z. Shao, “Real-time analytics at facebook,” _XLDB_ , 2011.
  * (2) M. Nardelli, V. Cardellini, V. Grassi, and F. L. Presti, “Efficient operator placement for distributed data stream processing applications,” _IEEE TPDS_ , vol. 30, no. 8, pp. 1753–1767, 2019.
  * (3) G. R. Russo, V. Cardellini, and F. L. Presti, “Reinforcement learning based policies for elastic stream processing on heterogeneous resources,” in _ACM DEBS_ , 2019, p. 31–42.
  * (4) M. Hirzel, R. Soulé, S. Schneider, B. Gedik, and R. Grimm, “A catalog of stream processing optimizations,” _ACM Computing Surveys_ , vol. 46, no. 4, 2014.
  * (5) L. Eskandari, J. Mair, Z. Huang, and D. Eyers, “I-scheduler: Iterative scheduling for distributed stream processing systems,” _Future Generation Computing Systems_ , vol. 117, pp. 219–233, 2021.
  * (6) G. Mencagli, P. Dazzi, and N. Tonci, “Spinstreams: A static optimization tool for data stream processing applications,” in _ACM Middleware_ , 2018, p. 66–79.
  * (7) M. Luthra, B. Koldehofe, N. Danger, P. Weisenberger, G. Salvaneschi, and I. Stavrakakis, “Tcep: Transitions in operator placement to adapt to dynamic network environments,” _Journal of Computer and System Sciences_ , vol. 122, pp. 94–125, 2021.
  * (8) T. Li, Z. Xu, J. Tang, and Y. Wang, “Model-free control for distributed stream data processing using deep reinforcement learning,” _PVLDB_ , vol. 11, no. 6, p. 705–718, 2018.
  * (9) C. Wang, X. Meng, Q. Guo, Z. Weng, and C. Yang, “Automating characterization deployment in distributed data stream management systems,” _IEEE TKDE_ , vol. 29, no. 12, pp. 2669–2681, 2017.
  * (10) A. Alnafessah, G. Russo Russo, V. Cardellini, G. Casale, and F. Lo Presti, _AI‐Driven Performance Management in Data‐Intensive Applications_ , 2021, pp. 199–222.
  * (11) G. Li, X. Zhou, and L. Cao, “Machine learning for databases,” ser. AIMLSystems, 2021.
  * (12) B. Hilprecht and C. Binnig, “Zero-shot cost models for out-of-the-box learned cost prediction,” 2022, arXiv. [Online]. Available: https://arxiv.org/abs/2201.00561
  * (13) B. Hilprecht and C. Binnig, “One Model to Rule them All: Towards Zero-Shot Learning for Databases,” in _CIDR_ , 2022.
  * (14) J. Karimov, T. Rabl, A. Katsifodimos, R. Samarev, H. Heiskanen, and V. Markl, “Benchmarking distributed stream data processing systems,” in _ICDE_ , 2018, pp. 1507–1518.
  * (15) P. Carbone, A. Katsifodimos, S. Ewen, V. Markl, S. Haridi, and K. Tzoumas, “Apache flink™: Stream and batch processing in a single engine,” _IEEE Data Eng. Bull._ , vol. 38, no. 4, pp. 28–38, 2015.
  * (16) G. Cugola and A. Margara, “Processing flows of information: From data stream to complex event processing,” _ACM Computing Surveys_ , vol. 44, no. 3, 2012\.
  * (17) B. Hilprecht, A. Schmidt, M. Kulessa, A. Molina, K. Kersting, and C. Binnig, “Deepdb: Learn from data, not from queries!” _PVLDB_ , vol. 13, no. 7, pp. 992–1005, 2020.
  * (18) M. V. Bordin, D. Griebler, G. Mencagli, C. F. R. Geyer, and L. G. L. Fernandes, “Dspbench: A suite of benchmark applications for distributed data stream processing systems,” _IEEE Access_ , vol. 8, pp. 222 900–222 917, 2020\.
  * (19) A. Koliousis, M. Weidlich, R. Castro Fernandez, A. L. Wolf, P. Costa, and P. Pietzuch, “Saber: Window-based hybrid stream processing for heterogeneous architectures,” in _ACM SIGMOD_ , 2016, p. 555–569.
  * (20) T. De Matteis and G. Mencagli, “Elastic scaling for distributed latency-sensitive data stream operators,” in _PDP_ , 2017, pp. 61–68.
  * (21) L. Aniello, R. Baldoni, and L. Querzoni, “Adaptive online scheduling in storm,” in _ACM DEBS_ , 2013, p. 207–218.
  * (22) B. Chandramouli, J. Goldstein, R. Barga, M. Riedewald, and I. Santos, “Accurate latency estimation in a distributed event processing system,” in _ICDE_ , 2011, p. 255–266.
  * (23) A. Toshniwal, S. Taneja, A. Shukla, K. Ramasamy, J. M. Patel, S. Kulkarni, J. Jackson, K. Gade, M. Fu, J. Donham, N. Bhagat, S. Mittal, and D. Ryaboy, “Storm@twitter,” in _ACM SIGMOD_ , 2014, p. 147–156.
  * (24) A. Alexandrov, R. Bergmann, S. Ewen, J.-C. Freytag, F. Hueske, A. Heise, O. Kao, M. Leich, U. Leser, V. Markl, F. Naumann, M. Peters, A. Rheinländer, M. J. Sax, S. Schelter, M. Höger, K. Tzoumas, and D. Warneke, “The stratosphere platform for big data analytics,” _VLDBJ_ , vol. 23, no. 6, p. 939–964, 2014.
  * (25) S. Imai, S. Patterson, and C. A. Varela, “Maximum sustainable throughput prediction for data stream processing over public clouds,” in _IEEE/ACM CCGRID_ , 2017, pp. 504–513.
  * (26) F. Lombardi, L. Aniello, S. Bonomi, and L. Querzoni, “Elastic symbiotic scaling of operators and resources in stream processing systems,” _IEEE TPDS_ , vol. 29, no. 3, pp. 572–585, 2018.