# Energy-Efficient and Federated Meta-Learning via Projected Stochastic
Gradient Ascent

Anis Elgabli, Chaouki Ben Issaid, Amrit S. Bedi†, Mehdi Bennis, Vaneet
Aggarwal∗ Centre for Wireless Communications (CWC) University of Oulu, Finland
†US Army Research Laboratory, MD, USA
∗School of Electrical and Computer Engineering, Purdue University
Emails: {anis.elgabli, chaouki.benissaid<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

In this paper, we propose an energy-efficient federated meta-learning
framework. The objective is to enable learning a meta-model that can be fine-
tuned to a new task with a few number of samples in a distributed setting and
at low computation and communication energy consumption. We assume that each
task is owned by a separate agent, so a limited number of tasks is used to
train a meta-model. Assuming each task was trained offline on the agent’s
local data, we propose a lightweight algorithm that starts from the local
models of all agents, and in a backward manner using projected stochastic
gradient ascent (P-SGA) finds a meta-model. The proposed method avoids complex
computations such as computing hessian, double looping, and matrix inversion,
while achieving high performance at significantly less energy consumption
compared to the state-of-the-art methods such as MAML and iMAML on conducted
experiments for sinusoid regression and image classification tasks.

###### Index Terms:

Meta-learning, federated learning, energy-efficient distributed machine
learning, stochastic gradient descent

##  I Introduction

In many artificial intelligence problems, the aim is to develop systems that
use previous experience to acquire new skills more effectively. To fulfill
this objective, meta-learning has been proposed as a framework that enables an
agent to quickly learn a new task by leveraging an optimized initializer that
is trained using similar tasks [1, 2, 3]. One way to formulate the meta-
learning problem is to cast it as a bi-level optimization problem [4, 5, 6],
involving two loops: (i) an inner loop optimization that represents adaptation
to a given task, and (ii) an outer loop optimization with the objective to
train a meta-model. In [5], the authors proposed the model-agnostic meta-
learning (MAML) formulation, which is shown to perform well in terms of
convergence and fast adaptation to new task. However, despite its convergence
and fast adaptation properties, solving MAML formulation requires
backpropagation through the inner optimization process. Consequently, it
requires higher-order derivative (Hessian’s computation) which imposes high
computational cost and vanishing gradients. These issues limit the scalability
of MAML especially when the local models are multiple SGD steps from the
initializer. First-order MAML (FOMAML) [5] and Reptile [7] are two first-order
MAML-type algorithms which avoid second-order derivatives. Though both
algorithms reduce the computational cost, they both ignore the dependence of
task-models on the meta-models.

In [6], MAML with implicit gradients (iMAML) was proposed with the goal of
reducing the computational cost of MAML by leveraging implicit
differentiation. The authors derived an analytical expression for the meta-
gradient that depends only on the solution to the inner optimization and not
the path taken by the inner optimization algorithm. However, this expression
involves matrix inversion that has been approximated by solving a problem that
involves using conjugate gradient (CG) method. Though iMAML reduces the
computation cost compared to MAML, it still involves high computation cost due
to: (i) the two loop procedure, (ii) the hessian computation which could be
computationally expensive for large models, and (iii) the matrix inversion
procedure using CG. Due to these limitations, running iMAML in a distributed
environment that involves local computation and communication using energy-
limited IoT devices is not feasible. Moreover, iMAML assumes a large number of
labeled tasks since at every iteration, it samples a batch of these tasks,
which may not be practical in cases where there are only few tasks owned by
different agents. Therefore, in this paper, we propose a distributed and
energy-efficient meta-learning approach that overcomes these limitations and
ensures efficient learning.

Standard federated learning solves the problem of $N$ workers trying to learn
a shared global model with the aid of a parameter server (PS). Each worker is
exposed to limited data samples and is not willing to share its data due to
privacy and/or communication constraints [8, 9, 10]. The PS starts with an
initial guess of the global model, shares it with all contributing workers
where each worker performs a local computation step before sharing its output.
The local computation step could be (i) calculating the gradient of the local
function with respect to the global model using a randomly selected mini-batch
[11] or (ii) performing a number of local stochastic gradient descent (SGD)
steps using the global model as initializer [12]. Then the PS aggregates the
outputs of all workers and performs one global step which is one SGD step
using the aggregated gradients of all workers for (i), or model averaging for
(ii). Under some assumptions, this leads to convergence to a global model
shared across workers.

Similarly, in federated (distributed) meta-learning, every task is owned by an
agent and each iteration involves communication with the PS. At each
iteration, every agent updates its local model and uploads it to the PS where
the meta-model is updated in a global step and pushed to all agents. Since
this involves local computation and communication energy consumption, it is
crucial to minimize the computation cost and save energy for communication.
Motivated by this, we propose a novel formulation that can be solved very
efficiently without involving double loop and Hessian computation.
Particularly, at every iteration, we show that our inner problem can be solved
in a simple and closed form expression that involves one SGD step and
projection. Experiments using sinusoid regression and image classification
tasks show that our proposed approach significantly outperforms iMAML in terms
of energy consumption.

The rest of the paper is organized as follows. In section II, we briefly
revisit MAML and iMAML and explain their key steps at high level. In section
III, we describe our problem formulation in details, then in section IV we
introduce our proposed approach. In section V, we evaluate the performance of
our proposed method and compare it iMAML on sinusoid regression and image
classification tasks. Finally, we conclude the paper in section VI.

Notations: Throughout this paper, $\|\cdot\|$ denote the Euclidean norm of a
vector. The notations $\nabla f$, $\nabla^{2}f$ stand for the gradient and the
Hessian of the function $f$, respectively.

Figure 1: Schematic illustration of our proposed algorithm (Meta-Backward)
compared to MAML and iMAML.

##  II Review of MAML and IMAML

In this section, we will describe the concept and briefly highlight the key
steps of MAML and iMAML.

###  II-A MAML

Let $\\{\mathcal{T}_{i}\\}_{i=1}^{N}$ be a set of meta-training tasks assumed
to be drawn from $P(\mathcal{T})$. For each task $\mathcal{T}_{i}$, we can
sample two disjoint datasets: a training dataset
$\mathcal{D}^{\mathrm{tr}}_{i}=\\{(\mathbf{x}_{i}^{k},\mathbf{y}_{i}^{k})\\}_{k=1}^{K}$
and a test dataset
$\mathcal{D}^{\mathrm{test}}_{i}=\\{(\mathbf{x}_{i}^{j},\mathbf{y}_{i}^{j})\\}_{j=1}^{J}$,
where $\mathbf{x}\in\mathcal{X}$ and $\mathbf{y}\in\mathcal{Y}$ denote the
inputs and outputs, respectively.

Introducing the loss function $\mathcal{L}(\bm{\phi},\mathcal{D})$ where
$\bm{\phi}\in\mathbb{R}^{d}$, each task $\mathcal{T}_{i}$ aims to learn task-
specific parameters $\bm{\phi}_{i}$ using $\mathcal{D}^{\mathrm{tr}}_{i}$ by
minimizing its test loss,
$\mathcal{L}({\bm{\phi}}_{i},\mathcal{D}^{\mathrm{test}}_{i})$. MAML seeks to
learn $\\{\bm{\phi}_{i}\\}_{i=1}^{N}$ using a set of meta-parameters
$\bm{\theta}\in\mathbb{R}^{d}$ and the training datasets from each task. The
objective is to learn the meta-parameter that achieve good task specific
parameters after adaptation. The meta-learning problem can be cast as the
following bi-level optimization problem

$\displaystyle\min_{{\bm{\theta}},\\{\bm{\phi}_{i}\\}_{i=1}^{N}}\frac{1}{N}\sum_{i=1}^{N}\
\mathcal{L}\left(\bm{\phi}_{i}({\bm{\theta}}),\
\mathcal{D}^{\mathrm{test}}_{i}\right).$ (1)

Giving a new task $\mathcal{T}_{i}\sim P(\mathcal{T})$, MAML can achieve a
good generalization performance by using the adaptation procedure with the
learned parameters ${\bm{\phi}}_{i}$ using ${\bm{\theta}}$ and
$\mathcal{D}^{\mathrm{tr}}_{i}$. More specifically, in the inner level of
MAML, ${\bm{\phi}}_{i}$ can be updated using gradient descent initialized at
${\bm{\theta}}$.

$\displaystyle{\bm{\phi}}_{i}={\bm{\theta}}-\alpha\nabla_{\bm{\theta}}\mathcal{L}({\bm{\theta}},\mathcal{D}^{\mathrm{tr}}_{i}),$
(2)

where $\alpha$ is the learning rate. Note that MAML also allows updating
${\bm{\phi}}_{i}$ by running multiple gradient descent steps starting from
${\bm{\theta}}$. However, a major concern with MAML is that updating the meta-
model according to

$\displaystyle{\bm{\theta}}\leftarrow{\bm{\theta}}-\eta\
\frac{1}{N}\sum_{i=1}^{N}\frac{d\bm{\phi}_{i}({\bm{\theta}})}{d{\bm{\theta}}}\
\nabla_{\bm{\phi}}\mathcal{L}(\bm{\phi}_{i}({\bm{\theta}}),\mathcal{D}^{\mathrm{test}}_{i}),$
(3)

when $\bm{\phi}_{i}({\bm{\theta}})$ is $K$ SGD steps from ${\bm{\theta}}$
requires backpropagating through the dynamics of the $K$ steps, which leads to
a significant computation and memory burden.

###  II-B MAML with Implicit Gradients (iMAML)

Due to requirements on memory and higher-order derivatives, MAML with Implicit
Gradients (iMAML) was proposed in [6] to address the challenge of
differentiating through the long optimization path in MAML. In addition to
that, iMAML proposes to incorporate regularization into the problem
formulation to enable appropriate leaning in the inner level, while preventing
over-fitting. In this case, the inner-level problem is formulated as

$\displaystyle\underset{{\bm{\phi}}}{\min}~{}\mathcal{L}({\bm{\phi}},\mathcal{D}^{\mathrm{tr}}_{i})+\frac{\lambda}{2}~{}||{\bm{\phi}}-{\bm{\theta}}||^{2},$
(4)

where $\lambda$ is a scalar hyperparameter that controls the regularization
strength. The introduced regularization term in (4) enforces ${\bm{\phi}}_{i}$
to remain close to ${\bm{\theta}}$. Ideally, the problem in (4) is solved
using iterative algorithms and the optimal $\bm{\phi}$ is determined. In this
case, the term $d\bm{\phi}_{i}({\bm{\theta}})/d{\bm{\theta}}$ can be
explicitly derived as

$\displaystyle\frac{d\bm{\phi}_{i}({\bm{\theta}})}{d{\bm{\theta}}}=\left(\bm{I}+\frac{1}{\lambda}~{}\nabla_{\bm{\phi}}^{2}\mathcal{L}({\bm{\phi}}_{i},\mathcal{D}^{\mathrm{tr}}_{i})\right)^{-1}.$
(5)

According to iMAML, the problem in (4) is solved up to $\delta$-optimal
solution, and the matrix inversion in (5) is determined by approximating a
solution of an equivalent problem using a few conjugate gradient (CG) steps.

##  III Problem Formulation

In the standard MAML problem defined over $N$ different tasks, the goal is to
find an initial point $\theta^{\star}$ such that when the local task $i$ runs
its local $K$ SGD steps, it provides a better personalization for it. Hence,
mathematically, we can write the problem of MAML as

$\displaystyle\min_{\bm{\theta},\\{\bm{\phi}_{i}\\}_{i=1}^{N}}\frac{1}{N}\sum_{i=1}^{N}$
$\displaystyle\
\mathcal{L}(\bm{\phi}_{i}^{K}(\bm{\theta}),\mathcal{D}_{i}^{test})$ (6)
$\displaystyle\text{s.t.}\ \ \bm{\phi}_{i}^{K}(\bm{\theta})=$ $\displaystyle
SGD_{K}^{\alpha}\left(\bm{\theta},\mathcal{L}(\bm{\phi}_{i}(\bm{\theta}),\mathcal{D}_{i}^{tr})\right),\forall
i$ (7)

where $SGD$ denotes that stochastic gradient descent is used for each task
with $\bm{\theta}$ as the starting point, i.e.,

$\displaystyle\bm{\phi}_{i}^{K}(\bm{\theta})=\bm{\phi}_{i}^{0}-\alpha\sum_{k=0}^{K-1}\nabla_{\bm{\phi}_{i}}\mathcal{L}(\bm{\phi}_{i},\mathcal{D}_{i}^{tr})\Big{|}_{\bm{\phi}_{i}=\bm{\phi}_{i}^{k}(\bm{\theta})},\forall
i,$ (8)

where $\bm{\phi}_{i}^{0}=\bm{\theta}$ and $\alpha$ is the learning rate. Now,
let us see the importance of the starting point $\bm{\theta}$ for each task in
more detail. The starting point actually controls the number of iterations
required to converge to the local optima $\bm{\phi}_{i}^{\star}(\bm{\theta})$.
If the local function is strongly convex, then the number of iterations
required for $\epsilon$-optimal solution is given by [13]

$\displaystyle
K_{i}\geq\mathcal{O}\left(\frac{\kappa}{\epsilon}\log\left(\frac{\mathcal{L}(\bm{\theta},\mathcal{D}_{i}^{tr})-\mathcal{L}(\bm{\phi}_{i}^{K_{i}}(\bm{\theta}),\mathcal{D}_{i}^{tr})}{\epsilon}\right)\right).$
(9)

Hence for each task $i$, the number of steps $(K_{i})$ required to achieve
$\epsilon$-optimal depends upon the term
$\mathcal{L}(\bm{\theta},\mathcal{D}_{i}^{tr})-\mathcal{L}(\bm{\phi}_{i}^{K_{i}}(\bm{\theta}),\mathcal{D}_{i}^{tr})$
controlled by the initialization $\bm{\theta}$ for each task $i$. The goal of
standard meta learning is to learn a common initializer $\bm{\theta}$ such
that for each task $i$, after running a number of SGD steps using few data
samples, we are able to obtain a good solution. So the placement of the
initializer $\bm{\theta}$ in the space of local solutions
$\bm{\phi}_{i}^{\star}(\bm{\theta})$ is the task.

We take a one step back and try to look into the problem in a backward manner.
For instance, if we are given the local $\epsilon$-optimal solutions
$\bm{\phi}_{i,\epsilon}^{\star}$ (note that they are independent of
$\bm{\theta}$) for each task $i$, then what should be the initializer point
$\bm{\theta}$ so that each $\bm{\phi}_{i,\epsilon}^{\star}$ is just $K_{i}$
SGD steps away from $\bm{\theta}$? Since the final number of steps to converge
to local optimal depends upon
$\mathcal{L}(\bm{\theta},\mathcal{D}_{i}^{tr})-\mathcal{L}(\bm{\phi}_{i,\epsilon}^{\star},\mathcal{D}_{i}^{tr})$,
the best initializer point can be obtained by solving the following
optimization problem

$\displaystyle\min_{\bm{\theta},\\{\bm{\phi}_{i}\\}_{i=1}^{N}}\ \ $
$\displaystyle\frac{1}{N}\sum_{i=1}^{N}\left(\mathcal{L}(\bm{\theta},D_{i}^{tr})-\mathcal{L}(\bm{\phi}_{i,\epsilon}^{\star},\mathcal{D}_{i}^{tr})\right)^{2}$
(10) s.t. $\displaystyle
SGD_{K_{i}}^{\alpha}\left(\bm{\theta},\mathcal{L}(\bm{\phi}_{i}(\bm{\theta}),\mathcal{D}_{i}^{tr})\right)=\bm{\phi}_{i,\epsilon}^{\star},\forall
i$ (11)

where $\bm{\phi}_{i,\epsilon}^{\star}$ is known. Solving (10)-(11) is as hard
as solving the original MAML problem. Motivated by the fact that
$\bm{\phi}_{i}^{\star}$ is known and computed offline for every agent, we
propose an approximate formulation for (10)-(11) that can be solved using the
proposed greedy backward algorithm described in Section IV, which is shown to
significantly outperform iMAML as illustrated in Section V. Our formulated
problem is given by

$\displaystyle\min_{\\{\bm{\phi}_{i}\\}_{i=1}^{N}}\ \ $
$\displaystyle\sum_{i=1}^{N}\sum_{k=0}^{K-1}\left(\mathcal{L}(\bm{\phi}_{i}^{k})-\mathcal{L}(\bm{\phi}_{i}^{k+1}+\alpha\nabla_{\bm{\phi}_{i}^{k+1}}\mathcal{L}(\bm{\phi}_{i},\mathcal{D}_{i}))\right)^{2}$
(12) s.t.
$\displaystyle\left\|\bm{\phi}_{i}^{k}-\frac{1}{N}\sum_{i=1}^{N}\bm{\phi}_{i}^{k+1}\right\|^{2}\leq\delta_{k},\forall
i,k=0,\cdots,K-1$ (13)
$\displaystyle\bm{\phi}_{i}^{K}=\bm{\phi}_{i,\epsilon}^{\star},\forall i$ (14)

Note that $\bm{\theta}=\frac{1}{N}\sum_{i=1}^{N}\bm{\phi}_{i}^{0}$, and
$\delta_{0}\leq\delta_{1}\leq\dots\leq\delta_{K-1}$. When
$\delta_{k}\rightarrow 0$, $\bm{\phi}_{i}^{0}\rightarrow\bm{\theta},\forall
i$.

Let’s now intuitively explain the relation between the formulation in
(10)-(11) and the one introduced in (12)-(13). Originally, starting from
$\bm{\phi}_{i}^{k+1}$, $\bm{\phi}_{i}^{k}$ is computed as

$\displaystyle\bm{\phi}_{i}^{k}=\bm{\phi}_{i}^{k+1}+\alpha\nabla_{\bm{\phi}_{i}^{k}}\mathcal{L}(\bm{\phi}_{i},\mathcal{D}_{i}).$
(15)

However, when moving in backward manner, the term
$\nabla_{\bm{\phi}_{i}^{k}}\mathcal{L}(\bm{\phi}_{i},\mathcal{D}_{i})$ cannot
be computed. To overcome this problem, we replace
$\nabla_{\bm{\phi}_{i}^{k}}\mathcal{L}(\bm{\phi}_{i},\mathcal{D}_{i})$ with
$\nabla_{\bm{\phi}_{i}^{k+1}}\mathcal{L}(\bm{\phi}_{i},\mathcal{D}_{i})$ which
are close to each other under smoothness assumption. However, the objective in
(10)-(11) does not reflect the fact that all agents need to meet at a common
initializer after $K$ backward steps. To solve this issue, we introduce
constraint (13) which enforces that the local models of all agents get closer
to each other at each step. Finally, the constraint (14) ensures that
$\bm{\phi}_{i}^{K}$ is the $\epsilon$-optimal solution of agent $i$
($\epsilon$-optimal in the norm 2 sense). Note that
$\bm{\phi}_{i,\epsilon}^{\star}$ is computed offline at each agent $i$ using
an itertative optimization solver such that

$\displaystyle\|\bm{\phi}_{i,\epsilon}^{\star}-\bm{\phi}_{i}^{\star}\|\leq\epsilon.$
(16)

##  IV Proposed Algorithm: Meta-Backward

Fig. 1 illustrates the idea of federated meta-learning and highlights the key
differences of our proposed approach compared to MAML and iMAML. We set
$\bm{\phi}_{i}^{K}=\bm{\phi}_{i,\epsilon}^{\star}$ and define
$\Phi^{k}=\frac{1}{N}\sum_{i=1}^{N}\bm{\phi}_{i}^{k}$. We run our proposed
backward meta-learning algorithm to find the meta model.

At backward step $k$, we need to solve the following problem to find
$\bm{\phi}_{i}^{k},\forall i$

$\displaystyle\min_{\\{\bm{\phi}_{i}\\}_{i=1}^{N}}$
$\displaystyle\sum_{i=1}^{N}\left(\mathcal{L}(\bm{\phi}_{i})-\mathcal{L}(\bm{\phi}_{i}^{k+1}+\alpha\nabla_{\bm{\phi}_{i}^{k+1}}\mathcal{L}(\bm{\phi}_{i}^{k+1},\mathcal{D}_{i}))\right)^{2}$
(17) s.t. $\displaystyle\|\bm{\phi}_{i}-\Phi^{k+1}\|^{2}\leq\delta_{k},\forall
i$ (18)

Intuitively, starting from $\bm{\phi}_{i}^{k+1},\forall i$, we want to find
$\bm{\phi}_{i}^{k}$ in an increasing direction that is closer to the average
across tasks (agents). Therefore, by running $K$ backward steps while
decreasing $\delta_{k}$, we eventually converge to a common initializer (meta
model) that approximates the solution of (12)-(14). We start by defining the
feasible set

$\displaystyle{\cal
X}_{i}=\\{\bm{\phi}_{i}:\parallel\bm{\phi}_{i}-\Phi^{k+1}\parallel^{2}\leq\delta_{k}\\}.$
(19)

With that, we can rewrite (17)-(18) as follows:

$\displaystyle\min_{\\{\bm{\phi}_{i}\\}_{i=1}^{N}}\ \ $
$\displaystyle\sum_{i=1}^{N}\parallel\mathcal{L}(\bm{\phi}_{i})-\mathcal{L}(\bm{\phi}_{i}^{k+1}+\alpha\nabla_{\bm{\phi}_{i}^{k+1}}\mathcal{L}(\bm{\phi}_{i}^{k+1},\mathcal{D}_{i}))\parallel^{2}$
(20) s.t. $\displaystyle\bm{\phi}_{i}\in{\cal X}_{i},\forall i$ (21)

Defining the indicator function

$I_{{\cal X}_{i}}(\bm{\phi}_{i})=\left\\{\begin{array}[]{l}0,\quad\text{ if
$\bm{\phi}_{i}\in{\cal X}_{i}$}\\\ +\infty,\quad\text{ otherwise}\\\
\end{array}\right.$ (22)

we can rewrite (20)-(21) as

$\displaystyle\min_{\\{\bm{\phi}_{i}\\}_{i=1}^{N}}\sum_{i=1}^{N}f_{i}(\bm{\phi}_{i}),$
(23)

where the function $f_{i}(\cdot)$ is defined as

$\displaystyle f_{i}(\bm{\phi}_{i})$
$\displaystyle=\parallel\mathcal{L}(\bm{\phi}_{i})\\!-\\!\mathcal{L}(\bm{\phi}_{i}^{k+1}\\!+\\!\alpha\nabla_{\bm{\phi}_{i}^{k+1}}\mathcal{L}(\bm{\phi}_{i}^{k+1},\mathcal{D}_{i}),\mathcal{D}_{i}\big{)}\parallel_{2}^{2}\\!+\\!I_{{\cal
X}_{i}}(\bm{\phi}_{i}).$ (24)

Projection gradient descent can be used to solve (23). Interestingly, it can
be show that by choosing

$\displaystyle\bm{\phi}_{i}^{k,0}=\bm{\phi}_{i}^{k+1}+\alpha\nabla_{\bm{\phi}_{i}^{k+1}}\mathcal{L}(\bm{\phi}_{i}^{k+1},\mathcal{D}_{i}),$
(25)

we do not need to iterate more, so by projecting $\bm{\phi}_{i}^{k,0}$ into
the feasible set ${\cal X}_{i}$, we find $\bm{\phi}_{i}^{k}$ for each task
$i$. The projection problem for each task $i$ is formulated as

$\displaystyle\min_{\bm{\phi}_{i}^{k}}\ \ $
$\displaystyle\frac{1}{2}\parallel\bm{\phi}_{i}^{k}-\bm{\phi}_{i}^{k,0}\parallel^{2}$
(26) s.t.
$\displaystyle\parallel\bm{\phi}_{i}^{k}-\Phi^{k+1}\parallel^{2}\leq\delta_{k}$

The Lagrangian function of (26) is given by

$\displaystyle
L(\bm{\phi}_{i},\Phi,\mu)=\frac{1}{2}\parallel\bm{\phi}_{i}^{k}-\bm{\phi}_{i}^{k,0}\parallel^{2}+\mu(\parallel\bm{\phi}_{i}^{k}-\Phi^{k+1}\parallel^{2}-\delta_{k}).$
(27)

Therefore, the K.K.T conditions of (26) are

$\displaystyle(\bm{\phi}_{i}^{k}-\bm{\phi}_{i}^{k,0})+2\mu(\bm{\phi}_{i}^{k}-\Phi^{k+1})$
$\displaystyle=\mathbf{0}$ (28)
$\displaystyle\mu(\parallel\bm{\phi}_{i}^{k}-\Phi^{k+1}\parallel^{2}-\delta_{k})$
$\displaystyle=0$ (29) $\displaystyle\mu$ $\displaystyle\geq 0$ (30)

Solving (28)-(30) yields the following solution

  * •

If $\mu=0$, then $\bm{\phi}_{i}^{k}=\bm{\phi}_{i}^{k,0}$.

  * •

If $\mu>0$, then

$\displaystyle\parallel\bm{\phi}_{i}^{k}-\Phi^{k+1}\parallel^{2}=\delta_{k}.$
(31)

Deriving $\bm{\phi}_{i}^{k}$ from (28) as a function of $\mu$, and
substituting it in (31) yilds $\mu$ that solves

$\displaystyle\mu^{2}+\mu-\frac{1}{4\delta_{k}}\parallel\bm{\phi}_{i}^{k,0}-\Phi^{k+1}\parallel^{2}+\frac{1}{4}=0.$
(32)

To sum up, compared to the state of the art algorithms, the proposed algorithm
is computationally efficient since it has a closed form update expression in
each iteration avoiding double loop. Moreover, it does not require any
computational expensive operations such as matrix inversion. Instead, it only
requires computing the gradient once and performing projection. The detailed
steps of the proposed algorithm are explained in Algorithm 1.

1: Input: $N,K,f_{i}(\bm{\phi}_{i})\ \text{for all}\ i$

2: for $i=0,1,2,\cdots,N$ do

3: Find $\phi_{i,\epsilon}^{\star}$ such that (16) is satisfied

4: Set $\phi_{i}^{K}=\phi_{i,\epsilon}^{\star}$

5: Transmit $\phi_{i}^{K}$ to the PS

6: end for

7: PS computes $\Phi^{K}=\frac{1}{N}\sum_{i=1}^{N}\phi_{i}^{K}$

8: for $k=K-1,\cdots,1,0$ do

9: Agent:  Each task (agent) finds $\phi_{i}^{k}$ that minimizes (23) given
$\phi_{i}^{k+1}$ and $\Phi^{k+1}$.

10: PS:  computes $\Phi^{k}=\frac{1}{N}\sum_{i=1}^{N}\phi_{i}^{k}$

11: transmits $\Phi^{k}$ to all agents

12: end for

Algorithm 1 Meta-Backward: Energy-Efficient Federated Meta learning

##  V Evaluation

In this section, we evaluate the performance of our proposed algorithm (Meta-
Backward) compared to iMAML [6]. We start by describing the evaluation
environment and then we discuss the evaluation results.

Figure 2: Accuracy, time, and energy of Meta-Backward and iMAML in the
sinusoid example: (a) CDF of the loss, (b) Computation and communication time
and (c) Computation and communication energy.
Figure 3:  Accuracy, time, and energy of Meta-Backward and iMAML in the image
classification example: (a) CDF of the loss, (b) Computation and communication
time and (c) Computation and communication energy.

###  V-A Experimental settings

Tasks:  We consider sinusoid regression and image classification tasks. For
the regression problem, each task involves regressing from the input to the
output of a sine wave. In particular, we train a meta model using 3 sinusoidal
regression tasks with amplitudes $2$, $6$, and $10$, respectively, and then
fine tune it to any sinusoid wave with amplitude ranges from $0.1$ to $10$
using $K$ samples. The loss is the mean-squared error between the prediction
$f(\mathbf{x})$ and true value. The regressor we use is the same one used in
[5], which is a neural network model with $2$ hidden layers of size $40$ with
ReLU nonlinear activation functions. We choose $K=40$ for the sinusoid
regression problem.

For the image classification task, we use the MNIST dataset [14] comprising of
60000 training and 10000 test samples, each of which represents a hand-written
$0-9$ digit image. We train using 2 tasks, where first task is trained to
classify digits $0$ to $2$, and the second task is trained to classify digits
$7$ to $9$. For both tasks, we use $400$ and $100$ samples for training and
validation respectively. The two tasks are used to train a meta model that is
then fine tuned to classify any $5$ randomly selected digits in the range $0$
to $9$ using $K$ samples from each un-seen class. We choose $K=10$ for the
classification problem considered in our experiment, so our problem is 5-way
10 shot classification. For the NN structure, we consider a 3-layer fully
connected multi-layer perceptron (MLP) comprising an input layer with 784
neurons, two hidden layers with 8 neurons each, and an output layer with 10
neurons, resulting in the model size $d=6442$. We use the rectified linear
unit (ReLu) activation function, softmax output, and cross entropy loss.

Baselines:  We compare our proposed approach with iMAML under different
settings where iMAML-$X$-$Y$ denotes running $X$ global SGD iterations to
update the meta-model according to (3), and for each global iteration, each
agent (task) runs $Y$ local SGD iterations to approximate the solution of the
inner problem. For iMAML, we approximate the matrix inversion indicated in (5)
using the same method described in [6] which involves $k$ conjugate gradient
(CG) steps where we choose $k=5$ and $\lambda=2$ as in [6].

Performance metrics:  We consider the following metrics when comparing the
performance of the proposed approach with the considered baselines.

  * •

Cumulative Distribution Function (CDF) of the test loss/accuracy: After
generating the meta-model of each algorithm, we fine-tune it to a new task
using $K$ samples from each class. We randomly select a new task each time for
$500$ and $100$ times for sinusoid regression and image classification
problems respectively, and we report the CDF of the test loss/accuracy for all
runs. As described earlier, we choose $K=40$ for the sinusoid regression
problem and $K=10$ for the classification problem. Since there are $5$ classes
in each new task for the classification problem, the total number of samples
is $50$.

  * •

Computation time and energy: For the computation time, we use the time module
in python, while the pyRAPL [15] package is used to evaluate the computation
energy. pyRAPL measures the energy footprint of a host machine when executing
a piece of Python code.

  * •

Communication time and energy: We assume communication over Additive White
Gaussian Noise (AWGN) where the bandwidth is $B$ (hz) per agent, the
transmission power is $P$ (W), and the noise power spectral density is $N_{o}$
(W/hz). Each element in the model is transmitted using $32$ bits. Hence, using
Shannon formula, the transmission rate $R$ is

$R=B\text{log}_{2}\left(1+\frac{P}{N_{o}B}\right).$ (33)

Then, we find the time ($\tau$) needed to send $32\cdot d$ bits, where $d$ is
the model size. Knowing the transmission time and power, the communication
energy per iteration and per agent is computed as $P\cdot\tau$. The total
communication energy is then calculated by summing the per agent energy. The
choices of $B$, $P$ and $N_{0}$ are specified for each example separately.

Framework and Hardware:  To run the experiments, we use Python 3.5 and
TensorFlow 2.0 operated in a MacBook Air computer (1.8 GHz Intel Core i5 CPU,
8 GB 1,600 MHz DDR3 RAM).

###  V-B Results Discussion

Now, we describe our results for both sinusoid regression and image
classification problems reported in Figs. 2 and 3.

####  V-B1 Sinusoidal Regression

We first point out that we use 50 projected stochastic gradient ascent updates
with a mini-batch size of $100$ samples and a fixed step size $\alpha=0.01$
for our proposed approach. For iMAML, we use the same choice for the mini-
batch size and $\alpha$ for both the inner and outer loops. Note that we
assume every task has been trained offline separately. To make the comparison
fair, we use the average of the local models of all tasks as the initializer
towards training the meta-model for all compared algorithms.

We evaluate the performance of all algorithms by fine-tuning the learned meta-
model on $K=40$ data-points chosen randomly. In this experiment, we choose
$B=$5Khz, $P=2$W, and $N_{o}=$1E-4, so the signal to noise ratio (SNR) is
equal to $13.86$ dB. The qualitative results, shown in Fig. 2-(a) show that
Meta-Backward is able to achieve the minimum loss in more than $80\%$ of the
experiments. In fact, iMAML requires more than 10 times local iterations
(computation cost) and global iterations (communication cost) to approach the
performance of Meta-Backward. Figs. 2 (b) and (c) confirm these findings and
show that Meta-Backward achieves this performance gain at significantly lower
latency and energy compared to the other baselines. This can be explained by
the simple and closed-form solution in each communication round performed by
Meta-Backward compared to iMAML that requires at least 50 local iterations per
communication round and involves complex operations such as matrix
manipulation.

####  V-B2 Image classification

In terms of communication setting, we use the same settings described for the
sinusoid regression problem. As seen from Figs 3-(a-c), the results confirm
our findings from the sinusoid experiment highlighting that iMAML needs to pay
significantly higher costs in terms of communication and computation energy to
outperform our proposed approach. In other words, the proposed approach can
achieve reasonable performance at very low energy cost using very few samples
from the new task.

##  VI Conclusion

In this paper, an energy-efficient meta-learning approach has been proposed.
The proposed approach enables learning a meta-model in a distributed setting
and is shown to achieve high accuracy (low loss) for a randomly generated new
similar regression/classification task with very few samples, at significantly
less energy consumption compared to iMAML. The practicality of the approach
was demonstrated using linear regression and image classification. Future work
will be focused on providing theoretical guarantees of the proposed approach
in addition to investigating other use cases.

## References

  * [1] J. Schmidhuber, “Evolutionary principles in self-referential learning,” _Diploma thesis, Institut f. Informatik, Tech. Univ. Munich_ , 1987.
  * [2] S. Thrun and L. Pratt, _Learning to learn_. Springer Science & Business Media, 1998.
  * [3] D. K. Naik and R. Mammone, “Meta-neural networks that learn by learning,” in _International Joint Conference on Neural Netowrks (IJCNN)_ , 1992.
  * [4] D. Maclaurin, D. Duvenaud, and R. Adams, “Gradient-based hyperparameter optimization through reversible learning,” in _International Conference on Machine Learning_ , 2015, pp. 2113–2122.
  * [5] C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in _International Conference on Machine Learning_. PMLR, 2017, pp. 1126–1135.
  * [6] A. Rajeswaran, C. Finn, S. Kakade, and S. Levine, “Meta-learning with implicit gradients,” _Advances in neural information processing systems_ , 2019.
  * [7] A. Nichol, J. Achiam, and J. Schulman, “On first-order meta-learning algorithms,” _arXiv preprint arXiv:1803.02999_ , 2018.
  * [8] A. Elgabli, J. Park, C. B. Issaid, and M. Bennis, “Harnessing wireless channels for scalable and privacy-preserving federated learning,” _IEEE Transactions on Communications_ , 2021.
  * [9] J. Park, S. Samarakoon, A. Elgabli, J. Kim, M. Bennis, S.-L. Kim, and M. Debbah, “Communication-efficient and distributed learning over wireless networks: Principles and applications,” _Proceedings of the IEEE_ , vol. 109, no. 5, pp. 796–819, 2021.
  * [10] T. Chen, G. B. Giannakis, T. Sun, and W. Yin, “Lag: Lazily aggregated gradient for communication-efficient distributed learning.”
  * [11] J. Chen, X. Pan, R. Monga, S. Bengio, and R. Jozefowicz, “Revisiting distributed synchronous SGD,” _arXiv preprint arXiv:1604.00981_ , 2016\.
  * [12] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in _Artificial Intelligence and Statistics_. PMLR, 2017, pp. 1273–1282.
  * [13] L. Bottou, F. E. Curtis, and J. Nocedal, “Optimization methods for large-scale machine learning,” _Siam Review_ , vol. 60, no. 2, pp. 223–311, 2018.
  * [14] Y. LeCun and C. Cortes, “MNIST handwritten digit database,” 2010.
  * [15] “pyRAPL,” https://github.com/powerapi-ng/pyRAPL, 2019.


# Uniqueness in determination of the fractional order in the TFDE using one
measurement

Yi Zhang Xianzheng Jia Gongsheng Li111Corresponding author, Email:
<EMAIL_ADDRESS>
_School of Mathematics and Statistics, Shandong University of Technology_
_Zibo, Shandong 255049, PRC_

Abstract: This article deals with an inverse problem of identifying the
fractional order in the 1D time fractional diffusion equation (TFDE in short)
using the measurement at one space-time point. Based on the expression of the
solution to the forward problem, the inverse problem is transformed to a
nonlinear algebraic equation. By choosing suitable initial values and the
measured point, the nonlinear equation has a unique solution by the
monotonicity of the Mittag-Lellfer function. Theoretical testifications are
presented to demonstrate the unique solvability of the inverse problem.

_Keywords:_ Time fractional diffusion equation (TFDE); inverse problem of
fractional order; Mittag-Lellfer function; nonlinear algebraic equation;
conditional uniqueness

MSC(2010) 35R11; 35R30

##  1 Introduction

Many diffusion processes in nature and engineering, such as contaminants
transport in the soil, oil flow in porous media, long distance transport of
pollutants in the groundwater, etc., are referred to as anomalous diffusion,
where the particle plume spreads slower or faster than predicted by the
classical integer-order diffusion model. In recent three decades, fractional
diffusion equations have been found to be efficient mathematical models to
describe some anomalous diffusion phenomena (see [1, 6, 7, 17, 35, 43, 48],
e.g.). The TFDE model, that is obtained from the classical diffusion equation
by replacing the first-order time derivative by a fractional derivative, is
given as

$\partial_{t}^{\alpha}u-D\triangle u=0,(x,t)\in\Omega_{T},$ $None$

where $\Omega_{T}=\Omega\times(0,T)$ with $T>0$, $\Omega\subset{\bf R}^{N}$ is
an open bounded domain with smooth boundary, and $u=u(x,t)$ denotes the state
variable at space point $x$ and time $t$, $D>0$ is the diffusion coefficient,
$\alpha\in(0,1)$ is the fractional order, and the fractional derivative
$\partial_{t}^{\alpha}u$ is defined by Caputo’s definition:

$\partial_{t}^{\alpha}u=\frac{1}{\Gamma(1-\alpha)}\int_{0}^{t}\frac{\partial
u(x,s)}{\partial s}\frac{ds}{(t-s)^{\alpha}}.$ $None$

See, e.g., Podlubny [38] and Kilbas et al [24] for the definitions and
properties of the fractional derivatives.
There are quite a lot of researches on equation (1.1) from numerics, and some
works from theoretical aspects, we refer to [8, 11, 12, 13, 16, 33, 34, 36,
39] for the theoretical analysis on the solution of the forward problem, and
also see the monographs [38], [24], [25] and [14]. On the other hand, we refer
to [2, 10, 15, 21, 22, 29, 31, 32, 37, 42, 44] for the researches on inverse
problems arising from the TFDEs. As we know the fractional order in a
fractional model is a key parameter to characterize the heavy-tail
subdiffusion with memory effect. However, it is always unknown for real-life
problems which leading to inverse fractional order problems. The inverse
problems of identifying parameters including fractional orders have been
studied during the last decade. We refer to Alimor and Ashurov [3], Ashurov
and Umarov [4], Ashurov and Zunnunov [5], Cheng et al [9], Hatano et al [18],
Janno [19], Janno and Kinash [20], Jin and Kian [23], Li et al [26], Li et al
[30], Li and Yamamoto [28], Sun et al [40], Tatar and Ulusoy [41], Yamamoto
[45, 46], Yu et al [47], and so on.
It is noted that in the existing work on inverse fractional order problems,
most of them were studied by using subdomain measurements or one-point
measurements at $t\in(0,T)$, or using subboundary data also at $t\in(0,T)$ for
arbiutrary given $T>0$. An interesting problem is can we determine the
fractional orders in theory only using limited discretized data? Exactly
speaking, there is one order $\alpha\in(0,1)$ in equation (1.1) which is
unknown, can we determine it uniquely only using one measurement?
It is difficult to give an answer in theory for the above question, but the
situation could be changed if having suitable conditions. Here we are
concerned with the inverse problem of determining the fractional order
$\alpha$ in Eq. (1.1) using one measurement at one space-time point. By the
eigenfunction expansion method, the solution of the forward problem is
expressed by the Mittag-Lellfer function, and the inverse problem is
transformed to a nonlinear algebraic equation. By choosing suitable initial
values and the measured point, the nonlinear equation can be solved uniquely
by the monotonicity of the Mittag-Lellfer function. The unique solvability of
the inverse problem is testified by theoretical examples.
The rest of the paper is organized as follows.
In Section 2, some preliminaries on the Mittag-Lellfer function, and
solvability on a nonlinear algebraic equation are introduced. The forward
problem and its solution, and the inverse problem of identifying the
fractional order using one measurement are given in Section 3. In Section 4, a
local well-posedness of the inverse problem is obtained by the theory of
solution to the nonlinear algebraic equation, and a conditional uniqueness is
proved by suitably choosing the initial value and the measured time, and
numerical testifications are presented. Conclusions are given in Section 5.

##  2 Preliminaries

We give some basic facts on the Gamma function, the Mittag-Leffler function
and their properties, and the theory of solving a nonlinear algebraic
equation. We refer to the monographs [14] and [27] for details.

###  2.1 The Gamma function and Mittag-Leffler function

The Gamma function is an analytic continuation of the factorial function in
the entire complex plane, which is defined by

$\Gamma(z)=\int_{0}^{\infty}x^{z}e^{-x}\ dx,$ $None$

with $\Re(z)>0$. For the Gamma function there hold the formula
$\Gamma(z+1)=z\Gamma(z)$, and the Stirling’s approximate formula

$\Gamma(z+1)\sim\sqrt{2\pi z}\ e^{-z}\ z^{z},\ z\rightarrow+\infty,$ $None$

and the derivative’s formula

$\frac{{{\Gamma^{{}^{\prime}}}(z)}}{{\Gamma(z)}}=-\gamma-\frac{1}{z}+\sum\limits_{n=1}^{\infty}{(\frac{1}{n}-\frac{1}{{n+z}})},$
$None$

where ${\Gamma^{{}^{\prime}}}(z)$ denotes the derivative of the Gamma
function, and $\gamma=0.5772157\dots$ is the Euler constant. On the Gamma
function, we have the following assertion.

###### Lemma 1

For the parameter $\alpha\in(0,1)$, there holds

$\lim\limits_{j\rightarrow\infty}\frac{\Gamma(\alpha j)}{\Gamma(\alpha
j+\alpha)}=0.$ $None$

_Proof_ By (2.2) we have for large $z>0$

$\begin{array}[]{lll}\frac{\Gamma(\alpha j)}{\Gamma(\alpha
j+\alpha)}&\sim&\frac{{{{[\sqrt{2\pi}{e^{-\alpha j}}{{(\alpha j)}^{\alpha
j}}]}\mathord{\left/{\vphantom{{[\sqrt{2\pi}{e^{-\alpha j}}{{(\alpha
j)}^{\alpha j}}]}{\sqrt{\alpha j}}}}\right.\kern-1.2pt}{\sqrt{\alpha
j}}}}}{{{{[\sqrt{2\pi}{e^{-\alpha j-\alpha}}{{(\alpha j+\alpha)}^{\alpha
j+\alpha}}]}\mathord{\left/{\vphantom{{[\sqrt{2\pi}{e^{-\alpha
j-\alpha}}{{(\alpha j+\alpha)}^{\alpha j+\alpha}}]}{\sqrt{\alpha
j+\alpha}}}}\right.\kern-1.2pt}{\sqrt{\alpha j+\alpha}}}}}\
(j\rightarrow\infty)\\\ &\sim&{e^{\alpha}}\ {\left({\frac{{\alpha j}}{{\alpha
j+\alpha}}}\right)^{\alpha j}}\ \frac{1}{{{{(\alpha j+\alpha)}^{\alpha}}}}\
\sqrt{\frac{{\alpha j+\alpha}}{{\alpha j}}}\ (j\rightarrow\infty)\\\
&\sim&{e^{\alpha}}\
{\left[{{{\left({\frac{1}{{1+{1\mathord{\left/{\vphantom{1j}}\right.\kern-1.2pt}j}}}}\right)}^{j}}}\right]^{\alpha}}\
\frac{1}{{{{(\alpha j+\alpha)}^{\alpha}}}}\ \sqrt{1+\frac{1}{j}}\
(j\rightarrow\infty).\\\ \end{array}$ $None$

Thanks to
$\mathop{\lim}\limits_{j\to\infty}{\left({\frac{1}{{1+{1\mathord{\left/{\vphantom{1j}}\right.\kern-1.2pt}j}}}}\right)^{j}}={e^{-1}}$,
we get

$\begin{array}[]{lll}\mathop{\lim}\limits_{j\to\infty}\frac{\Gamma(\alpha
j)}{\Gamma(\alpha
j+\alpha)}&=&\mathop{\lim}\limits_{j\to\infty}\left[{{e^{\alpha}}\
{e^{-\alpha}}\ \frac{1}{{{{(\alpha j+\alpha)}^{\alpha}}}}\
\sqrt{1+\frac{1}{j}}}\right]\\\
&=&\mathop{\lim}\limits_{j\to\infty}\frac{1}{(\alpha
j+\alpha)^{\alpha}}=0.\end{array}$ $None$

Following Mittag-Leffler’s classical definition, the one-parametric Mittag-
Leffler function is defined by the power series

$E_{\alpha}(z)=\sum\limits_{j=0}^{\infty}{\frac{{{z^{j}}}}{{\Gamma(\alpha
j+1)}}},\ z\in{\cal C},\ \alpha>0.$ $None$

Obviously there is $E_{1}(z)=e^{z}$ as $\alpha=1$, i.e., $E_{\alpha}(z)$ is a
generalization of the exponential function $e^{z}$.
On analytical properties of the Mittag-Leffler functions, we have the
following assertions.

###### Lemma 2

(i) There exists a constant $c>0$ such that
$|E_{\alpha}(z)|\leq\frac{c}{1+|z|}$, $\alpha>0$.
(ii) For real number $z\in{\bf R}$, the function $E_{\alpha}(z)$ is strictly
monotone on $z>0$, and decreasing on $z<0$.

###### Lemma 3

Let $G(\alpha)=E_{\alpha}(-ct^{\alpha})$ for $\alpha\in(0,1)$ and given
$0<c,t<\infty$. There holds

$G^{\prime}(\alpha)=\sum\limits_{j=1}^{\infty}(-c)^{j}\ j\ t^{\alpha
j}\frac{\ln(t)+\gamma_{j}}{\Gamma(\alpha j+1)},$ $None$

where $\gamma_{j}=\gamma+\frac{1}{\alpha
j+1}-\sum\limits_{n=1}^{\infty}{(\frac{1}{n}-\frac{1}{{n+\alpha j+1}})}$
according to (2.3).

_Proof_ By (2.7), it is easy to deduce that the derivative
$G^{\prime}(\alpha)$ ($0<\alpha<1$) is expressed by (2.8), where $\gamma_{j}$
is given by (2.3), and we only need to prove the convergence of the series at
the right-hand side of (2.8).
Denote

$y_{j}=\frac{(-c)^{j}\ j\ t^{\alpha j}}{\Gamma(\alpha j+1)}\ln(t),\
\bar{y}_{j}=\frac{(-c)^{j}\ j\ t^{\alpha j}\gamma_{j}}{\Gamma(\alpha j+1)},\
j=1,2,\cdots,$ $None$

and we have
$G^{\prime}(\alpha)=\sum\limits_{j=1}^{\infty}(y_{j}+\bar{y}_{j})$. For given
$c$ and $t$ as parameters, there holds

$\left|\frac{y_{j+1}}{y_{j}}\right|=c\ t^{\alpha}\frac{(j+1)\ \Gamma(\alpha
j+1)}{j\ \Gamma(\alpha j+\alpha+1)}=c\ t^{\alpha}\frac{\Gamma(\alpha
j)}{\Gamma(\alpha j+\alpha)},$ $None$

here we utilize the formula $\Gamma(z+1)=z\Gamma(z)$. Then by Lemma 1 we get

$\lim\limits_{j\rightarrow\infty}|\frac{y_{j+1}}{y_{j}}|=0,$ $None$

by which the series $\sum\limits_{j=1}^{\infty}y_{j}$ is convergent. Noting
that

$\lim\limits_{j\rightarrow\infty}\frac{\gamma_{j+1}}{\gamma_{j}}=1,$ $None$

we deduce that $\sum\limits_{j=1}^{\infty}\bar{y}_{j}$ is also convergent by
the same arguments. So the derivative $G^{\prime}(\alpha)$ is well-defined for
$\alpha\in(0,1)$.

###  2.2 Solution to a nonlinear algebraic equation

Consider to solve a group of nonlinear algebraic equations

$F(x)=y,$ $None$

where $F:D\subset{\bf R}^{N}\rightarrow{\bf R}^{N}$ is continuous differential
on $D$, and $D$ is an open bounded domain in ${\bf R}^{N}$. On the local
unique solvability of the equation (2.13), we give the following assertion
[27].

###### Lemma 4

For the equation (2.13), assume that there exists a nonsingular matrix
$B\in{\cal L}({\bf R}^{N})$, and $x^{0}\in D$ with a closed sphere
$S_{0}=\bar{S}(x^{0},\delta)$ such that

$||F(x)-F(y)-B(x-y)||\leq\beta||x-y||,\ \forall x,y\in S_{0},$ $None$

where $0<\beta<\|B^{-1}\|^{-1}$. Then $F$ is a homeomorphsim mapping from
$S_{0}$ to $F(S_{0})$.

We now consider to solve a nonlinear algebraic equation in 1D case

$g(x)=y,$ $None$

where $g:D\subset{\bf R}\rightarrow{\bf R}$ is continuous differential on $D$,
and $D$ is an open bounded interval in ${\bf R}$. Based on Lemma 4 there holds

###### Corollary 1

Suppose that the function $g:D\subset{\bf R}\rightarrow{\bf R}$ is continuous
differentiable at $x_{0}\in int(D)$, and $g^{\prime}(x_{0})\neq 0$, and
$V_{0}=\\{x:|x-x_{0}|\leq\delta\\}$ for any $\delta>0$ as a neighborhood of
$x_{0}$. Then $g$ is a local homeomorphism mapping at $x_{0}$, i.e., the
equation $g(x)=y$ has a unique solution for given $y\in g(V_{0})$. Moreover,
the inverse function $g^{-1}:g(V_{0})\rightarrow V_{0}$ is also continuous.

_Proof_ Denote $a=g^{\prime}(x_{0})$ and $a\neq 0$. By the definition of the
derivative, we have

$\lim\limits_{x\rightarrow x_{0}}\frac{g(x)-g(x_{0})}{x-x_{0}}=a.$ $None$

Thus $\forall\varepsilon>0$ and $\varepsilon<|a|$, there exists $\delta>0$
such that

$|g(x)-g(x_{0})-a(x-x_{0})|\leq\varepsilon|x-x_{0}|,\ \forall x,x_{0}\in
V_{0}.$ $None$

Noting $\varepsilon<|a|$, by Lemma 4 follows that the mapping $g$ is a local
homeomorphism from $V_{0}$ to $g(V_{0})$, and the equation $g(x)=y$ has a
unique solution for given $y\in g(V_{0})$ and the inverse function $g^{-1}$ is
continuous.

##  3 The forward problem and the inverse problem

###  3.1 The forward problem

Consider the homogeneous time-fractional diffusion equation in 1D case

$\partial_{t}^{\alpha}u-Du_{xx}=0,(x,t)\in\Omega_{T},$ $None$

where $\Omega_{T}=\Omega\times(0,T)$ and $\Omega=(0,l)$ for $l>0$. For the
equation (3.1), given the homogeneous Dirichelet boundary condition

$u(0,t)=u(l,t)=0,\ 0<t\leq T,$ $None$

and the initial distribution

$u(x,0)=f(x),\ x\in\Omega,$ $None$

where $f(x)\in L^{2}(\Omega)$. A forward problem is composed by the equation
(3.1) with the initial boundary value conditions (3.2)-(3.3). By using the
eigenfunction expansion method, there exists a unique solution (see [39, 46]
for instance):

$u(x,t)\in C([0,T],L^{2}(\Omega))\cap C((0,T],H^{2}(\Omega)\cap
H_{0}^{1}(\Omega)),$ $None$

and it can be expressed by the Mittag-Lellfer function given as

$u(x,t)=\sum\limits_{n=1}^{\infty}f_{n}\
{E_{\alpha}}(-D{\lambda_{n}}{t^{\alpha}})\ {\varphi_{n}}(x),$ $None$

where $f_{n}=\frac{2}{l}(f,\varphi_{n})$, and
$\lambda_{n}=\frac{n^{2}\pi^{2}}{l^{2}}$, $\varphi_{n}(x)=\sin(\frac{n\pi
x}{l})$, and $E_{\alpha}(\cdot)$ is given by (2.7).

###  3.2 The inverse problem

Although we get the solution to the forward problem given by (3.5), it can not
be put into practice if there are unknown parameters in the model. Actually,
the fractional order $\alpha$ in Eq. (3.1) is an important index
characterizing the anomalous diffusion with memory effect in time, but it
cannot be measured which leads to inverse fractional order problems.
Suppose that the fractional order in Eq. (3.1) is unknown, and we are to
determine it by limited data in time at one space point. Let $x_{0}\in\Omega$
be fixed, and we have the measured data at time $t_{k}$ given as

$u(x_{0},t_{k}),\ k=1,2,\cdots,K,$ $None$

where $K\geq 1$. The inverse problem is to identify the order $\alpha\in(0,1)$
using the data (3.6) based on the forward problem (3.1)-(3.3).
Denote $I=(0,1)$, and $u(\alpha)(x,t)$ as the unique solution to the forward
problem for any given $\alpha\in I$. An interesting problem is:
Can we determine $\alpha\in I$ uniquely only using one measurement of
$u(x_{0},t_{1})$ at $t_{1}\in(0,T)$ ?
Generally speaking, it is difficult to answer the above question in theory.
Nevertheless, we can determine it in some special cases. Denote
$d=u(x_{0},t_{1})$ as the measurement. Then noting the solution’s expression
(3.5), we get a nonlinear algebraic equation:

$F(\alpha)=u(\alpha)(x_{0},t_{1})=\sum\limits_{n=1}^{\infty}f_{n}\
{E_{\alpha}}(-D{\lambda_{n}}{t_{1}^{\alpha}})\ {\varphi_{n}}(x_{0})=d.$ $None$

As a result the inverse fractional order problem is transformed to solving of
the nonlinear equation (3.7).

##  4 Unique solvability of the inverse problem

###  4.1 Local well-posedness

We now prove the inverse problem is locally solvable and has unique solution
for $\alpha\in I$ by choosing appropriate measured point and the measured time
$t_{1}>0$. For that we need the existence of the derivative of the function
$F(\alpha)$ on $\alpha\in I$, and some limitations for the initial value
functions.
Without loss of generality, we set $l=\pi$. It is known that the eigenfunction
system of the forward problem is $\\{n^{2},\sin(nx)\\}$ for $n=1,2,\cdots$.
For $f(x)\in L^{2}(0,\pi)$, assume that there exists a limited number set
$\Sigma_{N}=\\{n_{1},n_{2},\cdots,n_{N}\\}\subset{\bf N}$ such that

$(f(x),\sin(nx))\left\\{\begin{array}[]{lll}\neq 0,n\in\Sigma_{N},\\\
=0,\hbox{others},\end{array}\right.$ $None$

which means that there holds

$f(x)=\sum\limits_{n\in\Sigma_{N}}f_{n}\sin(nx),$ $None$

where $f_{n}=\frac{2}{\pi}(f(x),\sin(nx))$.
By (3.5) we have

$u(x,t)=\sum\limits_{n\in\Sigma_{N}}f_{n}E_{\alpha}(-D\lambda_{n}t^{\alpha})\sin(nx),\
0<t\leq T.$ $None$

Then the equation (3.7) reduces to

$F(\alpha)=\sum\limits_{n\in\Sigma_{N}}f_{n}E_{\alpha}(-D\lambda_{n}t_{1}^{\alpha})\
\varphi_{n}(x_{0})=d,$ $None$

and we get by using (2.8) in Lemma 3

$F^{\prime}(\alpha)=\sum\limits_{n\in\Sigma_{N}}{f_{n}}\varphi_{n}(x_{0})\sum\limits_{j=1}^{\infty}\frac{(-1)^{j}\
j\ D^{j}\lambda_{n}^{j}t_{1}^{\alpha j}}{\Gamma(\alpha
j+1)}[\ln(t_{1})+\gamma_{j}],\ 0<\alpha<1.$ $None$

Henceforth, by using Corollary 1 there holds

###### Theorem 1

For suitable $x_{0}\in\Omega$ and $t_{1}>0$, there exists $\alpha_{0}\in I$
such that $F^{\prime}(\alpha_{0})\neq 0$, and then the equation (4.4) has a
unique solution in a neighborhood of $\alpha_{0}$, and the solution is
continuously dependent upon the measured data locally, i.e., the inverse
fractional order problem is of local well-posedness.

_Proof_ By suitably choosing the measured point $x_{0}$, we have
$\varphi_{n}(x_{0})\neq 0$ for $n\in\Sigma_{N}$, and then there exists one
$\alpha_{0}\in I$ such that $F^{\prime}(\alpha_{0})\neq 0$. Then by using
Corollary 1, the nonlinear equation (4.4) has a unique solution at the
neighborhood of $\alpha_{0}$, i.e., the inverse fractional order problem is
locally solvable. Moreover, noting the local continuity of the inverse
function $F^{-1}$, the inverse problem is of local stability.
Furthermore, we can get a conditional uniqueness under the nonnegative
conditions for the initial values.

###  4.2 Conditional uniqueness

By Theorem 1 the nonlinear equation (4.4) is locally solvable for
$d=u(x_{0},t_{1})$ where $0<t_{1}\leq T$ and $x_{0}\in\Omega$ be fixed. There
are no global uniqueness for solving nonlinear equations in general case, but
the situation could be changed if special conditions are overposed by which a
conditional uniqueness is obtained. For convenience we also set
$\Omega=(0,\pi)$. There holds

###### Theorem 2

Assume that the the initial value function has the form of (4.2), and there
hold $f_{n}\varphi_{n}(x)\geq 0$ for $n\in\Sigma_{N}$ and $x\in\Omega$. Then
the fractional order can be uniquely determined by one measurement
$u(x_{0},t_{1})$ if $\varphi_{n}(x_{0})\neq 0$ and $t_{1}>0$.

_Proof_ Under the given conditions, there are $\varphi_{n}(x)=\sin(nx)$ and
$\lambda_{n}=n^{2}$ for $n=1,2,\cdots$. By (4.3) the solution of the forward
problem is given as

$u(\alpha)(x,t)=\sum\limits_{n\in\Sigma_{N}}f_{n}E_{\alpha}(-Dn^{2}t^{\alpha})\sin(nx),$
$None$

and we get the nonlinear equation on the order $\alpha\in(0,1)$

$F(\alpha)=\sum\limits_{n\in\Sigma_{N}}E_{\alpha}(-Dn^{2}t_{1}^{\alpha})\
f_{n}\sin(nx_{0})=d,$ $None$

where $d=u(x_{0},t_{1})$ is the additional measurement.
By choosing $x_{0}\in\Omega$ such that $\sin(nx_{0})>0$ for $n\in\Sigma_{N}$,
and noting the condition $f_{n}\varphi_{n}(x)\geq 0$ for $n\in\Sigma_{N}$, we
can deduce that the function $F(\alpha)$ is of strict monotonicity on
$\alpha\in(0,1)$ due to the strict monotonicity and positivity of the Mittag-
Lelffer function. Therefore the equation (4.7) has only one solution
$\alpha^{*}\in I=(0,1)$. The proof is completed.

###  4.3 Theoretical testification

####  4.3.1 Example 1

In this example we set $D=0.1$, and the initial function
$f(x)=\frac{1}{2}\sin(2x)$ for $x\in[0,\pi]$, and we choose
$x_{0}=\frac{\pi}{4}$ as the measured point, and $t_{1}=2$ as the measured
time, i.e., the additional measurement is $d=u(\frac{\pi}{4},2)$. Noting $n=2$
and $f_{n}=\frac{1}{2}$, and $\sin(nx_{0})=1$, we have by (4.7)

$F(\alpha)=\frac{1}{2}{E_{\alpha}}(-\frac{2^{1+\alpha}}{5}),\ 0<\alpha<1,$
$None$

and we need to solve the equation

$F(\alpha)-d=0,$ $None$

using the additional data $u(x_{0},t_{1})=d$.
Let the exact order be $\alpha^{exa}=0.75$, by which we get the additional
data $d=0.25818$ by solving the forward problem. In order to see the unique
solvability of the equation, we plot the continuous function $F(\alpha)-d$ on
$\alpha\in[0,1]$ in Figure 1, where $F(0)$ and $F(1)$ are defined by

$F(0)=\frac{1}{2}\sum\limits_{k=0}^{\infty}(-\frac{2}{5})^{k},\
F(1)=\frac{1}{2}e^{-\frac{4}{5}}.$ $None$

It can be seen clearly that the function $F(\alpha)-d$ is strictly monotone on
$\alpha\in[0,1]$, and the inverse order problem is of uniqueness.

Figure 1. The picture of $F(\alpha)-d$ on $\alpha\in[0,1]$ in Ex. 1.

####  4.3.2 Example 2

We set $D=0.05$, and the initial function $f(x)=2\sin(x)+\frac{1}{2}\sin(3x)$
for $x\in[0,\pi]$ in this example. For given $\alpha\in(0,1)$, noting that
$D=0.05$, and $\lambda_{1}=1,\lambda_{3}=3$, the solution of the forward
problem in this example is expressed as follows

$u(\alpha)(x,t)=2E_{\alpha}(-0.05t^{\alpha})\sin(x)+\frac{1}{2}E_{\alpha}(-0.45t^{\alpha})\sin(3x).$
$None$

For the inverse order problem, let the exact fractional order be
$\alpha^{exa}=0.5$. We choose $x_{0}=\frac{\pi}{6}$ as the measured point, and
$t_{1}=10$ as the measured time, i.e., the additional measurement is
$d=u(\frac{\pi}{6},10)=1.0112$. We have the nonlinear equation combing with
the additional data

$F(\alpha)=E_{\alpha}(-0.05\cdot 10^{\alpha})+\frac{1}{2}E_{\alpha}(-0.45\cdot
10^{\alpha})=d.$ $None$

As done in the above, the function $F(\alpha)-d$ on $\alpha\in[0,1]$ are
plotted in Figure 2.

Figure 2. The picture of $F(\alpha)-d$ on $\alpha\in[0,1]$ in Ex. 2.

From Figure 2 it can be seen again that the equation $F(\alpha)-d=0$ has a
unique solution, and the inverse order problem is uniquely solvable by one
measurement.

##  5 Conclusion

An inverse problem of identifying the fractional order in the 1D time
fractional diffusion equation by using one measurement is investigated. Based
on the expression of the solution to the forward problem, the inverse problem
is reduced to a nonlinear algebraic equation on the fractional order, and the
unique solvability can be obtained by the monotonicity and positivity of the
Mittag-Lellfer function of real variables under the positive conditions for
the initial values and the measured point. Theoretical examples are given to
illustrate the conditional uniqueness of the inverse problem. It is noted that
the derivative of the function $F(\alpha)$ on $\alpha\in(0,1)$ can be computed
by (4.8) or (4.12) respectively, and some gradient-type iterative algorithms
can be applied to solve the nonlinear equation for which we will give details
in another work in the near future.

## Acknowledgements

This work is supported by Natural Science Foundation of Shandong Province,
China (No. ZR2019MA021), and National Natural Science Foundation of China
(nos. 11871313, 11371231).

## References

  * [1] E. E. Adams and L. W. Gelhar, Field study of dispersion in a heterogeneous aquifer 2: Spatial moments analysis, Water Resources Research, 28(1992) 3293–3307.
  * [2] S. Alimov and R. Ashurov, Inverse problem of determining an order of the Riemann-Liouville time-fractional derivative, Prog. Fractional Differ. Appl., 1(2015) 1–7.
  * [3] S. Alimov and R. Ashurov, Inverse problem of determining an order of the Caputo time fractional derivative for a subdiffusion equation, Journal of Inverse and Ill-Posed Prolems, 28(2020) 651–658.
  * [4] R. Ashurov and S. Umarov, Determination of the order of fractional derivative for subdiffusion equations, Fractional Calculus and Applied Analysis, 23(2020) 1647–1662.
  * [5] R. Ashurov and R. Zunnunov, Initial-boundary value and inverse problems for subdiffusion equations in ${\bf R}^{N}$, Fractional Differ. Calculus, 10(2020) 291–306.
  * [6] D. A. Benson, The Fractional Advection-Dispersion Equation: Development and Application, Dissertation of Doctorial Degree, University of Nevada, Reno, USA, 1998.
  * [7] B. Berkowitz, H. Scher and S. E. Silliman, Anomalous transport in laboratory-scale heterogeneous porus media, Water Resources Research, 36(2000) 149–158.
  * [8] Z. Q. Chen, M. M. Meerschaert and E. Nane, Space-time fractional diffusion on bounded domains, J. Math. Anal. Appl., 393(2012) 479–488.
  * [9] J. Cheng, J. Nakagawa, M. Yamamoto and T. Yamazaki, Uniqueness in an inverse problem for a one-dimensional fractional diffusion equation, Inverse Problems, 25(2009) 115002.
  * [10] G. S. Chi, G. S. Li and X. Z. Jia, Numerical inversions of source term in FADE with Dirichlet boundary condition by final observations, Computers and Mathematics with Applications, 62(2011) 1619-1626.
  * [11] X. L. Ding and J. J. Nieto, Analytical solutions for the multi-term time-space fractional reaction-diffusion on equations on an infinite domain, Fractional Calculas and Applied Analysis, 18(2015) 697–716.
  * [12] S. D. Eidelman and A. N. Kochubei, Cauchy problem for fractional diffusion equations, J. Diff. Eqn., 199(2004) 211–255.
  * [13] R. Gorenflo, Y. Luchko and P. P. Zabrejko, On solvability of linear fractional differential equations in Banach spaces, Fract. Calc. Appl. Anal., 2(1999)163–176.
  * [14] R. Gorenflo, A. A. Kilbas, F. Mainardi, and S. Rogosin, Mittag-Leffler Functions, Related Topics and Applications (2nd Ed.), Springer, Berlin, 2020.
  * [15] N. Guerngar, E. Nane, R. Tinaztepe, S. Ulusoy and H. W. Van Wyk, Simultaneous inversion for the fractional exponents in the space-time fractional diffusion equation, Fractional Calculus and Applied Analysis, 24(2021).
  * [16] A. Hanyga, Multidimensional solutions of time-fractional diffusion-wave equations, Proc. R. Soc. Lond. A, 458(2002)933–957.
  * [17] Y. Hatano and N. Hatano, Dispersive transport of ions in column experiments: an explanation of long-tailed profiles, Water Resources Research, 34(1998) 1027–1033.
  * [18] Y. Hatano, J. Nakagawa, S. Wang, and M. Yamamoto, Determination of order in fractional diffusion equation, J. Math. Ind., 5A(2013) 51–57.
  * [19] J. Janno, Determination of the order of fractional derivative and a kernel in an inverse problem for a genaralized time fractional diffusion equation, Electron. J. Differ. Equ., 199(2016) 1–28.
  * [20] J. Janno and N. Kinash, Reconstruction of an order of derivative and a source term in a fractional diffusion equation from final measurements, Inverse Problems, 34(2018) 025007.
  * [21] X. Z. Jia, D. L. Zhang, G. S. Li, et al, Numerical inversion of the fractional orders in the space-time fractional advection-dispersion equation with variable coefficients (in Chinese). Mathematica Numerica Sinica, 36(2014) 113–132.
  * [22] B. T. Jin and W. Rundell, A tutorial on inverse problems for anomalous diffusion processes, Inverse Problems, 31(2015) 035003.
  * [23] B. T. Jin and Y. Kian, Recovery of the order of derivation for fractional diffusion equations in an unknown medium, 2021, arXiv: 2101.09165.
  * [24] A. A. Kilbas, H. M. Srivastava and J. J. Trujillo, Theory and Applications of Fractional Differential Equations, Elsevier, Amsterdam, 2006.
  * [25] A. Kubica, K. Ryszewska, M. Yamamoto, Theory of Time-Fractional Differential Equations an Introduction, Springer, Berlin, 2020.
  * [26] G. S. Li, D. L. Zhang, X. Z. Jia and M. Yamamoto, Simultaneous inversion for the space-dependent diffusion coefficient and the fractional order in the time-fractional diffusion equation, Inverse Problems, 29(2013) 065014.
  * [27] Q. Y. Li, Z. Z. Mo, L. Q. Qi, Numerical Solutions for Nonlinear Equations (in Chinese), Science Press, Beijing, 2005.
  * [28] Z. Y. Li and M. Yamamoto, Uniqueness for inverse problems of determining orders of multi-term time-fractional derivatives of diffusion equation, Applicable Analysis, 94(2015) 570–579.
  * [29] Z. Y. Li, O. Y. Imanuvilov and M. Yamamoto, Uniqueness in inverse boundary value problems for fractional diffusion equations, Inverse Problems, 32(2016) 015004.
  * [30] Z. Y. Li, K. Fujishiro, G. S. Li, Uniqueness in the inversion of distributed orders in ultraslow diffusion equations, Journal of Computational and Applied Mathematics, 369(2020) 112564.
  * [31] J. J. Liu and M. Yamamoto, A backward problem for the time-fractional diffusion equation, Applicable Analysis, 89(2000) 1769–1788.
  * [32] Y. K. Liu, W. Rundell and M. Yamamoto, Strong maximum principle for fractional diffusion equations and an application to an inverse source problem, Fractional Calculus and Applied Analysis, 19(2016) 888–906.
  * [33] Y. Luchko, Maximum principle for the generalized time-fractional diffusion equation, J. Math. Anal. Appl., 351(2009)218–223.
  * [34] Y. Luchko, Some uniqueness and existence results for the initial-boundary-value problems for the generalized time-fractional diffusion equation, Comput. Math. Appl. 59(2010)1766–1772.
  * [35] F. Mainardi, Fractional Calculus and Waves in Linear Viscoelasticity: An Introduction to Mathematical Models, Imperial College Press, London, 2010.
  * [36] R. Metzler and J. Klafter, Boundary value problems for fractional diffusion equations, Physica A, 278(2000) 107–125.
  * [37] D. A. Murio, Stable numerical solution of fractional-diffusion inverse heat conduction probllem, Computers and Mathematics with Applications, 53(2007) 1492–1501.
  * [38] I. Podlubny, Fractional Differential Equations, Academic, San Diego, 1999.
  * [39] K. Sakamoto and M. Yamamoto, Initial value/boundary value problems for fractional diffusion-wave equations and applications to some inverse problems, Journal of Mathematical Analysis and Applications, 382(2011) 426–447.
  * [40] L. L. Sun, Y. S. Li and Y. Zhang, Simultaneous inversion for the potential term and the fractional orders in a multi-term time-fractional diffusion equation, Inverse problems, 37(2021) 055007.
  * [41] S. Tatar and S. Ulusoy, A uniqueness result for an inverse problem in a space-time fractional diffusion equation, Electronic Journal of Differential Equations, 258(2013): 1–9.
  * [42] T. Wei, X. L. Li and Y. S. Li, An inverse time-dependent source problem for a time-fractional diffusion equation, Inverse problems, 32(2016) 085003.
  * [43] Y. W. Xiong, G. H. Huang and Q. Z. Huang, Modeling solute transport in one dimensional homogeneous and heterogeneous soil columns with continuous time random walk, J. Contam. Hydrol., 86(2006) 163-175.
  * [44] M. Yamamoto, Y. Zhang, Conditional stability in determining a zeroth-order coefficient in a half-order fractional diffusion equation by a Carleman estimate, Inverse Problmes, 28(2012) 105010.
  * [45] M. Yamamoto, Uniqueness in determining the orders of time and spatial fractional derivatives, 2020, arXiv: 2006.15046.
  * [46] M. Yamamoto, Uniqueness in determining fractional orders of derivatives and initial values, Inverse Problems, 37(2021) 095006.
  * [47] B. Yu, X. Jiang and H. Qi, An inverse problem to estimate an unknown order of a Riemann-Liouville fractional derivative for a fractional Stokes’ first problem for a heated generalized second grade fluid, Acta Mech. Sin., 31(2015) 153–161.
  * [48] L. Zhou and H. M. Selim, Application of the fractional advection-dispersion equations in porous media, Soil. Sci. Soc. Am. J., 67(2003) 1079–1084.