# Computation Offloading for Uncertain Marine Tasks by Cooperation of UAVs and
Vessels

Jiahao You†, Ziye Jia†, Chao Dong†, Lijun He∗, Yilu Cao†, and Qihui Wu†
†The Key Laboratory of Dynamic Cognitive System of Electromagnetic Spectrum
Space, Ministry of Industry and Information Technology, Nanjing University of
Aeronautics and Astronautics, Nanjing, Jiangsu, 210016, China
∗School of software, Northwestern Polytechnical University, Xi’an, Shaanxi,
710129, China
{yjiahao, jiaziye, dch, caoyilu<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

With the continuous increment of maritime applications, the development of
marine networks for data offloading becomes necessary. However, the limited
maritime network resources are very difficult to satisfy real-time demands.
Besides, how to effectively handle multiple compute-intensive tasks becomes
another intractable issue. Hence, in this paper, we focus on the decision of
maritime task offloading by the cooperation of unmanned aerial vehicles (UAVs)
and vessels. Specifically, we first propose a cooperative offloading
framework, including the demands from marine Internet of Things (MIoTs)
devices and resource providers from UAVs and vessels. Due to the limited
energy and computation ability of UAVs, it is necessary to help better apply
the vessels to computation offloading. Then, we formulate the studied problem
into a Markov decision process, aiming to minimize the total execution time
and energy cost. Then, we leverage Lyapunov optimization to convert the long-
term constraints of the total execution time and energy cost into their short-
term constraints, further yielding a set of per-time-slot optimization
problems. Furthermore, we propose a Q-learning based approach to solve the
short-term problem efficiently. Finally, simulation results are conducted to
verify the correctness and effectiveness of the proposed algorithm.

###### Index Terms:

Maritime networks, Cooperative computation offloading framework, Lyapunov
optimization, Q-learning.

##  I Introduction

With the increasing demands from marine Internet of things (MIoTs) [1], the
limited marine network resources cannot satisfy multiple tasks. Fortunately,
the development of space-air-ground network can help figure out such resource
insufficient issues. Besides, the multi-access edge computing (MEC) [2, 3, 4]
technique can be introduced to handle the computation related requirements
from MIoTs, e.g. marine rescue. Due to the easy deployment and flexible
movement of unmanned aerial vehicles (UAVs), it can be employed to assist
MIoTs for rapid rescue [5]. However, UAVs cannot complete the compute-
intensive tasks due to the limited energy and computation resources.
Therefore, how to assist UAVs to handle compute-intensive tasks is
significant. Vessels can provide abundant computing and energy resources to
address the resource limitation of UAVs. Thus, the cooperation of UAVs and
vessels provides more flexible services for the maritime MEC tasks. However,
the following challenges still should be handled: 1) the position adjustment
of UAV cannot follow the uncertainty of user demands; 2) how to select an
appropriate vessel for a UAV to offload the selected tasks; 3) the arrival of
marine tasks is uncertain, and UAVs have no knowledge of the task information.
Hence, it is significant to design a reasonable offloading scheme involving
both UAVs and vessels for the marine MEC tasks.

There exist a couple of recent works related to MEC in UAV based scenarios.
For instance, Jia et al. [6] consider the cooperation of LEO satellites and
HAPs. The matching among users, HAPs, and satellites are solved with a
matching algorithm. In [7], Yang et al. use multi-armed bandits learning to
optimize offloading delay and energy cost. Thus, the reliability of marine
communications is ensured. Zhao et al. [8] study cooperative offloading
strategies under MEC for multiple UAVs, and model the problem as a Markov
decision process (MDP). The resource management of computation and
communication is implemented by multi-agent deep reinforcement learning to
minimize the sum of latency and energy cost. Ning et al. [9] build a
5G-enabled UAV-to-community offloading system to maximize the throughput. This
work designs an auction based algorithm to deal with the trajectory design, as
well as a community based scheduling algorithm to meet the transmission rate
demands. In [10], Sacco et al. solve the problem of task offloading via a
distributed architecture. The system learns the best actions from environments
to minimize the latency and energy cost to improve network service levels.
However, the aforementioned works lack considering the uncertain MEC tasks in
the marine environment, which is a significant issue.

In this work, we focus on the MEC related problem cooperated by UAVs and
vessels for marine applications. In detail, we propose an integrated space-
air-ground-marine architecture in order to clearly depict the resources and
demands in the marine scenario. Based on the proposed architecture, we further
formulate the studied problem into a mixed-integer program, to minimize both
the total execution time and energy cost. However, the formulated problem is
NP hard and prone to dimensional catastrophes at large scales [11]. To this
end, we first utilize the Lyapunov optimization technique to transform the
formulation problem into a set of per-time-slot optimization problems. Then,
we propose a Q-learning based method to solve each per-time-slot optimization
problem efficiently. Finally, numerical results verify that the proposed
algorithm can effectively guarantee solution effectiveness.

The rest of the paper is organized as follows. The system model and problem
formulation are presented in Section II. Then, a virtual queue-based
Q-learning approach (VQQ) is proposed in Section III. Numerical results are
provided in Section IV, and finally conclusions are drawn in Section V.

##  II SYSTEM MODEL AND PROBLEM FORMULATION

As shown in Fig. 1, an integrated space-air-ground-marine architecture
consisting of satellites, UAVs, vessels, and base stations (BSs) is provided.
Satellites provide signaling and navigation for the target area. BSs mainly
provide tasks commands for vessels and UAVs. The vessels are equipped with
multiple receiving antennas, powerful computing capability, etc. Hence,
vessels can perform as the mobile base stations. The MIoT devices in Fig 1
build multiple tasks with computation demands. However, the limited computing
and energy ability of MIoT cannot complete the local computation. Therefore,
UAVs can collect the data from MIoT for lightweight computing. Besides, due to
the limited payloads of UAVs, the compute-intensive demands can be relayed to
vessels by the UAV.

The components in the system include MIoTs, UAVs, and vessels. MIoTs are
denoted as $i\in\mathcal{M}=\\{1,2,3,...,I\\}$, UAVs are represented as
$j\in\mathcal{U}=\\{1,2,3,...,J\\}$, and vessels are indicated by
$k\in\mathcal{V}=\\{1,2,3,...,K\\}$. The vessels are equipped with
$N_{k}^{max}$ antennas, denoting the maximum number of UAVs that can
communicate simultaneously. The total time is depicted as $\mathcal{T}$ and
divided into $T$ time periods with length $l_{0}$ of each time slot. We
utilize a 3D Cartesian coordinate system to represent the positions of MIoTs,
UAVs, and vessels, as $m_{i}(t)=(x_{i}^{m},y_{i}^{m},h_{i}^{m})$,
$u_{j}(t)=(x_{j}^{u},y_{j}^{u},h_{j}^{u})$, and
$v_{k}(t)=(x_{k}^{v},y_{k}^{v},h_{k}^{v})$, respectively. The tasks generated
by the MIoTs in time slot $t$ is
$\lambda_{i}^{m}(t)=\\{l_{i}^{m}(t),d_{i}^{m}(t)\\}$, in which, $l_{i}^{m}(t)$
(in bits) indicates the input data size, and $d_{i}^{m}(t)$ (in CPU
cycles/bit) denotes the number of CPU cycles to process data. Considering the
practical situation of limited UAV capacity, the maximum capacity of UAV $j$
is denoted as $S_{j}^{max}$.

In time slot $t$, the task offloading decision should be made for each new
arrival task, and we define the task offloading decision
$\mathcal{O}=[o_{i,j}]_{I\times J}$, $\mathcal{S}=[s_{i,k}]_{J\times K}$ as:

$o_{i,j}(t)=\begin{cases}1,&\text{ if task }\lambda_{i}^{m}(t)\text{ is
offloaded to UAV }j,\\\ 0,&\text{ otherwise,}\end{cases}$ (1)

and

$s_{i,k}(t)=\begin{cases}1,&\text{ if task }\lambda_{i}^{m}(t)\text{ is
offloaded to vessel }k,\\\ 0,&\text{ otherwise. }\end{cases}$ (2)

The connection decision $\mathcal{Q}=[q_{i,j}]_{I\times J}$ between MIoT $i$
and UAV $j$ is:

$q_{i,j}(t)=\begin{cases}1,&\text{if MIoT }i\text{ is connected to UAV }j,\\\
0,&\text{ otherwise. }\end{cases}$ (3)

Besides, some tasks should be offloaded to vessels relayed by UAVs, the
connection $\mathcal{P}=[p_{j,k}]_{J\times K}$ between UAV $j$ and vessel $k$
is defined as:

$p_{j,k}(t)=\begin{cases}1,&\text{ if UAV }j\text{ is connected to vessel
}k,\\\ 0,&\text{ otherwise. }\end{cases}$ (4)

Figure 1: System overview.

Note that we assume UAVs communicate in full-duplex mode, i.e., the upload and
download of tasks can be performed simultaneously. Moreover, since the size of
computed data is much smaller than the original data, the orthogonal frequency
division multiple access technique is leveraged in backhaul transmission.
Hence, the result can be sent back within one time slot, and the corresponding
delay is omitted.

(a) Tasks are generated from MIoTs. (b) Task 1 is being processed on the UAV.
Task 2 is offloaded to the vessel for processing. (c) Task 1 is completed and
transmitted back to MIoT. Task 2 is being processed on the UAV. Task 3 is
offloaded to the vessel for processing. Task 4 is being processed on the UAV.
(d) Task 2 is completed and transmitted back to UAV. Task 3 is being processed
on the vessel. Task 4 completion is returned to the MIoT.

Figure 2: An example of task offloading

###  II-A Communication Model

According to [5], $\xi_{a,b}(t)$ indicates the path loss from node $a$ to node
$b$, in which ($a\in\mathcal{M}$, $b\in\mathcal{U}$) or ($a\in\mathcal{U}$,
$b\in\mathcal{V}$). The path loss between two nodes in the marine environment
is:

$\displaystyle\xi_{a,b}(t)$
$\displaystyle=\frac{\zeta_{L}-\zeta_{NL}}{1+\alpha\exp\\{-\beta[\gamma_{a,b}(t)-\alpha\\}}$
(5)
$\displaystyle+20\lg\left(\frac{4\pi\|x_{a}-x_{b}\|f_{c}}{C_{0}}\right)+\zeta_{NL},$

in which

$\displaystyle\gamma_{a,b}(t)=\arctan\left(\frac{|h_{a}-h_{b}|}{\|x_{a}-x_{b}\|_{2}}\right),$
(6a)

where $x_{a},x_{b}$ denote the positions of transmitter $a$ and receiver $b$,
respectively. $h_{a}$ and $h_{b}$ represent the heights of transmitter and
receiver, respectively. $f_{c}$ indicates the carrier frequency, $C_{0}$ is
the speed of light. $\zeta_{L}$, $\zeta_{NL}$, $\alpha$, and $\beta$ represent
environmental parameters.

Following Shannon formula, the average rate between transmitter $a$ and
receiver $b$ is calculated as:

$R_{a,b}(t)=B_{0}\log_{2}[1+\frac{P_{a}\xi_{a,b}(t)}{N_{G}}],$ (7)

where $B_{0}$ is the bandwidth of the channel, $P_{a}$ represents the
transmitting power, and $N_{G}$ indicates the power of the additive white
Gaussian noise.

###  II-B Computation Model

####  II-B1 UAV based computation model

For each task generated by MIoT $i$ in time slot $t$, the total execution time
includes the execution time of the UAV-based computation, the queuing delay
and the task transmission time:

$\displaystyle T_{i,j}^{uc}(t)$
$\displaystyle=T_{i,j}^{ue}+T_{i,j}^{ut}+T_{i,j}^{uqd}$ (8)
$\displaystyle=\frac{d_{i}^{m}(t)}{f_{j}^{u}}+\frac{l_{i}^{m}(t)}{R_{i,j}(t)}+T_{i,j}^{vqd},$

where $T_{i,j}^{ue}$ is the execution time of UAV-based computation.
$T_{i,j}^{ut}$ represents the transmission time from MIoT $i$ to UAV $j$, and
$T_{i,j}^{uqd}$ denotes the queuing delay. $f_{j}^{u}$ indicates the computing
ability of UAV $j$, i.e., the CPU clock.

####  II-B2 Vessel based computation model

If a task is offloaded from UAV $j$ to vessel $k$, such a process requires
three procedures: task transmission, task execution, and task backhaul. Since
the backhaul time is much smaller than the transmission and execution time, it
is omitted and the total execution time is expressed as:

$\displaystyle T_{i,j,k}^{vc}$
$\displaystyle=T_{j,k}^{ve}+T_{i,j}^{ut}+T_{j,k}^{vt}+T_{j,k}^{vqd}$ (9)
$\displaystyle=\frac{d_{i}^{m}(t)}{f_{k}^{v}}+\frac{l_{i}^{m}(t)}{R_{i,j}(t)}+\frac{l_{i}^{m}(t)}{R_{j,k}(t)}+T_{i,k}^{vqd},$

where $T_{j,k}^{ve}$ is the computation time cost on vessel $k$, and
$T_{j,k}^{vt}$ denotes the task transmission time from UAV $j$ to vessel $k$.
$f_{k}^{v}$ represents the computing ability of vessel $k$, i.e., the CPU
clock.

As above, the total execution time for task generated by MIoT $i$ in time slot
$t$ is:

$T_{i}^{a}(t)=\sum_{j=1}^{J}\sum_{k=1}^{K}o_{i,j}(t)T_{i,j}^{uc}(t)+s_{i,k}(t)T_{i,j,k}^{vc}.$
(10)

During task offloading, the constraints related to the total execution time
should be considered, to ensure the effectiveness. Hence, the total execution
time related constraint is listed as:

$\lim_{T\rightarrow\infty}\frac{1}{T}\sum_{t=1}^{T}\frac{T_{i}^{a}(t)}{\overline{T_{i}^{a}}(t)}\leq\varphi_{max}^{t},\\\
$ (11)

where $\overline{T_{i}^{a}}(t)=\frac{1}{t}\sum_{i=0}^{t-1}T_{i}^{a}(t)$, and
$\varphi_{max}^{t}$ is the bounds of the total execution time.

###  II-C Energy Model

There exist three main energy-consuming modes of UAVs: hovering, trajectory
and for computation functions. The energy required for UAV $j$ trajectory is
[12]:

$P_{j}^{t}=\frac{v_{j}}{v_{j}^{\max}}\left[P_{j}^{\max}-P_{j}^{h}\right],$
(12)

where $v_{j}$ and $v_{j}^{max}$ denote the current speed and maximum speed,
respectively. $P_{j}^{\max}$ is the power at maximum speed and $P_{j}^{h}$ is
the power for hovering. The hovering energy cost of the UAV is:

$P_{j}^{h}=C_{0}\sqrt{\frac{\left(M_{j}^{u}\right)^{3}}{r_{j}^{2}\kappa_{j}}},$
(13)

where $g$ represents the acceleration of gravity, and $\theta$ denotes the air
density. $M_{j}^{u}$ represents the mass, $r_{j}$ is the propeller radius, and
$\kappa_{j}$ indicates the number of propellers. $C_{0}$ is the environmental
parameter, and $C_{0}=\sqrt{\frac{g^{3}}{2\pi\theta}}$.

The energy cost of a UAV to compute the task generated by MIoT $i$ at time
slot $t$ is:

$E_{i,j}^{uc}(t)=d_{i}^{m}(t)\varsigma_{j}^{u},$ (14)

where $\varsigma_{j}^{u}$ is the energy cost per unit computational resources.
The energy consumed by vessel $k$ to compute task $\lambda_{i}^{m}(t)$ is

$E_{i,j,k}^{vc}=d_{i}^{m}(t)\varsigma_{k}^{v},$ (15)

where $\varsigma_{k}^{v}$ is the energy cost of per unit of computational
resources of vessel $k$.

Hence, the energy cost of the task generated by MIoT $i$ in time slot $t$ is:

$\displaystyle E_{i}^{a}(t)$
$\displaystyle=\underbrace{\sum_{j=1}^{J}o_{i,j}(t)P_{j}^{t}\frac{\|u_{j}(t)-u_{j}(t+1)\|}{v_{j}}+P_{j}^{h}l_{0}}_{\text{energy
cost for UAV movement}}$ (16)
$\displaystyle+\underbrace{\sum_{j=1}^{J}\sum_{k=1}^{K}o_{i,j}(t)E_{i,j}^{uc}+\frac{s_{i,k}(t)P_{j}L_{i}^{m}(t)}{R_{j,k}(t)}}_{\text{UAV
based computation and communication cost}}$
$\displaystyle+\underbrace{\sum_{j=1}^{J}\sum_{k=1}^{K}s_{i,k}(t)E_{i,j,k}^{vc}}_{\text{vessel
based calculation cost}}.$

Besides, energy related constraints should be considered in terms of time
constraints. Therefore, task energy related constrained is represented as:

$\lim_{T\rightarrow\infty}\frac{1}{T}\sum_{t=1}^{T}\frac{E_{i}^{a}(t)}{\overline{E_{i}^{a}}(t)}\leq\varphi_{max}^{e},$
(17)

where $\overline{E_{i}^{a}}(t)=\frac{1}{t}\sum_{i=0}^{t-1}E_{i}^{a}(t)$, and
$\varphi_{max}^{e}$ is the bounds of energy constraints.

###  II-D Problem Formulation

The optimization problem is formulated to minimize the total execution time as
well as energy cost:

$\displaystyle\textbf{P0:}\min_{\mathcal{O},\mathcal{S},\mathcal{P},\mathcal{Q}}$
$\displaystyle\sum_{t=1}^{T}\sum_{i=1}^{I}\theta_{0}E_{i}^{a}(t)+\theta_{1}T_{i}^{a}(t),$
(18a) s.t.
$\displaystyle\sum_{j=1}^{J}o_{i,j}(t)+\sum_{k=1}^{K}s_{i,k}(t)=1,\forall
i,t,$ (18b) $\displaystyle\sum_{i=1}^{I}o_{i,j}(t)*l_{i}^{m}(t)+S_{j}(t)\leq
S_{j}^{max},\forall j,t,$ (18c) $\displaystyle\sum_{j=1}^{J}p_{j,k}(t)\leq
N_{k}^{max},\forall k,t,$ (18d)
$\displaystyle\|u_{j}(t)\\!-\\!u_{j}(t\\!+\\!1)\|\\!\leq\\!l_{0}V_{j}^{max}\\!,\\!\forall
t\in[0,T\\!-\\!1],j,$ (18e) $\displaystyle
o_{i,j}(t),s_{i,k}(t),q_{i,j}(t),p_{j,k}(t)\in\\{0,1\\},\forall i,j,k,t,$
(18f) $\displaystyle(\ref{e11})\ \text{and}\ (\ref{e17}),\forall i,j,k,$

where $\theta_{0}$ and $\theta_{1}$ indicate the weights of energy cost and
the total execution time, respectively. Constraint (18b) ensures that a task
can be implemented by only one UAV or vessel. Constraint (18c) is a storage
constraint, denoting at any time slot, a UAV cannot store more tasks than its
own storage capacity. Constraint (18d) denotes a vessel connection constraint
i.e., the maximum number of connections of a vessel cannot exceed
$N_{k}^{max}$. Constraint (18e) indicates that the maximum moving distance for
a UAV in two adjacent time slots is limited by the speed of UAV. Constraint
(18f) indicates the binary indicators. Due to the binary variables, P0 is non-
convex and the solution space grows exponentially with the problem scale,
which is intractable to solve by a traditional optimization approach.

##  III DEEP REINFORCEMENT LEARNING-BASED TASK OFFLOADING

Since both the constraints of energy cost and time cost hold for a long time
$\mathcal{T}$, i.e., long-term constraints. Hence, $\mathcal{T}$ is decoupled
as multiple time slots, and each time slot is short-term $t$. Then, by
leveraging the Lyapunov optimization, the long-term stochastic optimization
problem can be decoupled into short-term deterministic optimization
subproblems.

###  III-A Problem Transformation

In order to transform the constraints in P0 within the same time slot, we
convert (11) and (17) into virtual queues, detailed as:

$V_{i}^{t}(t+1)=\max\\{V_{i}^{t}(t)+\frac{T_{i}^{a}(t)}{\overline{T_{i}^{a}}(t)}-\varphi_{max}^{t},0\\},\\\
$ (19)

and

$V_{i}^{e}(t+1)=\max\\{V_{i}^{e}(t)+\frac{E_{i}^{a}(t)}{\overline{E_{i}^{a}}(t)}-\varphi_{max}^{e},0\\}.$
(20)

Thus, the original problem P0 is converted to the cross-time slot problem P1.
Therefore, P1 can be solved by per-time-slot strategy, which guarantee the
time and energy constraints:

P1:
$\displaystyle\min_{\mathcal{O}(t),\mathcal{S}(t),\mathcal{Q}(t),\mathcal{P}(t)}\sum_{t=1}^{T}\sum_{i=1}^{I}\mathfrak{A}_{i,j,k}(t),$
(21) $\displaystyle\text{s.t. }18(b)-18(f),$

where $\mathfrak{A}$ is:

$\displaystyle\mathfrak{A}_{i,j,k}(t)=V_{i}^{t}(t)(\frac{T_{i}^{a}(t)}{\overline{T_{i}^{a}}(t)}-\varphi_{max}^{t})+V_{i}^{e}(t)(\frac{E_{i}^{a}(t)}{\overline{E_{i}^{a}}(t)}-\varphi_{max}^{e}).$
(22)

To figure out P1, it is necessary to design a task offloading scheme, and each
MIoT generates a task in different time slots and offloads to a UAV or a
vessel, to minimize $\mathfrak{A}_{i,j,k}(t)$. Hence, P1 is transformed as a
MDP problem [13].

Algorithm 1 A virtual queue-based Q-learning approach.

0: State $S(t)$, epochs $N$, and $\epsilon$.

0: $\mathcal{O}(t)$, $\mathcal{S}(t)$, $\mathcal{Q}(t)$, and $\mathcal{P}(t)$.

1: Randomly generate initial values of $s(t)$ and $\mu$, and create neural
networks with $\mu$.

2: for each $k=1:N$ do

3: for each $t=1:T$ do

4: if rand$(0,1)<\epsilon$ then

5: Stochastically select action.

6: else

7: Select the action with the largest Q value.

8: end if

9: Change the connections according to $\mathcal{O}(t)$ and $\mathcal{S}(t)$.

10: Offload tasks according to $\mathcal{Q}(t)$ and $\mathcal{P}(t)$.

11: Compute the reward of the task and loss function.

12: Update $\mu=\mu+\phi\triangledown_{\mu}loss(\mu)$.

13: Update $s(t+1)$ and move to the next state.

14: end for

15: end for

In particular, the process of MDP is defined as: state, action, reward and
transfer probability, as follows:

  1. 1.

State: The state at time slot $t$ is defined as
$s(t)=\\{T_{i}^{a}(t),E_{i}^{a}(t),V_{i}^{t}(t),V_{e}^{t}(t)\\}$, which
represents the energy cost and time consumption in the current time slot.

  2. 2.

Action: The action in time slot $t$ is defined as
$a(t)=\\{\mathcal{O}(t),\mathcal{S}(t),\mathcal{Q}(t),\mathcal{P}(t)\\}$,
which denotes the offloading and connection decision.

  3. 3.

Reward: As for the optimization problem, we set the reward as
$r(t)=-\mathfrak{A}_{i,j,k}(t)$.

  4. 4.

Transfer probability: The probability of moving from the current state to the
next state is $p(s(t+1)\mid s(t),a(t))$.

In MDP, the state space solution tends to deteriorate due to the curse of
dimensionality. Then, we propose the reinforcement learning-based model to
figure out the intractable issue.

###  III-B Algorithm Design

The Q-learning based mechanism is utilized to solve the current problem via
selecting the Q value of action $a(t)$ in state $s(t)$, and the Q value is
defined as:

$\displaystyle Q(s(t),a(t))=$ $\displaystyle Q(s(t),a(t))+\phi[r({t+1})+$ (23)
$\displaystyle\psi\max_{a}Q(s({t+1}),a)-Q(s(t),a(t))],$

where $\phi,R,\psi$ denote the learning rate, reward and discount factors,
respectively.

In order to simplify the scale of the computation and reduce the subsequent
storage of Q values, we design the deep Q-network method, leveraging a deep
neural network to map the relationship between the current state and the
action. Such a relationship denotes Q values, and the deep neural network is
utilized to gradually approximate the current parameter $\mu$:

$Q(s(t),a(t),\mu)\approx
Q_{\mathcal{O},\mathcal{S},\mathcal{P},\mathcal{Q}}(s(t),a(t)).$ (24)

Further, the loss function is defined as:

$loss(\mu)=(r(t)+\psi\max_{a}Q(s({t+1}),a)-Q(s(t),a(t)))^{2}.$ (25)

The VQQ is detailed in Algorithm 1. Firstly, we assume that there are $N$
epochs and each episode consists of $T$ time slots. At the beginning of each
time slot, the offloading policy and linking policy are chosen according to
the random probability $\epsilon-$greedy (line 4-line 7). Then, the tasks are
offloaded to the selected UAV or vessel according to the action (line 9-line
10). Based on (21) and (25), the reward and loss function are calculated for
this time slot $t$ (line 11). Finally, the value of the next state is updated
(line 12-line 13).

##  IV SIMULATIONS AND PERFORMANCE ANALYSIS

Figure 3: Effect of number of MIoT on latency. Figure 4: Effect of number of
MIoT on time cost.

Simulations are conducted in the following scenario: 3 vessels, 10 UAVs and
$I$ MIoTs are evenly distributed in a region of 5km$\times$5km. The vessels
and MIoTs heights are set as 0 and the UAV’s height set is 100m. Python is
employed to implement the proposed algorithm VQQ. In addition, other
parameters are set as: $T=$ 100, $l_{0}=$ 1s, $l_{i}^{m}(t)\in$ [2.4,
2.6]Mbits, $d_{i}^{m}(t)\in$ [2.4, 3.6]Mbits, $B_{0}=$ 1Mhz, $\zeta_{L}=$ 2.3,
$\zeta_{NL}=$ 34, $\alpha=$ 5.0188, $\beta=$ 0.3511, $P_{a}=$ 10W, $N_{G}=$
-114dBm, $f_{c}=$ 1Mhz, $f_{j}^{u}=$ 1MHz, $f_{k}^{v}=$ 1GHz,
$\varsigma_{j}^{u}=$ 1.87J/Mb, $\varsigma_{k}^{v}=$ 1.87J/Mb,
$\varphi_{max}^{t}=$ 0.99, $\varphi_{max}^{e}=$ 0.99, $S_{j}^{max}=$ 20Mbits,
$N_{k}^{max}=$5, $\phi\in$ [0,1], $\psi\in$ [0,1], and $V_{j}^{max}=$12m/s
[14, 5].

To evaluate the performance of VQQ, the optimization results for VQQ, the
algorithm Dream from [14] and greedy algorithm are compared in Fig. 3 and Fig.
4. It is obvious that the latency and energy cost increase with the increment
number of MIoT. Besides, compared with the Dream and greedy algorithm, both
the latency and energy cost of VQQ are lower. It is explained that Dream does
not take into account the location change of UAVs, and the greedy algorithm
always offloads the task to the nearest UAVs. At 30 MIoTs, the latency and
energy cost depends mainly on the speed of task transmission. The VQQ performs
slightly better than Dream and significantly better than the greedy algorithm.
Thus, the effectiveness of the VQQ is proved from Fig. 3 and Fig. 4.
Additionally, Fig. 5 and Fig. 6 show the convergence at $I\in\\{10,20,30\\}$.
With the influence from the environment and the reward from an action, VQQ can
quickly converge and gradually become stable. Thus, the fast convergence
reveals the efficiency of VQQ.

Figure 5: Normalized time consumption v.s. episode. Figure 6: Normalized
energy cost v.s. episode.

##  V CONCLUSIONS

In this paper, we investigate the cooperation between UAVs and vessels in
offloading decisions for maritime communications. Then, we focus on minimizing
the total execution time and energy cost. However, the problem is in the form
of mixed-integer programming, and NP hard to solve. Hence, to optimize the
problem of offloading under uncertain tasks. We decouple the long-term
constraints and use them as a reward for Q-learning. Then, we design the
algorithm of VQQ to efficiently obtain the optimal solution. Finally, the
simulation results verify the efficiency and convergence of VQQ by compared
with two reference algorithms.

## References

  * [1] C. Zhu, W. Zhang, Y.-H. Chiang, N. Ye, L. Du, and J. An, “Software-defined maritime fog computing: Architecture, advantages, and feasibility,” _IEEE Netw._ , vol. 36, no. 2, pp. 26–33, Mar. 2022.
  * [2] Z. Jia, Q. Wu, C. Dong, C. Yuen, and Z. Han, “Hierarchical aerial computing for internet of things via cooperation of HAPs and UAVs,” _IEEE Internet Things J._ , Feb. 2022, early access.
  * [3] C. Dong, Y. Shen, Y. Qu, K. Wang, J. Zheng, Q. Wu, and F. Wu, “UAVs as an intelligent service: Boosting edge intelligence for air-ground integrated networks,” _IEEE Netw_ , vol. 35, no. 4, pp. 167–175, Aug. 2021.
  * [4] Z. Jia, M. Sheng, J. Li, D. Niyato, and Z. Han, “LEO-satellite-assisted UAV: Joint trajectory and data collection for Internet of remote things in 6G aerial access networks,” _IEEE Internet Things J._ , vol. 8, no. 12, pp. 9814–9826, Jun. 2021.
  * [5] Y. Wang, W. Feng, J. Wang, and T. Q. S. Quek, “Hybrid satellite-UAV-terrestrial networks for 6G ubiquitous coverage: A maritime communications perspective,” _IEEE J. Sel. Areas Commun._ , vol. 39, no. 11, pp. 3475–3490, Nov. 2021.
  * [6] Z. Jia, M. Sheng, J. Li, D. Zhou, and Z. Han, “Joint HAP access and HAP satellite backhaul in 6G: Matching game-based approaches,” _IEEE J. Sel. Areas Commun._ , vol. 39, no. 4, pp. 1147–1159, Aug. 2021.
  * [7] T. Yang, S. Gao, J. Li, M. Qin, X. Sun, R. Zhang, M. Wang, and X. Li, “Multi-armed bandits learning for task offloading in maritime edge intelligence networks,” _IEEE Trans. Veh. Technol._ , vol. 71, no. 4, pp. 4212–4224, Apr. 2022.
  * [8] N. Zhao, Z. Ye, Y. Pei, Y.-C. Liang, and D. Niyato, “Multi-agent deep reinforcement learning for task offloading in UAV-assisted mobile edge computing,” _IEEE Trans. Wirel. Commun._ , vol. 21, no. 9, pp. 6949–6960, Sep. 2022.
  * [9] Z. Ning, P. Dong, M. Wen, X. Wang, L. Guo, R. Y. K. Kwok, and H. V. Poor, “5G-enabled UAV-to-community offloading: Joint trajectory design and task scheduling,” _IEEE J. Sel. Areas Commun._ , vol. 39, no. 11, pp. 3306–3320, Nov. 2021.
  * [10] A. Sacco, F. Esposito, G. Marchetto, and P. Montuschi, “Sustainable task offloading in UAV networks via multi-agent reinforcement learning,” _IEEE Trans. Veh. Technol._ , vol. 70, no. 5, pp. 5003–5015, May 2021.
  * [11] K. Shuai, Y. Miao, K. Hwang, and Z. Li, “Transfer reinforcement learning for adaptive task offloading over distributed edge clouds,” _IEEE Trans. on Cloud Comput._ , Jul. 2022, early access.
  * [12] A. A. Al-Habob, O. A. Dobre, S. Muhaidat, and H. V. Poor, “Energy-efficient information placement and delivery using UAVs,” _IEEE Internet Things J._ , Aug. 2022, early access.
  * [13] S. Zhu, L. Gui, D. Zhao, N. Cheng, Q. Zhang, and X. Lang, “Learning-based computation offloading approaches in UAVs-assisted edge computing,” _IEEE Trans. Veh. Technol._ , vol. 70, no. 1, pp. 928–944, Jan. 2021.
  * [14] Z. Jia, Z. Zhou, X. Wang, and S. Mumtaz, “Learning-based queuing delay-aware task offloading in collaborative vehicular networks,” in _IEEE International Conference on Communications_ , Montreal, QC, Canada, Jun. 2021.


$\displaystyle\leq\frac{1}{s}\left(-\frac{\theta}{2}+\frac{C_{\mathtt{N}}}{s}\right)\|\eta_{t,s}\,g_{J}\psi_{t}\|_{0}^{2}+\frac{C_{\mathtt{N}}}{s^{\mathtt{N}}}\|\,g_{J}\psi_{t}\|_{0}^{2}$

So, for $s\geq 1$ sufficiently large, the first term in the right hand side
above is negative and, using also that $e^{-{\rm i}t\mathsf{H}}$ is unitary
and commutes with $g_{J}\equiv g_{J}(\mathsf{H})$, we get

$\frac{d}{dt}\|(\chi_{t,s})^{\frac{1}{2}}\,g_{J}\psi_{t}\|_{0}^{2}\leq\frac{C_{\mathtt{N}}}{s^{\mathtt{N}}}\|\,g_{J}\psi_{0}\|_{0}^{2}\
.$

Integrating this inequality between $0$ and $t$ we find $\forall t>0$

$\|\chi^{\frac{1}{2}}\Big{(}\frac{\mathsf{A}-a-\vartheta
t}{s}\Big{)}\,g_{J}\psi_{t}\|_{0}^{2}\leq\|\chi^{\frac{1}{2}}\Big{(}\frac{\mathsf{A}-a}{s}\Big{)}\,g_{J}\psi\|_{0}^{2}+\frac{C_{\mathtt{N}}\,t}{s^{\mathtt{N}}}\|\,g_{J}\psi\|_{0}^{2},$

uniformly for $a\in{\mathbb{R}}$ and $s\geq 1$ sufficiently large. We evaluate
this inequality at $a=-\frac{\vartheta}{2}t$ and $s=\sqrt{t}$, obtaining for
$t\geq 1$ sufficiently large, the minimal velocity estimate

$\|\chi^{\frac{1}{2}}\Big{(}\frac{\mathsf{A}-\frac{\vartheta}{2}t}{\sqrt{t}}\Big{)}\,g_{J}\psi_{t}\|_{0}\leq\|\chi^{\frac{1}{2}}\Big{(}\frac{\mathsf{A}+\frac{\vartheta}{2}t}{\sqrt{t}}\Big{)}\,g_{J}\psi\|_{0}+C_{\mathtt{N}}\,t^{-\frac{{\mathtt{N}}}{4}+\frac{1}{2}}\|\,g_{J}\psi\|_{0}\
.$ (C.20)

To conclude, take $k\in{\mathbb{N}}$ and consider
$\|\left\langle\mathsf{A}\right\rangle^{-k}\,g_{J}\,\psi_{t}\|_{0}$. Clearly

$\displaystyle\|\left\langle\mathsf{A}\right\rangle^{-k}\,g_{J}\,\psi_{t}\|_{0}$
$\displaystyle\leq\|\chi^{\frac{1}{2}}\Big{(}\frac{\mathsf{A}-\frac{\vartheta}{2}t}{\sqrt{t}}\Big{)}\,\left\langle\mathsf{A}\right\rangle^{-k}\,g_{J}\,\psi_{t}\|_{0}$
(C.21)
$\displaystyle\quad+\|\Big{(}1-\chi^{\frac{1}{2}}\Big{(}\frac{\mathsf{A}-\frac{\vartheta}{2}t}{\sqrt{t}}\Big{)}\,\Big{)}\,\left\langle\mathsf{A}\right\rangle^{-k}\,g_{J}\,\psi_{t}\|_{0}$
(C.22)

We estimate first (C.22). By Theorem B.2 (ii) we have

$\|\Big{(}1-\chi^{\frac{1}{2}}\Big{(}\frac{\mathsf{A}-\frac{\vartheta}{2}t}{\sqrt{t}}\Big{)}\,\Big{)}\,\left\langle\mathsf{A}\right\rangle^{-k}\|_{{\mathcal{L}}({\mathcal{H}})}\leq\sup_{\lambda\in{\mathbb{R}}}\left|\Big{(}1-\chi^{\frac{1}{2}}\Big{(}\frac{\lambda-\frac{\vartheta}{2}t}{\sqrt{t}}\Big{)}\Big{)}\,\left\langle\lambda\right\rangle^{-k}\right|\leq
C_{k}\left\langle t\right\rangle^{-k}.$ (C.23)

To prove the last inequality, use that for $\lambda\geq\frac{\vartheta}{4}t$
then $\left\langle\lambda\right\rangle^{-k}\leq\left\langle
t\right\rangle^{-k}$, whereas when $\lambda<\frac{\vartheta}{4}t$ then, being
$\lambda\mapsto 1-\chi^{\frac{1}{2}}(\lambda)$ monotone increasing and
exponentially decaying at $-\infty$,

$1-\chi^{\frac{1}{2}}\Big{(}\frac{\lambda-\frac{\vartheta}{2}t}{\sqrt{t}}\Big{)}\leq
1-\chi^{\frac{1}{2}}\Big{(}-\frac{\vartheta}{4}\sqrt{t}\Big{)}\leq
C\left(e^{-\frac{\vartheta}{4}\sqrt{t}}\right)^{\frac{1}{2}}\leq
C_{k}\left\langle t\right\rangle^{-k}.$

Next we estimate (C.21) using the minimal velocity estimate. As
$\left\langle\mathsf{A}\right\rangle^{-k}$ is a bounded operator,

$\displaystyle\|\chi^{\frac{1}{2}}\Big{(}\frac{\mathsf{A}-\frac{\vartheta}{2}t}{\sqrt{t}}\Big{)}\,\left\langle\mathsf{A}\right\rangle^{-k}\,g_{J}\,\psi_{t}\|_{0}$
$\displaystyle\leq\|\chi^{\frac{1}{2}}\Big{(}\frac{\mathsf{A}-\frac{\vartheta}{2}t}{\sqrt{t}}\Big{)}\,g_{J}\,\psi_{t}\|_{0}$
$\displaystyle\stackrel{{\scriptstyle\eqref{min61}}}{{\leq}}\|\chi^{\frac{1}{2}}\Big{(}\frac{\mathsf{A}+\frac{\vartheta}{2}t}{\sqrt{t}}\Big{)}\,g_{J}\psi\|_{0}+C_{\mathtt{N}}\,t^{-\frac{{\mathtt{N}}}{4}+\frac{1}{2}}\|\,g_{J}\psi\|_{0}$
$\displaystyle\leq\|\chi^{\frac{1}{2}}\Big{(}\frac{\mathsf{A}+\frac{\vartheta}{2}t}{\sqrt{t}}\Big{)}\,\left\langle\mathsf{A}\right\rangle^{-k}\|_{{\mathcal{L}}({\mathcal{H}})}\,\|\left\langle\mathsf{A}\right\rangle^{k}\,g_{J}\psi\|_{0}+C_{\mathtt{N}}\,t^{-\frac{{\mathtt{N}}}{4}+\frac{1}{2}}\|\,g_{J}\psi\|_{0}$

Again we have

$\|\chi^{\frac{1}{2}}\Big{(}\frac{\mathsf{A}+\frac{\vartheta}{2}t}{\sqrt{t}}\Big{)}\,\left\langle\mathsf{A}\right\rangle^{-k}\|_{{\mathcal{L}}({\mathcal{H}})}\leq\sup_{\lambda\in{\mathbb{R}}}\left|\chi^{\frac{1}{2}}\Big{(}\frac{\lambda+\frac{\vartheta}{2}t}{\sqrt{t}}\Big{)}\,\left\langle\lambda\right\rangle^{-k}\right|\leq
C_{k}\left\langle t\right\rangle^{-k},$ (C.24)

since for $\lambda\leq-\frac{\vartheta}{4}t$ one has
$\left\langle\lambda\right\rangle^{-k}\leq C\left\langle t\right\rangle^{-k}$,
whereas in case $\lambda>-\frac{\vartheta}{4}t$, as
$\lambda\mapsto\chi^{\frac{1}{2}}(\lambda)$ is monotone decreasing
exponentially fast at $+\infty$, one has

$\chi^{\frac{1}{2}}\Big{(}\frac{\lambda+\frac{\vartheta}{2}t}{\sqrt{t}}\Big{)}\leq\chi^{\frac{1}{2}}\Big{(}\frac{\vartheta}{4}\sqrt{t}\Big{)}\leq
C\left(e^{-\frac{\vartheta}{4}\sqrt{t}}\right)^{\frac{1}{2}}\leq
C_{k}\left\langle t\right\rangle^{-k}.$

Altogether, from (C.21), (C.22) we have proved that for $t\geq 1$ sufficiently
large,

$\displaystyle\|\left\langle\mathsf{A}\right\rangle^{-k}\,g_{J}\,\psi_{t}\|_{0}$
$\displaystyle\leq C_{k}\left\langle
t\right\rangle^{-k}\|g_{J}\psi_{t}\|_{0}+C_{k}\left\langle
t\right\rangle^{-k}\|\left\langle\mathsf{A}\right\rangle^{k}g_{J}\psi\|_{0}+C_{\mathtt{N}}\,t^{-\frac{{\mathtt{N}}}{4}+\frac{1}{2}}\|\,g_{J}\psi\|_{0}$
$\displaystyle\leq C_{k}\left\langle
t\right\rangle^{-k}\,\|\left\langle\mathsf{A}\right\rangle^{k}g_{J}\psi\|_{0}$

provided ${\mathtt{N}}=4k+2$. This proves the estimate (C.1) for $t\geq 1$
sufficiently large, and it is also clearly true for $t$ in any bounded
interval.

∎

## References

  * [1] P. Anselone.  Collectively compact operator approximation theory and applications to integral equations.  Prentice-Hall, Inc., Englewood Cliffs, N. J., 1971.
  * [2] J. Arbunich, F. Pusateri, I.M. Sigal, A. Soffer.  Growth of Sobolev norms for linear Schrödinger operators.  ArXiv e-print, arXiv:2011.04570, 2020\.
  * [3] V. Bach, J. Fröhlich, I.M. Sigal, and A. Soffer.  Positive commutators and the spectrum of Pauli-Fierz Hamiltonian of atoms and molecules.  Comm. Math. Phys., 207(3):557–587, 1999.
  * [4] D. Bambusi.  Reducibility of 1-d Schrödinger equation with time quasiperiodic unbounded perturbations, I.  Trans. Amer. Math. Soc., 370(3):1823–1865, 2018.
  * [5] D. Bambusi, B. Grébert, A. Maspero, and D. Robert.  Growth of Sobolev norms for abstract linear Schrödinger equations.  J. Eur. Math. Soc. (JEMS), 2020. doi: 10.4171/JEMS/1017
  * [6] D. Bambusi, B. Grébert, A. Maspero, and D. Robert.  Reducibility of the quantum harmonic oscillator in d-dimensions with polynomial time-dependent perturbation.  Anal. PDE, 11(3):775–799, 2018.
  * [7] D. Bambusi, B. Langella, and R. Montalto.  Growth of Sobolev norms for unbounded perturbations of the Laplacian on flat tori.  ArXiv e-print, arXiv:2012.02654, 2020.
  * [8] M. Berti and A. Maspero.  Long time dynamics of Schrödinger and wave equations on flat tori.  J. Diff. Eq., 267(2):1167 – 1200, 2019.
  * [9] J. Bourgain.  Growth of Sobolev norms in linear Schrödinger equations with quasi-periodic potential.  Comm. Math. Phys., 204(1):207–247, 1999.
  * [10] J. Bourgain.  On growth of Sobolev norms in linear Schrödinger equations with smooth time dependent potential.  J. Anal. Math., 77:315–348, 1999.
  * [11] O. Chodosh.  Infinite matrix representations of isotropic pseudodifferential operators.  Methods Appl. Anal., 18(4):351–371, 2011.
  * [12] Y. Colin de Verdière.  Spectral theory of pseudodifferential operators of degree 0 and an application to forced linear waves.  Anal. PDE, 13(5):1521–1537, 2020.
  * [13] Y. Colin de Verdière and Laure Saint-Raymond.  Attractors for two-dimensional waves with homogeneous Hamiltonians of degree 0.  Comm. Pure Appl. Math., 73(2):421–462, 2020.
  * [14] H. Cycon, R. Froese, W. Kirsch, and B. Simon.  Schrödinger operators with application to quantum mechanics and global geometry.  Texts and Monographs in Physics. Springer-Verlag, Berlin, study edition, 1987.
  * [15] E. Davies.  The functional calculus.  J. London Math. Soc. (2), 52(1):166–176, 1995.
  * [16] J.-M. Delort.  Growth of Sobolev norms of solutions of linear Schrödinger equations on some compact manifolds.  Int. Math. Res. Not. IMRN, (12):2305–2328, 2010.
  * [17] J.-M. Delort.  Growth of Sobolev norms for solutions of time dependent Schrödinger operators with harmonic oscillator potential.  Comm. Partial Differential Equations, 39(1):1–33, 2014.
  * [18] J. Dereziński and C. Gérard.  Scattering theory of classical and quantum $N$-particle systems.  Texts and Monographs in Physics. Springer-Verlag, Berlin, 1997.
  * [19] M. Dimassi and J. Sjostrand.  Spectral Asymptotics in the Semi-Classical Limit.  London Mathematical Society Lecture Note Series. Cambridge University Press, 1999.
  * [20] S. Dyatlov and M. Zworski.  Microlocal analysis of forced waves.  Pure Appl. Anal. , 1(3): 359–384, 2019.
  * [21] P. Duclos, O. Lev, and P. Sťovíček.  On the energy growth of some periodically driven quantum systems with shrinking gaps in the spectrum.  J. Stat. Phys., 130(1):169–193, 2008.
  * [22] E. Faou and P. Raphael.  On weakly turbulent solutions to the perturbed linear harmonic oscillator.  ArXiv e-print, arXiv:2006.08206, 2020\.
  * [23] C. Gérard and I. M. Sigal.  Space-time picture of semiclassical resonances.  Comm. Math. Phys., 145(2):281–328, 1992.
  * [24] E. Grenier, T. Nguyen, F. Rousset, and A. Soffer.  Linear inviscid damping and enhanced viscous dissipation of shear flows by using the conjugate operator method.  J. Funct. Anal., 278(3):108339, 27, 2020.
  * [25] E. Haus and A. Maspero.  Growth of Sobolev norms in time dependent semiclassical anharmonic oscillators.  J. Funct. Anal. , 278(2), 108316, 2020.
  * [26] B. Helffer and J. Sjöstrand.  Équation de Schrödinger avec champ magnétique et équation de Harper.  In Schrödinger operators, volume 345 of Lecture Notes in Phys., 118–197. Springer, Berlin, 1989.
  * [27] L. Hörmander.  The Weyl calculus of pseudodifferential operators.  Comm. Pure Appl. Math., 32(3):360–444, 1979.
  * [28] L. Hörmander.  The analysis of linear partial differential operators I-IV.  Grundlehren der mathematischen Wissenschaften 256. Springer-Verlag, 1985\.
  * [29] W. Hunziker and I. M. Sigal.  Time-dependent scattering theory of n-body quantum systems.  Reviews in Mathematical Physics, 12(08):1033–1084, 2000.
  * [30] W. Hunziker, I. M. Sigal, and A. Soffer.  Minimal escape velocities.  Comm. Partial Differential Equations, 24(11-12):2279–2295, 1999\.
  * [31] A. Jensen, É. Mourre, and P. Perry.  Multiple commutator estimates and resolvent smoothness in quantum scattering theory.  Ann. Inst. H. Poincaré Phys. Théor., 41(2):207–225, 1984\.
  * [32] T. Kato.  Perturbation theory for linear operators.  Classics in Mathematics. Springer-Verlag, Berlin, 1995.
  * [33] A. Maspero and D. Robert.  On time dependent Schrödinger equations: Global well-posedness and growth of Sobolev norms.  J. Fun. Anal., 273(2):721 – 781, 2017.
  * [34] A. Maspero.  Lower bounds on the growth of Sobolev norms in some linear time dependent Schrödinger equations.  Math. Res. Lett., 26(4):1197–1215, 2019.
  * [35] R. Montalto.  Growth of Sobolev norms for time dependent periodic Schrödinger equations with sublinear dispersion.  J. Diff. Eq., 266(8):4953 – 4996, 2019.
  * [36] E. Mourre.  Absence of singular continuous spectrum for certain selfadjoint operators.  Comm. Math. Phys., 78(3):391–408, 1980/81.
  * [37] G. Nenciu.  Adiabatic theory: stability of systems with increasing gaps.  Annales de l’I. H. P, 67-4:411–424, 1997.
  * [38] M. Reed and B. Simon.  Methods of modern mathematical physics. I.  Academic Press, Inc., New York, second edition, 1980.  Functional analysis.
  * [39] J. Saranen and G. Vainikko.  Periodic integral and pseudodifferential equations with numerical approximation.  Springer Monographs in Mathematics. Springer-Verlag, Berlin, 2002.
  * [40] M. Shubin.  Pseudodifferential operators and spectral theory.  Springer-Verlag, Berlin, second edition, 2001.
  * [41] I.M. Sigal and A. Soffer.  Local decay and velocity bounds for quantum propagation.  preprint (Princeton), 1988. http://www.math.toronto.edu/sigal/publications/SigSofVelBnd.pdf
  * [42] E. Skibsted.  Propagation estimates for $N$-body Schroedinger operators.  Comm. Math. Phys., 142(1):67–98, 1991.
  * [43] C. Sogge.  Hangzhou Lectures on Eigenfunctions of the Laplacian.  Princeton University Press, 2014.
  * [44] L. Thomann.  Growth of Sobolev norms for linear Schrödinger operators.  ArXiv e-print, arXiv:2006.02674, 2020\.