# Task-oriented Semantics-aware Communications for Robotic Waypoint
Transmission: the Value and Age of Information Approach

Wenchao Wu, Yuanqing Yang, Yansha Deng, and A. Hamid Aghvami W. Wu, Y. Yang,
Y. Deng, and A. Hamid Aghvami (Emeritus Professor) are with the Department of
Engineering, King’s College London, Strand, London WC2R 2LS, U.K. (e-mail:
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>(Corresponding author: Yansha Deng).

###### Abstract

The ultra-reliable and low-latency communication (URLLC) service of the fifth-
generation (5G) mobile communication network struggles to support safe robot
operation. Nowadays, the sixth-generation (6G) mobile communication network is
proposed to provide hyper-reliable and low-latency communication to enable
safer control for robots. However, current 5G/ 6G research mainly focused on
improving communication performance, while the robotics community mostly
assumed communication to be ideal. To jointly consider communication and
robotic control with a focus on the specific robotic task, we propose task-
oriented and semantics-aware communication in robotic control (TSRC) to
exploit the context of data and its importance in achieving the task at both
transmitter and receiver. At the transmitter, we propose a deep reinforcement
learning algorithm to generate optimal control and command (C&C) data and a
proactive repetition scheme (DeepPro) to increase the successful transmission
probability. At the receiver, we design the value of information (VoI) and age
of information (AoI) based queue ordering mechanism (VA-QOM) to reorganize the
queue based on the semantic information extracted from the AoI and the VoI.
The simulation results validate that our proposed TSRC framework achieves a
91.5$\%$ improvement in the mean square error compared to the traditional
unmanned aerial vehicle control framework.

###### Index Terms:

Task-oriented and semantics-aware, robotic control, AoI, VoI, DRL, proactive
repetition scheme, C&C data.

##  I Introduction

The field of robotics has experienced significant advancements in the past few
decades, leading to widespread applications, such as agriculture [1],
education [2], and surgery [3]. Wireless connectivity is important for the
control of mobile robots (i.e., ground vehicles and unmanned aerial vehicles
(UAVs)) [4], especially for tasks that require remote control and sensing [5].
Remote control of robots can benefit from the development of the fifth-
generation (5G) mobile communication network [6], due to its ultra-reliable
low latency communication (URLLC) service with a latency requirement as low as
1 ms [7, 8, 9, 10]. Unfortunately, this URLLC service defined in 5G is still
unable to support the real-time safe operation of mobile robots [11]. In the
context of high-speed mobile robots, even a mere 1 ms transmission delay can
result in significant movement that cannot be neglected. In such case, the
mobile robot may potentially collide with obstacles or pose a risk to the
surrounding objects [12].

Compared to 5G, the sixth-generation (6G) mobile communication network [13]
aims to provide a more stringent service requirement, known as hyper-reliable
and low-latency communication, which demands a peak data rate ranging from 50
Gbps to 200 Gbps, latency within 0.1ms to 1ms, and reliability ranging from
1-$10^{-5}$ to 1-$10^{-7}$ [14]. 6G technology is expected to facilitate the
remote control of mobile robots with accuracy, robustness, energy efficiency,
and safety [11]. Until now, existing research has been mainly focused on
improving the communication performance, such as the reliability [15], the
data rate [16], and the latency [17].

On one hand, existing wireless communication works tend to ignore the
behaviour and actuation performance of robotics, and only focus on improving
the communication performance. On the other hand, existing robotics papers
[18, 19, 20] mostly assumed wireless communication to be ideal with no packet
loss or latency. Even in the case where robots are controlled remotely, as
demonstrated in [20], the transmitter (remote controller) is responsible for
periodically publishing control commands, and the receiver (mobile robot)
stores the commands in a queue based on their time of arrival. However, the
potential loss and delay of the packets caused by the practical wireless
channels are always ignored by the robotics community. Therefore, there is a
big research gap on how to jointly consider and optimize wireless
communication and robotic control with a focus on specific robotic tasks, such
as tracking, navigation, etc.

To fill this gap, task-oriented/ goal-oriented communication has been proposed
in [21] to design the communication system/ network with a focus on the
effectiveness of communication in accomplishing a specific goal/ task. In the
robotics field, the effectiveness of controlling the robots wirelessly can be
measured by effectiveness level performance metrics, such as the mean square
error (MSE) [22] and the tracking accuracy [23]. In [24], the authors proposed
a deep reinforcement learning (DRL) method to choose the optimal segment
length and sampling rate for the physical world trajectory transmission of a
robotic arm, with the aim to improve the tracking error measured by the MSE
between the trajectory of the device and its digital model. Even though [24]
optimized task-oriented performance (i.e., tracking error), the meaning and
importance/ significance behind the meaning of bits are completely ignored.

To enable more accurate, context-aware, and efficient communication, semantic
communication has been introduced to solve the semantic-level problem [25].
Different from the well-studied technical-level problem that focuses on how to
reliably transmit all the bits, the semantic-level problem considers how to
convey semantic information precisely and serves as the foundation for the
effectiveness-level problem [26]. The semantic information has two main
definitions. One is defined as the meaning of the information, which pertains
to what the information is intended to express and how it relates to the world
around it, entailing a focus on syntax [26]. This definition has mainly
considered for text data type, where the bilingual evaluation understudy
(BLEU) score is introduced to evaluate the sentence similarity for machine
translation [27], signal-to-distortion ratio (SDR) and perceptual evaluation
of speech distortion (PESQ) are used to evaluate speech signal reconstruction
[28].

The second definition pertains to the significance and importance of
information [29], which evaluates the importance of extracted semantic
information in accomplishing a specific task and is closely coupled with the
considered task. Depending on the task/ goal, the significance of information
can be evaluated based on various new metrics, including the age of
information (AoI), the value of information (VoI), just noticeable difference
[21], the skeleton of the avatar [30], and et. The AoI is defined as the
duration elapsed since the information was first generated and is widely used
to effectively characterize latency in the real-time status update system
[31]. In [32], the authors proposed a semantics-aware approach to address the
problem of active fault detection in an IoT scenario. A stochastic
approximation algorithm was employed to jointly consider the importance and
purpose of the information to detect faults and how to acquire fresh
information more effectively. In this approach, the AoI was utilized to
evaluate the freshness of the sensor information, and the semantic information
was measured by a function as VoI, which was related to the AoI and the
entropy of the probability distribution over the operational status. The
concept of VoI is commonly defined as the value/ significance of the
information in completing communication tasks and can be utilized as a metric
to quantify the significance of semantic information [33, 34]. Depending on
the task/ goal, VoI has been represented based on various metrics, such as the
importance of the extracted features to the accurate classification [33], the
cost for wrong actuation [35], and structural index similarity (SSIM) [36].

To incorporate semantic level consideration into the task-oriented
communication framework, a generic task-oriented and semantics-aware (TOSA)
communication framework was proposed in [21]. This framework jointly
considered the semantic level information and effectiveness level performance
metrics, for different tasks with various data types. Although TOSA has been
applied in various domains [37, 30, 38], there is little research on the
command and control data for robotics. In [39], the TOSA communication
framework was studied for the downlink control and command (C&C) signal
transmission, which characterized semantic information as a function of AoI
and the similarity between adjacent C&C signals. This study effectively
reduced redundant packet transmissions by utilizing a DRL-based TOSA
optimization method. However, it lacks the design for effectiveness level
metrics and the receiver design.

To fill this gap, we propose task-oriented and semantics-aware communication
in robotic control (TSRC) in a real-time UAV waypoint transmission task.
Compared to traditional transmitter/receiver design and previous works [20,
24, 32, 35, 39], we design the DRL at the transmitter to generate the optimal
C&C data to accomplish the task, adopt the proactive repetition scheme for C&C
data transmission, order the queue of received C&C data based on their
semantic information extracted from the AoI and the VoI, and utilize the MSE
as our effectiveness level metric. To the best of our knowledge, our
contributions can be summarized as follows:

  * 1)

We consider a scenario involving a base station (BS) and a UAV with a dynamic
communication environment. In this setup, the BS acts as the transmitter,
periodically transmitting C&C data to the UAV to manoeuvre it to fly along a
trajectory. Our goal is to collaboratively consider the communication and
control procedures on both the BS and UAV sides, with the aim of minimizing
the positional error between the actual flight trajectory of the UAV and the
target trajectory.

  * 2)

We propose task-oriented and semantics-aware communication in robotics control
(TSRC), which includes DRL-based C&C data generation with the proactive scheme
for repetition (DeepPro) and the VoI and the AoI based queue ordering
mechanism (VA-QOM). At the semantic level, the TSRC employs the DeepPro on the
BS side, facilitating the generation of more appropriate C&C data to
accomplish the task and elevating the probability of successful transmission
compared to conventional frameworks. Meanwhile, on the UAV side, the VA-QOM is
deployed by utilizing the combined function of AoI and VoI to extract the
semantic information of received C&C data in the queue, which can measure the
importance of each C&C data at the present moment and has the potential to be
extended to other time-sensitive control applications. At the effectiveness
level, we choose the MSE between the actual flight trajectory of the UAV and
the target one as the performance metric.

  * 3)

To evaluate the effectiveness of the proposed TSRC framework, we implemented
it in a real-time UAV trajectory control task and compared it with the
traditional UAV control framework via simulation. The simulation results
indicate that the proposed TSRC framework achieves 91.5$\%$ improvement in the
mean square error compared to the traditional UAV control framework.

The rest of the paper is organized as follows: In Section $\mathrm{II}$, we
present the system model and problem formulation. Section $\mathrm{III}$ and
Section $\mathrm{IV}$ introduce the design of the traditional UAV control
framework and the proposed TSRC framework, respectively. In Section
$\mathrm{V}$, the simulation results is presented. Finally, Section
$\mathrm{VI}$ concludes this paper.

##  II System Model and Problem formulation

In this section, we introduce a real-time UAV waypoint transmission task,
where a BS controls a UAV to fly alongside a trajectory in real time by
sending C&C data. Subsequently, we model the communication environment for
downlink C&C data transmission. Based on that, we present the problem
formulation and the evaluation metric.

###  II-A Real-time UAV Waypoint Transmission Task

Figure 1: User case.

We consider a real-time UAV waypoint transmission task, where a BS serves as a
remote controller to generate C&C data to control a UAV to fly alongside a
trajectory in real time. The trajectory is composed of a series of waypoints,
which are specific geographic locations and serve as reference points for
navigation. In the task, the BS only has knowledge of the UAV’s target
waypoint in the next time slot. As a result, the BS must transmit the
corresponding C&C data in real time. By executing the received C&C data, the
UAV flies across various positions or waypoints at different timestamps. One
user case is shown in Fig. 1, where the BS needs to control the UAV in real
time to scan QR codes on the parcels located at various positions. To
effectively complete the task within the time constraint, the UAV aims to
reach each waypoint on time.

###  II-B Channel Model

We assume that the BS remains stationary throughout the communication, and the
UAV flies within a circular horizontal disk with a radius of R and a height of
H, as shown in Fig. 1.

Considering both line-of-sight (LoS) and non-line-of-sight (NLoS) conditions
for flying UAVs, we utilize free-space path loss and Rayleigh fading to model
the path loss from the BS to the UAV as

$h=\left\\{\begin{aligned} &{(\frac{4\pi
df_{c}}{c})}^{\alpha}\eta_{\mathrm{LoS}}\beta,\quad P_{\mathrm{LoS}}\\\
&{(\frac{4\pi df_{c}}{c})}^{\alpha}\eta_{\mathrm{NLoS}}\beta,\quad
P_{\mathrm{NLoS}},\end{aligned}\right.$ (1)

where $d$ is the distance between the UAV and the BS, $f_{c}$ denotes the
transmission frequency, $c$ is the speed of light, $\alpha$ represents the
path loss exponent, $\beta$ denotes the Rayleigh small-scale fading which
follows $\mathcal{CN}(0,1)$, and $\eta_{\mathrm{LoS}}$ and
$\eta_{\mathrm{NLoS}}$ represent the path loss coefficients of LoS and NLoS
cases, respectively. In Eq. (1), the the LoS probability $P_{\mathrm{LoS}}$ is
derived as

$P_{\mathrm{LoS}}=\frac{1}{1+ae^{-b(\theta-a)}},$ (2)

where $\theta=\frac{180}{\pi}\arcsin\frac{\mathrm{H}}{d}$ is the evaluation
angle of the UAV, $\mathrm{H}$ is the flight height of the UAV, and $a$ and
$b$ are positive constants related to the environment. Based on Eq. (1) and
Eq. (2), the downlink channel from the BS to the UAV can be derived as

$h=(P_{\mathrm{LoS}}\eta_{\mathrm{LoS}}+P_{\mathrm{NLoS}}\eta_{\mathrm{NLoS}}){(\frac{4\pi
df_{c}}{c})}^{\alpha}\beta.$ (3)

Based on Eq. (3), the signal-to-noise ratio (SNR) can be formulated as

$\mathrm{SNR}=\frac{\mathrm{P}h}{\sigma^{2}},$ (4)

where $\mathrm{P}$ is the transmit power of the BS, $h$ is the channel gain,
and $\sigma^{2}$ is the Additive White Gaussian Noise (AWGN) power. Then, we
can derive the transmission time for each generated C&C data as

$t^{\mathrm{CC}}=\frac{\mathrm{N_{CC}}}{B\log(\mathrm{SNR+1})},$ (5)

where $\mathrm{N_{CC}}$ is the size of each C&C data, and $B$ is bandwidth. In
downlink transmission, the UAV can only successfully decode the C&C data when
the SNR of C&C transmission exceeds a specific threshold denoted as
$\gamma_{\mathrm{th}}$. The parameter that determines whether C&C data can be
successfully decoded is expressed as

$\delta^{\mathrm{CC}}=\begin{cases}0,\quad\mathrm{SNR}\leq\gamma_{\mathrm{th}}\\\
1,\quad\mathrm{SNR}>\gamma_{\mathrm{th}}.\end{cases}$ (6)

The C&C data can be successfully decoded if $\delta^{\mathrm{CC}}=1$,
otherwise, the C&C data can not be decoded. Meanwhile, if one C&C data is
successfully decoded, we assume that it is not distorted and can be accurately
recovered.

###  II-C The General Communication and Control Process

To provide an overview of the general communication and control process for
the task, we introduce the C&C data transmission protocol and the queue
employed to store the received C&C data.

####  II-C1 C&C Data Transmission

The BS starts to generate and transmit the C&C data periodically at the
beginning of $i^{\mathrm{th}}$ transmission time interval (TTI) $t_{i-1}$,
where $i\in\\{1,2,...,N^{\mathrm{TTI}}\\}$ and
$N^{\mathrm{TTI}}\in\mathbbm{N}$ is the maximum value of $i$. The time
duration of each TTI is fixed as a constant $T$. If the UAV successfully
receives and decodes the C&C data $\boldsymbol{m}_{i}$, it will send an
acknowledgement (ACK) to the BS. Otherwise, it will send a negative
acknowledgement (NACK) to the BS. The parameter that determines to send ACK or
NACK is presented as

$\delta^{\mathrm{ACK}}_{i}=\begin{cases}0,\ \delta^{\mathrm{CC}}_{i}=0\\\ 1,\
\delta^{\mathrm{CC}}_{i}=1,\end{cases}$ (7)

where $\delta^{\mathrm{CC}}_{i}$ is the transmission result (success or
failure) for $\boldsymbol{m}_{i}$. If $\delta^{\mathrm{ACK}}_{i}=1$, the UAV
sends ACK. Otherwise, the UAV sends NACK.

####  II-C2 Queue

The queue of a UAV is a crucial component of the UAV’s control system, serving
as a structured list of commands and instructions to be executed. When the UAV
successfully receives and decodes one C&C data, it will store the C&C data in
the queue with a predefined size $Q_{\mathrm{max}}$. The size
$Q_{\mathrm{max}}$ is not fixed in all scenarios and can be customized for the
needs of a particular task. If the number of received C&C data exceeds
$Q_{\mathrm{max}}$, the C&C data with lower priority will be discarded. The
priority can be measured by various metrics, such as the arrival time and the
semantic information. The UAV ranks the C&C data in the queue based on their
priorities and executes the C&C data according to the order. This adaptability
allows for efficient task management and execution, ensuring that the UAV can
effectively handle different scenarios and workloads as needed.

####  II-C3 The Overall Process

We assume that the time duration of the whole process is
$[0,TN^{\mathrm{TTI}}]$. The UAV’s actual position and target position at
$t\in[0,TN^{\mathrm{TTI}}]$ can be represented as
$\boldsymbol{p}_{\frac{t}{T}}$ and $\boldsymbol{g}_{\frac{t}{T}}$,
respectively. At the beginning of $i^{\mathrm{th}}$ TTI $t_{i-1}=(i-1)T$, the
BS generates and transmits the C&C data $\boldsymbol{m}_{i}$ with the
execution time of the length of one TTI $T$, expressed as

$\boldsymbol{m}_{i}=(v_{i}^{x},v_{i}^{y},v_{i}^{z}),$ (8)

where $v_{i}^{x}$, $v_{i}^{y}$, and $v_{i}^{z}$ are the UAV’s planned velocity
on the x-axis, y-axis, and z-axis for the $i^{\mathrm{th}}$ TTI, respectively.
Upon successfully receiving and decoding the C&C data $\boldsymbol{m}_{i}$,
the UAV sends an ACK back ($\delta^{\mathrm{ACK}}_{i}=1$) to the BS.
Otherwise, the UAV sends a NACK ($\delta^{\mathrm{ACK}}_{i}=0$) instead. When
the UAV successfully receives and decodes $\boldsymbol{m}_{i}$, it will store
it in the queue and rank all the received C&C data in the queue. By executing
the C&C data in the queue according to their order, the UAV’s position is
updated. It is important to note that, at the end of $i^{\mathrm{th}}$ TTI
$t_{i}=iT$, the UAV sends its real-time position
$\boldsymbol{p}_{i}=(x_{i},y_{i},z_{i})$ back to the BS, where $x_{i}$,
$y_{i}$, and $z_{i}$ are the UAV’s position on the x-axis, y-axis, and z-axis,
respectively. To focus on the downlink transmission, we assume the uplink
transmission (i.e., ACK, NACK, and $\boldsymbol{p}_{i}$) from the UAV to the
BS is ideal without packet loss or delay.

###  II-D Problem Formulation and Performance Metric

Due to the delay and packet loss in the downlink transmission of C&C data,
there is often a discrepancy between the UAV’s actual trajectory and the UAV’s
target trajectory. Consequently, we utilize the mean square error (MSE) to
measure the gap between the two trajectories. Specifically, we aim to minimize
the MSE between the positions of the actual trajectory and their corresponding
positions of the target trajectory at the end of each TTI. We assume that each
TTI can be uniformly divided into $N^{\mathrm{M}}$ time duration. For
$j^{\mathrm{th}}$ time duration in $i^{\mathrm{th}}$ TTI where
$j\in\\{1,...,N^{\mathrm{M}}\\}$, its timestamp is represented as
$[(i-1)T+\frac{T}{N^{\mathrm{M}}}j]$. In this way, we can obtain the
corresponding positions of the UAV’s actual trajectory
$\boldsymbol{p}_{(i-1+\frac{j}{N^{\mathrm{M}}})}$ and the UAV’s target
trajectory $\boldsymbol{g}_{(i-1+\frac{j}{N^{\mathrm{M}}})}$. When
$N^{\mathrm{M}}$ is infinity, the average MSE of these two positions
represents the MSE of the two trajectories. As a result, the problem can be
formulated as

$\begin{split}&\mathcal{P}1:\min\lim_{N^{\mathrm{M}}\rightarrow\infty}\frac{1}{N^{\mathrm{TTI}}N^{\mathrm{M}}}\sum_{\begin{subarray}{c}i=1\end{subarray}}^{\begin{subarray}{c}N^{\mathrm{TTI}}\end{subarray}}\sum_{\begin{subarray}{c}j=1\end{subarray}}^{\begin{subarray}{c}N^{\mathrm{M}}\end{subarray}}||\boldsymbol{p}_{(i-1+\frac{j}{N^{\mathrm{M}}})}-\boldsymbol{g}_{(i-1+\frac{j}{N^{\mathrm{M}}})}||^{2}\\\
&\mathrm{s.t.}\ N^{\mathrm{M}}\in\mathbbm{N},\end{split}$ (9)

where $||.||$ is the Frobenius norm used to calculate the distance of the two
positions.

Similar to the problem formulation, we utilize the MSE between the UAV’s
actual trajectory and target trajectory as the performance metric. To simplify
the calculation, instead of obtaining the positions of infinity time duration,
we set $N^{\mathrm{M}}$ as a fixed constant and the performance metric can be
calculated as

$\mathrm{MSE}=\frac{1}{N^{\mathrm{M}}N^{\mathrm{TTI}}}\sum_{i=1}^{N^{\mathrm{TTI}}}\sum_{j=1}^{N^{\mathrm{M}}}||\boldsymbol{p}_{(i-1+\frac{j}{N^{\mathrm{M}}})}-\boldsymbol{g}_{(i-1+\frac{j}{N^{\mathrm{M}}})}||^{2}$
(10)

##  III Traditional UAV Control Framework

In this section, we introduce the traditional UAV control framework as the
baseline scheme. As shown in Fig. 2, it includes the design at the BS side and
the UAV side. The BS consists of two main components: the C&C data generator
and the transmitter. The C&C data generator is responsible for generating the
C&C data at the beginning of each TTI. The transmitter includes an encoder, a
modulator, and an antenna, which is used to convert the C&C data to the
waveform and broadcast it to the wireless channel.

Figure 2: Traditional UAV control framework

Input: $a$, $b$, $f_{c}$, $\alpha$, $\eta_{\mathrm{LoS}}$,
$\eta_{\mathrm{NLoS}}$, $\sigma^{2}$, $\mathrm{N_{CC}}$, $B$, and
$\gamma_{\mathrm{th}}$.

Output: UAV actual position set $\mathcal{P}$ with its corresponding time set
$\mathcal{T}$.

1 Initialization: The UAV’s initial position $\boldsymbol{p}_{0}$ with its
corresponding time $t_{0}$, UAV’s received C&C data set $\mathcal{C}$.

2 for  _$i\leftarrow 1$ to $N^{\mathrm{TTI}}$_ do

3 Update the UAV’s position $\boldsymbol{p}_{i-1}$.

4 Initialize UAV’s received C&C data set in $i^{\mathrm{th}}$ TTI
$\mathcal{C}_{i}$ to $\emptyset$.

5 Generate $\boldsymbol{m}_{i}$.

6 Calculate $P_{\mathrm{LoS}}$ using Eq. (2).

7 Calculate $h_{i}$ using Eq. (3).

8 Calculate SNR using Eq. (4).

9 Calculate $t_{i}^{\mathrm{CC}}$ using Eq. (5).

10 Calculate $\delta^{\mathrm{CC}}_{i}$ using Eq. (6).

11 if  _$\delta^{\mathrm{CC}}_{i}=1$_ then

12 Add $\boldsymbol{m}_{i}$ in $\mathcal{C}$.

13

14 end if

15 for  _all C &C data in $\mathcal{C}$_ do

16 if  _the C &C data arrived in $[t_{i-1},t_{i})$_ then

17 Add this C&C data in $\mathcal{C}_{i}$.

18

19 end if

20

21 end for

22 Reorder the C&C data in $\mathcal{C}_{i}$ by their arrival time.

23 for  _all C &C data in $\mathcal{C}_{i}$_ do

24 Update the UAV’s position.

25

26 end for

27

28 end for

Algorithm 1 Traditional UAV Control Framework

At the UAV side, it comprises a receiver and a flight controller. The receiver
consists of an antenna, a demodulator, and a decoder, which is able to recover
the C&C data from the received waveform if the C&C data can be successfully
decoded (i.e., $\mathrm{SNR}>\gamma_{\mathrm{th}}$). In the traditional UAV
control framework, the queue size $Q_{\mathrm{max}}$ is set to one, and the
queue is ordered by the arrival time of the C&C data. Therefore, only the
newly arrived C&C data $\boldsymbol{m}$ is stored in the queue of the UAV’s
memory, and delivered to the flight controller. The UAV’s flight controller is
able to convert the received C&C data $\boldsymbol{m}$ into direct commands
(velocity $\boldsymbol{v}$ and the execution time $\tau$), which are used to
control the UAV to accomplish the specific task.

Figure 3: Exemplar timeline in the traditional UAV control framework.

By assuming the C&C data processing time at both the BS and the UAV sides can
be neglected, we plot an exemplar timeline of the BS and the UAV in the
traditional UAV control framework shown in Fig. 3. At the beginning of
$(i-1)^{\mathrm{th}}$ TTI $t_{i-2}$, the BS transmits the C&C data
$\boldsymbol{m}_{i-1}$ to the UAV. Once the UAV successfully decodes
$\boldsymbol{m}_{i-1}$ at $t_{i-1}^{\mathrm{A}}$, it will fly in a constant
speed $\boldsymbol{v}=\boldsymbol{m}_{i-1}$. If the UAV successfully decodes
new C&C data $\boldsymbol{m}_{i}$ when executing the ongoing C&C data
$\boldsymbol{m}_{i-1}$ at $t_{i}^{\mathrm{A}}$, it will discard the ongoing
C&C data $\boldsymbol{m}_{i-1}$ and start to execute the new C&C data
$\boldsymbol{m}_{i}$. After finishing executing the C&C data
$\boldsymbol{m}_{i}$, the UAV remains hovering until one new C&C data is
successfully decoded.

The design of the traditional UAV control framework is shown in
$\mathbf{Algorithm\ 1}$. At the beginning of $i^{\mathrm{th}}$ TTI $t_{i-1}$,
the UAV’s current position $\boldsymbol{p}_{i-1}$ is updated. By utilizing the
channel model, we can obtain the transmission result $\delta_{i}$ and
transmission time $t_{i}^{\mathrm{CC}}$ for the C&C data $\boldsymbol{m}_{i}$.
Subsequently, all the successfully decoded C&C data in $i^{\mathrm{th}}$ TTI
$[t_{i-1},t_{i})$ can be obtained, which can be utilized to update the UAV’s
position. The algorithm iterates until all the real-time C&C data are
transmitted.

##  IV The Proposed TSRC Framework

Figure 4: The proposed TSRC framework

In this section, we propose TSRC framework which incorporates the semantic
level design into the effectiveness level metric. As shown in Fig. 4, compared
with the traditional UAV control framework, our proposed TSRC framework
includes the DRL-based C&C data generator with proactive repetition
transmission, namely, DeepPro, at the BS, and the VoI and the AoI based queue
ordering mechanism (VA-QOM) at the UAV.

###  IV-A DeepPro at the BS

At the BS, we design a DeepPro scheme, which is expected to generate optimal
C&C data for the real-time UAV waypoint transmission task via the DRL-based
C&C data generator and increase the probability of successful transmission for
each C&C data via the proactive repetition transmission.

####  IV-A1 DRL-based C&C Data Generator

To generate optimal C&C data at the beginning of each TTI, we need to obtain
the influence of this C&C data on the task performance, which is related to
the gap between the UAV’s actual positions and the UAV’s target positions of
future TTIs. We define the state of the $i^{\mathrm{th}}$ TTI as
$S_{i}=\\{x_{i-1},y_{i-1},z_{i-1},t_{i-1}\\}$, where $x_{i-1}$, $y_{i-1}$, and
$z_{i-1}$ are the UAV’s position on the x-axis, y-axis, and z-axis at the
beginning of $i^{\mathrm{th}}$ TTI $t_{i-1}=(i-1)T$, respectively. We aim to
tackle the problem of optimizing the C&C data generation defined by the action
parameter $A_{i}=\\{v_{i}^{x},v_{i}^{y},v_{i}^{z}\\}$ at $i^{\mathrm{th}}$
TTI, where $v_{i}^{x}\in V^{\mathrm{X}}$, $v_{i}^{y}\in V^{\mathrm{Y}}$, and
$v_{i}^{z}\in V^{\mathrm{Z}}$ are the UAV’s planned velocity in x-axis,
y-axis, and z-axis, respectively. At the beginning of $i^{\mathrm{th}}$ TTI
$t_{i-1}$, the agent at the BS makes the decision by accessing all prior
historical observations
$\\{U_{1},...,U_{i^{\mathrm{{}^{\prime}}}},...,U_{i-1}\\}$ for all previous
TTIs $i^{\mathrm{{}^{\prime}}}\in\\{1,...,i-1\\}$, where the observation
$U_{i^{\mathrm{{}^{\prime}}}}$ consists of the following variables: the UAV’s
actual position $\boldsymbol{p}_{i^{\mathrm{{}^{\prime}}}-1}$ and the UAV’s
target position $\boldsymbol{g}_{i^{\mathrm{{}^{\prime}}}-1}$ at
$t_{i^{\mathrm{{}^{\prime}}}-1}$. It is noted that the observation in each TTI
$i$ includes all histories of such measurements and past actions, which is
denoted as $O_{i}=\\{A_{1},U_{1},A_{2},U_{2},...,A_{i-1},U_{i-1}\\}$.

To obtain the impact of the C&C data $\boldsymbol{m}_{i}$ generated at the
start of $i^{\mathrm{th}}$ TTI $t_{i-1}$, the BS aims at minimizing the
distance between the UAV’s actual positions and the UAV’s target positions in
future TTIs. The optimization relies on the selection of parameters in $A_{i}$
according to the current observation history $O_{i}$ with respect to the
stochastic policy $\pi$, which can be formulated as

$\mathcal{P}2:\quad\max_{\pi(A_{i}|O_{i})}\sum_{\begin{subarray}{c}b=i\end{subarray}}^{\begin{subarray}{c}\infty\end{subarray}}\gamma^{b-i}\mathbbm{E}_{\pi}[-||\boldsymbol{p}_{b}-\boldsymbol{g}_{b}||],$
(11)

where $\gamma\in(0,1]$ is the discount factor for the performance accrued in
future TTIs, and $||\boldsymbol{p}_{b}-\boldsymbol{g}_{b}||$ is the distance
between the UAV’s actual position $\boldsymbol{p}_{b}$ and target position
$\boldsymbol{g}_{b}$ at the end of $b^{\mathrm{th}}$ TTI $t_{b}$. Based on
that, the reward $R_{i}$ is formulated as

$R_{i}=-||\boldsymbol{p}_{i}-\boldsymbol{g}_{i}||.$ (12)

It is noticed that the smaller the distance is after selecting one action, the
higher the reward this action has.

The C&C data selection process at the BS is the Markov Decision Process
because the state $S_{i}$ is only related to its previous state and past
action. We define $\mathcal{S}$ as a set of actual environment’s states with
variables of the UAV’s positions at a certain time $\\{x,y,z,t\\}$ and channel
parameters. The fact is that the BS cannot fully observe actual states in
$\mathcal{S}$, while it can only access variables of $\\{x,y,z,t\\}$ without
channel parameters. Meanwhile, the packet loss or delay decided by the channel
parameters influences the transition between any two successive states. Since
the channel parameter cannot be assessed by the BS, the process of state
transition is not fully observed, which introduces a generally intractable
Partially Observable Markov Decision Process (POMDP) problem.

To tackle this problem, we design the deep-Q network (DQN) algorithm, as one
powerful DRL approach. The agent of the DQN algorithm is deployed at the BS to
choose optimal actions by exploring the environment. During the training
process, learning takes place over multiple episodes, with each episode
including $N^{\mathrm{TTI}}$ TTIs.

From the environment perspective, when it receives an action $A_{i}$ from the
agent which is in state $S_{i}$, it responds by providing the action’s reward
$R_{i}$ and the agent’s next state $S_{i+1}$. From the agent perspective, it
receives the sample $(S_{i},A_{i},R_{i},S_{i+1})$ and stores it in the replay
memory. The replay memory serves as a buffer that stores past experiences for
the agent to learn from. Once the replay memory stored enough samples, the
agent randomly selects a batch of samples for training. During the training
process, the current state $S_{i}$ and its corresponding action $A_{i}$ are
inputted to the Q-network, which calculates the predicted value using the
function $Q(S_{i},A_{i};\boldsymbol{\theta}_{i})$. Here,
$\boldsymbol{\theta}_{i}$ represents the parameter vector of the Q-network.
Simultaneously, the next state $S_{i+1}$ and the reward $R_{i}$ are passed to
the target Q-network, which is a separate network used for calculating the
target values. Then, they pass the results to the loss function calculation
parts, and the gradient of the loss function is calculated and utilized to
update the parameter vector in the Q network. The gradient of the loss
function is given as

$\nabla
L(\boldsymbol{\theta}_{i})=\mathbbm{E}_{S_{i},A_{i},R_{i},S_{i+1}}[(R_{i}+\gamma\max_{A}Q(S_{i+1},A;\boldsymbol{\theta}^{-}_{i})\\\
-Q(S_{i},A_{i};\boldsymbol{\theta}_{i}))\nabla_{\boldsymbol{\theta}}Q(S_{i},A_{i};\boldsymbol{\theta}_{i})],$
(13)

where $\boldsymbol{\theta}_{i}$ is updated by

$\boldsymbol{\theta}_{i+1}=\boldsymbol{\theta}_{i}-\lambda_{\mathrm{RMS}}\nabla
L(\boldsymbol{\theta}_{i}),$ (14)

where $\lambda_{\mathrm{RMS}}$ is the learning rate of RMSprop. In the
training process, it is noted that the parameter vector
$\boldsymbol{\theta}^{-}$ in the target Q-network is updated by copying the
parameter vector $\boldsymbol{\theta}$ from the Q-network for every
$N^{\boldsymbol{\theta}}$ episodes. By selecting a batch of samples from the
replay memory instead of a single sample, using two networks with the same
structure but different parameters (the Q-network and the target Q-network),
and updating the parameter vector $\boldsymbol{\theta}^{-}$ periodically, the
stability of the learning process is improved.

Input: Action space $\mathcal{A}$, $N^{\mathrm{I}}$,
$N^{\boldsymbol{\theta}}$, $\lambda_{\mathrm{RMS}}$, $\gamma$, and $\epsilon$.

1 Initialization of replay memory $M$, $\boldsymbol{\theta}$, and
$\boldsymbol{\theta}^{-}$.

2 for  _$Iteration\leftarrow 1$ to $N^{\mathrm{I}}$_ do

3 for  _$i\leftarrow 1$ to $N^{\mathrm{TTI}}$_ do

4 Update the wireless channel model.

5 if  _the probability is less than $\epsilon$_ then

6 Randomly select the action $A_{i}\in\mathcal{A}$.

7

8 else

9 Select the action $A_{i}=\arg\max_{A}Q(S_{i},A;\boldsymbol{\theta}_{i})$.

10

11 end if

12 The BS observes the state $S_{i+1}$ and calculate the reward of the action
$R_{i}$.

13 Store the transition $(S_{i},A_{i},R_{i},S_{i+1})$ in the replay memory
$M$.

14 Randomly sample transitions from replay memory $M$ with the number of batch
size.

15 Calculate gradient descent using Eq. (13).

16 Update $\boldsymbol{\theta}$ using Eq. (14).

17

18 end for

19 Update $\boldsymbol{\theta^{-}=\theta}$ in every $N^{\boldsymbol{\theta}}$
episodes.

20

21 end for

Algorithm 2 DQN-based C&C Data Generation

To balance the relationship between exploration and exploitation, we adopt the
$\epsilon-$greedy approach in our learning process with the condition that
$\epsilon\in[0,1]$. In each TTI, the agent randomly generates a probability
and compares it with $\epsilon$. If the probability is less than $\epsilon$,
the agent randomly chooses an action in this TTI. If not, the agent chooses
the optimal action.

Based on that, we implement the DQN algorithm, which is presented in
$\mathbf{Algorithm\ 2}$. At the beginning of $i^{\mathrm{th}}$ TTI, the agent
at the BS selects an action $A_{i}$ which is utilized to form the C&C data
$\boldsymbol{m}_{i}$, and the BS transmit $\boldsymbol{m}_{i}$ to the UAV. At
the end of this TTI $t_{i}$, the UAV sends its position to the BS. Then, the
agent can obtain the next state $S_{i+1}$ and reward $R_{i}$. As the algorithm
iterates, the parameters can be updated and optimal actions can be selected.

####  IV-A2 Proactive Repetition Scheme

Due to the unstable communication channel, the packet loss and delay of
downlink C&C data transmission can be harmful to the task accomplishment. In
order to increase the successful transmission of the C&C data, we introduce
the proactive scheme which is proposed in uplink transmission and has been
proven to have the lowest latent access failure probability (the failure
probability within a limited time) among the reactive scheme, K-repetition
scheme and itself [40]. In the proactive scheme for uplink transmission, the
user equipment (UE) repeats the transmission for a maximum number of
$K_{\mathrm{max}}$ repetitions, where the period to send each repetition is
denoted as $T_{\mathrm{rep}}$. The UE is configured to repeat the transmission
for $K_{\mathrm{max}}$ repetitions but can receive feedback after each
repetition. This allows the UE to terminate repetitions earlier once receiving
positive feedback (ACK).

Different from the uplink transmission without the latency constraint, in our
work, we focus on downlink transmission with stringent delay limitations.
Thus, we do not take retransmission into consideration. The exemplar timeline
between the BS and the UAV for the downlink C&C data transmission is shown in
Fig. 5.

Figure 5: An example of the timeline of UAV and BS

We make the assumption that the data (i.e., C&C data, ACK, etc.) processing
time is ignored. Then, the important parameters of the proactive repetition
scheme are $K_{\mathrm{max}}$ and $T_{\mathrm{rep}}$. In this example, the
maximum repetition number $K_{\mathrm{max}}$ is set to be 4. In the
$i^{\mathrm{th}}$ TTI, only the $2^{\mathrm{nd}}$ repetition is successfully
received by the UAV and its corresponding ACK arrival time is larger than the
transmitting time of the $4^{\mathrm{th}}$ repetition. As a result, there is
no repetition terminated at the BS and the number of repetitions is
$K_{\mathrm{max}}$. By contrast, in the $(i+1)^{\mathrm{th}}$ TTI, the ACK of
the $1^{\mathrm{st}}$ repetition is successfully received before the
transmitting time of the $3^{\mathrm{rd}}$ repetition. Hence, the
$3^{\mathrm{rd}}$ repetition is terminated and there are only two repetitions.
It is worth noticing that the time duration between the first repetition and
the maximum repetition should not exceed the threshold which is set to be the
length of one TTI $T$.

Input: $K_{\mathrm{max}}$, $T_{\mathrm{rep}}$.

1 Obtain the C&C data $\boldsymbol{m}_{i}$.

2 Initialize the arrival time $t_{i}^{\mathrm{A}}$ of $\boldsymbol{m}_{i}$ to
$t_{i}$.

3 for  _$k\leftarrow 1$ to $K_{\mathrm{max}}$_ do

4 if  _$t_{i}^{\mathrm{A}}\leq t_{i,k}=t_{i-1}+(k-1)T_{\mathrm{rep}}$_ then

5 break.

6

7 end if

8 Update the UAV’s position $\boldsymbol{p}_{i,k}$ at $t_{i,k}$.

9 Calculate the transmission result $\delta_{i,k}^{\mathrm{CC}}$ and
transmission time $t_{i,k}^{\mathrm{CC}}$ of $k^{\mathrm{th}}$ repetition.

10 if  _$\delta_{i,k}^{\mathrm{CC}}=1$._ then

11 if  _there is no $\boldsymbol{m}_{i}$ in $\mathcal{C}$._ then

12 Add $\boldsymbol{m}_{i}$ in $\mathcal{C}$.

13

14 end if

15

16 end if

17 if  _$t_{i}^{\mathrm{A}} >t_{i,k}+t_{i,k}^{\mathrm{CC}}$._ then

18 $t_{i}^{\mathrm{A}}=t_{i,k}+t_{i,k}^{\mathrm{CC}}$.

19

20 end if

21 if  _$k=K_{\mathrm{max}}$_ then

22 $t^{\mathrm{End}}=t_{i}$.

23

24 else

25 $t^{\mathrm{End}}=t_{i,(k+1)}$.

26

27 end if

28 Initialize $\mathcal{C}^{\mathrm{Tmp}}$ to $\emptyset$.

29 for  _C &C data in $\mathcal{C}$_ do

30 if  _this C &C data arrives in $[t_{i,k},t^{\mathrm{End}})$_ then

31 Add this C&C data to $\mathcal{C}^{\mathrm{Tmp}}$.

32

33 end if

34

35 end for

36

37 Reorder the C&C data in $\mathcal{C}^{\mathrm{Tmp}}$ by their arrival time.

38 for  _all C &C data in $\mathcal{C}^{\mathrm{Tmp}}$_ do

39 Update the UAV’s position.

40

41 end for

42

43 end for

44Initialize $\mathcal{C}^{\mathrm{Tmp}}$ to $\emptyset$.

45 for  _C &C data in $\mathcal{C}$_ do

46 if  _the arrived time for the C &C data is in $[t^{\mathrm{End}},t_{i}$)_
then

47 Add this C&C data in $\mathcal{C}^{\mathrm{Tmp}}$.

48

49 end if

50

51 end for

52Reorder the C&C data in $\mathcal{C}^{\mathrm{Tmp}}$ by their arrival time.

53 for  _all C &C data in $\mathcal{C}^{\mathrm{Tmp}}$_ do

54 Update the UAV’s position.

55

56 end for

Algorithm 3 Proactive Repetition Scheme

The implementation of the proactive repetition scheme is presented in
$\mathbf{Algorithm\ 3}$. After one C&C data $\boldsymbol{m}_{i}$ is generated,
its $k^{\mathrm{th}}$ ($k\in\\{1,...,K_{\mathrm{max}}\\}$) repetition occurs
at $t_{i,k}=t_{i-1}+(k-1)T_{\mathrm{rep}}$. In the time duration between two
consecutive repetitions, the UAV’s position may change, which may influence
the transmission process (i.e., transmission result $\delta^{\mathrm{CC}}$ and
transmission time $t^{\mathrm{CC}}$) of future repetitions. As a result, we
need to calculate the UAV’s position in the time duration
$[t_{i,k},t^{\mathrm{End}})$, where $t^{\mathrm{End}}$ can be expressed as

$t^{\mathrm{End}}=\begin{cases}t_{i,(k+1)},&\ k<K_{\mathrm{max}}\\\ t_{i},&\
k=K_{\mathrm{max}}.\end{cases}$ (15)

If $k=K_{\mathrm{max}}$, this means there is no future repetition of
$\boldsymbol{m}_{i}$, the end of this time duration $t^{\mathrm{End}}$ is the
beginning of the next TTI $t_{i}$. Otherwise, $t^{\mathrm{End}}$ is the
transmission time of the next potential repetition $t_{i,(k+1)}$.

Since the queue at the UAV only stores the earliest successfully decoded C&C
data $\boldsymbol{m}_{i}$ for $i^{\mathrm{th}}$ TTI despite the number of
successful repetitions it receives, we also need to obtain the smallest
arrival time of all successfully decoded repetitions and consider it as the
arrival time $t_{i}^{\mathrm{A}}$ of $\boldsymbol{m}_{i}$. Based on the
arrival time of all C&C data in the successfully decoded C&C data set
$\mathcal{C}$, we can obtain the C&C data arrived during
$[t_{i,k},t^{\mathrm{End}})$ and update the UAV’s position. This loop
continues to iterate until the number of repetitions reaches
$K_{\mathrm{max}}$ or the repetition is terminated when the ACK of the
earliest successfully decoded repetition arrives at the BS
($t_{i}^{\mathrm{A}}<t_{i,k}$).

####  IV-A3 DeepPro

The DeepPro at the BS is designed by integrating the proactive repetition
scheme into DRL-based C&C data generation. In $i^{\mathrm{th}}$ TTI, the DRL-
based C&C data generator generates a C&C data $\boldsymbol{m}_{i}$ based on
$\mathbf{Algorithm\ 2}$, and the proactive repetition scheme repeats the
transmission of $\boldsymbol{m}_{i}$ several times and the UAV’s position is
updated based on $\mathbf{Algorithm\ 3}$. Next, the UAV’s position
$\boldsymbol{p}_{i}$ at the end of the $i^{\mathrm{th}}$ TTI $t_{i}$ is
obtained and utilized to acquire $S_{i+1}$ and $R_{i}$. Finally, the sample
$(S_{i},A_{i},S_{i+1},R_{i})$ is fed in the DQN training process in
$\mathbf{Algorithm\ 2}$.

###  IV-B VA-QOM

With the aim of executing the optimal C&C data in the queue to accomplish the
task, we design a VA-QOM scheme to control the UAV to fly to the target
position as close as possible, where the size of the queue is set to be
$Q_{\mathrm{max}}$. Different from the traditional UAV control framework where
the queue is ordered by the arrival time, we organise the queue of received
C&C data according to their semantic information based on its importance,
which is described by the AoI and the VoI.

####  IV-B1 AoI Design

Since the packet loss and transmission delay may cause negative impacts (i.e.,
delayed response time and wrong actions of the UAV) on the accomplishment of
our time-sensitive task, it is crucial to utilize the AoI to evaluate the
freshness of the packet. In our task, the AoI of the C&C data
$\boldsymbol{m}_{i}$ can be formulated as
$\mathrm{AoI}(\boldsymbol{m}_{i})=t-t_{i-1}$, where $t$ is the current time
and $t_{i-1}$ is the $\boldsymbol{m}_{i}$ generation time instance. We can
observe that the higher the AoI is, the less fresh the C&C data is.

####  IV-B2 VoI Design

In our task, we aim to decrease the difference between the UAV’s actual
trajectory and its target trajectory, which is equivalent to decreasing the
distance between the UAV’s actual positions and the UAV’s target positions at
each TTI. It is important to note that different C&C data may have different
importance or value to the task to be executed, and the value of the same C&C
data may change according to time. Concretely, at the same time, different C&C
data may control the UAV to move closer or further to its target position,
which implies different values. For the same C&C data, even though the content
of data (i.e., planned velocity) is the same, the target position of the UAV
may vary from time due to dynamics during transmission. To quantify the value
of the C&C data in the current time $t$, we introduce the VoI, denoted as
$\mathrm{VoI}(\boldsymbol{m}_{i})$.

We assume that when the UAV ranks the queue and executes the C&C data
according to the order at $t$, there is $n$ ($n\in\mathbbm{N}$) C&C data in
the queue
$\\{\boldsymbol{m}^{\mathrm{Q}_{1}},...,\boldsymbol{m}^{\mathrm{Q}_{l}},...\boldsymbol{m}^{\mathrm{Q}_{n}}$
($l\in\\{1,...,n\\}$). Based on the assumption, for any queue updating time
$t$, the index of TTI of this queue updating time $t$ is written as

$i^{\mathrm{E}}=\lfloor\frac{t}{T}\rfloor+1,t\in[0,TN^{\mathrm{TTI}})$ (16)

where $\lfloor.\rfloor$ is the floor function. Whenever the queue is updated,
the first C&C data in the queue will be executed at the flight controller.

At time $t\in[t_{i^{\mathrm{E}}-1},t_{i^{\mathrm{E}}})$, the UAV aims to fly
to the its target position $\boldsymbol{g}_{i^{\mathrm{E}}}$ as close as
possible at $t_{i^{\mathrm{E}}}$. As the UAV has no direct knowledge about its
target position $\boldsymbol{g}_{i^{\mathrm{E}}}$ and cannot know its actual
future position $\boldsymbol{p}_{i^{\mathrm{E}}}$, we propose a method for the
UAV to estimate them. The estimated target position of the UAV at
$t_{i^{\mathrm{E}}}$ is denoted as $\hat{\boldsymbol{g}}_{i^{\mathrm{E}}}$.
For the estimated actual position of the UAV at $t_{i^{\mathrm{E}}}$, due to
that different C&C data $\boldsymbol{m}^{\mathrm{Q}_{l}}$ in the queue may
control the UAV to fly to different positions, different C&C data
$\boldsymbol{m}^{\mathrm{Q}_{l}}$ has different UAV’s estimated actual
position $\hat{\boldsymbol{p}}_{i^{\mathrm{E}},\mathrm{Q}_{l}}$. For the C&C
data $\boldsymbol{m}^{\mathrm{Q}_{l}}$ in the queue, if its estimated actual
position $\hat{\boldsymbol{p}}_{i^{\mathrm{E}},\mathrm{Q}_{l}}$ is closer to
the estimated target position $\hat{\boldsymbol{g}}_{i^{\mathrm{E}}}$, it will
be more valuable. As a result, we define the VoI of
$\boldsymbol{m}^{\mathrm{Q}_{l}}$ as

$\mathrm{VoI}(\boldsymbol{m}^{\mathrm{Q}_{l}})=-||\hat{\boldsymbol{p}}_{i^{\mathrm{E}},\mathrm{Q}_{l}}-\hat{\boldsymbol{g}}_{i^{\mathrm{E}}}||,$
(17)

where $||.||$ is the Frobenius norm used to calculate the distance between the
estimated actual position and estimated target position at
$t_{i^{\mathrm{E}}}$. The lower this distance is, the more valuable or more
important this C&C data is.

Next, we present how we obtain $\hat{\boldsymbol{g}}_{i^{\mathrm{E}}}$ and
$\hat{\boldsymbol{p}}_{i^{\mathrm{E}},\mathrm{Q}_{l}}$, respectively. We
define $\hat{\boldsymbol{g}}_{i^{\mathrm{E}}}$ as the latest position where
the BS expects the UAV to reach, which can be obtained in two steps, as shown
in Fig. 6.

Figure 6: The process of estimating the UAV’s target position
$\hat{\boldsymbol{g}}_{i^{\mathrm{E}}}$. Figure 7: The process of estimating
the UAV’s actual position
$\hat{\boldsymbol{p}}_{i^{\mathrm{E}},\mathrm{Q}_{l}}$.

First, we calculate the AoI of all C&C data $\boldsymbol{m}^{\mathrm{Q}_{l}}$
in the queue to find the latest generated C&C data
$\boldsymbol{m}^{\mathrm{Q}_{l^{*}}}$. For $l^{\mathrm{th}}$ C&C data
$\boldsymbol{m}^{\mathrm{Q}_{l}}$, its AoI is calculated as

$\mathrm{AoI}(\boldsymbol{m}^{\mathrm{Q}_{l}})=t-t_{i^{\mathrm{Q}_{l}}-1},$
(18)

where $t_{i^{\mathrm{Q}_{l}}-1}$ is the generation time of
$\boldsymbol{m}^{\mathrm{Q}_{l}}$. Based on that, we can obtain the index of
the C&C data in the queue with the smallest AoI as

$l^{*}=\arg\min_{l}\mathrm{AoI}(\boldsymbol{m}^{\mathrm{Q}_{l}}).$ (19)

Second, due to the C&C data $\boldsymbol{m}^{\mathrm{Q}_{l^{*}}}$ is the
planned velocity vector
$(v_{i^{\mathrm{Q}_{l^{*}}}}^{x},v_{i^{\mathrm{Q}_{l^{*}}}}^{y},z_{i^{\mathrm{Q}_{l^{*}}}}^{z})$
of the UAV and its planned execution time is $T$, we are able to attain the
planned movement of the UAV as $T\boldsymbol{m}^{\mathrm{Q}_{l^{*}}}$. Based
on that, we estimate the target position for the UAV at time
$t_{i^{\mathrm{E}}}$ as

$\hat{\boldsymbol{g}}_{i^{\mathrm{E}}}=\boldsymbol{p}_{i^{\mathrm{Q}_{l^{*}}}-1}+T\boldsymbol{m}^{\mathrm{Q}_{l^{*}}},$
(20)

where $\boldsymbol{p}_{i^{\mathrm{Q}_{l^{*}}}-1}$ is the UAV’s actual position
at the generation time $t_{i^{\mathrm{Q}_{l^{*}}}-1}$ of the C&C data
$\boldsymbol{m}^{\mathrm{Q}_{l^{*}}}$ in the queue.

Then, we estimate the UAV’s actual position
$\hat{\boldsymbol{p}}_{i^{\mathrm{E}},\mathrm{Q}_{l}}$ for the C&C data
$\boldsymbol{m}^{\mathrm{Q}_{l}}$ in two steps, as shown in Fig. 7. First, as
the queue is updated at $t$, the UAV will execute the C&C data
$\boldsymbol{m}^{\mathrm{Q}_{l}}$ from $t$ to $t_{i^{\mathrm{E}}}$, with the
movement as

$\Delta\hat{\boldsymbol{p}}_{i^{\mathrm{E}},\mathrm{Q}_{l}}=\boldsymbol{m}^{\mathrm{Q}_{l}}(t_{i^{\mathrm{E}}}-t).$
(21)

Second, since the UAV know its current position $\boldsymbol{p}_{\frac{t}{T}}$
at $t$, we estimate the actual position of the UAV at time
$t_{i^{\mathrm{E}}}$ as

$\hat{\boldsymbol{p}}_{i^{\mathrm{E}},\mathrm{Q}_{l}}=\boldsymbol{p}_{\frac{t}{T}}+\Delta\hat{\boldsymbol{p}}_{i^{\mathrm{E}},\mathrm{Q}_{l}}.$
(22)

After estimating $\hat{\boldsymbol{g}}_{i^{\mathrm{E}}}$ and
$\hat{\boldsymbol{p}}_{i^{\mathrm{E}},\mathrm{Q}_{l}}$, the VoI
$\mathrm{VoI}(\boldsymbol{m}^{\mathrm{Q}_{l}})$ of the C&C data
$\boldsymbol{m}^{\mathrm{Q}_{l}}$ can be calculated using Eq. (17).

####  IV-B3 Semantic Information Extraction

As the optimal C&C data for one TTI is generated at the beginning of this TTI,
the C&C data in the queue is most valuable for the UAV if its AoI is less than
$T$. Otherwise, its semantic information only depends on its VoI. To normalize
the range of semantic information, we define the function of semantic
information for $\boldsymbol{m}^{\mathrm{Q}_{l}}$ as

$\mathrm{SI}(\boldsymbol{m}^{\mathrm{Q}_{l}})=u[T-\mathrm{AoI}(\boldsymbol{m}^{\mathrm{Q}_{l}})]+u[\mathrm{AoI}(\boldsymbol{m}^{\mathrm{Q}_{l}})-T]e^{\mathrm{VoI}(\boldsymbol{m}^{\mathrm{Q}_{l}})},$
(23)

where the function $u(.)$ is formulated as

$u(x)=\begin{cases}1,\quad x>0\\\ 0,\quad x\leq 0.\end{cases}$ (24)

If $\mathrm{AoI}(\boldsymbol{m}^{\mathrm{Q}_{l}})<T$,
$\boldsymbol{m}^{\mathrm{Q}_{l}}$ is just generated at the beginning of this
TTI, thus it has the highest semantic information which is equal to 1.
Otherwise, $\boldsymbol{m}^{\mathrm{Q}_{l}}$ is assumed to be outdated and its
$\mathrm{VoI}(\boldsymbol{m}^{\mathrm{Q}_{l}})$ decides its importance to the
task. As the $\mathrm{VoI}(\boldsymbol{m}^{\mathrm{Q}_{l}})$ increases, its
semantic information increases and the upper bound is 1. At the beginning of
$i^{\mathrm{th}}$ TTI $t_{i-1}$ or when the UAV successfully decodes one new
C&C data at $t_{i}^{\mathrm{A}}$, the UAV starts to order the queue based on
the semantic information in a descending order, as shown in
$\mathbf{Algorithm\ 4}$.

Input: The UAV’s queue $\mathcal{Q}$, at the beginning of $i^{\mathrm{th}}$
TTI $t=t_{i-1}$ or at the time $t=t_{i}^{\mathrm{A}}$ when the UAV
successfully decodes a C&C data.

Output: The UAV’s reordered queue $\mathcal{Q}^{\mathrm{R}}$

1 Calculate $i^{\mathrm{E}}$ using Eq. (16).

2 for  _$l\leftarrow 1$ to $n$_ do

3 Calculate $\Delta\hat{\boldsymbol{p}}_{i^{\mathrm{E}},\mathrm{Q}_{l}}$ from
$t$ to $t_{i^{\mathrm{E}}}$ using Eq. (21).

4 Calculate $\hat{\boldsymbol{p}}_{i^{\mathrm{E}},\mathrm{Q}_{l}}$ at
$t_{i^{\mathrm{E}}}$ using Eq. (22).

5 Calculate $\mathrm{AoI}(\boldsymbol{m}^{\mathrm{Q}_{l}})$ using Eq. (18).

6

7 end for

8Obtain the index $l^{*}$ using Eq. (19).

9 Obtain $\hat{\boldsymbol{g}}_{i^{\mathrm{E}}}$ at $t_{i^{\mathrm{E}}}$ using
Eq. (20).

10 for  _$l\leftarrow 1$ to $n$_ do

11 Calculate $\mathrm{VoI}(\boldsymbol{m}^{\mathrm{Q}_{l}})$ using Eq. (17).

12 Calculate $\mathrm{SI}(\boldsymbol{m}^{\mathrm{Q}_{l}})$ using Eq. (23).

13

14 end for

15Reorder the queue $\mathcal{Q}$ to $\mathcal{Q}^{\mathrm{R}}$ based on the
semantic information in descending order.

Algorithm 4 VA-QOM

###  IV-C Communication and Control Process

As shown in Fig. 8, we plot the exemplar timeline of the BS and the UAV to
introduce the communication and control process in our proposed TSRC
framework. At the beginning of $i^{\mathrm{th}}$ TTI $t_{i-1}$, the agent at
the BS selects the action $A_{i}$ from past observations to generate the C&C
data $\boldsymbol{m}_{i}$ which is transmitted by the proactive repetition
scheme. At $t_{i-1}$ or at $t_{i}^{\mathrm{A}}$ When the UAV successfully
decodes this C&C data $\boldsymbol{m}_{i}$, the UAV reorders its queue based
on the semantic information and executes the C&C data in the queue according
to its order to update its position. At the end of $i^{\mathrm{th}}$ TTI
$t_{i}$, the UAV sends its real-time position $\boldsymbol{p}_{i}$ back to the
BS. The agent at the BS utilizes $\boldsymbol{p}_{i}$ to obtain the reward
$R_{i}$ and the next state $S_{i+1}$ which are stored in the observation. This
process continues looping until all the C&C data are transmitted.

Figure 8: Exemplar timeline in our proposed TSRC framework.

##  V Simulation Results

In this section, we examine the effectiveness of our proposed TSRC framework
and compare it with the traditional UAV control framework (TUCF) via
simulation. To assess the individual contributions of the TSRC components, we
also conduct separate simulations for the DeepPro and VA-QOM. The start
position of the UAV is
$\boldsymbol{p}_{0}=\boldsymbol{g}_{0}=(80\mathrm{m},80\mathrm{m},20\mathrm{m})$
at time $t=0\mathrm{s}$. We set $T$ and $N^{\mathrm{TTI}}$ as 1 ms and 99,
respectively. For parameters in the channel model, we adopt $f_{c}$,
$\gamma_{\mathrm{th}}$, $\sigma^{2}$, and $\mathrm{P}$ as 5 GHz, 5.5 dB, -104
dBm, and 18 dBm, respectively. The hyperparameters $\epsilon$, $\gamma$,
$\lambda_{\mathrm{RMS}}$ of the DQN algorithm are set to 1, 0.1, and
$10^{-4}$, respectively. The size of one C&C data is 104 bytes, and the queue
size $Q_{\mathrm{max}}$ and the parameter $N^{\mathrm{M}}$ are 10 and 9,
respectively. The velocity sets are $V^{\mathrm{X}},\
V^{\mathrm{Y}}\in\\{-5000,-4000,...,5000\\}$ and $V^{\mathrm{Z}}\in\\{0\\}$,
respectively. The results are obtained by simulating each algorithm 1000 times
and taking the average value.

(a) Trajectory comparison.

(b) Error comparison.

(c) MSE comparison.

Figure 9: The comparison of various schemes.

We compare different schemes (TUCF, VA-QOM, DeepPro, and TSRC) via simulation
using two types of trajectories, as shown in Fig. 9. The first trajectory is a
simple one but typical for our use case, while the second is more complex with
a randomly generated trajectory. Fig. 9 (a) plots the trajectory comparison
over different schemes. We can observe that the trajectory of the TUCF is
notably most distant from the target trajectory. The TUCF and the VA-QOM share
a similar trajectory shape because their generated C&C data is the same.
However, the VA-QOM, with its ability to update target positions and
prioritise the queue, achieves a trajectory closer to the target one compared
to the TUCF. Due to the severe packet loss and delay, the update in the VA-QOM
is always not on time, which still results in a huge gap between the
trajectory of the VA-QOM and the target one. The DeepPro’s trajectory is
closer to the target one than the VA-QOM because it can generate optimal C&C
data via the DQN algorithm and decrease the risk of packet loss by the
proactive repetition scheme. By integrating the DeepPro and the VA-QOM, the
trajectory of our proposed TSRC framework is the closest to the target
trajectory, indicating superior performance.

Fig. 9 (b) plots the error comparison for different schemes over time. It can
be noticed that our proposed TSRC achieves the lowest overall error, with the
DeepPro and VA-QOM also showing lower overall errors compared to TUCF.
Interestingly, the error of the DeepPro and the TSRC remains stable over time,
whereas the error of the TUCF and the VA-QOM fluctuate substantially over
time. The reason is that the TUCF and VA-QOM are more sensitive to the
communication environment with the same C&C data sequence, while the DeepPro
and the TSRC can dynamically adapt their generated C&C data to the
communication environment automatically. Fig. 9 (c) presents the MSE
comparison over different schemes. The TUCF achieves the highest MSE. Compared
to TUCF, both the DeepPro and VA-QOM substantially reduce MSE by 86.92$\%$ and
40.61$\%$, respectively. The TSRC, based on the integration of the DeepPro and
VA-QOM, achieves the lowest MSE, resulting in a remarkable 91.51$\%$ reduction
compared to TUCF.

Fig. 10 presents the impact of the proactive repetition scheme on the MSE for
different schemes (TUCF, DeepPro, VA-QOM and TSRC).

(a) $T_{\mathrm{rep}}=5*10^{-5}$ (s).

(b) $K_{\mathrm{max}}=3$.

Figure 10: The MSE comparison over different values of $K_{\mathrm{max}}$ and
$T_{\mathrm{rep}}$.

It is noticeable that our proposed TSRC scheme achieves the lowest MSE in
various values of $K_{\mathrm{max}}$ and $T_{\mathrm{rep}}$, while the TUCF
scheme has the highest MSE. Interestingly, the changes in $K_{\mathrm{max}}$
and $T_{\mathrm{rep}}$, which are highly related to the packet loss and delay
of the C&C data, have a more obvious impact on the MSE of the TUCF and VA-QOM
schemes compared to the DeepPro and TSRC. Since the DeepPro and TSRC are able
to generate optimal C&C data to adapt to varying packet loss and delay
conditions, their MSE remains more stable during such changes. Fig. 10 (a)
plots the MSE for various schemes over different values of $K_{\mathrm{max}}$
with a fixed $T_{\mathrm{rep}}=5*10^{-5}$. We can observe that increasing
$K_{\mathrm{max}}$ leads to a decrease in MSE for all schemes. This reduction
is due to a decrease in packet loss of each C&C data as $K_{\mathrm{max}}$
increases. Additionally, the MSE gap between the TUCF and DeepPro schemes
diminishes because DQN is more likely to select the C&C data which is the same
as the TUCF when $K_{\mathrm{max}}$ is higher. Fig. 10 (b) shows the MSE for
various schemes against different values of $T_{\mathrm{rep}}$ with a fixed
$K_{\mathrm{max}}=3$. It can be observed that with the $T_{\mathrm{rep}}$
increasing, there is a rise in the MSE of all schemes. The reason is that with
a larger $T_{\mathrm{rep}}$, the BS needs more time to transmit the next
repetition if there is no ACK back, which increases the average arrival time
of each C&C data.

Figure 11: The relationship between the MSE and the average number of
transmissions for each C&C data when $T_{\mathrm{rep}}=5*10^{-5}$.

Fig. 11 plots the relationship between the MSE and the average number of
actual transmissions for each C&C data over time of all schemes when
$T_{\mathrm{rep}}=5*10^{-5}$ with various $K_{\mathrm{max}}$. We can observe
that as the average number of transmissions per C&C data increases, the MSE
for all schemes decreases. Among all schemes, our proposed TSRC framework
maintains the lowest MSE, while the TUCF exhibits the highest MSE when the
average number of transmissions is nearly equivalent. This indicates that the
design of the DeepPro and VA-QOM is able to enhance task performance, and the
combination of them (TSRC) could significantly improve task performance while
maintaining a similar number of transmissions. Moreover, we can see that our
proposed TSRC framework requires fewer transmissions than the TUCF while
achieving a comparable MSE, demonstrating its potential to reduce the number
of transmissions while adhering to task performance constraints compared to
the TUCF.

##  VI Conclusion

In this paper, we proposed task-oriented and semantics-aware communication in
robotic control (TSRC) which incorporates both semantic-level and
effectiveness-level for a real-time UAV waypoint transmission task. At the
semantic level, we proposed the value of information (VoI) and the age of
information (AoI) based queue ordering mechanism (VA-QOM) scheme at the UAV to
rank the importance of data packets in the queue in accomplishing a task based
on the AoI and the newly defined VoI. At the effectiveness level, we proposed
a DeepPro scheme at the BS, which facilitates the generation of optimal
control and command (C&)C data to accomplish the task and elevates the
probability of successful transmission for each C&C data by optimizing the
mean square error (MSE). Our simulation results shed light on that our
proposed TSRC framework can decrease the number of average C&C data
transmissions when maintaining a similar MSE and enhance the MSE when
maintaining a similar number of C&C data transmissions.

## References

  * [1] Y. Liu, X. Ma, L. Shu, G. P. Hancke, and A. M. Abu-Mahfouz, “From Industry 4.0 to Agriculture 4.0: Current Status, Enabling Technologies, and Research Challenges,” IEEE Trans. Ind. Informatics, vol. 17, pp. 4322–4334, Jun. 2021.
  * [2] A. Alam, “Educational Robotics and Computer Programming in Early Childhood Education: A Conceptual Framework for Assessing Elementary School Students’ Computational Thinking for Designing Powerful Educational Scenarios,” in Proc. 2022 Int. Conf. Smart Technologies Syst. Next Gener. Comput. (ICSTSN), pp. 1–7, Mar. 2022.
  * [3] T. Haidegger, “Autonomy for Surgical Robots: Concepts and Paradigms,” IEEE Trans. Med. Robot. Bionics, vol. 1, pp. 65–76, Apr. 2019.
  * [4] S. H. Alsamhi, O. Ma, and M. S. Ansari, “Convergence of Machine Learning and Robotics Communication in Collaborative Assembly: Mobility, Connectivity and Future Perspectives,” J. Intell. Robot. Syst., vol. 98, pp. 541–566, Oct. 2020.
  * [5] Y. Zeng, R. Zhang, and T. J. Lim, “Wireless communications with unmanned aerial vehicles: opportunities and challenges,” IEEE Commun. Mag., vol. 54, pp. 36–42, May 2016.
  * [6] M. Shafi, A. F. Molisch, P. J. Smith, T. Haustein, P. Zhu, P. De Silva, F. Tufvesson, A. Benjebbour, and G. Wunder, “5G: A Tutorial Overview of Standards, Trials, Challenges, Deployment, and Practice,” IEEE J. Sel. Areas Commun., vol. 35, pp. 1201–1221, Apr. 2017.
  * [7] H. Ren, C. Pan, Y. Deng, M. Elkashlan, and A. Nallanathan, “Joint Power and Blocklength Optimization for URLLC in a Factory Automation Scenario,” IEEE Trans. Wireless Commun., vol. 19, pp. 1786–1801, Dec. 2020.
  * [8] C. Pan, H. Ren, Y. Deng, M. Elkashlan, and A. Nallanathan, “Joint Blocklength and Location Optimization for URLLC-Enabled UAV Relay Systems,” IEEE Commun. Lett., vol. 23, pp. 498–501, Jan. 2019.
  * [9] B. Chang, L. Zhang, L. Li, G. Zhao, and Z. Chen, “Optimizing Resource Allocation in URLLC for Real-Time Wireless Control Systems,” IEEE Trans. Veh. Technol., vol. 68, pp. 8916–8927, Jul. 2019.
  * [10] “Minimum requirements related to technical performance for IMT-2020 radio interface(s),” tech. rep., doc. ITU-R M, Oct. 2016.
  * [11] C. D. Alwis, A. Kalla, Q.-V. Pham, P. Kumar, K. Dev, W.-J. Hwang, and M. Liyanage, “Survey on 6G Frontiers: Trends, Applications, Requirements, Technologies and Future Research,” IEEE Open J. Commun. Soc., vol. 2, pp. 836–886, Apr. 2021.
  * [12] J. Morales, G. Rodriguez, G. Huang, and D. Akopian, “Toward UAV Control via Cellular Networks: Delay Profiles, Delay Modeling, and a Case Study Within the 5-mile Range,” IEEE Trans. Aerosp. Electron. Syst., vol. 56, pp. 4132–4151, Apr. 2020.
  * [13] G. Gui, M. Liu, F. Tang, N. Kato, and F. Adachi, “6G: Opening New Horizons for Integration of Comfort, Security, and Intelligence,” IEEE Wireless Commun., vol. 27, pp. 126–132, Mar. 2020.
  * [14] “Framework and overall objectives of the future development of IMT for 2030 and beyond,” tech. rep., ITU-T Tech. Rep., Jun. 2023.
  * [15] A. Ranjha, G. Kaddoum, and K. Dev, “Facilitating URLLC in UAV-Assisted Relay Systems With Multiple-Mobile Robots for 6G Networks: A Prospective of Agriculture 4.0,” IEEE Trans. Ind. Informatics, vol. 18, pp. 4954–4965, Nov. 2022.
  * [16] S. Zhang, H. Zhang, and L. Song, “Beyond D2D: Full Dimension UAV-to-Everything Communications in 6G,” IEEE Trans. Veh. Technol., vol. 69, pp. 6592–6602, Apr. 2020.
  * [17] W. Sun, H. Zhang, R. Wang, and Y. Zhang, “Reducing Offloading Latency for Digital Twin Edge Networks in 6G,” IEEE Trans. Veh. Technol., vol. 69, pp. 12240–12251, Aug. 2020.
  * [18] T. Johannink, S. Bahl, A. Nair, J. Luo, A. Kumar, M. Loskyll, J. A. Ojea, E. Solowjow, and S. Levine, “Residual Reinforcement Learning for Robot Control,” in Proc. 2019 Int. Conf. Robotics Autom. (ICRA), pp. 6023–6029, May 2019.
  * [19] C. Yang, D. Huang, W. He, and L. Cheng, “Neural Control of Robot Manipulators With Trajectory Tracking Constraints and Input Saturation,” IEEE Trans. Neural Netw. Learn. Syst., vol. 32, pp. 4231–4242, Aug. 2021.
  * [20] P. Iyer, S. Tarekar, and S. Dixit, “Hand Gesture Controlled Robot,” in Proc. 2019 9th Int. Conf. Emerging Trends Eng. Technol. - Signal Inf. Process. (ICETET-SIP-19), pp. 1–5, Nov. 2019.
  * [21] H. Zhou, X. Liu, Y. Deng, N. Pappas, and A. Nallanathan, “Task-Oriented and Semantics-Aware 6G Networks,” arXiv e-prints, p. arXiv:2210.09372, Oct. 2022.
  * [22] M. Nasr-Azadani, J. Abouei, and K. N. Plataniotis, “Single- and Multiagent Actor-Critic for Initial UAV’s Deployment and 3-D Trajectory Design,” IEEE Internet Things J., vol. 9, pp. 15372–15389, Feb. 2022.
  * [23] Z. Wang, R. Zhou, C. Hu, and Y. Zhu, “Online Iterative Learning Compensation Method Based on Model Prediction for Trajectory Tracking Control Systems,” IEEE Trans. Ind. Informatics, vol. 18, pp. 415–425, Jun. 2022.
  * [24] Z. Meng, C. She, G. Zhao, and D. De Martini, “Sampling, Communication, and Prediction Co-Design for Synchronizing the Real-World Device and Digital Model in Metaverse,” IEEE J. Sel. Areas Commun., vol. 41, pp. 288–300, Nov. 2023.
  * [25] X. Luo, H.-H. Chen, and Q. Guo, “Semantic Communications: Overview, Open Issues, and Future Research Directions,” IEEE Wireless Commun., vol. 29, pp. 210–219, Jan. 2022.
  * [26] D. Gündüz, Z. Qin, I. E. Aguerri, H. S. Dhillon, Z. Yang, A. Yener, K. K. Wong, and C.-B. Chae, “Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications,” IEEE J. Sel. Areas Commun., vol. 41, pp. 5–41, Nov. 2023.
  * [27] H. Xie, Z. Qin, G. Y. Li, and B.-H. Juang, “Deep Learning Enabled Semantic Communication Systems,” IEEE Trans. Signal Process., vol. 69, pp. 2663–2675, Apr. 2021.
  * [28] Z. Weng and Z. Qin, “Semantic Communication Systems for Speech Transmission,” IEEE J. Sel. Areas Commun., vol. 39, pp. 2434–2444, Jun. 2021.
  * [29] M. Kountouris and N. Pappas, “Semantics-Empowered Communication for Networked Intelligent Systems,” IEEE Commun. Mag., vol. 59, pp. 96–102, Jun. 2021\.
  * [30] Z. Wang, Y. Deng, and A. H. Aghvami, “Task-oriented and Semantics-aware Communication Framework for Augmented Reality,” arXiv e-prints, p. arXiv:2306.15470, June 2023.
  * [31] R. D. Yates, Y. Sun, D. R. Brown, S. K. Kaul, E. Modiano, and S. Ulukus, “Age of Information: An Introduction and Survey,” IEEE J. Sel. Areas Commun., vol. 39, pp. 1183–1210, Mar. 2021.
  * [32] G. J. Stamatakis, N. Pappas, A. Fragkiadakis, and A. Traganitis, “Semantics-Aware Active Fault Detection in IoT,” in Proc. 2022 20th Int. Symp. Model. Optim. Mobile, Ad hoc, Wireless Networks (WiOpt), pp. 161–168, Sep. 2022.
  * [33] W. Yang, H. Du, Z. Q. Liew, W. Y. B. Lim, Z. Xiong, D. Niyato, X. Chi, X. Shen, and C. Miao, “Semantic Communications for Future Internet: Fundamentals, Applications, and Challenges,” IEEE Commun. Surv. Tutorials, vol. 25, pp. 213–250, Nov. 2023.
  * [34] A. Kosta, N. Pappas, A. Ephremides, and V. Angelakis, “Age and value of information: Non-linear age case,” in Proc. 2017 IEEE Int. Symp. Inf. Theory (ISIT), pp. 326–330, Jun. 2017.
  * [35] E. Fountoulakis, N. Pappas, and M. Kountouris, “Goal-oriented Policies for Cost of Actuation Error Minimization in Wireless Autonomous Systems,” IEEE Commun. Lett., pp. 1–1, Jun. 2023.
  * [36] D. Huang, F. Gao, X. Tao, Q. Du, and J. Lu, “Toward Semantic Communications: Deep Learning-Based Image Semantic Coding,” IEEE J. Sel. Areas Commun., vol. 41, pp. 55–71, Nov. 2023.
  * [37] A. Gupta, P. Zhang, G. Lalwani, and M. Diab, “CASA-NLU: Context-Aware Self-Attentive Natural Language Understanding for Task-Oriented Chatbots,” arXiv e-prints, p. arXiv:1909.08705, Sept. 2019.
  * [38] H. Sun, J. Bao, Y. Wu, and X. He, “Mars: Modeling context & state representations with contrastive learning for end-to-end task-oriented dialog,” in Findings of the Association for Computational Linguistics: ACL 2023 (A. Rogers, J. Boyd-Graber, and N. Okazaki, eds.), (Toronto, Canada), pp. 11139–11160, Association for Computational Linguistics, July 2023\.
  * [39] Y. Xu, H. Zhou, and Y. Deng, “Task-Oriented Semantics-Aware Communication for Wireless UAV Control and Command Transmission,” IEEE Commun. Lett., pp. 1–1, Jun. 2023.
  * [40] Y. Liu, Y. Deng, M. Elkashlan, A. Nallanathan, and G. K. Karagiannidis, “Analyzing Grant-Free Access for URLLC Service,” IEEE J. Sel. Areas Commun., vol. 39, pp. 741–755, Aug. 2021.